GSoC 2019 Final Submission

Project: Add PRU drivers to RTEMS

Student: Nils Hölscher

Mentors: Chris Johns, Amaan Cheval, Kuan-Hsun Chen, Sarvesh Patkar

Original Proposal

Project Overview

This project added PRU support to RTEMS, using the Beaglebone Black (BBB). The BBB has a Texas Instruments AM3358 SoC with a Programmable Real-Time Unit (PRU). The PRU is able to connect to the SoCs i/o within one cycle. This should enable the RTEMS community to develop heavily i/o dependent tasks on the Texas Instruments SoCs with PRUs.

However currently only tested on the BBB, with its PRUSS-v2.

Project Goals

The project goals were to to port the Linux UIO drivers to RTEMS. However this changed after I found out that I missed a part of the Linux driver in my project proposal. I missed the “actual” UIO driver in Linux and started porting just the library for accessing this driver. After realising this I looked into Linux UIO drivers and realized porting them would not be easy. So we redefined the goal to porting the FreeBSD driver, as this would be easier to integrate into RTEMS and these drivers are also more feature rich. My mentors and me also agreed on having a RTEMS-Shell command for accessing the PRU during development would be nice, so we also added this to the goals.

My Work

Repositories:

Gists:

Project Phases

This section will describe each Phase of my neat GSoC project. Most of the information is taken from our weekly GSoC meetings at RTEMS-IRC with the GSoC’ers and some mentors. The short summaries of our meeting can be found here.

Phase I [May 27 – June 28]

During this phase my time was very compromised, because in Germany semester has just started. And I also wasted quite some time with the aforementioned misconceptions.

  • The Project started with coding some test applications for the PRU and to run them via Linux. This was necessary to ensure that the applications behave the same way on RTEMS later on.
  • I spend a lot time with trying to route PRU access to the GPIO pins of the BBB via device tree overlays. Later I realised that Linux uses a management software that resets all device tree definitions after boot and decided to go on with the project, since this software is not part of RTEMS.
  • After finishing running my sample code on Linux it was time to move on to RTEMS and build a running SD-Card. At this point I had to figure out the boot process with U-Boot, RTEMS and device tree overlays.
  • Figuring out the right device tree and its overlay took some time and help. So I started writing the SHELL command whilst waiting for answers concerning device tree.
  • In the las week during this phase I got everything to boot and was able to test the Linux “drivers” without success.

Phase II [June 29 – July 26]

In this phase I realized my misconceptions and my focus changed to the FreeBSD drivers for RTEMS.

  • Because my Linux “drivers” weren’t working I spent the first week enabling debugging via Network. Luckily my mentor Chris Johns was just working on this so I didn’t need to get a JTAG controller.
  • Also I had to reassess my device tree again, because the one from Linux, I was using, caused trouble. In the end I changed over to FreeBSDs device tree.
  • After I realized that the /dev/uio devices were missing under RTEMs, it was obvious that I missed some part of the PRU driver in my proposal. Following this I had to discuss with the community and it was decided that I would go with the FreeBSD drivers. This rendered parts of my work useless. ):
  • Now that everything was decided I ported the FreeBSD drivers to RTEMS-libbsd.
  • When testing the ported drivers I encountered the projects biggest blocker. The PRU driver needed a driver, that was able to set the PRU clock. But that driver always initialised after the PRU driver and so it could not attach itself to the system.

Phase III [July 27 – August 26]

  • The blocker kept me busy for nearly three weeks. And because my debugging solution depended on the RTEMS-libbsd networking modules I wasn’t able to debug this whole process of interest. So I had to read into the whole FreeBSD init process and investigate the differences to RTEMS-libbsd. During this work I wrote down what I learned about the init process and had my program print everything during initialization.
  • I wrote a Blog entry about the whole process and a few paragraphs for the rtems Documentation project.
  • When I found the solution to this problem, it got merged into RTEMS-libbsd pretty fast.
  • In the last two week I ported my SHELL-command, now using the FreeBSd driver and tested all I could during the time I was left with.
  • I successfully tested my loop example running for exactly 10 second.
  • Please see Future Work for what still needs to be done.

My Experiences

I had a nice time during GSoC and the community was very friendly and helpful.

During my time working, I learned a lot of things, for example I have never heard of device trees before and also learned my share about the FreeBSD init process.

But for me the greatest feat is to have openly communicated with an open source community, taking away the fear to do so in the future.

If I would do the same project again I would take more time preparing my initial proposal. Also I would directly start communicating on the mailing list and not just with my mentors, as this will generate more input by having a wider audience. Also when starting GSoC the whole community should be seen as mentors.

Future Work

I still wasn’t able to read IRQ’s triggered by the PRU (see here). However this doesn’t seem to be related to the driver but to some issue with the app I use for testing. When this is resolved I still need to see my drivers merged and add Documentation.

After resolving this and having my code merged I think it might be nice to build something using pru and maybe propose to my university staff to teach with RTEMS in their courses. I would call myself a part of the RTEMS community and see what the future brings.

The libBSD init process in rtems with focus on driver modules

After porting FreeBSD’s PRU driver to rtems-libbsd, I encountered a problem concerning the init process. The PRU driver needs TI’s prcm module to be loaded when initialising itself. However the current rtems-libbsd init process doesn’t allow me to sort that dependency. So I had to investigate the whole process, since I don’t have JTAG . );

Linking the drivers.

In FreeBSD Modules are registered using the DRIVER_MODUL macro. This macro hands down all its information via a macro chain and some structs containing additional information. The DRIVER_MODULE is defined in bus.h and is actually a call of EARLY_DRIVER_MODULE_ORDERED with static order (SI_ORDER_MIDDLE) and pass level (BUS_PASS_DEFAULT). All data concerning the driver is stored in two structs and the module is forwarded to DECLARE_MODULE in module.h. The last macro of this chain is SYSINIT.

The init process

When using rtems with rtems-libbsd the whole libBSD init process get triggered by rtems_bsd_initialize() in rtems-kernel-init.c. This functions sets parameters for the BSD library and calls mi_startup() from init_main.c. This function iterates over the objects, linked by SYSINIT, and executes a bubble sort on them. After this the modules are ordered by subsystem and in their subsystem they are ordered by si-order. All objects are called in order with their defined function and data here. Focusing on driver modules this function is module_register_init() in kern_module.c. This function retrieves the data stored in additional structs during linking and uses the MOD_EVENT macro. This macro calls the modules event handler and hands threw the mod itself and its arguments. For bus related modules as our drivers are the handler refers to driver_module_handler in subr_bush.c. Here all bus devices are registered for activation.

Registering bus-devices.

I am currently investigating this topic. My guess is that the loop attaching devices to the bus does not run over the fdt multiple times as stated int he FreeBSD manual. UPDATE: LibBSD does indeed run over fdt only once. But after trying to add another run over fdt I discovered that all drivers are registered with the same pass level, which definitely shouldn’t be the case. The method calling a fdt pass for a pass level is called here.

Booting RTEMS with UBoot and the right FDT overlay

I have spinned quite some time on figuring out the whole device tree “mysterium” with uboot and rtems. But once you have a functioning SD-Image with a nice UBoot setup it works like a charm and updating kernel images is just copy paste.

Creating SD Image with UBoot and rtems

My current workflow is to use the rtems-boot-image script in rtems-tools to create a SD Image, which can be flashed to SD Card via dd. This requires a functional UBoot install. The command looks like this:

rtems-boot-image -o sd-card.img -b u-boot-beaglebone -s 32m -k rtems.exe -d am335x-boneblack.dtb ../u-boot/MLO ../u-boot/u-boot.img

-o sd-card.img: Name of the OutputFile.

-b u-boot-beaglebone: Name of the Board.

-s 32m: Size of the Image (32MB).

-k rtems.exe: Path to the rtems kernel file.

-d am335x-boneblack.dtb: Path to device tree.

../u-boot/MLO: Path to first stage boot loader.

../u-boot/u-boot.img: Path to second stage boot loader.

When this command has finished the SD card can be flashed with the dd command or a tool like etcher.

The .dtb File

The device tree is taken from FreeBSD since the linux device trees tend to be useless or buggy with rtems. The device tree sources can be found here and be build with the tools linked here. Before building the environment variable “MACHINE” has to be set to target platform, on a BBB it’s export MACHINE=arm. I also had to change all includes in the .dts files to absolute paths. With this we have the FreeBSD device tree overlay for the BBB.

Applying fdt overlay via UBoot

In this step I provide my uEnv.txt, that apply the fdt overlay needed to use the pruss units on BBB. Since I don’t think to give a better explanation than the guide I used myself, I will only link to it.

setenv bootdelay 5
uenvcmd=run boot_rtems; reset;
rtems_banner=echo ""; echo "RTEMS u-boot-beaglebone (arm-ti-am335x_evm)"; echo " rtems-boot-image v5.0.not_released"; echo "";
boot_rtems=run rtems_banner; echo "Loading fileio.exe.img"; load mmc 0 0x82000000 fileio.exe.img; echo "Loading am335x-boneblack.dtb"; load mmc 0 0x88000000 am335x-boneblack.dtb; echo "Loading AM335X-PRU-UIO-00A0.dtbo"; load mmc 0 0x880f0000 AM335X-PRU-UIO-00A0.dtbo; echo "Applying fdt overlay"; fdt addr 0x88000000; fdt resize 65536; fdt apply 0x880f0000;  bootm 0x82000000 - 0x88000000

Conclusion

Since I spent way too much time figuring this one out. I hope this helps everyone, who also has problems figuring out how to use UBoot, rtems and fdt overlays. Sadly documentation is sparse.

Locating PRUSS driver in RTEMS

After discussing with my mentors and on devel@rtems.org, I decided to locate the device driver in bsps/arm/beagle/pruss/prussdrv.c and __prussdrv.h. The headers are located in bsps/arm/beagle/include/bsp/prussdrv.h and pruss_intc_mapping.h.

This post is mainly meant to document how to port the drivers to another bsp that uses an TI Sitara AM33xx or AM18xx SoC.

To compile the files I also had to register them in the automake files.

I added the following code to /bsps/arm/beagle/headers.am

include_bsp_HEADERS += ../../../../../../bsps/arm/beagle/include/bsp/prussdrv.h
include_bsp_HEADERS += ../../../../../../bsps/arm/beagle/include/bsp/pruss_intc_mapping.h

And the driver files have to be added to /c/src/lib/libbsp/arm/beagle/Makefile.am

# PRUSS
librtemsbsp_a_SOURCES += ../../../../../../bsps/arm/beagle/pruss/prussdrv.c

When these drivers are copied to another bsp the paths have to look like:

…/bsps/arm/YOUR_BOARD/…

Changes to the original drivers

All changes to the drivers that differ from the original code, taken from am335x_pru_package, are marked with #defines using __rtems__ or having an /* __rtems__ */ comment behind them. These are the coding conventions taken from rtems-libbsd and can be found here.

Update: Device Tree and Pins

After discussing my last blog post with my co GSoC’er Vijay Kumar Banerjee, we came to some conclusions concerning the device tree overlay.

Vijay Kumar Banerjee is also working with rtems on the BBB visit his Blog, if you are interested.

I came to the conclusion, that the device tree overlay I provided before works. However the pins I want to set up are already used by another unit on the ocp-bus. My previous suspicion, that another application uses these pins already, came true.

The output when trying to access the pins

[   38.186334] pinctrl-single 44e10800.pinmux: pin PIN13 already requested by ocp:P8_11_pinmux; cannot claim for 4a300000.pruss
[   38.463257] pinctrl-single 44e10800.pinmux: pin-13 (4a300000.pruss) status -22
[   38.709802] pinctrl-single 44e10800.pinmux: could not request pin 13 (PIN13) from group pinmux_pru_pru_pins  on device pinctrl-single
[   39.086278] pruss_uio 4a300000.pruss: Error applying setting, reverse things back
[   84.039763] watchdog: watchdog0: watchdog did not stop!
[   84.370168] reboot: Restarting system

This was the output I got when I tried to use the routed pins, and found out that the pins were already routed to another unit on the ocp-bus and therefore couldn’t be routed to pruss.

The bone-pinmux-helper does not help me.

When I investigated the BBB’s device tree overlays I found that all the pins are routed to the bone-pinmux-helper. Please take a look at its overlays:

[...]
	fragment@4 {
		target = <&ocp>;
		__overlay__ {

			P8_11_pinmux {
				compatible = "bone-pinmux-helper";
				status = "okay";
				pinctrl-names = "default", "gpio", "pruout";
				pinctrl-0 = <&P8_11_default_pin>;
				pinctrl-1 = <&P8_11_gpio_pin>;
				pinctrl-2 = <&P8_11_pruout_pin>;
			};
[...]

Conclusion

With this I know that my overlay files work and can be used on rtems, since rtems has no bone-pinmux-helper. Now I will move away from Linux and start working on rtems.

Setting up PRUs Pins via Device Tree Files

To route the BBBs GPIO header pins to the PRU, I decided to go with the device tree overlay. The overlay gets loaded by UBoot at start up to set up the board. Sadly I wasn’t able to get this it to work on Debian Jessie. My guess is that some other programs, loaded after start interfere with the pin setup on the board after UBoot finished setting it up. Additionally there is no Documentation by the BB community concerning this way of setting up the pins. But using the capemanager or other software installed on Debian to set up the pins would have been meaningless to me, since I won’t have these on RTEMS. So here is what I did so far.

Enabling the UIO drivers via device tree overlay

The device tree overlay file for enabling the PRUs UIO driver is already compiled and installed in /lib/firmware and can be activated by uncommenting it in the /boot/uEnv.txt. This was already described in my older blog post about using the UIO driver on Linux.

Routing the PRU pins via device tree overlay

I had to write my own device tree overlay, due to missing documentation in the BBs community. The pins are not routed to the PRU by default and are also used by other devices on the BBB, so they have to be rerouted and/or other peripherals have to be deactivated to free the pins for the PRU without breaking other drivers. So here is the device tree overlay I came up with to rout two pins to the PRU. It can also be found here.


/dts-v1/;
/plugin/;

/ {
	compatible = "ti,beaglebone", "ti,beaglebone-black", "ti,beaglebone-green";

	// identification
	part-number = "PRU-UIO-EXAMPLE-PINS";
	version = "00A0";

	
   fragment@0 {  
    target = <&am33xx_pinmux>;  
    __overlay__ {  
      example_pins: pinmux_pru_pru_pins {
       pinctrl-single,pins = <  
         0x34 0x06 /* Pin 8_11 out gpmc_ad13 Mode 6 */
         0x38 0x26 /* Pin 8_16 in gpmc_ad14 Mode 6 */
       >;  
      };  
    };  
   };

   
   fragment@1 {
    target-path="<&pruss>";
    __overlay__ {
      pinctrl-names = "default";  
      pinctrl-0 = <&example_pins>;
      status = "okay";  
     };
   };
};

Fragment@0 routes the pins 8_11 in output mode and the pin 8_16 in input mode to the PRU. Information about the pins addresses and and their modes can be found at my GSoC wiki on RTEMS and the documentation provided by Texas Instruments.

Fragment@1 hand the pins to the PRU, here called pruss.

The example program I used to test the pins can be found here.


; blink.p: demonstration of PRU on the BeagleBone Black
; blink LED connected to P8_11 ten times
.origin 0
.entrypoint TOP
TOP:
  MOV r1, 10 ; blink counter
BLINK:
  SET r30, r30, 15 ; set GPIO output 15
  MOV r0, 0x00a00000 ; delay counter
DELAY:
  SUB r0, r0, 1
  QBNE DELAY, r0, 0 ; loop until r0 == 0 (delay)
  CLR r30, r30, 15  ; clear GPIO output 15
  MOV r0, 0x00a00000 ; delay counter
DELAY2:
  SUB r0, r0, 1
  QBNE DELAY2, r0, 0 ; loop until r0 == 0 (delay)
  SUB r1, r1, 1
  QBNE BLINK, r1, 0 ; loop until r1 = 0 (blink counter)
  MOV r31.b0, 32 + 3
  HALT

Conclusion

Like I mentioned before I wasn’t able to use the pins with an example programm on the PRU. And because of lacking informations by the BB community I decided to move on to Porting the drivers to RTEMS and to check later if the device tree overlay works for RTEMS. If this setup also won’t work on RTEMS I will have to start again but I don’t think I should spend more time on setting up an unsupported way of routing the pins on Linux.

First PRU UIO example on BBB with Linux

In this entry I intend to show how to get started with PRU on the BBB using Debian Jessie.

The first problem I encountered while following the Guide in [1], was that Pin Management in BBB-Linux has changed, since this guide has been written. It has changed from slot files [2] to U-Boot overlays [3]. Additionally the userspace drivers have been replaced by remoteproc over time, so I wasn’t able to find newer Guides and had to close the gaps in the deprecated guide I used [1].

I also recommend updating the BBB to the newest distribution [4].

1. Enabling PRU UIO driver in /boot/uEnv.txt.

To use the PRU UIO driver in BB-Linux, their Uboot overlay has to be uncommented in /boot/uEnv.txt.

Before: #uboot_overlay_pru=/lib/firmware/AM335X-PRU-UIO-00A0.dtbo

After: uboot_overlay_pru=/lib/firmware/AM335X-PRU-UIO-00A0.dtbo

The sources of AM335X-PRU_UIO_00A0.dtbo can be found here [5].

If you intend to use PRU1 as well, you should disable HDMI.

2. Installing PRU UIO drivers.

Get the source files from [6].

git clone https://github.com/beagleboard/am335x_pru_package

Compile and install the assembler and PRU driver.

cd am335x_pru_package

sudo make all

sudo make install

cd ..

3. Getting example program from deprecated guide.

Download or copy/paste the pru programm loop.p and the loader loader.c script from [1].

Links to my Gists:

loop.p: https://gist.github.com/nilhoel1/1cbaf8a2e583317c2bfa669a1970d4b7

loader.c: https://gist.github.com/nilhoel1/7056d4bb0ce56f0b073742e5fd98e73d

The PRU program loop.p loops for 20 times, with a 500 ms delay in each loop. So executing loop.p should result in a runtime of 10 seconds as shown below.

20 * ((2 x 0.000000005 sec) * 50,000,000) = 10 sec

4. Compiling and assembling the examples.

The PRU assembly can be assembled with:

pasm -b loop.p

Before loader.c can be compiled we have to update the environment variable LD_LIBRARY_PATH with:

export LD_LIBRARY_PATH=”$LD_LIBRARY_PATH:/usr/local/lib”

Now the loader can be compiled:

gcc -o loader loader.c -lprussdrv  

5. Running the example.

The example code provided from [1] will flash the P8_11 Pin ten times. The example can be executed with:

./loader loopd.bin

To see how long loop runs execute it with:

time ./loader loopd.bin

It should run for 10 seconds.

The next step will be to run a PRU program with access to BBBs IO Pins and to flash a LED with it.

Resources:

  1. http://www.righto.com/2016/08/pru-tips-understanding-beaglebones.html Deprecated Guide on PRU UIO.
  2. https://elinux.org/Beagleboard:BeagleBoneBlack_Debian#Where_did_the_slots_file_go.3F Old Pin management in BBB-Linux.
  3. https://elinux.org/Beagleboard:BeagleBoneBlack_Debian#U-Boot_Overlays New Pin management in BBB-Linux.
  4. https://markayoder.github.io/PRUCookbook/02start/start.html Guide for updating BBB.
  5. https://github.com/beagleboard/bb.org-overlays/blob/master/src/arm/AM335X-PRU-UIO-00A0.dts PRU UIO Pin settings.
  6. https://github.com/beagleboard/am335x_pru_package PRU UIO drivers.