The libBSD init process in rtems with focus on driver modules

After porting FreeBSD’s PRU driver to rtems-libbsd, I encountered a problem concerning the init process. The PRU driver needs TI’s prcm module to be loaded when initialising itself. However the current rtems-libbsd init process doesn’t allow me to sort that dependency. So I had to investigate the whole process, since I don’t have JTAG . );

Linking the drivers.

In FreeBSD Modules are registered using the DRIVER_MODUL macro. This macro hands down all its information via a macro chain and some structs containing additional information. The DRIVER_MODULE is defined in bus.h and is actually a call of EARLY_DRIVER_MODULE_ORDERED with static order (SI_ORDER_MIDDLE) and pass level (BUS_PASS_DEFAULT). All data concerning the driver is stored in two structs and the module is forwarded to DECLARE_MODULE in module.h. The last macro of this chain is SYSINIT.

The init process

When using rtems with rtems-libbsd the whole libBSD init process get triggered by rtems_bsd_initialize() in rtems-kernel-init.c. This functions sets parameters for the BSD library and calls mi_startup() from init_main.c. This function iterates over the objects, linked by SYSINIT, and executes a bubble sort on them. After this the modules are ordered by subsystem and in their subsystem they are ordered by si-order. All objects are called in order with their defined function and data here. Focusing on driver modules this function is module_register_init() in kern_module.c. This function retrieves the data stored in additional structs during linking and uses the MOD_EVENT macro. This macro calls the modules event handler and hands threw the mod itself and its arguments. For bus related modules as our drivers are the handler refers to driver_module_handler in subr_bush.c. Here all bus devices are registered for activation.

Registering bus-devices.

I am currently investigating this topic. My guess is that the loop attaching devices to the bus does not run over the fdt multiple times as stated int he FreeBSD manual. UPDATE: LibBSD does indeed run over fdt only once. But after trying to add another run over fdt I discovered that all drivers are registered with the same pass level, which definitely shouldn’t be the case. The method calling a fdt pass for a pass level is called here.

Advertisements

Booting RTEMS with UBoot and the right FDT overlay

I have spinned quite some time on figuring out the whole device tree “mysterium” with uboot and rtems. But once you have a functioning SD-Image with a nice UBoot setup it works like a charm and updating kernel images is just copy paste.

Creating SD Image with UBoot and rtems

My current workflow is to use the rtems-boot-image script in rtems-tools to create a SD Image, which can be flashed to SD Card via dd. This requires a functional UBoot install. The command looks like this:

rtems-boot-image -o sd-card.img -b u-boot-beaglebone -s 32m -k rtems.exe -d am335x-boneblack.dtb ../u-boot/MLO ../u-boot/u-boot.img

-o sd-card.img: Name of the OutputFile.

-b u-boot-beaglebone: Name of the Board.

-s 32m: Size of the Image (32MB).

-k rtems.exe: Path to the rtems kernel file.

-d am335x-boneblack.dtb: Path to device tree.

../u-boot/MLO: Path to first stage boot loader.

../u-boot/u-boot.img: Path to second stage boot loader.

When this command has finished the SD card can be flashed with the dd command or a tool like etcher.

The .dtb File

The device tree is taken from FreeBSD since the linux device trees tend to be useless or buggy with rtems. The device tree sources can be found here and be build with the tools linked here. Before building the environment variable “MACHINE” has to be set to target platform, on a BBB it’s export MACHINE=arm. I also had to change all includes in the .dts files to absolute paths. With this we have the FreeBSD device tree overlay for the BBB.

Applying fdt overlay via UBoot

In this step I provide my uEnv.txt, that apply the fdt overlay needed to use the pruss units on BBB. Since I don’t think to give a better explanation than the guide I used myself, I will only link to it.

setenv bootdelay 5
uenvcmd=run boot_rtems; reset;
rtems_banner=echo ""; echo "RTEMS u-boot-beaglebone (arm-ti-am335x_evm)"; echo " rtems-boot-image v5.0.not_released"; echo "";
boot_rtems=run rtems_banner; echo "Loading fileio.exe.img"; load mmc 0 0x82000000 fileio.exe.img; echo "Loading am335x-boneblack.dtb"; load mmc 0 0x88000000 am335x-boneblack.dtb; echo "Loading AM335X-PRU-UIO-00A0.dtbo"; load mmc 0 0x880f0000 AM335X-PRU-UIO-00A0.dtbo; echo "Applying fdt overlay"; fdt addr 0x88000000; fdt resize 65536; fdt apply 0x880f0000;  bootm 0x82000000 - 0x88000000

Conclusion

Since I spent way too much time figuring this one out. I hope this helps everyone, who also has problems figuring out how to use UBoot, rtems and fdt overlays. Sadly documentation is sparse.

Locating PRUSS driver in RTEMS

After discussing with my mentors and on devel@rtems.org, I decided to locate the device driver in bsps/arm/beagle/pruss/prussdrv.c and __prussdrv.h. The headers are located in bsps/arm/beagle/include/bsp/prussdrv.h and pruss_intc_mapping.h.

This post is mainly meant to document how to port the drivers to another bsp that uses an TI Sitara AM33xx or AM18xx SoC.

To compile the files I also had to register them in the automake files.

I added the following code to /bsps/arm/beagle/headers.am

include_bsp_HEADERS += ../../../../../../bsps/arm/beagle/include/bsp/prussdrv.h
include_bsp_HEADERS += ../../../../../../bsps/arm/beagle/include/bsp/pruss_intc_mapping.h

And the driver files have to be added to /c/src/lib/libbsp/arm/beagle/Makefile.am

# PRUSS
librtemsbsp_a_SOURCES += ../../../../../../bsps/arm/beagle/pruss/prussdrv.c

When these drivers are copied to another bsp the paths have to look like:

…/bsps/arm/YOUR_BOARD/…

Changes to the original drivers

All changes to the drivers that differ from the original code, taken from am335x_pru_package, are marked with #defines using __rtems__ or having an /* __rtems__ */ comment behind them. These are the coding conventions taken from rtems-libbsd and can be found here.

Update: Device Tree and Pins

After discussing my last blog post with my co GSoC’er Vijay Kumar Banerjee, we came to some conclusions concerning the device tree overlay.

Vijay Kumar Banerjee is also working with rtems on the BBB visit his Blog, if you are interested.

I came to the conclusion, that the device tree overlay I provided before works. However the pins I want to set up are already used by another unit on the ocp-bus. My previous suspicion, that another application uses these pins already, came true.

The output when trying to access the pins

[   38.186334] pinctrl-single 44e10800.pinmux: pin PIN13 already requested by ocp:P8_11_pinmux; cannot claim for 4a300000.pruss
[   38.463257] pinctrl-single 44e10800.pinmux: pin-13 (4a300000.pruss) status -22
[   38.709802] pinctrl-single 44e10800.pinmux: could not request pin 13 (PIN13) from group pinmux_pru_pru_pins  on device pinctrl-single
[   39.086278] pruss_uio 4a300000.pruss: Error applying setting, reverse things back
[   84.039763] watchdog: watchdog0: watchdog did not stop!
[   84.370168] reboot: Restarting system

This was the output I got when I tried to use the routed pins, and found out that the pins were already routed to another unit on the ocp-bus and therefore couldn’t be routed to pruss.

The bone-pinmux-helper does not help me.

When I investigated the BBB’s device tree overlays I found that all the pins are routed to the bone-pinmux-helper. Please take a look at its overlays:

[...]
	fragment@4 {
		target = <&ocp>;
		__overlay__ {

			P8_11_pinmux {
				compatible = "bone-pinmux-helper";
				status = "okay";
				pinctrl-names = "default", "gpio", "pruout";
				pinctrl-0 = <&P8_11_default_pin>;
				pinctrl-1 = <&P8_11_gpio_pin>;
				pinctrl-2 = <&P8_11_pruout_pin>;
			};
[...]

Conclusion

With this I know that my overlay files work and can be used on rtems, since rtems has no bone-pinmux-helper. Now I will move away from Linux and start working on rtems.

Setting up PRUs Pins via Device Tree Files

To route the BBBs GPIO header pins to the PRU, I decided to go with the device tree overlay. The overlay gets loaded by UBoot at start up to set up the board. Sadly I wasn’t able to get this it to work on Debian Jessie. My guess is that some other programs, loaded after start interfere with the pin setup on the board after UBoot finished setting it up. Additionally there is no Documentation by the BB community concerning this way of setting up the pins. But using the capemanager or other software installed on Debian to set up the pins would have been meaningless to me, since I won’t have these on RTEMS. So here is what I did so far.

Enabling the UIO drivers via device tree overlay

The device tree overlay file for enabling the PRUs UIO driver is already compiled and installed in /lib/firmware and can be activated by uncommenting it in the /boot/uEnv.txt. This was already described in my older blog post about using the UIO driver on Linux.

Routing the PRU pins via device tree overlay

I had to write my own device tree overlay, due to missing documentation in the BBs community. The pins are not routed to the PRU by default and are also used by other devices on the BBB, so they have to be rerouted and/or other peripherals have to be deactivated to free the pins for the PRU without breaking other drivers. So here is the device tree overlay I came up with to rout two pins to the PRU. It can also be found here.


/dts-v1/;
/plugin/;

/ {
	compatible = "ti,beaglebone", "ti,beaglebone-black", "ti,beaglebone-green";

	// identification
	part-number = "PRU-UIO-EXAMPLE-PINS";
	version = "00A0";

	
   fragment@0 {  
    target = <&am33xx_pinmux>;  
    __overlay__ {  
      example_pins: pinmux_pru_pru_pins {
       pinctrl-single,pins = <  
         0x34 0x06 /* Pin 8_11 out gpmc_ad13 Mode 6 */
         0x38 0x26 /* Pin 8_16 in gpmc_ad14 Mode 6 */
       >;  
      };  
    };  
   };

   
   fragment@1 {
    target-path="<&pruss>";
    __overlay__ {
      pinctrl-names = "default";  
      pinctrl-0 = <&example_pins>;
      status = "okay";  
     };
   };
};

Fragment@0 routes the pins 8_11 in output mode and the pin 8_16 in input mode to the PRU. Information about the pins addresses and and their modes can be found at my GSoC wiki on RTEMS and the documentation provided by Texas Instruments.

Fragment@1 hand the pins to the PRU, here called pruss.

The example program I used to test the pins can be found here.


; blink.p: demonstration of PRU on the BeagleBone Black
; blink LED connected to P8_11 ten times
.origin 0
.entrypoint TOP
TOP:
  MOV r1, 10 ; blink counter
BLINK:
  SET r30, r30, 15 ; set GPIO output 15
  MOV r0, 0x00a00000 ; delay counter
DELAY:
  SUB r0, r0, 1
  QBNE DELAY, r0, 0 ; loop until r0 == 0 (delay)
  CLR r30, r30, 15  ; clear GPIO output 15
  MOV r0, 0x00a00000 ; delay counter
DELAY2:
  SUB r0, r0, 1
  QBNE DELAY2, r0, 0 ; loop until r0 == 0 (delay)
  SUB r1, r1, 1
  QBNE BLINK, r1, 0 ; loop until r1 = 0 (blink counter)
  MOV r31.b0, 32 + 3
  HALT

Conclusion

Like I mentioned before I wasn’t able to use the pins with an example programm on the PRU. And because of lacking informations by the BB community I decided to move on to Porting the drivers to RTEMS and to check later if the device tree overlay works for RTEMS. If this setup also won’t work on RTEMS I will have to start again but I don’t think I should spend more time on setting up an unsupported way of routing the pins on Linux.

First PRU UIO example on BBB with Linux

In this entry I intend to show how to get started with PRU on the BBB using Debian Jessie.

The first problem I encountered while following the Guide in [1], was that Pin Management in BBB-Linux has changed, since this guide has been written. It has changed from slot files [2] to U-Boot overlays [3]. Additionally the userspace drivers have been replaced by remoteproc over time, so I wasn’t able to find newer Guides and had to close the gaps in the deprecated guide I used [1].

I also recommend updating the BBB to the newest distribution [4].

1. Enabling PRU UIO driver in /boot/uEnv.txt.

To use the PRU UIO driver in BB-Linux, their Uboot overlay has to be uncommented in /boot/uEnv.txt.

Before: #uboot_overlay_pru=/lib/firmware/AM335X-PRU-UIO-00A0.dtbo

After: uboot_overlay_pru=/lib/firmware/AM335X-PRU-UIO-00A0.dtbo

The sources of AM335X-PRU_UIO_00A0.dtbo can be found here [5].

If you intend to use PRU1 as well, you should disable HDMI.

2. Installing PRU UIO drivers.

Get the source files from [6].

git clone https://github.com/beagleboard/am335x_pru_package

Compile and install the assembler and PRU driver.

cd am335x_pru_package

sudo make all

sudo make install

cd ..

3. Getting example program from deprecated guide.

Download or copy/paste the pru programm loop.p and the loader loader.c script from [1].

Links to my Gists:

loop.p: https://gist.github.com/nilhoel1/1cbaf8a2e583317c2bfa669a1970d4b7

loader.c: https://gist.github.com/nilhoel1/7056d4bb0ce56f0b073742e5fd98e73d

The PRU program loop.p loops for 20 times, with a 500 ms delay in each loop. So executing loop.p should result in a runtime of 10 seconds as shown below.

20 * ((2 x 0.000000005 sec) * 50,000,000) = 10 sec

4. Compiling and assembling the examples.

The PRU assembly can be assembled with:

pasm -b loop.p

Before loader.c can be compiled we have to update the environment variable LD_LIBRARY_PATH with:

export LD_LIBRARY_PATH=”$LD_LIBRARY_PATH:/usr/local/lib”

Now the loader can be compiled:

gcc -o loader loader.c -lprussdrv  

5. Running the example.

The example code provided from [1] will flash the P8_11 Pin ten times. The example can be executed with:

./loader loopd.bin

To see how long loop runs execute it with:

time ./loader loopd.bin

It should run for 10 seconds.

The next step will be to run a PRU program with access to BBBs IO Pins and to flash a LED with it.

Resources:

  1. http://www.righto.com/2016/08/pru-tips-understanding-beaglebones.html Deprecated Guide on PRU UIO.
  2. https://elinux.org/Beagleboard:BeagleBoneBlack_Debian#Where_did_the_slots_file_go.3F Old Pin management in BBB-Linux.
  3. https://elinux.org/Beagleboard:BeagleBoneBlack_Debian#U-Boot_Overlays New Pin management in BBB-Linux.
  4. https://markayoder.github.io/PRUCookbook/02start/start.html Guide for updating BBB.
  5. https://github.com/beagleboard/bb.org-overlays/blob/master/src/arm/AM335X-PRU-UIO-00A0.dts PRU UIO Pin settings.
  6. https://github.com/beagleboard/am335x_pru_package PRU UIO drivers.

Adding PRU-ICSS drivers to RTEMS on Texas Instrument Sitara SoCs

So my little project for RTEMS with GSoC has started. In this post I introduce my project. This blog will cover my progress during GSoC and some guides to use my code and recreate my results.

Abstract:

This Project intends to add the PRU support to RTEMS, using the Beaglebone Black (BBB). The BBB has an Texas Instruments AM3358 SoC with an  Programmable Real-Time Unit (PRU). The PRU is able to connect to the SoCs i/o within one cycle. This will enable the RTEMS community to develop heavily i/o dependent tasks on the Texas Instruments SoCs with PRUs. More information can be found on RTEMS wiki page concerning this project. I intend to extend the wiki during summer to be a good starting point for RTEMS users wanting to use PRU.

My main goal is to port the linux userspace driver used by the BeagleBoard.org community. Please visit RTEMS GSoC page the full GSoC proposal I wrote.