Elixir on Industrial Hardware, porting Nerves to e-RT3
Tags: nerves, ert3, buildroot, elixir
Published: 2021-06-10 12:44:57

## Introduction

The title says it all, I just tried porting Nerves and got it working!

So, for someone who are going to try porting like me, here is a summary of the key points of the process.


## Target machine

The target machine is [OS-free CPU module](https://www.yokogawa.com/solutions/products-platforms/control-system/ert3-embedded-controller/ert3-products/ert3-products-cpu/) of [e-RT3 Products](https://www.yokogawa.com/solutions/products-platforms/control-system/ert3-embedded-controller/ert3-products/) which is made by Yokogawa Electric Corp.

[myasu](https://qiita.com/myasu) has been writing several articles about it on [Qiita](https://qiita.com/tags/e-rt3), one of Japanese tech web sites.

One day he gave me the idea of porting, 

* If we could port Nerves onto this, we can get the **industrial robust hardware Nerves** 💡

It Sounds very nice!! So this hardware became the target machine for us.


### About e-RT3

This [OS-free CPU module](https://www.yokogawa.com/solutions/products-platforms/control-system/ert3-embedded-controller/ert3-products/ert3-products-cpu/) is a very unique controller that allows users to freely select the OS they want to run. For this reason, its booting specifications docs and the Linux kernel source are provided.

The specifications/sources are open so that users can freely select the OS they want to run, which made this porting successful. **It's cool the info is open.**😉

In this article, I will refer to it as "e-RT3" instead of "OS-free CPU module"(cause little bit long).


## Porting outline

The porting procedure of Nerves is described in [README of nerves_system_br](https://github.com/nerves-project/nerves_system_br/blob/0229532e3843bc3b4e534807636c767c380ae4b4/README.md) as follows:

> 1. Create a minimal Buildroot `defconfig` that boots and runs on the board. This doesn't use Nerves at all.
> 2. If the `defconfig` requires a writable root filesystem, figure out how to make it read-only. This should be pretty easy unless you're using `systemd`. Since Nerves uses a custom init system, keep in mind for later that `systemd` may be helping initialize something on the board that will need to be done manually later.
> 3. Take a look at the Flash memory layout and compare that to the layouts used in one of the supported systems. We use  [fwup](https://github.com/fhunleth/fwup) to create images. There's a lot of variety in how one can lay out Flash memory and deal with things like failbacks. At this point, just see if you can get `fwup` to create an image.
> 4. Clone one of the official systems that seems close for your board. Update the `nerves_defconfig` based on the Buildroot `defconfig` that works.
> 5. Build the system using `mix` or manually by running the `create-build.sh` script.

I went through the steps in this order, but in retrospect, the first thing I needed to do was to understand the **how the target machine boots**.

Once I understood this, I was able to finish step 2 above by learning how to use Buildroot and creating the kernel and dtb. Steps 3, 4, and 5, I use [nerves_system_rpi3](https://github.com/nerves-project/nerves_system_rpi3) as base and made modifications. This is a difficult part to describe what I did.


## First, understand how the target machine(e-RT3) boots

When e-RT3 is turned on, it reads the boot loader, U-Boot, in its internal ROM and starts up.

The U-Boot checks if there is a uEnv.txt in the SD slots 1 and 2, and if there is, e-RT3 operates according to the file. The uEnv.txt is like a script for U-boot.

The provided Ubuntu image has a uEnv.txt, which was very helpful for me as a working reference.


### Key points of uEnv.txt

The key points of uEnv.txt are,

* Where the kernel and dtb should be located.
  * This is determined by the configuration of the SD image you are creating.
* Where (address) in RAM should they be loaded?
  * This is determined by the specification of target machine. In case of e-RT3, documented.
* Adjust kernel parameters (root=rootfs device, etc.)
  * In case of e-RT3, Ubuntu's kernel image has a uEnv.txt. I could use it with modifications.

This is the first step.

Once you know the above, you can use uEnv.txt to load the kernel and dtb into the target machine's RAM, pass the kernel parameters, and boot the kernel with uBoot's bootm/z command.


## Next, use Buildroot to create the kernel, dtb, and rootfs.

Once you know how the target boots, you will know that you need to create the kernel, dtb, and rootfs. These can be created by downloading, configuring and compiling the Linux kernel sources, but Nerves use [Buildroot](https://buildroot.org/) to do this.

Buildroot is a tool of configuring Linux systems (kernel, dtb, rootfs) for embedded use.

The `make menuconfig` used in the Linux kernel sources is also available in Buildroot (wrapped for Buildroot) and can be configured.

Depending on the configuration, Buildroot will download what it needs and cross-compile it on the host machine.

What it needs are, for example

* The specified version of the Linux kernel sources
* Kernel patches (ex. real-time patches)
* A cross compiler (a compiler that creates binaries for the target machine)
* [fwup](https://github.com/fwup-home/fwup) for image creation
* ...etc. 

Buildroot allows us to build a linux system with only configuration if the [board](https://github.com/buildroot/buildroot/tree/master/board) is registered in Buildroot.

e-RT3 is not registered, so I needed to create the corresponding configuration.


### e-RT3 Linux kernel source investigation and kernel generation

By investigating the provided kernel sources, I found that it would be configured as follows.

![provided kernel source](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3gz5asfoumsjofz6vgbt.png)

e-RT3 uses the Xilinx Zynq-7000, and it seemed that Xilinx [linux-xlnx](https://github.com/Xilinx/linux-xlnx) code is  partially applied.

The two bottom rows can be downloaded by configuring Buildroot.
So, if I could prepare a patch for the upper two, I could  make a kernel that would cover the functions of e-RT3.
(The [aufs](https://github.com/sfjro/aufs4-standalone/tree/aufs4.14.73+) patch is not necessary for Nerves, so I ignored it.)

Therefore, I made a kernel source consisting of the lower three rows shown in the above figure, and made a mixed patch of the upper two rows from the difference between that and the provided kernel source, so that the final kernel source would be shown in the figure below.

![kernel source for Nerves](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b20zr3miq64zp0hx4s35.png)

After creating the mixed patch, follow the Buildroot reference to create the necessary configuration files in the board directory, and then generate the kernel. (I'll spare you the details.

I only read the following two references which can be accessed from https://buildroot.org/docs.html. Both are excellent, easy to understand documents.

* [The Buildroot user manual](https://buildroot.org/downloads/manual/manual.html)
  * I read Chapter 9 carefully.
* [Bootlin's Buildroot Training slides](https://bootlin.com/doc/training/buildroot/buildroot-slides.pdf)
  * It's about 350 pages, but you don't have to read all of it; just do a search on the keywords you want to know, you'll find the answer to your question with a high probability.

The br2-external for e-RT3 I created is available here, [br2-external-f3rp70](https://github.com/pojiro/br2-external-f3rp70).


### booting the created kernel

Burn image (buildroot-dir/output/images/f3rp70.img) to SD,

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/369713/35e922d2-e44a-2dd4-c870-9c56489727c7.png)

Connect the serial cable and turn on, and I could confirm  booting the minimal Linux created by Buildroot.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/369713/a9a0e3c5-f03f-d9e0-f55a-4dd64237c60a.png)



## Finally, fit it in so that it becomes Nerves

It's difficult to describe what I did, but [nerves_system_f3rp70](https://github.com/pojiro/nerves_system_f3rp70) that I created is based on [nerves_system_rpi3](https://github.com/nerves-project/nerves_system_rpi3). So you can see what I changed by looking at the diff between  them.

NOTE: In the fitting, I renamed the Buildroot's defconfig to nerves_defconfig for nerves_system_f3rp70.


### Buildroot minimal Linux becomes Nerves!!

Hi, Nerves!!!

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/369713/553b229d-9eae-fb75-93e2-23fd8009af66.png)


## Once Nerves is ported

* Elixir's strengths in concurrency and process monitoring/recovering are fully revealed.
* We can use Elixir libraries to their fullest extent.

For example,

* The monitoring/recovering process that was done with systemd or shell scripts in the normal Linux environment can be replaced with Elixir's Supervisor/GenServer.
* Nginx and Apache can be replaced by Elixir's Phoenix.

And there are no configuration files scattered around /etc.

**These are strengths of Nerves which use Elixir and hide the Linux details.**

* Addendum: I've published it on hex.pm, yey -> https://hex.pm/packages/nerves_system_f3rp70


### e-RT3 specific support

The e-RT3's IO needs to be accessed through libm3.so which is  provided from Yokogawa.

For example, DIO is not a GPIO, we can't use [circuits_gpio](https://github.com/elixir-circuits/circuits_gpio). So, we need to use NIFs as myasu did in [Real-time OS controller e-RT3 (controlled from Elixir)](https://qiita.com/myasu/items/0dbf308947b878423112).

It will be a good idea that creating a library such as circuits_gpio.


## Summary

* **Industrial robust hardware, e-RT3 from Yokogawa Electric Corp**

meets

* **Nerves, a game changer of embedded Linux development experience**

🎉🎉🎉

This experience has given me the confidence that I can do simple porting if

* I get the boot specification
* I have working kernel sources (including dts).


### Acknowledgements

* **myasu** for the porting idea.
* **Yokogawa Electric Corp** for providing info openly
* **Nerves** on our side

Thank you!!