Trial and Error My Journey Porting Nerves to the M5Stack Core MP135

# Introduction

In this article, I want to share the trial-and-error experiences I had while porting Nerves to the M5Stack CoreMP135.
The Nerves documentation provides detailed information about porting, and I recommend referring to it. I also followed its guidelines during my work.

From my experience, I found that the porting steps are not overly difficult for someone familiar with Nerves’ mechanisms.

However, as someone who was not familiar with Nerves’ mechanisms, I repeatedly found myself in situations where I couldn’t figure out why things weren’t working, which was quite challenging.
I hope my first-time experience will help you understand the general workflow of Nerves porting and the areas you need to consider, serving as a reference when proceeding with your own porting work.

Recently, various SBCs have been emerging. I hope that porting Nerves to new hardware like this will contribute to the development of new IoT products.

This article was created by translating my original Japanese article using ChatGPT. While I believe there are no major translation errors, please feel free to let me know if there are points that need correction or improvement.

# M5Stack CoreMP135

Let me briefly introduce the M5Stack CoreMP135, the target for the porting work.
M5Stack is a company specializing in small modular products for IoT and embedded development, offering a wide range of unique products. Among them, the "CoreMP135," released in May 2024, stands out. Unlike other Core series products from M5Stack, the CoreMP135 supports Linux.

Details are available on the product page [M5Stack CoreMP135](https://shop.m5stack.com/products/m5stack-coremp135-w-stm32mp135d), but its ability to run Linux allows for more advanced operations and processing compared to typical microcontroller boards.

# Porting Workflow

The [official documentation](https://github.com/nerves-project/nerves_system_br/blob/0229532e3843bc3b4e534807636c767c380ae4b4/README.md) says the following:

```
We only officially support easily obtained hardware, but that doesn't mean that
Nerves only works on these boards. If it's possible to use Buildroot to create a
Linux root filesystem for your hardware, then it's possible that Nerves can be
made to run. The general steps to supporting a new board are the following:

1. Create a minimal Buildroot `defconfig` that boots and runs on the board. This
   doesn't use Nerves at all.
2. If the `defconfig` requires a writable root filesystem, figure out how to
   make it read-only. This should be pretty easy unless you're using `systemd`.
   Since Nerves uses a custom init system, keep in mind for later that `systemd`
   may be helping initialize something on the board that will need to be done
   manually later.
3. Take a look at the Flash memory layout and compare that to the layouts used
   in one of the supported systems. We use
   [fwup](https://github.com/fhunleth/fwup) to create images. There's a lot of
   variety in how one can lay out Flash memory and deal with things like
   failbacks.  At this point, just see if you can get `fwup` to create an image.
4. Clone one of the official systems that seems close for your board. Update
   the `nerves_defconfig` based on the Buildroot `defconfig` that works.
5. Build the system using `mix` or manually by running the `create-build.sh`
   script.
```

# Building with Buildroot

As mentioned in the official documentation, the first step is to create firmware that runs Linux using Buildroot. For details about Buildroot, see [Buildroot's official site](https://buildroot.org/).

M5Stack's official site includes instructions for building Linux for the CoreMP135. Referring to those, I built Linux using Buildroot.

```
git clone https://github.com/m5stack/CoreMP135_buildroot.git
git clone https://github.com/m5stack/CoreMP135_buildroot-external-st.git
cd CoreMP135_buildroot
make BR2_EXTERNAL=../CoreMP135_buildroot-external-st/ m5stack_coremp135_defconfig
make -j4
```

An SD card image is generated at `./output/images/sdcard.img`.

I confirmed that the system boots by writing this image to an SD card and starting the device.

There were several times when the system didn’t boot as expected, but comparing it to a working SD card helped identify where the issues were.

# Creating the Repository

The CPU of the M5Stack CoreMP135 is the STM32MP135DAE7 from STMicroelectronics. Since the OSD32MP1 CPU supported by Nerves is based on the STM32MP1, it seemed like a good starting point. I decided to base my work on it.

Following the Nerves documentation on creating a custom system, I copied the `nerves_system_osd32mp1` repository to create `nerves_system_m5stack_core_mp135`.

https://hexdocs.pm/nerves/customizing-systems.html

It’s a good idea to name your system `nerves_system_XXX` as this will eventually become the package name specified in `@app`.

# Verifying the Build

At this stage, the build won’t work, but you can test whether the toolchain is functioning by building the system as described in [Building the System](https://hexdocs.pm/nerves/customizing-systems.html#building-the-system).

Create a new Nerves project and edit `mix.exs`.

```
mix nerves.new your_project
```

Changes to `mix.exs`:

```Elixir:mix.exs
#=vvv= Update your_project/mix.exs to accept your new :custom_rpi3 target

# ...
@all_targets [:osd32mp1, :m5stack_core_mp135]
#                        =^^^^^^^^^^^^^^^^^^=

defp deps do
  [
    # Dependencies for all targets
    # ...

    # Dependencies for specific targets
    {:nerves_system_osd32mp1, "~> 0.15", runtime: false, targets: :osd32mp1},

    # Add the entry below vvv
    {:m5stack_core_mp135,
     path: "../nerves_system_m5stack_core_mp135",
     runtime: false,
     targets: :m5stack_core_mp135,
     nerves: [compile: true]},
  ]
end
```

Building with Buildroot takes quite a bit of time. On my machine (32GB RAM, AMD Ryzen 7, NVMe), it took about an hour.

When I tested it by writing to an SD card and booting, nothing happened.

# The Role of `nerves_system_br`

Buildroot is a tool for creating Linux firmware. It builds all the packages necessary for running a Linux OS and creates a single firmware image.

The `_br` in `nerves_system_br` likely stands for "Buildroot," and this package provides functionality for creating Nerves firmware.

`nerves_system_br` contains common packages that are hardware-agnostic, while target-specific differences are found in packages like `nerves_system_rpi0`.

Porting involves creating such a target-specific package.

# Buildroot Functionality

Buildroot creates firmware images to write to an SD card. These images include not only the Linux kernel but also the bootloader, device tree, root filesystem, and applications such as Elixir and Nerves processes.

The boot process from power-on to a running Linux OS goes as follows:

- Power on
- CPU executes the program written in ROM
- Reads the boot sector of the SD card (hardware-dependent)
- Bootloader is loaded from the boot sector
- Bootloader loads the Linux kernel and starts it
- Kernel mounts the root filesystem
- Nerves processes start

When porting, it’s a good idea to verify the following steps in order:

1. The bootloader (U-Boot) works and starts the kernel.
2. The kernel starts and operates correctly.
3. Nerves processes start correctly.

# Configuring Buildroot

Buildroot configuration is done in the `nerves_defconfig` file in the `nerves_system_m5stack_core_mp135` directory. `nerves_system_br` refers to this file to create the firmware.

Porting involves preparing this file and its referenced files.

Using M5Stack’s Buildroot configuration as a reference, I examined how to structure the `nerves_defconfig` file.

I compared M5Stack’s configuration (referred to as "M5 configuration") with the `nerves_system_osd32mp1` configuration (referred to as "Nerves configuration") to determine the most appropriate options.

For the bootloader, the M5 configuration uses the ARM_TRUSTED_FIRMWARE package. While there may be ways to avoid using ARM_TRUSTED_FIRMWARE, doing so would require a deep understanding of the CPU and ROM specifications. For now, I decided to follow the M5 configuration.

For the kernel, I used the M5 configuration to avoid driver-related issues.

| Item         | Nerves Configuration | M5 Configuration   | Chosen Configuration |
|--------------|-----------------------|--------------------|-----------------------|
| Actual File  | [defconfig](https://github.com/m5stack/CoreMP135_buildroot-external-st/blob/st/2023.02.10/configs/m5stack_coremp135_515_defconfig) | [nerves_defconfig](https://github.com/nerves-project/nerves_system_osd32mp1/blob/main/nerves_defconfig) | - |
| Toolchain    | Nerves project        | Buildroot          | Nerves Configuration |
| Kernel       | Linux from kernel.org | Linux from STM Git | M5 Configuration     |
| Bootloader   | Buildroot’s U-Boot    | STM Git’s U-Boot (with TF-A and OPTEE) | M5 Configuration |
| Device Tree  | For MP1               | For MP135          | M5 Configuration     |
| Other Packages | -                   | -                  | Nerves Configuration |

# Kernel Build Configuration

For the kernel configuration in `nerves_defconfig`, I used the M5 configuration, as shown below:

```
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_TARBALL=y
BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,STMicroelectronics,linux)v5.15-stm32mp-r2.1.tar.gz"
BR2_LINUX_KERNEL_DEFCONFIG="multi_v7"
BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES="$(LINUX_DIR)/arch/arm/configs/fragment-01-multiv7_cleanup.config $(LINUX_DIR)/arch/arm/configs/fragment-02-multiv7_addons.config $(NERVES_DEFCONFIG_DIR)/m5stack/linux-disable-etnaviv.config $(NERVES_DEFCONFIG_DIR)/m5stack/linux-enable-fbdev-emul.config $(NERVES_DEFCONFIG_DIR)/m5stack/linux-enable-m5stack.config $(NERVES_DEFCONFIG_DIR)/m5stack/fragment-03-systemd.config"
BR2_LINUX_KERNEL_DTS_SUPPORT=y
BR2_LINUX_KERNEL_DTB_OVERLAY_SUPPORT=y
BR2_LINUX_KERNEL_INSTALL_TARGET=y
BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
BR2_LINUX_KERNEL_CUSTOM_DTS_PATH="$(NERVES_DEFCONFIG_DIR)/m5stack/linux-dts/*"
BR2_LINUX_KERNEL_INTREE_DTS_NAME="stm32mp135f-coremp135"
```

The kernel is downloaded from STMicroelectronics' repository.
While `nerves_system_osd32mp1` uses the Linux kernel from kernel.org and `nerves_system_rpi4` uses the Raspberry Pi-provided kernel, it’s crucial to use a kernel suitable for the hardware. For now, I’ll proceed with the current configuration.

Kernel configuration can also be done via `BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE`, but I followed the M5 configuration approach and used `BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES`.

Some fragments like `fragment-03-systemd.config` may not be necessary when using Nerves. It’s likely sufficient to retain M5-specific hardware settings and limit other configurations to what Nerves requires. However, to avoid unexpected issues, I decided to first get it running with the proven M5 configuration.

# Applying Patches

Buildroot builds packages in the following order:

1. Downloads the package
2. Extracts the package
3. Applies patches
4. Builds the package

In `nerves_defconfig`, the `BR2_GLOBAL_PATCH_DIR` variable specifies the directory where patches are stored, with each package having its own directory for patches. Patches can be applied by placing them here.

In the Nerves configuration, it is set as follows:
```
BR2_GLOBAL_PATCH_DIR="${BR2_EXTERNAL_NERVES_PATH}/patches"
```

Since I needed patches for arm-trusted-firmware and optee, I created a `patches` directory in `nerves_system_m5stack_core_mp135` to store the patches. I also updated the path to include this directory:

```
BR2_GLOBAL_PATCH_DIR="${BR2_EXTERNAL_NERVES_PATH}/patches ${NERVES_DEFCONFIG_DIR}/patches"
```

`${NERVES_DEFCONFIG_DIR}` refers to the directory of `nerves_system_m5stack_core_mp135`.

# Building

Running `mix firmware` triggers the Buildroot build process.

If Buildroot completes successfully, firmware creation proceeds. However, during development, it’s common for the process to stop partway. Here’s how to troubleshoot those issues.

Buildroot stores built files in `.nerves/artifacts`. Specifically, they are saved in a directory like the following:

```
/path/nerves_system_m5stack_core_mp135/.nerves/artifacts/nerves_system_m5stack_core_mp135-portable-0.0.1
```

You can navigate to this directory and run `make` to re-execute the Buildroot build. This allows you to check error messages and debug.

During this porting effort, I made U-Boot display the Nerves logo on the LCD screen. This required several rebuilds of U-Boot. To rebuild only the U-Boot package, delete the `build/u-boot-custom/` directory and then run `make`. This significantly reduces build time.

You can also rebuild the Linux kernel with `make linux-rebuild`. This is useful when you need to rebuild only the driver portion of Linux.

Once the build process is complete, run `mix firmware` to create the firmware.


# Testing the Firmware

Run the following commands to write the firmware to the SD card, insert it into the M5Stack CoreMP135, and power it on:

```
mix firmware
mix burn
```

# Trial and Error

When powering on with the initial firmware, nothing appeared on the screen. The Core MP135 has an LCD panel, but nothing was displayed on it, nor was there any HDMI output.

It didn’t work as expected, and since nothing was displayed, it was unclear what was causing the issue or where to start investigating.

Since M5Stack’s official Buildroot works, we can rely on it and proceed step by step.

Two main points guided my investigation:

### Determine the Current Execution Stage

The following considerations helped in making this judgment:

- The M5 Buildroot firmware displays a logo on the LCD panel during the U-Boot process. Whether this is displayed indicates whether U-Boot is running.
- By configuring the Linux boot parameters to output kernel logs to the serial port, you can determine whether the kernel is starting through console output.

Although U-Boot should also provide serial console output, I couldn’t confirm it. It’s possible that the output was directed to another UART port.

### Identify Differences from the Working Configuration

I swapped binary files created with M5 Buildroot and compared partition structures and configuration files to identify issues.

For example, I replaced the zImage in the M5 Buildroot’s `image` directory with the zImage from the Nerves Build and created an SD card from it. Since all other files were from the working configuration, the system should work if there were no issues with the zImage. This allowed me to confirm that the zImage itself was not the problem.

# Common Porting Issues

If firmware construction using Buildroot is functioning, U-Boot and the kernel should generally work. However, during Nerves porting, the following issues often arise:

### `fwup.conf` Configuration

Many startup issues stem from the `fwup.conf` configuration, which required several adjustments. The partition structure in M5 Buildroot differs from that in Nerves, which is usually the case during porting. Since Nerves defines the partition structure in `fwup.conf`, it’s essential to understand how to write `fwup.conf` and adjust it to reflect the original Buildroot firmware.

M5’s configuration uses ARM-TRUSTED-FIRMWARE during boot, necessitating adjustments in `fwup.conf` to account for differences from MP1.

### U-Boot Environment Variable Storage Location

Both the M5 and Nerves configurations save U-Boot environment variables to specific sectors on the SD card, but their storage locations differ. The Nerves configuration must match the storage location.

In Nerves, this is specified in the following section of `fwup.conf`:

```fwup.conf
uboot-environment uboot-env {
    block-offset = ${UBOOT_ENV_OFFSET}
    block-count = ${UBOOT_ENV_COUNT}
}
```

`UBOOT_ENV_OFFSET` and `UBOOT_ENV_COUNT` are defined in `fwup_include/fwup-common.conf`. U-Boot’s configuration must specify the same location, converting between byte and sector units as necessary:

```
CONFIG_ENV_OFFSET=0x100000
CONFIG_ENV_SECT_SIZE=0x10000
```

In this porting effort, an incorrect setting here caused the kernel to fail to boot. U-Boot initialized the environment variable area during startup, overwriting the kernel image file (zImage) due to the incorrect storage location. Although simple in hindsight, identifying the issue took considerable time.

### Kernel Configuration

Once U-Boot runs correctly, the kernel should start. You can confirm this stage by configuring the kernel to output logs to the serial console. Boot parameters for the kernel are specified in `extlinux.conf`:

```extlinux.conf
label stm32mp135f-coremp135-buildroot
  kernel zImage
  devicetree stm32mp135f-coremp135.dtb
  append root=/dev/mmcblk0p5 rootwait console=tty1 console=ttySTM0,115200n8 quiet
```

In the kernel used here, the serial console device is `ttySTM0`, so `console=ttySTM0` was specified. This setting outputs kernel boot logs to the serial console, providing insight into its status. If no logs are displayed, it’s likely an issue with U-Boot or an earlier stage of the boot process.

In this porting effort, once the kernel started, the `iex` prompt was displayed, and the system became usable.

### Writing to the Filesystem

Although the `iex` prompt appeared and Nerves was functional, the SSH key changed with every boot. This was due to a lack of write access to the filesystem. The kernel configuration was missing support for [F2FS](https://www.kernel.org/doc/html/latest/filesystems/f2fs.html), which Nerves expects.

The following configuration resolved the issue:

```
CONFIG_F2FS_FS=y
```

# Post-Nerves Initialization

There were few target-specific issues after Nerves initialized. You can find the settings referenced by Nerves in the following directory:

```
nerves_system_m5stack_core_mp135/etc/rootfs_overlay
```

For this porting effort, I made the following changes:

### Board ID

Since this board lacks a unique hardware ID, I modified the settings to default to `0000` if no U-Boot environment variable is specified:

```boardid.config
-b uboot_env -u nerves_serial_number
-b uboot_env -u serial_number
-b force -f "0000"
```

### `erlinit.config`

The `-c` option in `erlinit.config` specifies the console for the `iex>` prompt. To use UART (serial port) by default, set it to something like `-c ttySTM0`:

```erlinit.config
#-c ttySTM0    # UART pins on the GPIO connector
-c tty1     # HDMI or LCD Display output
```

The default behavior is determined by the target’s `erlinit.config`. This configuration can also be overridden via the application’s Mix config:

https://hexdocs.pm/nerves/advanced-configuration.html#overriding-erlinit-config-from-mix-config

Although the process was long, I successfully booted the kernel and initialized Nerves.

# Summary

- Lack of Nerves knowledge prolonged debugging, but understanding the key points made porting relatively straightforward.
- If Buildroot can construct firmware, porting to Nerves isn’t particularly challenging.
- U-Boot often requires adjustments during porting.
- After the kernel boots, there are few target-specific modifications.

With the increasing availability of SBCs, including those with NPUs and FPGAs, consider porting Nerves to such unique boards for building embedded systems.

[Changes from MP1 in this porting effort](https://github.com/masahiro-999/nerves_system_m5stack_core_mp135/commit/5d0bea1c7d5e16e40037d68ee318cf65aad31d15)

### References

- [nerves_system_m5stack_core_mp135](https://github.com/masahiro-999/nerves_system_m5stack_core_mp135)
- [STM32MP1 Platform Boot](https://www.stmcu.jp/wp/wp-content/uploads/files/presentation-ja/STM32MP1/60_STM32MP1-Software-Platform-boot-(BOOT)_J.pdf)
- [Elixir on Industrial-Grade Hardware: Nerves Porting on e-RT3](https://qiita.com/pojiro/items/81f3aafb77eed47f3baa)