Honeycomb-lx2k - First day
Tags: aarch64, arm, gentoo, linux
Published: 2020-06-01 18:57:05

I managed to almost finish assembling a `server-grade` aarch64 system.

Here some notes

## Hardware
- The [honeycomb](https://shop.solid-run.com/product/SRLX216S00D00GE064H04CH/).
- SO-DIMMS (I picked the one suggested [here](https://developer.solid-run.com/knowledge-base/lx2160a-cex7-tested-memory-so-dimms/))
- A [quieter fan](https://noctua.at/en/products/fan/nf-a4x20-pwm) (the fan coming with the board is extremely noisy)
- Some storage if you do not want to use the on-board eMMC 
- A case, I ended up getting a slim case but anything that supports mini-itx works.
- micro-usb cable
- sd card to flash the firmware and boot.

I do not plan to do anything GPU specific so I'm not using any GPU at least for now.

I plan to compile a lot so I put in a nice nvme.

## Setup

You need to connect the PSU, the power and reset pins, mount the ram and the storage.

Connect the mini-usb to your laptop (assuming you have already the right usb-serial driver) and start the machine.

Once I powered up the PSU 2 green leds got up and the fan started to spin at a low rate. Nothing on console.

> Pressing the `reset` button once started the boot process and the fan started to go at full speed. It is **painfully** noisy, I replaced it before continuing.
> **UPDATE**: Looks like my board is faulty, normal hardware boots and reboots correctly.

I followed [those instructions](https://developer.solid-run.com/knowledge-base/honeycomb-clearfog-cx-installation-and-tips/)

```
=> load mmc 0:1 0xa4000000 ubuntu-core.img
440401920 bytes read in 21592 ms (19.5 MiB/s)
=> nvme scan
=> nvme info
Device 0: Vendor: 0x1987 Rev: RKT303.3 Prod: 203E0705061500174291
            Type: Hard Disk
            Capacity: 976762.3 MB = 953.8 GB (2000409264 x 512)
=> nvme write 0xa4000000 0 0xd2000

nvme write: device 0 block # 0, count 860160 ... 860160 blocks written: OK
=> boot
```

In the [readme](https://github.com/SolidRun/lx2160a_build/blob/master/README.md) it mentions that it can boot from the eMMC but it does not seem to work.

I made sure the system it is working and I put ubuntu in the eMMC as well so I have a fallback.

## Booting

> I'm not sure if it is a board quirk or what I got is not really functional but this board does not seem to `reboot` nor `halt` correctly.
> It does boot up only from a *cold* boot and only if I first press `power` it and then press `reset` after, it is fairly annoying. I hope it is an issue that can be addressed later.
> **UPDATE**: The board is faulty, hopefully it will be repaired.

It uses a nice u-boot setup:
- the default `bootcmd` probes every device and looks for a `/extlinux/extlinux.conf`
- The kernel command line uses the `PARTUUID` to boot the right drive

```
  TIMEOUT 30
  DEFAULT linux
  MENU TITLE linux-lx2160a boot options
  LABEL primary
    MENU LABEL primary kernel
    LINUX /boot/Image
    FDT /boot/fsl-lx2160a-cex7.dtb
    APPEND console=ttyAMA0,115200 earlycon=pl011,mmio32,0x21c0000 default_hugepagesz=1024m hugepagesz=1024m hugepages=2 pci=pcie_bus_perf root=PARTUUID=30303030-01 rw rootwait
```

I installed the same image on both the `eMMC` and the `nvme`, so I ended up editing it to use directly `/dev/nvme0n1p1`

```
  TIMEOUT 30
  DEFAULT linux
  MENU TITLE linux-lx2160a boot options
  LABEL nvme0
    MENU LABEL primary kernel (nvme0)
    LINUX /boot/Image
    FDT /boot/fsl-lx2160a-cex7.dtb
    APPEND console=ttyAMA0,115200 earlycon=pl011,mmio32,0x21c0000 default_hugepagesz=1024m hugepagesz=1024m hugepages=2 pci=pcie_bus_perf root=/dev/nvme0n1p1 rw rootwait
  LABEL mmc1
    MENU LABEL primary kernel (mmc1)
    LINUX /boot/Image
    FDT /boot/fsl-lx2160a-cex7.dtb
    APPEND console=ttyAMA0,115200 earlycon=pl011,mmio32,0x21c0000 default_hugepagesz=1024m hugepagesz=1024m hugepages=2 pci=pcie_bus_perf root=PARTUUID=30303030-01 rw rootwait
```

## Gentoo

The bootloader and kernel provided are recent enough so all I did was to install Gentoo the lazy way from the eMMC ubuntu

``` sh
$ apt install btrfs-progs
$ mkfs.btrfs /dev/nvme0n1p1
$ mount /dev/nvme0n1p1 /mnt
$ cd /mnt
# change 20200508 with what's available when you try
$ wget http://distfiles.gentoo.org/experimental/arm64/stage3-arm64-20200508.tar.bz2
$ mount --rbind /dev dev
$ mount --make-rslave dev
$ mount -t proc /proc proc
$ mount --rbind /sys sys
$ mount --make-rslave sys
$ mount --rbind /tmp tmp
$ cp /etc/resolv.conf etc
$ chroot . /bin/bash
$ env-update && . /etc/profile
$ emerge-webrsync
$ emerge superadduser openssh vim
# Set the root password
$ passwd 
# enable root login
$ vim /etc/ssh/sshd_config 
# or create your user
$ superadduser your_user
$ ln -s /etc/init.d/net.{lo,eth0}
$ rc-update add sshd default
$ reboot
```

Simple enough.

My `make.conf` changes so far:

``` sh
ACCEPT_KEYWORDS=~arm64
MAKEOPTS="-j 16"
EMERGE_DEFAULT_OPTS="-j 8 --load-average 8.0"
```

## Coming next

Tomorrow I'll probably build a new kernel since its defaults seem lacking some optional features portage would like to use for sandboxing, if something strange happens I might blog more about it.

If not I'll write an update regarding rav1e since we have `0.3.2` out (not yet in crates.io for some ongoing discussion regarding being compatible with rustc-1.36 or rustc-1.38) and probably talk a bit about the new rate control API and hopefully give a walk-through the current API (both rust and C).