【Nerves】Zinqでハードウエア情報を含んだLinuxカーネルを起動するまでの道のり

# はじめに

Zinqを使用したEBAZ4205でNervesを動作させる事が目標です。
U-bootに必要なファイルをXSAファイルから作成するのと、カーネルのビルドに必要なDTSファイルを作成します。
これらを使ってLinuxカーネルの起動を行います。

# DTSファイル作成

XSAファイルからDTSファイルを作成します。XSAファイルは、Vivadoで作成したプラットフォームファイルです。

作成に必要なツールをxilinxのリポジトリから取得します。

```
$ mkdir ebaz4205_test02_dt_work
$ cd ebaz4205_test02_dt_work
$ mkdir dts
$ git clone https://github.com/Xilinx/device-tree-xlnx.git
$ cd device-tree-xlnx
$ git checkout xlnx_rel_v2024.1
```

XSCTコンソール（WindowsにVivadoをインストールしている場合、XSCTコンソールを開くショートカットができてた）を開いて、次のコマンドを実行します。

```
xsct% cd <device-tree-xlnxのパス>
xsct% hsi open_hw_design <プラットフォームファイル.xsa>
xsct% hsi set_repo_path ./
xsct% hsi create_sw_design device-tree -os device_tree -proc ps7_cortexa9_0
xsct% hsi generate_target -dir ../dts
xsct% exit
```

作成された
```
$ ls -la
合計 52
drwxr-xr-x   4 masa masa  4096 12月  8 14:08 .
drwxr-xr-x  11 masa masa  4096 12月  8 13:57 ..
drwxr-xr-x 117 masa masa  4096 12月  8 13:57 device-tree-xlnx
-rw-r--r--   1 masa masa  3904 12月  8 14:08 device-tree.mss
drwxr-xr-x   3 masa masa  4096 12月  8 14:08 include
-rw-r--r--   1 masa masa  1048 12月  8 14:08 pcw.dtsi
-rw-r--r--   1 masa masa   297 12月  8 13:57 skeleton.dtsi
-rw-r--r--   1 masa masa   485 12月  8 14:08 system-top.dts
-rw-r--r--   1 masa masa   194 12月  8 14:08 system.dts
-rw-r--r--   1 masa masa 13982 12月  8 13:57 zynq-7000.dtsi
```

Nervesのプロジェクトにコピー
```
$ cp -a dts/ ~/nerves/nerves_system_zynq_ebaz4205/board
```

# u-boot

Buildrootの標準のu-bootを使ってみたのですが、うまく起動できませんでした。DTSファイルや、BOOT.BINとの兼ね合いかもしれません。
Xilinxのリポジトリから、u-bootを取得して、使用してみます（結果論ですがこれではうまくいきました）。

```
BR2_TARGET_UBOOT=y
BR2_TARGET_UBOOT_BUILD_SYSTEM_KCONFIG=y
BR2_TARGET_UBOOT_CUSTOM_GIT=y
BR2_TARGET_UBOOT_CUSTOM_REPO_URL="https://github.com/Xilinx/u-boot-xlnx.git"
BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="xilinx-v2024.1"
BR2_TARGET_UBOOT_BOARD_DEFCONFIG="xilinx_zynq_virt"
# BR2_TARGET_UBOOT_FORMAT_BIN is not set
BR2_TARGET_UBOOT_FORMAT_REMAKE_ELF=y
BR2_TARGET_UBOOT_NEEDS_UTIL_LINUX=y
```

# Kernel

カーネルもxilinxのレポジトリにあるものを使用してみます。

```
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/Xilinx/linux-xlnx.git"
BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="xlnx_rebase_v5.15_LTS"
BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y
BR2_LINUX_KERNEL_DTS_SUPPORT=y
BR2_LINUX_KERNEL_USE_CUSTOM_DTS=y
BR2_LINUX_KERNEL_CUSTOM_DTS_PATH="${NERVES_DEFCONFIG_DIR}/board/dts/system-top.dts ${NERVES_DEFCONFIG_DIR}/board/dts/zynq-7000.dtsi ${NERVES_DEFCONFIG_DIR}/board/dts/pcw.dtsi"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="${NERVES_DEFCONFIG_DIR}/linux/linux-5.15.defconfig"
BR2_LINUX_KERNEL_PATCH="${NERVES_DEFCONFIG_DIR}/linux"
BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
```

# BOOT.BINの作成

Zynqでは、SDカードから起動するために、BOOT.BINを読み込んで起動する仕様となっているので、BOOT.BINが必要です。
BOOT.BINは、bootgenを使って作成します。
このツールはXilinxのツールです。
Buildrootで、
`BR2_PACKAGE_HOST_BOOTGEN=y`
としておくと、使えるようになります。とてもありがたいです。

bootgen実行時に必要な情報は、`bootgen.bif`に記載します。次の通りです。

```:bootgen.bif
the_ROM_image:
{
	[bootloader] zynq_fsbl.elf
	system.bit
	u-boot.elf
	[load=0x00100000] system-top.dtb
}
```

`BOOT.BIN`を作成するコマンドを、`post-createfs.sh`に追加して行います。
```
echo "Creating BOOT.BIN..."
cd $BINARIES_DIR
cp -f $NERVES_DEFCONFIG_DIR/board/bootgen.bif .
cp -f $NERVES_DEFCONFIG_DIR/board/zynq_fsbl.elf .
cp -f $NERVES_DEFCONFIG_DIR/board/system.bit .
$HOST_DIR/bin/bootgen -arch zynq -image bootgen.bif -w -o BOOT.BIN
```

system.bitとzynq_fsbl.elfは、vivadoで作成したものをboardディレクトリから、$BINARIES_DIRにコピーして、bootgenを実行するようにします。

これで、BOOT.BINが作成されました。

# 起動

起動すると、次の順に処理が行われます

- SDカードからBOOT.BINが読み込まれる
- BOOT.BINのzynq_fsbl.elfが実行される
- BOOT.BINのsystem.bitが読み込まれ、PLが初期化される
- BOOT.BINのu-boot.elfが実行される
- BOOT.BINのsystem-top.dtbが読み込まれ、カーネルが起動される

という流れになります。

起動時のメッセージを見ると、`Device Tree blob`が読み込まれていることがわかります。
```
Kernel image @ 0x200000 [ 0x000000 - 0x4e7880 ]
## Flattened Device Tree blob at 00100000
   Booting using the fdt blob at 0x100000
Working FDT set to 100000
   Loading Device Tree to 0dac0000, end 0dac5c26 ... OK
Working FDT set to dac0000
```


カーネルの起動メッセージに、axiバスに関するものも表示され、DTBが機能していることがわかります。
```
platform axi: Fixed dependency cycle(s) with /axi/interrupt-controller@f8f01000
platform replicator: Fixed dependency cycle(s) with /axi/etb@f8801000
amba f8801000.etb: Fixed dependency cycle(s) with /replicator
```

<details>
<summary>
起動メッセージ全体を表示する
</summary>

```
U-Boot 2024.01 (Dec 04 2024 - 21:19:45 +0000)

CPU:   Zynq 7z010
Silicon: v3.1
DRAM:  ECC disabled 256 MiB
Core:  23 devices, 16 uclasses, devicetree: board
Flash: 0 Bytes
NAND:  0 MiB
MMC:   mmc@e0100000: 0
Loading Environment from FAT... *** Error - No Valid Environment Area found
*** Warning - bad env area, using default environment

In:    serial@e0001000
Out:   serial@e0001000
Err:   serial@e0001000
Net:
ZYNQ GEM: e000b000, mdio bus e000b000, phyaddr 0, interface gmii

Warning: ethernet@e000b000 (eth0) using random MAC address - a6:0c:d3:fc:53:97
eth0: ethernet@e000b000
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
2776 bytes read in 12 ms (225.6 KiB/s)
## Executing script at 03000000
Trying to load boot images from mmc0
** No boot file defined **
Wrong Image Format for bootm command
ERROR: can't get kernel image!
5142656 bytes read in 457 ms (10.7 MiB/s)
** No boot file defined **
Wrong Image Format for bootm command
ERROR: can't get kernel image!
** No boot file defined **
Wrong Image Format for bootm command
ERROR: can't get kernel image!
Kernel image @ 0x200000 [ 0x000000 - 0x4e7880 ]
## Flattened Device Tree blob at 00100000
   Booting using the fdt blob at 0x100000
Working FDT set to 100000
   Loading Device Tree to 0dac0000, end 0dac5c26 ... OK
Working FDT set to dac0000

Starting kernel ...

Booting Linux on physical CPU 0x0
Linux version 6.10.14-xilinx-v2022.1 (buildroot@buildroot) (armv7-nerves-linux-gnueabihf-gcc (crosstool-NG UNKNOWN) 13.2.0, GNU ld (crosstool-NG UNKNOWN) 2.40) #1 SMP PREEMPT Wed Dec  4 21:19:45 UTC 2024
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: xlnx,zynq-7000
Memory policy: Data cache writealloc
cma: Reserved 128 MiB at 0x05800000 on node -1
Zone ranges:
  Normal   [mem 0x0000000000000000-0x000000000fffffff]
  HighMem  empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x000000000fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
percpu: Embedded 13 pages/cpu s22036 r8192 d23020 u53248
Kernel command line: console=ttyPS0,115200 uio_pdrv_genirq.of_id=generic-uio root=/dev/mmcblk0p2 rootwait
printk: log_buf_len individual max cpu contribution: 2097152 bytes
printk: log_buf_len total cpu_extra contributions: 2097152 bytes
printk: log_buf_len min size: 2097152 bytes
printk: log_buf_len: 4194304 bytes
printk: early log buf free: 2095892(99%)
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
Built 1 zonelists, mobility grouping on.  Total pages: 65536
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 90728K/262144K available (7168K kernel code, 6636K rwdata, 2120K rodata, 1024K init, 2195K bss, 40344K reserved, 131072K cma-reserved, 0K highmem)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
rcu: Preemptible hierarchical RCU implementation.
rcu:    RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
slcr mapped to (ptrval)
GIC physical location is 0xf8f01000
L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
L2C-310 erratum 769419 enabled
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 8 ways, 512 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
rcu: srcu_init: Setting srcu_struct sizes based on contention.
zynq_clock_init: clkc starts at (ptrval)
Zynq clock init
sched_clock: 64 bits at 167MHz, resolution 6ns, wraps every 4398046511103ns
clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0x26703d7dd8, max_idle_ns: 440795208065 ns
Switching to timer-based delay loop, resolution 6ns
Console: colour dummy device 80x30
Calibrating delay loop (skipped), value calculated using timer frequency.. 333.33 BogoMIPS (lpj=1666666)
CPU: Testing write buffer coherency: ok
CPU0: Spectre v2: using BPIALL workaround
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x100000 - 0x100060
rcu: Hierarchical SRCU implementation.
rcu:    Max phase no-delay instances is 1000.
smp: Bringing up secondary CPUs ...
CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
CPU1: Spectre v2: using BPIALL workaround
smp: Brought up 1 node, 2 CPUs
SMP: Total of 2 processors activated (666.66 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 512 (order: 3, 32768 bytes, linear)
pinctrl core: initialized pinctrl subsystem
NET: Registered PF_NETLINK/PF_ROUTE protocol family
DMA: preallocated 256 KiB pool for atomic coherent allocations
thermal_sys: Registered thermal governor 'step_wise'
cpuidle: using governor menu
platform axi: Fixed dependency cycle(s) with /axi/interrupt-controller@f8f01000
platform replicator: Fixed dependency cycle(s) with /axi/etb@f8801000
amba f8801000.etb: Fixed dependency cycle(s) with /replicator
platform replicator: Fixed dependency cycle(s) with /axi/tpiu@f8803000
amba f8803000.tpiu: Fixed dependency cycle(s) with /replicator
platform replicator: Fixed dependency cycle(s) with /axi/funnel@f8804000
amba f8804000.funnel: Fixed dependency cycle(s) with /axi/ptm@f889d000
amba f8804000.funnel: Fixed dependency cycle(s) with /axi/ptm@f889c000
amba f8804000.funnel: Fixed dependency cycle(s) with /replicator
amba f8804000.funnel: Fixed dependency cycle(s) with /axi/ptm@f889c000
amba f889c000.ptm: Fixed dependency cycle(s) with /axi/funnel@f8804000
amba f8804000.funnel: Fixed dependency cycle(s) with /axi/ptm@f889d000
amba f889d000.ptm: Fixed dependency cycle(s) with /axi/funnel@f8804000
hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
hw-breakpoint: maximum watchpoint size is 4 bytes.
e0001000.serial: ttyPS0 at MMIO 0xe0001000 (irq = 26, base_baud = 6249999) is a xuartps
printk: legacy console [ttyPS0] enabled
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
EDAC MC: Ver: 3.0.0
FPGA manager framework
Advanced Linux Sound Architecture Driver Initialized.
vgaarb: loaded
clocksource: Switched to clocksource arm_global_timer
NET: Registered PF_INET protocol family
IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear)
Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
TCP bind hash table entries: 1024 (order: 2, 16384 bytes, linear)
TCP: Hash tables configured (established 1024 bind 1024)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp-with-tls transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 0 bytes, default 64
armv7-pmu f8891000.pmu: hw perfevents: no interrupt-affinity property, guessing.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
workingset: timestamp_bits=14 max_order=16 bucket_order=2
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) (SUMMARY)  c 2001-2006 Red Hat, Inc.
io scheduler mq-deadline registered
io scheduler kyber registered
io scheduler bfq registered
zynq-pinctrl 700.pinctrl: zynq pinctrl initialized
brd: module loaded
loop: module loaded
CAN device driver interface
macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000 irq 34 (a6:0c:d3:fc:53:97)
e1000e: Intel(R) PRO/1000 Network Driver
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
usbcore: registered new interface driver usb-storage
i2c_dev: i2c /dev entries driver
cdns-i2c e0004000.i2c: can't get pinctrl, bus recovery not supported
cdns-i2c e0004000.i2c: 400 kHz mmio e0004000 irq 35
cdns-i2c e0005000.i2c: can't get pinctrl, bus recovery not supported
cdns-i2c e0005000.i2c: 400 kHz mmio e0005000 irq 36
cdns-wdt f8005000.watchdog: Xilinx Watchdog Timer with timeout 10s, nowayout
EDAC MC: ECC not enabled
Xilinx Zynq CpuIdle Driver started
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
ledtrig-cpu: registered to indicate activity on CPUs
clocksource: ttc_clocksource: mask: 0xffff max_cycles: 0xffff, max_idle_ns: 537538477 ns
timer #0 at (ptrval), irq=39
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
pl353-smc e000e000.memory-controller: no matching children
mmc0: SDHCI controller on e0100000.mmc [e0100000.mmc] using ADMA
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
NET: Registered PF_PACKET protocol family
can: controller area network core
NET: Registered PF_CAN protocol family
can: raw protocol
can: broadcast manager protocol
can: netlink gateway - max_hops=1
Registering SWP/SWPB emulation handler
mmc0: new high speed SDHC card at address 1234
mmcblk0: mmc0:1234 SA32G 28.9 GiB
 mmcblk0: p1 p2 p3
Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level
clk: Disabling unused clocks
ALSA device list:
  No soundcards found.
VFS: Mounted root (squashfs filesystem) readonly on device 179:2.
devtmpfs: mounted
Freeing unused kernel image (initmem) memory: 1024K
Run /sbin/init as init process
F2FS-fs (mmcblk0p3): Magic Mismatch, valid(0xf2f52010) - read(0xffffffff)
F2FS-fs (mmcblk0p3): Can't find valid F2FS filesystem in 1th superblock
F2FS-fs (mmcblk0p3): Magic Mismatch, valid(0xf2f52010) - read(0xffffffff)
F2FS-fs (mmcblk0p3): Can't find valid F2FS filesystem in 2th superblock
erlinit: Cannot mount /dev/mmcblk0p3 at /root: Invalid argument
erlinit: Unable to create seed directory
random: crng init done
Erlang/OTP 27 [erts-15.1.2] [source] [32-bit] [smp:2:2] [ds:2:2:10] [async-threads:1]

F2FS-fs (mmcblk0p3): Magic Mismatch, valid(0xf2f52010) - read(0xffffffff)
F2FS-fs (mmcblk0p3): Can't find valid F2FS filesystem in 1th superblock
F2FS-fs (mmcblk0p3): Magic Mismatch, valid(0xf2f52010) - read(0xffffffff)
F2FS-fs (mmcblk0p3): Can't find valid F2FS filesystem in 2th superblock
F2FS-fs (mmcblk0p3): Found nat_bits in checkpoint
F2FS-fs (mmcblk0p3): Mounted with checkpoint version = 2b784f37
macb e000b000.ethernet eth0: PHY [e000b000.ethernet-ffffffff:00] driver [Generic PHY] (irq=POLL)
macb e000b000.ethernet eth0: configuring for phy/gmii link mode
Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)
??????    ????
??  ??????  ??
??  ??  ??  ??   N  E  R  V  E  S
??  ??????  ??
????    ??????
test_project_zynq_ebaz4205 0.1.0 (8f7ebf68-a4af-5445-fcb2-88c17ef90969) arm zynq_ebaz4205
  Serial       : 5397
  Uptime       : 19.007 seconds
  Clock        : 2024-12-04 21:20:04 UTC (unsynchronized)

  Firmware     : Valid (A)               Applications : 38 started
  Memory usage : 56 MB (25%)             Part usage   : 449 MB (2%)
  Hostname     : nerves-5397             Load average : 1.51 0.38 0.13


Nerves CLI help: https://hexdocs.pm/nerves/iex-with-nerves.html

Toolshed imported. Run h(Toolshed) for more info.
iex(1)> macb e000b000.ethernet eth0: unable to generate target frequency: 25000000 Hz
macb e000b000.ethernet eth0: Link is Up - 100Mbps/Full - flow control tx
```
</details>

# おわりに

XSAファイルから、Nerves起動に必要なファイルを作成し、カーネルが起動するところまで確認できました。
やっとこれで、EBAZ4205ボードを自在に使える状態になりました。
ハードウエアを作成して、Nervesから使ってみたいと思います。

# 参考
https://github.com/zeam-vm/nerves_system_zybo_z7_10
https://blog-between-hw-sw.com/zynq7-make-devicetree4linux/