Testing ublk on Ubuntu 22.04
Tags: ublk, linux
Published: 2023-01-16 06:45:25


ublk is a generic framework for implementing block device logic in the userspace. It's developed by our mentor Ming Lei. [Here](https://lwn.net/Articles/903855/) is an article that described the workings and future possibilities of ublk. Reading all about its possibilities, I'm super excited to work with this project.

ublk is comprised of two parts: kernel [ublk driver](https://www.kernel.org/doc/Documentation/block/ublk.rst) which is newly introduced to the Linux kernel since version 6.0 and userspace daemon ([ublksrv](https://github.com/ming1/ubdsrv)).  

When testing ublk driver on a Fedora 37 VM, it worked out of the box. But those who want to use it on Ubuntu, for now, will have to update their kernel to version 6.0 or up and configure it to enable ublk driver. I have Ubuntu 22.04 Jammmy Jellyfish and kernel 5.15. So the first step is to configure and build the kernel.

###1. Configure and build the Linux kernel

Download the latest stable kernel from [kernel.org](https://www.kernel.org/). The latest kernel as of this writing was 6.1.4.

Extract it:

```
tar xvf linux-6.1.4.tar.xz
```

Go to the extracted directory and copy the current kernel's configuration:

```
cd linux-6.1.4
cp -v /boot/config-$(uname -r) .config
```
The new kernel will have many new config options. To keep the old configuration and also to configure the new options to their default values, run the following:

```
make olddefconfig
```

Configure ublk_drv by running the GUI menuconfig, navigating to `Device drivers -> Block devices` and add `M` to the `Userspace block driver (Experimental)` to make it a loadable module. Click Save and Exit.

```
make menuconfig
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/egv4tyzqr304mspclrun.png)


Disable the conflicting security certificates. Without disabling them, the make will fail with error. This might not be the best solution, but the easiest one and serves my purpose.

```
scripts/config --disable SYSTEM_TRUSTED_KEYS
scripts/config --disable SYSTEM_REVOCATION_KEYS
```

Build and install. Use -j option to speed up the build process as it takes a lot of time. `nproc` will return the number of cores you can use.

```
make -j16
make modules
sudo make modules_install
sudo make install
```

Update grub bootloader:
```
sudo update-grub
```
Reboot

###2. Build and install liburing

libiring is library for the new Linux asynchronous IO interface called io_uring. This is the interface ublk uses and will require version 2.2 or up. 

You can install liburing with apt as shown below, but it was not the latest version. 

```
sudo apt install liburing2 liburing-dev
```

Therefore, I just built it from the cloned source.

```
$ git clone https://github.com/axboe/liburing
$ cd liburing
$ ./configure
$ make
$ sudo make install
```

###3. Build ubdsrv (userspace daemon)

```
git clone https://github.com/ming1/ubdsrv.git
autoreconf -i
./configure
make
```


###4. Try ublk

First, load `ublk_drv` as a module. After it's loaded, a device called `/dev/ublk_control` is created. Userspace daemon will communicate with this device.
```
sudo modprobe ublk_drv
```
Then I tried ublk by following the Quick start tutorial on ubdsrv's [source page](https://github.com/ming1/ubdsrv). For now, ublk commands need root permission to run.

####null disk

```
$ sudo ublk add -t null
dev id 0: nr_hw_queues 1 queue_depth 128 block size 512 dev_capacity 524288000
	max rq size 524288 daemon pid 235918 flags 0x2 state LIVE
	queue 0: tid 235919 affinity(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 )
	target {"dev_size":268435456000,"name":"null","type":0}
```
After the above command executed, ublk block device `ublkb0` and a char device `ublkc0` is added by the ublk driver in /dev/
```
$ ls /dev/ublk
ublkb0        ublkc0        ublk-control 
```
####loop device

A raw disk image created with `qemu-img` is used.
```
$ qemu-img create -f raw foo.img 4G
$ sudo ublk add -t loop -f foo.img 
[sudo] password for amar: ublk
dev id 1: nr_hw_queues 1 queue_depth 128 block size 4096 dev_capacity 8388608
	max rq size 524288 daemon pid 329999 flags 0x2 state LIVE
	queue 0: tid 330000 affinity(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 )
	target {"backing_file":"foo.img","dev_size":4294967296,"direct_io":1,"name":"loop","type":1}
$ ls /dev/ublk
ublkb0        ublkb1        ublkc0        ublkc1        ublk-control
```
You can format the block device `/dev/ublkb1` with xfs, mount it and can do anything you want with it.

```
$ sudo mkfs.xfs /dev/ublkb0 
meta-data=/dev/ublkb0            isize=512    agcount=4, agsize=262144 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1    bigtime=0 inobtcount=0
data     =                       bsize=4096   blocks=1048576, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
Discarding blocks...Done.
$ sudo mount /dev/ublkb0 /mnt/tmp/
$ sudo mkdir /mnt/tmp/test
$ ls /mnt/tmp/
test
$ sudo umount /mnt/tmp
```

###qcow2 disk 
qcow2 support is still experimental, but I was able to add the Fedora disk image created in my previous blog [post](https://dev.to/amarjargal/running-fedora-37-on-qemu-x8664-machine-133b).

```
$ sudo ublk add -t qcow2 -f ../fedora/fedora37.qcow2 
dev id 2: nr_hw_queues 1 queue_depth 128 block size 512 dev_capacity 62914560
	max rq size 524288 daemon pid 243562 flags 0x2 state LIVE
	queue 0: tid 243564 affinity(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 )
	target {"backing_file":"../fedora/fedora37.qcow2","cluster_bits":16,"dev_size":32212254720,"header_length":112,"l1_size":60,"name":"qcow2","refcount_order":4,"refcount_table_clusters":1,"type":2,"version":3}
```
`ublkb2p1`, `ublkb2p2` and `ublkb2p3` are added for each partition of the disk. Then we can mount the main partition and access the filesystem.
```
$ ls /dev/ublk
ublkb0        ublkb1        ublkb2        ublkb2p1      ublkb2p2      ublkb2p3      ublkc0        ublkc1        ublkc2        ublk-control
$ sudo mount /dev/ublkb2p3 /mnt/tmp/
$ ls /mnt/tmp/
afs  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
$ sudo umount /mnt/tmp
```

List all the devices:

```
$ sudo ublk list 
dev id 0: nr_hw_queues 1 queue_depth 128 block size 512 dev_capacity 524288000
	max rq size 524288 dae$ sudo umount /mnt/tmpmon pid 235918 flags 0x2 state LIVE
	queue 0: tid 235919 affinity(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 )
	target {"dev_size":268435456000,"name":"null","type":0}
dev id 1: nr_hw_queues 1 queue_depth 128 block size 4096 dev_capacity 8388608
	max rq size 524288 daemon pid 329999 flags 0x2 state LIVE
	queue 0: tid 330000 affinity(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 )
	target {"backing_file":"foo.img","dev_size":4294967296,"direct_io":1,"name":"loop","type":1}
dev id 2: nr_hw_queues 1 queue_depth 128 block size 512 dev_capacity 62914560
	max rq size 524288 daemon pid 243562 flags 0x2 state LIVE
	queue 0: tid 243564 affinity(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 )
	target {"backing_file":"../fedora/fedora37.qcow2","cluster_bits":16,"dev_size":32212254720,"header_length":112,"l1_size":60,"name":"qcow2","refcount_order":4,"refcount_table_clusters":1,"type":2,"version":3}

```
Remove all devices:
```
sudo ublk del -a
```