QEMU Storage Daemon
Tags: qemu, qsd
Published: 2023-01-06 17:34:57


[qemu-storage-daemon](https://www.qemu.org/docs/master/tools/qemu-storage-daemon.html) is a tool that provides disk image functionality for a VM without running the VM itself. It can export disk images, run block job operations, and perform other disk-related operations. This [article](https://lwn.net/Articles/911281/) was very helpful to get a better understanding of the QEMU storage daemon.

Here, I used two of the export methods for qsd to access the Fedora disk image that was created in the previous blog post.

##1. Fuse export##

[FUSE](https://www.kernel.org/doc/html/latest/filesystems/fuse.html) (Filesystem in Userspace) is an interface for userspace programs to export a filesystem to the Linux kernel.

There is a really nice tutorial on FUSE export on the [QEMU website](https://www.qemu.org/2021/08/22/fuse-blkexport/) which I followed through.

1. First of all, install some dependencies:
    ```bash
    sudo apt install libfuse3-3 libfuse3-dev
    ```

2. Then configure qemu with `--enable-fuse` option and build it again. 
    ```bash
    ./configure --enable-fuse
    ```

3. Also, uncomment `user_allow_other` in `/etc/fuse.conf`. By default, only user who exported the image will have an access to it. Enabling `user_allow_other` option here will allow non-root users to be able to use `allow_other` mount option which enables users other than the owner to be able to access the mounted fuse image.

4. Create a file that will be a mount point where the filesystem will be mounted on. It's also possible to mount the image on itself.
    ```
    touch mount-point
    ```

5. Export:
    ```bash
    qemu-storage-daemon \
       --blockdev node-name=prot-node,driver=file, 
    filename=fedora37.qcow2   \
       --blockdev node-name=fmt-node,driver=qcow2,file=prot-node   
       --export type=fuse,id=exp0,node-name=fmt-node, 
    mountpoint=mount-point,writable=on  
    ```
    After the above command is executed, the mount-point will appear as a raw VM disk. Then, we'll be able to use tools like `parted`, `kpartx`, `file`,`mount` as they can't parse qcow2 images and can only work with raw images.
    ```bash
    $ qemu-img info mount-point
    image: mount-point
    file format: raw
    virtual size: 30 GiB (32212254720 bytes)
    disk size: 1.58 GiB
    ```

6. List the partition table:
    ```bash
    $ parted mount-point p
    WARNING: You are not superuser.  Watch out for permissions.
    Model:  (file)
    Disk /home/amar/projects/outreachy/qemu/fedora/mount-point: 32.2GB
    Sector size (logical/physical): 512B/512B
    Partition Table: gpt
    Disk Flags: 
    
    Number  Start   End     Size    File system  Name     Flags
     1      1049kB  2097kB  1049kB               primary  bios_grub
     2      2097kB  1076MB  1074MB  xfs          primary
     3      1076MB  32.2GB  31.1GB  xfs          primary
    ```

7. Add device mappings over partitions:
    ```bash
    $ sudo kpartx -av mount-point 
    [sudo] password for amar: 
    add map loop31p1 (253:6): 0 2048 linear 7:31 2048
    add map loop31p2 (253:7): 0 2097152 linear 7:31 4096
    add map loop31p3 (253:8): 0 60810752 linear 7:31 2101248
    ```

8. Mount the main partition:
    ```bash
    $ sudo mount /dev/mapper/loop31p3 /mnt/tmp/
    $ ls /mnt/tmp/
    afs  boot  etc   lib    media  opt   root  sbin  sys  usr
    bin  dev   home  lib64  mnt    proc  run   srv   tmp  var
    ```

9. Unmount the partition after done using it.
    ```bash
    sudo umount /mnt/tmp
    ```

10. Stop qsd by just hitting `Ctrl+C` or if it's running in the background, just kill the process.

## 2. NBD export

[NBD](https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md) (Network Block Device) is a simple protocol that allows users to export a block device to a client over a network. [Here](https://medium.com/@aysadx/linux-nbd-introduction-to-linux-network-block-devices-143365f1901b) is a nice introduction of NBD as suggested by my mentors.

1. First, load NBD module:
    ```
    sudo modprobe nbd
    ```

2. Export the image file over NBD. With the below settings, NBD server will start running on localhost port 10809.

    ```bash
    qemu-storage-daemon     \
        --blockdev node-name=prot-node,driver=file,
    filename=fedora37.qcow2     \
        --blockdev node-name=fmt-node,driver=qcow2,
    file=prot-node     \
        --nbd-server addr.type=inet,addr.host=localhost,
    addr.port=10809     \
        --export type=nbd,id=exp0,node-name=fmt-node,
    name=guest-disk,writable=on
    ```

3. Mount the NBD exported device as a local device /dev/nbd0 by starting the `nbd-client`. For `-N` or `-name` option, use the name provided in the export command above. If this option is not supplied, `nbd-client` will complain about oldstyle negotiation.
    ```bash
    $ sudo nbd-client localhost 10809 /dev/nbd0 -N guest-disk
    [sudo] password for amar: 
    Negotiation: ..size = 30720MB
    Connected /dev/nbd0
    ```

4. Block devices for each partition of the original image will be automatically created.
    ```
    $ ls /dev/nbd0<tab>
    nbd0    nbd0p1  nbd0p2  nbd0p3
    ```

5. Mount the main partition
    ```bash
    $ sudo mount /dev/nbd0p3 /mnt/tmp/
    $ ls /mnt/tmp/
    afs  boot  etc   lib    media  opt   root  sbin  sys  usr
    bin  dev   home  lib64  mnt    proc  run   srv   tmp  var
    ```

6. After finished, unmount and disconnect. Proper clean-up is important because if not done correctly, the next export will not work.
    ```
    umount /mnt/tmp
    nbd-client -d /dev/nbd0
    ```