Using QCOW2 backing files & overlays for creating libvirt/kvm vm base images.
Tags: linux, devops, tutorial, virtualization
Published: 2020-05-08 12:13:00

I'm fascinated by and thankful for storage solutions supporting checkpoint technologies such as snapshotting. In essence when snapshot is created, storage state is saved and future changes do not alter the state. Snapshots can save time and associated storage costs in containerization, virtualization and plain OS workflows. QCOW2 backing files & overlays can also be used for creating virtual machine disk base images. 

Below is [kickstart](www.tomas.io/assets/base-image/ks-base.cfg) configuration for Fedora installation. It is losely based on minimum kickstart from [try-fedora](www.tomas.io/articles/try-fedora) article, with customizations to storage and user access. 

    #version=DEVEL

    # Keyboard layouts
    keyboard --vckeymap=us --xlayouts='us'

    # Use text mode install
    text

    # System language
    lang en_US.UTF-8

    # System timezone
    timezone UTC --isUtc

    # Network information
    network --bootproto=dhcp --device=link --activate

    # Firewall configuration
    firewall --enabled --service=ssh

    # Do not configure the X Window System
    skipx

    # System services
    services --enabled=sshd,chronyd

    # Lock root account
    rootpw --lock

    # Create user with authorized key for ssh access
    user --groups=wheel,users --name="ansible"
    # Contents of ansible.pub from 'ssh-keygen -b 2048 -t rsa -f ~/.ssh/ansible -q -N ""'
    sshkey --username="ansible" "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCKnhHiwSGuV65LWLG3D6JubYpBzQyUlAXh1bo5VDgfhU1ixGnkX6ucm/aEKNwZr3DTMZSbW+pmuDq/7plsxa8BDvZWFMHUc+w990EikwNDRizYwywUdmc7C4v81FO++j92SGnclX8lC+V9N/4FzFZz8otT6z8xHyNzFtmACCjBiHhb0/KvQZ7NyPC2y0GY5pA2152qSujZ1+Vx5DhAJBJAw8WhiGu+T6DVHQ29D1KTZ3hTy+bNP8o499N79eQSoX2RMoW8+B93cFAbQqw/tknSX4wi3vatyLvnw6jjeYVkWT+WzktcCZ8Yb0ql3OTQWxCLTJloAmbjDsmL9rWRZuv tomas@uola"

    # System bootloader configuration
    bootloader --location=mbr --boot-drive=vda --append="net.ifnames=0 biosdevname=0"

    # Format partitions created in %pre
    part /boot --fstype=ext4 --label=boot --onpart=vda1
    # create physical volume for LVM
    part pv.1 --onpart=vda2
    # create vg volume group
    volgroup vg --pesize=4096 pv.1
    # create root volume
    logvol swap --label=swap --name=swap --vgname=vg --size=1000
    logvol / --fstype=xfs --label=root --name=root --vgname=vg --grow --percent 100

    # Use net install
    url --url="https://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/"

    # Enable repos
    repo --name=os --baseurl="https://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/"
    repo --name=updates --baseurl="https://download.fedoraproject.org/pub/fedora/linux/updates/$releasever/Everything/$basearch/"

    # Packages
    %packages
    @^custom-environment
    %end

    # Pre install script
    %pre --logfile=/root/ks-pre.log

    # Setup disk partitions
    echo "Creating partitions on /dev/vda"
    parted --script /dev/vda \
        mklabel msdos \
        mkpart primary ext4 1MiB 300MiB \
        mkpart primary ext4 300MiB 100% \
        set 2 lvm on

    kpartx /dev/vda

    %end

    # Post install script
    %post --logfile=/root/ks-post.log

    # Allow users in wheel group execute sudo commands without password
    echo "Creating /etc/sudoers.d/wheel"
    echo "%wheel ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/wheel

    # Remove hostname
    echo "" > /etc/hostname

    %end

    # Halt after installation
    halt
    
Fedora base os image is created using network installation with a kickstart file.

    🌈[base-image]$ virt-install \
    --name f32-base \
    --ram 2048 \
    --os-type linux \
    --os-variant fedora31 \
    --graphics none \
    --disk=./f32-base.vda.x86_64.qcow2,bus=virtio,format=qcow2,size=7 \
    --location=https://download.fedoraproject.org/pub/fedora/linux/releases/32/Everything/x86_64/os \
    --initrd-inject ./ks-base.cfg \
    --extra-args="inst.ks=file:/ks-base.cfg console=ttyS0 net.ifnames=0 biosdevname=0"

f32-base vm should not be started after the installation as this will prevent storage file from being used as backing file.

<iframe allowfullscreen="true" webkitallowfullscreen="true" mozallowfullscreen="true" src="/assets/base-image/virt-install.html" width="100%" height="420"></iframe>


Qemu-img command is used to create virtual disks with various configurations, including backed storage.

    🌈[base-image]$ qemu-img create -b f32-base.vda.x86_64.qcow2 -f qcow2 saturn.vda.x86_64.qcow2 30G

Newly created backed images almost take no disk space with the benefit of having contents of the parent disk image.

    🌈[base-image]$ qemu-img info saturn.vda.x86_64.qcow2 --backing-chain

    image: saturn.vda.x86_64.qcow2
    file format: qcow2
    virtual size: 30 GiB (32212254720 bytes)
    disk size: 196 KiB
    cluster_size: 65536
    backing file: f32-base.vda.x86_64.qcow2
    Format specific information:
        compat: 1.1
        lazy refcounts: false
        refcount bits: 16
        corrupt: false

    image: f32-base.vda.x86_64.qcow2
    file format: qcow2
    virtual size: 7 GiB (7516192768 bytes)
    disk size: 1.37 GiB
    cluster_size: 65536
    Format specific information:
        compat: 1.1
        lazy refcounts: true
        refcount bits: 16
        corrupt: false


Virtual machine creation using backed storage.

    🌈[base-image]$ virt-install \
    --name saturn \
    --ram 1024 \
    --os-type linux \
    --os-variant generic \
    --graphics none \
    --disk=./saturn.vda.x86_64.qcow2,bus=virtio,format=qcow2 \
    --qemu-commandline="-netdev user,id=net0,hostname=saturn,hostfwd=tcp::22001-:22 -device pcnet,netdev=net0" \
    --import

Virt-install by default connects to qemu:///session which uses user networking (slirp) networking backend. Extra qemu arguments are specified to enable tcp port forwarding from host 22001 to guest 22.  

<iframe allowfullscreen="true" webkitallowfullscreen="true" mozallowfullscreen="true" src="/assets/base-image/virt-install-saturn.html" width="100%" height="420"></iframe>

Base image is setup to prevent console access, instead ansible user has authorized ssh key, which will allow ssh access via forwarded tcp port 22001 

    🌈[base-image]$ ssh ansible@127.0.0.1 -p 22001 -i ~/.ssh/ansible
    The authenticity of host '[127.0.0.1]:22001 ([127.0.0.1]:22001)' can't be established.
    ECDSA key fingerprint is SHA256:x4weSw+f9CFHpTQ1Rm/O4ynwXPLnowF+qGc6Vo3zodc.
    Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
    Warning: Permanently added '[127.0.0.1]:22001' (ECDSA) to the list of known hosts.
    
    [ansible@saturn ~]$ uname -a
    Linux saturn 5.6.8-300.fc32.x86_64 #1 SMP Wed Apr 29 19:01:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
    
    [ansible@saturn ~]$ sudo lvs -a
    LV   VG Attr       LSize    Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
    root vg -wi-ao----   <5.73g                                                    
    swap vg -wi-ao---- 1000.00m                                                    
    
    [ansible@saturn ~]$ sudo parted /dev/vda unit GB print free
    Model: Virtio Block Device (virtblk)
    Disk /dev/vda: 32.2GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Disk Flags: 

    Number  Start   End     Size    Type     File system  Flags
            0.00GB  0.00GB  0.00GB           Free Space
    1      0.00GB  0.31GB  0.31GB  primary  ext4         boot
    2      0.31GB  7.52GB  7.20GB  primary               lvm
            7.52GB  32.2GB  24.7GB           Free Space

    
    [ansible@saturn ~]$ free -h
                total        used        free      shared  buff/cache   available
    Mem:          973Mi       121Mi       678Mi       0.0Ki       173Mi       713Mi
    Swap:         999Mi          0B       999Mi
   
    [ansible@saturn ~]$ nmcli -p device
    =====================
    Status of devices
    =====================
    DEVICE  TYPE      STATE      CONNECTION         
    ----------------------------------------------------------------------
    eth0    ethernet  connected  eth0               
    eth1    ethernet  connected  Wired connection 1 
    lo      loopback  unmanaged  --                 
    
    [ansible@saturn ~]$ sudo dnf update
    Fedora 32 openh264 (From Cisco) - x86_64                     2.8 kB/s | 5.1 kB     00:01    
    Fedora Modular 32 - x86_64                                   2.9 MB/s | 4.9 MB     00:01    
    Fedora Modular 32 - x86_64 - Updates                         1.1 MB/s | 1.4 MB     00:01    
    Fedora 32 - x86_64 - Updates                                 2.4 MB/s | 7.8 MB     00:03    
    Fedora 32 - x86_64                                           4.9 MB/s |  70 MB     00:14    
    Dependencies resolved.
    Nothing to do.
    Complete!
    [ansible@saturn ~]$ exit
    logout
    Connection to 127.0.0.1 closed.

As seen from *parted* command vda disk has 24.7GB free space that can be used to create new partitions or extend existing ones. Multiple VM's can share same backing storage.