Project: containerd (Runtime / Container Runtime)
File: devmapper.md

Devmapper is a `containerd` snapshotter plugin that stores snapshots in filesystem images in a Device-mapper thin-pool. The devmapper plugin takes advantage of Device-mapper features like . To use the devmapper snapshotter plugin you need to prepare a Device-mapper `thin-pool` in advance and update containerd's configuration file. This file is typically located at `/etc/containerd/config.toml`. Here's a minimal sample entry that can be made in the configuration file: ```toml version = 2 [plugins] ... [plugins."io.containerd.snapshotter.v1.devmapper"] root_path = "/var/lib/containerd/devmapper" pool_name = "containerd-pool" baseimagesize = "8192MB" ... ``` The following configuration flags are supported: `root_path` - a directory where the metadata will be available (if empty default location for `containerd` plugins will be used) `pool_name` - a name to use for the Device-mapper thin-pool. Pool name should be the same as in `/dev/mapper/` directory `baseimagesize` - defines how much space to allocate when creating thin device snapshots from the base (pool) device `async_remove` - flag to async remove device using snapshot GC's cleanup callback (default: `false`) `discard_blocks` - whether to discard blocks when removing a device. This is especially useful for returning disk space to the filesystem when using loopback devices. (default: `false`) `fs_type` - defines the file system to use for snapshot device mount. Valid values are `ext4` and `xfs` (default: `"ext4"`) `fs_options` - optionally defines the file system options. This is currently only applicable to `ext4` file system. (default: `""`) `rootpath`, `poolname`, and `baseimagesize` are required snapshotter parameters. Give it a try with the following commands: ```bash ctr images pull --snapshotter devmapper docker.io/library/hello-world:latest ctr run --snapshotter devmapper docker.io/library/hello-world:latest test ``` The devicemapper snapshotter requires `dmsetup` (>= 1.02.110) command line tool to be installed and available on your computer. On Ubuntu, it can be installed with `apt-get install dmsetup` command. There are many ways how to configure a Device-mapper thin-pool depending on your requirements, disk configuration, and environment. Two common configurations are provided below, one for development environments and one for production environments. On local dev environment you can utilize loopback

This type of configuration is simple and suits well for development and testing (please note that this configuration is slow and not recommended for production uses). Run the following script to create a thin-pool device with associated metadata and data device files: ```bash set -ex DATA_DIR=/var/lib/containerd/devmapper POOL_NAME=devpool sudo mkdir -p ${DATA_DIR} sudo touch "${DATA_DIR}/data" sudo truncate -s 100G "${DATA_DIR}/data" sudo touch "${DATA_DIR}/meta" sudo truncate -s 10G "${DATA_DIR}/meta" DATADEV=$(sudo losetup --find --show "${DATADIR}/data") METADEV=$(sudo losetup --find --show "${DATADIR}/meta") SECTOR_SIZE=512 DATASIZE="$(sudo blockdev --getsize64 -q ${DATADEV})" LENGTHINSECTORS=$(bc <<< "${DATASIZE}/${SECTORSIZE}") DATABLOCKSIZE=128 LOWWATERMARK=32768 sudo dmsetup create "${POOL_NAME}" \ --table "0 ${LENGTHINSECTORS} thin-pool ${METADEV} ${DATADEV} ${DATABLOCKSIZE} ${LOWWATERMARK}" cat << EOF [plugins] [plugins."io.containerd.snapshotter.v1.devmapper"] poolname = "${POOLNAME}" rootpath = "${DATADIR}" baseimagesize = "10GB" discard_blocks = true EOF ``` Use `dmsetup` to verify that the thin-pool was created successfully: ```bash sudo dmsetup ls devpool (253:0) ``` Once `containerd` is configured and restarted, you'll see the following output: ``` INFO[2020-03-17T20:24:45.532604888Z] loading plugin "io.containerd.snapshotter.v1.devmapper"... type=io.containerd.snapshotter.v1 INFO[2020-03-17T20:24:45.532672738Z] initializing pool device "devpool" ``` Another way to setup a thin-pool is via the tool (formerly known as `docker-storage-setup`). It is a script to configure CoW file systems like devicemapper: ```bash set -ex BLOCK_DEV=/dev/sdf POOL_NAME=devpool VG_NAME=containerd git clone https://github.com/projectatomic/container-storage-setup.git cd container-storage-setup/ sudo make install-core echo "Using version $(container-storage-setup -v)" sudo tee /etc/sysconfig/docker-storage-setup <<EOF DEVS=${BLOCK_DEV} VG=${VG_NAME} CONTAINERTHINPOOL=${POOLNAME} EOF sudo container-storage-setup cat << EOF [plugins] [plugins.devmapper] root_path = "/var/lib/containerd/devmapper" poolname = "${VGNAME}-${POOL_NAME}" baseimagesize = "10GB" EOF ``` If successful `container-storage-setup` will output: ``` echo VG=containerd sudo container-storage-setup INFO: Volume group backing root filesystem could not be determined INFO: Writing zeros to first 4MB of device /dev/xvdf 4+0 records in 4+0 records out 4194304 bytes (4.2 MB) copied, 0.0162906 s, 257 MB/s INFO: Device node /dev/xvdf1 exists. Physical volume "/dev/xvdf1" successfully created. Volume group "containerd" successfully created Rounding up size to full physical extent 12.00 MiB Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data. Logical volume "devpool" created. Logical volume containerd/devpool changed. ... ``` And `dmsetup` will produce the following output: ```bash sudo dmsetup ls containerd-devpool (253:2) containerd-devpool_tdata (253:1) containerd-devpool_tmeta (253:0) ``` See also for additional information about production devmapper setups. For more information on Device-mapper, thin provisioning, etc., you can refer to the following resources: https://access.redhat.com/documentation/en-us/redhatenterpriselinux/6/html/logicalvolumemanageradministration/device_mapper https://en.wikipedia.org/wiki/Device_mapper https://docs.docker.com/storage/storagedriver/device-mapper-driver/ https://www.kernel.org/doc/Documentation/device-mapper/thin-provisioning.txt https://www.kernel.org/doc/Documentation/device-mapper/snapshot.txt