BtrFS, ZFS, and More
Tags: sysadmin, comparison
Published: 2019-07-03 21:58:01

## Butter Filesystem.  Hold the toast.

I've started experimenting with _BtrFS_ which aims to provide an "advanced and modern filesystem" (heavily compared to ZFS) on Linux.  With my new workstation I've started using BtrFS for my home directories (`/home`) and my build directories (`/mnt/slackbuilds`) to gain exposure to the filesystem and compare it to ZFS and EXT4 on LVM (all of my other data, including my root disk is on EXT4 on LVM).

I have used ZFS heavily in the past, and using BtrFS is significantly different as many of the fundamental concepts are different.  BtrFS has no concept of "pools" or "volume groups" -- instead there are "volumes."  BtrFS has no concept of "datasets" or "logical volumes" -- instead there are "subvolumes".

Here's a comparison between [ZFS](http://hub.opensolaris.org/bin/view/Community+Group+zfs/WebHome), [BtrFS](https://btrfs.wiki.kernel.org/index.php/Main_Page), and [EXT4](https://ext4.wiki.kernel.org/index.php/Main_Page) on [LVM](http://sources.redhat.com/lvm2/wiki/):

### Comparison

<table>
<tr><th></th><th>ZFS</th><th>BtrFS</th><th>EXT4 and LVM</th></tr>
<tr><td><b>Commands Involved</b></td><td><tt>zpool</tt>, <tt>zfs</tt></td><td><tt>mkfs.btrfs</tt>, <tt>btrfs</tt></td><td><tt>pvcreate</tt>, <tt>vgcreate</tt>, <tt>lvcreate</tt>, <tt>mkfs.ext4</tt></td></tr>
<tr><td><b>Pool of disks</b></td><td>&quot;zpool&quot;</td><td>&quot;volume&quot;</td><td>&quot;volume group&quot;</td></tr>
<tr><td><b>Mountable unit</b></td><td>&quot;dataset&quot;</td><td>&quot;volume&quot; and &quot;subvolume&quot;</td><td>&quot;logical volume&quot;</td></tr>
<tr><td><b>License</b></td><td>CDDL</td><td>GPL</td><td>GPL</td></tr>
<tr><td><b>Can be Boot filesystem</b></td><td>Yes</td><td>Yes (grub 2.00)</td><td>No</td></tr>
<tr><td><b>Can be Root filesystem</b></td><td>Yes</td><td>Yes</td><td>Yes</td></tr>
<tr><td><b>Can provide swapspace</b></td><td>Yes (zvols)</td><td>No</td><td>Yes (lvm)</td></tr>
<tr><td><b>OSes with Implementations</b></td><td>Solaris, OpenSolaris, Nexenta, FreeBSD, Mac OS X, Linux</td><td>Linux</td><td>Linux</td></tr>
<tr><td><b>Stability</b></td><td>Stable</td><td>Stable</td><td>Stable</td></tr>
<tr><td><b>CLI-System Integration [1]</b></td><td>Strong</td><td>Weak</td><td>Mild</td></tr>
<tr><td><b>Grow Online</b></td><td>Yes</td><td>Yes</td><td>Only when there are no snapshots</td></tr>
<tr><td><b>Shrink Pool</b></td><td>No</td><td>Online</td><td>Online</td></tr>
<tr><td><b>Shrink Filesystem</b></td><td>Online (reduce quota)</td><td>Online</td><td>Offline</td></tr>
<tr><td><b>Replace Disk (without parity)</b></td><td>Yes (must be compatible size disk)</td><td>Yes</td><td>Yes (copies space allocated on disk)</td></tr>
<tr><td><b>Filesystem Level Storage Pooling</b></td><td>Yes</td><td>Yes</td><td>No</td></tr>
<tr><td><b>Re-balance</b></td><td>No</td><td>Yes</td><td>Can be done manually (<tt>pvmove</tt>)</td></tr>
<tr><td><b>Checksumming</b></td><td>Yes</td><td>Yes</td><td>No</td></tr>
<tr><td><b>Autocorrect Checksum Errors</b></td><td>Yes</td><td>???</td><td>No</td></tr>
<tr><td><b>Compression</b></td><td>Yes</td><td>Yes</td><td>No</td></tr>
<tr><td><b>De-duplication</b></td><td>Yes (only synchronous)</td><td>Yes (only asynchronous)</td><td>No</td></tr>
<tr><td><b>Ditto Blocks</b></td><td>Yes</td><td>???</td><td>No</td></tr>
<tr><td><b>Tiered Caching</b></td><td>Yes</td><td>No</td><td>No</td></tr>
<tr><td><b>Writable Snapshots</b></td><td>Yes (clone)</td><td>Yes</td><td>Yes</td></tr>
<tr><td><b>Copy-on-Write</b></td><td>Fast, space-efficient</td><td>Fast, space-efficient</td><td>Slow, requires pre-allocating an LV</td></tr>
<tr><td><b>Redundancy</b></td><td>Mirroring and Parity (x1, x2, x3)</td><td>Mirroring</td><td>Mirroring, though the PVs can be RAID devices</td></tr>
<tr><td><b>Maximum Volume Size</b></td><td>16 Exabytes</td><td>16 Exabytes</td><td>1 Exabyte</td></tr>
<tr><td><b>Maximum File Size</b></td><td>16 Exabytes</td><td>16 Exabytes</td><td>16 Terabytes</td></tr>
<tr><td><b>Maximum Number of Snapshots</b></td><td><i>Unlimited</i></td><td><i>Unlimited</i></td><td>Effectively 32</td></tr>
</table>

[1] For lack of a better term -- how well the command line interface integrates with the system as a whole, this might be subjective.

For a more complete, but less focused comparison see [Wikipedia's Comparison of Filesystems](http://en.wikipedia.org/wiki/Comparison_of_file_systems)

### The Rosetta Stone
   1. _Task:_ Create a pool of storage from disks `/dev/A`, `/dev/B`, and `/dev/C` (striped or linear concat)
    1. Using ZFS:
     1. `# zpool create TESTPOOL A B C`
    1. Using BtrFS:
     1. `# mkfs.btrfs -d raid0 /dev/A /dev/B /dev/C`
    1. Using EXT4 on LVM:
     1. `# pvcreate /dev/A /dev/B /dev/C`
     1. `# vgcreate TESTPOOL /dev/A /dev/B /dev/C`
   1. _Task:_ Make storage from pool available to system
    1. Using ZFS:
     1. `# zfs set mountpoint=/data TESTPOOL`
    1. Using BtrFS:
     1. `# mkdir /data`
     1. `# mount -t btrfs /dev/A /data`
     1. Update `/etc/fstab`
    1. Using EXT4 on LVM:
     1. `# mkdir /data`
     1. `# lvcreate -L _SizeOfVolume_ -n DATA TESTPOOL`
     1. `# mkfs -t ext4 /dev/TESTPOOL/DATA`
     1. `# mount /dev/TESTPOOL/DATA /data`
     1. Update `/etc/fstab`
   1. _Task:_ Add an additional disk to the pool
    1. Using ZFS:
     1. `# zfs add TESTPOOL D`
    1. Using BtrFS:
     1. `# btrfs device add /dev/D /data`
     1. `# btrfs filesystem balance /data`
    1. Using EXT4 on LVM:
     1. `# pvcreate /dev/D`
     1. `# vgextend TESTPOOL /dev/D`
   1. _Task:_ Add additional space to a filesystem
    1. Using ZFS:
     1. No action needed
    1. Using BtrFS:
     1. No action needed
    1. Using EXT4 on LVM:
     1. `# lvextend -L _SizeToIncreaseTo_ /dev/TESTPOOL/DATA`
     1. `# resize2fs /dev/TESTPOOL/DATA`
   1. _Task:_ Remove a disk from the pool
    1. Using ZFS: _N/A_
    1. Using BtrFS:
     1. `# btrfs device delete /dev/A /data`
     1. `# btrfs filesystem balance /data`
    1. Using EXT4 on LVM:
     1. `# pvmove /dev/A`
     1. `# vgreduce TESTPOOL /dev/A`
   1. _Task:_ Replace operational disk
    1. Using ZFS:
     1. `# zfs replace TESTPOOL A D`
    1. Using BtrFS:
     1. `# btrfs device add /dev/D /data`
     1. `# btrfs device delete /dev/A /data`
     1. `# btrfs filesystem balance /data`
    1. Using EXT4 on LVM:
     1. `# pvcreate /dev/D`
     1. `# vgextend TESTPOOl /dev/D`
     1. `# pvmove TESTPOOL /dev/A /dev/D`
     1. `# vgreduce TESTPOOL /dev/A`
   1. _Task:_ Take a snapshot of a filesystem
    1. Using ZFS:
     1. `# zfs snapshot TESTPOOL@snapshot1`
    1. Using BtrFS:
     1. `# btrfs subvolume snapshot /data /data/snapshot1`
    1. Using EXT4 on LVM:
     1. `# lvcreate -s /dev/TESTPOOL/DATA -L _SizeToAllocate_ -n snapshot1`
   1. _Task:_ Rollback a snapshot
    1. Using ZFS:
     1. `# zfs rollback TESTPOOL@snapshot1`
    1. Using BtrFS:
     1. _Not sure..._
    1. Using EXT4 on LVM:
     1. `# lvconvert --merge /dev/TESTPOOL/snapshot1`
   1. _Task:_ Delete a snapshot
    1. Using ZFS:
     1. `# zfs destroy TESTPOOL@snapshot1`
    1. Using BtrFS:
     1. `# btrfs subvolume delete /data/snapshot1`
    1. Using EXT4 on LVM:
     1. `# lvremove /dev/TESTPOOL/snapshot1`