BTRFS Training Exercises

In 2015 I ran a training session on BTRFS and ZFS. Here are the notes for BTRFS updated for the current versions of everything when running Debian/Trixie on KVM. You can do all the same exercises on a non virtual machine, but it’s safest not to do this on a machine with important data. For the KVM setup I use /dev/vdc and /dev/vdd are the devices for test data.

On a Debian Buster or newer system all that you need to do to get ready for this is to install the btrfs-progs package.

Making the filesystem

1. Make the filesystem, this makes a filesystem that spans 2 devices (note you must use the-f option if there was already a filesystem on those devices):

   mkfs.btrfs /dev/vdc /dev/vdd

2. Use file(1) to see basic data from the superblocks:

   file -s /dev/vdc /dev/vdd

3. Mount the filesystem (can mount either block device, the kernel knows they belong together):

   mount /dev/vdd /mnt/tmp

4. See a BTRFS df of the filesystem, shows what type of RAID is used (note the use of “single” IE RAID-0 for data):

   btrfs filesystem df /mnt/tmp

5. See more information about FS device use:

   btrfs filesystem show /mnt/tmp

6. Balance the filesystem to change it to RAID-1 and verify the change, note that some parts of the filesystem were single and RAID-0 before this change):

   btrfs balance start -dconvert=raid1 -mconvert=raid1 -sconvert=raid1 --force /mnt/tmp
   btrfs filesystem df /mnt/tmp

7. See if there are any errors, shouldn’t be any (yet):

   btrfs device stats /mnt/tmp

8. Copy some files to the filesystem:

   cp -r /usr /mnt/tmp

9. Check the filesystem for basic consistency (only checks checksums):

   btrfs scrub start -B -d /mnt/tmp

Online corruption

1. Corrupt the filesystem:

   dd if=/dev/zero of=/dev/vdd bs=1024k count=2000 seek=50

2. Scrub again, should give a warning about errors:

   btrfs scrub start -B /mnt/tmp

3. Check error count:

   btrfs device stats /mnt/tmp
   dmesg

4. Corrupt it again:

   dd if=/dev/zero of=/dev/vdd bs=1024k count=2000 seek=50

5. Unmount it:

   umount /mnt/tmp

6. In another terminal follow the kernel log:

   dmesg -w

7. Mount it again and observe it reporting (and maybe correcting) errors on mount:

   mount /dev/vdd /mnt/tmp

8. Run a diff, observe kernel error messages and observe that diff reports no file differences (this will be really slow):

   diff -ru /usr /mnt/tmp/usr/

9. Run another diff, you probably won’t observe kernel error messages as all errors in data read by diff were probably fixed in the previous run. Observe that diff again reports no file differences but is faster as errors have been corrected:

   diff -ru /usr /mnt/tmp/usr/

10. Run another scrub, this may correct some errors which weren’t discovered by diff:

    btrfs scrub start -B -d /mnt/tmp

Offline corruption

1. Umount the filesystem, corrupt the start, then try mounting it again which will fail because the superblocks were wiped:

   umount /mnt/tmp
   dd if=/dev/zero of=/dev/vdd bs=1024k count=200
   mount /dev/vdd /mnt/tmp
   mount /dev/vdc /mnt/tmp

2. Note that the filesystem was not mountable due to a lack of a superblock. It might be possible to recover from this but that’s more advanced so we will restore the RAID.
   Mount the filesystem in a degraded RAID mode, this allows full operation.

   mount /dev/vdc /mnt/tmp -o degraded

3. Add /dev/vdd back to the RAID:

   btrfs device add /dev/vdd /mnt/tmp

4. Show the filesystem devices, observe that vdd is listed twice, the missing device and the one that was just added:

   btrfs filesystem show /mnt/tmp

5. Remove the missing device and observe the change:

   btrfs device delete missing /mnt/tmp
   btrfs filesystem show /mnt/tmp

6. Balance the filesystem, not sure this is necessary but it’s good practice to do it when in doubt:

   btrfs balance start /mnt/tmp

7. Umount and mount it, note that the degraded option is not needed:

   umount /mnt/tmp
   mount /dev/vdc /mnt/tmp

Compression

1. Get the properties of the root of the btrfs filesystem, set it to zstd compression (which is inherited when creating subdirectories) and inspect the change:

   btrfs property get /mnt/tmp
   btrfs property set /mnt/tmp compression zstd
   btrfs property get /mnt/tmp

2. Note that the kernel message log has the following when zstd is enabled, this is OK it just means that the filesystem can’t be mounted on ancient kernels with zstd enabled.

   setting incompat feature flag for COMPRESS_ZSTD (0x10)

3. View the space used, convert the filesystem to zstd compression, and view the new space used (note that the space used is about half what it was):

   df -h /mnt/tmp
   btrfs filesystem defrag -czstd -r /mnt/tmp
   df -h /mnt/tmp

RAID-5

1. Run “df -h /mnt/tmp” to see the space available.
2. Add more devices to the filesystem and check the space available:

   btrfs device add /dev/vdb /mnt/tmp
   btrfs device add /dev/vde /mnt/tmp
   df -h /mnt/tmp

3. Convert the filesystem to RAID-5 for data and RAID-1 3 copies for metadata. Note that RAID-5 for metadata caused corruption and data loss last time I tested it.

   btrfs balance start -dconvert=raid5 -mconvert=raid1c3 -sconvert=raid1c3 --force /mnt/tmp

4. Check the space available by “df -h /mnt/tmp”.

Experiment

1. Experiment with the “btrfs subvolume create” and “btrfs subvolume delete” commands (which act like mkdir and rmdir).
2. Experiment with “btrfs subvolume snapshot SOURCE DEST” and “btrfs subvolume snapshot -r SOURCE DEST” for creating regular and read-only snapshots of other subvolumes (including the root).

Related posts:

1. ZFS Training Exercises
2. Postfix Training
3. SE Linux Policy Writing Training