Skip to content

Internals#

Engineering notes for contributors. These pages explain why the on-disk format work looks the way it does — pointers into the relevant kernel / OpenZFS source, lists of the bugs the lib had to fix to read real-world output, and the algorithms ported into Go.

  • On-disk format fixes

    The full list of 14 format mismatches the go-filesystems/* libraries had to fix to read real mkfs.btrfs / zpool create output. Each entry cites the kernel / OpenZFS reference.

  • btrfs multi-device routing

    How chunk.go and multidev.go route per-IO reads across the six btrfs RAID profiles, modelled on fs/btrfs/volumes.c:btrfs_map_block.

  • ZFS multi-vdev + RAID-Z

    OpenFromDevices, multiVdevPool, raidzMapAlloc — the healthy-path RAID-Z geometry ported from module/zfs/vdev_raidz.c.

  • vdev label NVList

    XDR layout, encoding/decoding, and how ProbeLabel walks vdev_tree.children to surface the leaf-id ordering cloud-boot-init needs.

House rules for this section#

  • Cite kernel / OpenZFS source by file + function. Bonus points for git tag / version.
  • Quote on-disk byte layouts directly when the bug was an offset or a size mismatch — future readers need to see the numbers.
  • Link the test fixture that surfaced each bug. Every fixture in testdata/raid/*.tar.zst was generated by real mkfs.btrfs / zpool create in a Lima Debian 12 VM; the generation recipe is in memory:userland-fs-drivers (operator only — internal note).
  • Keep cross-references hot. If a chunk-mapping change touched superblock.go AND chunk.go AND every chunk-using read site, spell out all three.

Why pure-Go FS drivers?#

Three reasons that compound:

  1. Kernel slimming. With the userland-FS pivot the disk-<arch>.config kernel drops CONFIG_EXT4_FS, CONFIG_XFS_FS, CONFIG_BTRFS_FS entirely. The bootstrap Image shrinks by ~5 MiB.
  2. ZFS without DKMS. Pre-pivot, ZFS support required either the out-of-tree OpenZFS DKMS module (multi-MB binary blob, per-kernel-version compile) or the zfsutils-linux userspace stack. Post-pivot, neither: the pure-Go driver reads OpenZFS pools end-to-end. Worth roughly 30 MB of build artifact and several minutes of CI per architecture.
  3. Same code on every path. The loader (Path B), the init binary (Paths A + C), and any host-side tool that wants to inspect a chained-distro disk all share the SAME go-filesystems/* libraries. No drift between userspace-mounted reads and embedded-init reads.