Skip to content

Internals

title: Internals

Internals

Engineering notes for contributors. These pages explain why the on-disk format work looks the way it does — pointers into the relevant kernel / OpenZFS source, lists of the bugs the lib had to fix to read real-world output, and the algorithms ported into Go.

  • :material-bug:{ .lg .middle } On-disk format fixes

    The full list of 14 format mismatches the go-filesystems/* libraries had to fix to read real mkfs.btrfs / zpool create output. Each entry cites the kernel / OpenZFS reference.

  • :material-shape:{ .lg .middle } btrfs multi-device routing

    How chunk.go and multidev.go route per-IO reads across the six btrfs RAID profiles, modelled on fs/btrfs/volumes.c:btrfs_map_block.

  • :material-database-cog:{ .lg .middle } ZFS multi-vdev + RAID-Z

    OpenFromDevices, multiVdevPool, raidzMapAlloc — the healthy-path RAID-Z geometry ported from module/zfs/vdev_raidz.c.

  • :material-file-tree-outline:{ .lg .middle } vdev label NVList

    XDR layout, encoding/decoding, and how ProbeLabel walks vdev_tree.children to surface the leaf-id ordering cloud-boot-init needs.

House rules for this section

  • Cite kernel / OpenZFS source by file + function. Bonus points for git tag / version.
  • Quote on-disk byte layouts directly when the bug was an offset or a size mismatch — future readers need to see the numbers.
  • Link the test fixture that surfaced each bug. Every fixture in testdata/raid/*.tar.zst was generated by real mkfs.btrfs / zpool create in a Lima Debian 12 VM. The generation recipe is an operator-side note, kept unpublished on purpose — so it is named here rather than linked. The link that used to sit in this sentence pointed into a working directory that exists on one machine and returned 404 to every reader of this page.
  • Keep cross-references hot. If a chunk-mapping change touched superblock.go AND chunk.go AND every chunk-using read site, spell out all three.

Why pure-Go FS drivers?

Three reasons that compound:

  1. Kernel slimming. With the userland-FS pivot the disk-<arch>.config kernel drops CONFIG_EXT4_FS, CONFIG_XFS_FS, CONFIG_BTRFS_FS entirely. The bootstrap Image shrinks by ~5 MiB.
  2. ZFS without DKMS. Pre-pivot, ZFS support required either the out-of-tree OpenZFS DKMS module (multi-MB binary blob, per-kernel-version compile) or the zfsutils-linux userspace stack. Post-pivot, neither: the pure-Go driver reads OpenZFS pools end-to-end. Worth roughly 30 MB of build artifact and several minutes of CI per architecture.
  3. Same code on every path. The loader (Path B), the init binary (Paths A + C), and any host-side tool that wants to inspect a chained-distro disk all share the SAME go-filesystems/* libraries. No drift between userspace-mounted reads and embedded-init reads.