I recently bought new PC parts (full AMD yeah) and decided to finally switch to NixOS and full Wayland. After building it I realized that switching will be really painful because of how customized my install of Opensuse Tumbleweed has gotten over the years. So I decided to start with my laptop and apply the stuff I learn along the way to my main rig later.
I first thought about partitioning the single 1TB disk like this:
LUKS
└─ ZFS
├─ ... zfs datasets
└─ SWAP
The issue with this is that ZFS needs RAM for cache. When RAM gets full and data has to be swapped out this results in a deadlock which crashes the whole system.
To overcome this I decided to just create a GPT table and partition the LUKS content into SWAP and ZFS. Thanks to disko this was really easy to declaratively configure:
disko.devices = {
disk."builtin" = {
type = "disk";
device = "/dev/disk/by-id/...";
content = {
type = "table";
format = "gpt";
partitions = [
{
name = "ESP";
start = "1MiB";
end = "1GiB";
bootable = true;
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot/efi";
};
}
{
name = "luks";
start = "1GiB";
end = "100%";
content = {
type = "luks";
name = "crypted";
extraOpenArgs = ["--allow-discards"];
keyFile = "/tmp/luks.txt";
content = {
type = "table";
format = "gpt";
partitions = [
{
name = "zfs";
start = "0%";
end = "-20GiB";
content = {
type = "zfs";
pool = "rpool";
};
}
{
name = "swap";
start = "-20GiB";
end = "100%";
content = {
type = "swap";
randomEncryption = false;
};
}
];
};
};
}
];
};
};
}
After deploying this using nixos-anywhere the system didn't boot and complained that ZFS was not able to find the pool. This took a while to debug, especially since I wasn't able to mount the LUKS content while booted into an installer image either.
I partly followed this great blog post from Aaron Lauterer and after having finally read the end after hours of debugging I realized that partprobe is needed to get /dev/mapper/crypted1 and /dev/mapper/crypted2 to show up. Makes sense.
I tried googling how I can add partprobe to initrd, but I just found Arch-based answers using mkinitcpio. On NixOS it's quite a lot easier though:
boot.initrd = {
extraUtilsCommands = "copy_bin_and_libs ${pkgs.parted}/bin/partprobe";
luks.devices.crypted.postOpenCommands = ''
partprobe /dev/mapper/crypted
'';
};
That's all that was needed! This runs the partprobe command on /dev/mapper/crypted after I unlock LUKS and ZFS is finally able to find the pool and NixOS booted with working Swap!
One neat thing I also found was adding a little message to the preDeviceCommands section which gets printed before the decryption key prompt from LUKS. I made it a little more obvious using # so that non-tech-savvy people hopefully read at least that between the other startup logs ;)
preDeviceCommands = ''
echo "\
#############################
#
# Hi, this computer is called "${config.networking.hostName}".
# Owner: <your name>
# Email: <your email>
# Website: <your website or other ways to communicate with you>
#
#############################
"
'';
This way people can return the device even when found shut down/in the encrypted state.