My NixOS Journey Part 4

Part One | Part Two | Part Three | Part Four

Welcome to my journey.  I'm documenting how I make my way down the rabbit hole, failures, successes, lessons learned all in hopes that I don't do it again, and maybe you'll be able to take away something from this too.

Where to discuss this article or contact me

Matrix:
 #my-nixos-journey:beardedtek.com
 #nixnerds:jupiterbroadcasting.com
 @beardedtek:matrix.org
Telegram:
 @beardedtek
Twitter:
 @beardedtek
Email:
 contact@beardedtek.com

This Article's GitHub Repo

GitHub - BeardedTek/nixos-24.05-secureboot-installer

Contribute to BeardedTek/nixos-24.05-secureboot-installer development by creating an account on GitHub.

GitHubBeardedTek

Disk Image Available

A disk image is available in the Repo's Releases:
https://github.com/BeardedTek/nixos-24.05-secureboot-installer/releases/tag/2024-01-15-001

You Can't Secure Boot an Installer

Can't is a four letter word in my book.  If you say I can't, I'll figure out how I can. That's just how I'm built!  Lanzaboote is a nix-community project that enables Secure Boot, but it is fairly new.  It is NOT guaranteed to work out of the box, but on most modern hardware, it should work without a hitch.

Struggles

For a couple days I struggled trying to build a custom installer iso using the existing methods explained on the NixOS Wiki.  This, unfortunately was something that would not work due to the installer not being able to use systemd-boot.  A limitation of booting off CD/DVD.

QEMU to the Rescue!

I figured I could use a SATA SSD attached to a USB device should allow me to build a system with secure boot from where I could perform an installation.  If nothing else, it would allow me to play around and see if I could figure it out.  If I failed, I would have just learned a bit more about the installation process with secure boot.  A win even if I couldn't get it working.

First I added the USB to SATA adapter as a USB Disk.

Then I ensured it was booting via UEFI.

I then mounted the NixOS minimal 24.05 Unstable Install ISO as a SATA CDROM.

This started me down the right path, but took about a day and a half of failures before I started getting the results I wanted.

My Aha! Moment

With the help of some amazing people in Jupiter Broadcasting's Nix Nerds Matrix chat room I was able to find the all-hardware.nix and installer-device.nix Hardware Profiles in NixOS' Github organization.

configuration.nix:

{ config, pkgs, lib, ... }:
let
  sources = import ./nix/sources.nix;
  lanzaboote = import sources.lanzaboote;
in
{

  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
      ./users.nix
      ./packages.nix
      ./network.nix
      lanzaboote.nixosModules.lanzaboote
    ];

  boot.loader.systemd-boot.enable = lib.mkForce false;
  boot.lanzaboote = {
    enable = true;
    pkiBundle = "/etc/secureboot";
  };

  # Set your time zone.
  time.timeZone = "America/Anchorage";

  # Set nix Version
  system.stateVersion = "unstable";

  # Allow Unfree Software
  nixpkgs.config.allowUnfree = true;

}

hardware-configuration.nix

The magic lines in here are the imports:
/profiles/all-hardware.nix will allow all kernel modules for known hardware to be available.
`/profiles/installation-device.nix` will set the drive up as installation media including passwordless nixos user with passwordless sudo and all installation software necessary.

{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [
      (modulesPath + "/profiles/all-hardware.nix")
      (modulesPath + "/profiles/installation-device.nix")
    ];
  boot.initrd.availableKernelModules = [
    "ahci"
    "xhci_pci"
    "usb_storage"
    "sd_mod"
    "sata_nv"
    "ext4"
    "btrfs"
    "nvme"
    "ata_piix"
    "sata_uli"
    "sata_via"
    "sata_sis"
    "sd_mod"
    "sr_mod"
    "uhci_hcd"
    "ehci_hcd"
    "nouveau"
  ];

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/93167f50-68c6-4d6d-8d40-21d2860c7bae";
      fsType = "btrfs";
      options = [ "subvol=root" ];
    };

  fileSystems."/home" =
    { device = "/dev/disk/by-uuid/93167f50-68c6-4d6d-8d40-21d2860c7bae";
      fsType = "btrfs";
      options = [ "subvol=home" ];
    };

  fileSystems."/nix" =
    { device = "/dev/disk/by-uuid/93167f50-68c6-4d6d-8d40-21d2860c7bae";
      fsType = "btrfs";
      options = [ "subvol=nix" ];
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/4A63-D8D7";
      fsType = "vfat";
    };

  swapDevices = [ ];

  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
}

packages.nix

sbctl is used to generate secureboot keys and niv is used to setup sources for lanzaboote.

{ pkgs, ... }:
{
  # Allow non-free (Unfree) packages
  nixpkgs.config.allowUnfree = true;

  # List packages installed in system profile. To search, run:
  # $ nix search wget
   environment.systemPackages = with pkgs; [
     nano # Why would I ever want vim to touch my system :)
     sbctl
     niv
   ];
}

network.nix is not necessary in this context, but provided so you can customize nameservers, and your IP address.  You could set your IP manually and do your install via ssh.

{
  # Setup Networking
  networking = {
    hostName= "nixos";
    nameservers = [
      "8.8.8.8"
      "9.9.9.9"
      ];
    enableIPv6 = false;
    firewall = {
      enable = false;
    };
  };
}

A custom installation script

I then repurposed and improved my install script from a previous post to do this installation:

NOTE: There are not many checks built into this script.  It should "Just Work."™
It may fail, but should be a good start.  I will be testing this more thoroughly in the future and will update this blog post as I improve it.

UUID vs Device

I found out the hard way that this would not boot if you just used /dev/sdan the new installation media will not boot properly.  You need to use partition UUID's which are created after you partition the disk and create a file system.

#!/usr/bin/env bash
# Installs NixOS Installation media for Secure Boot

# It is recommended to run this script after issuing `sudo -i` or as root.
# If it is run as a regular user it will fail in really horrible ways and you'll have to start over.

# Be sure to set the following environment variable prior to install
# or you can set them here prior to installing
#
DISK=/dev/sda

if [[ $DISK = *nvme* ]]; then
  PART_MRKR="p"
fi

echo "Double check these details:"
echo "HOSTNAME       : ${HOSTNAME}"
echo "DISK           : ${DISK}"
echo "BOOT PARTITION : ${DISK}${PART_MRKR}1"
echo "ROOT PARTITION : ${DISK}${PART_MRKR}2"

# Partition the drive with 550M boot partition and the rest ext4
printf "label: gpt\n,550M,U\n,,L\n" | sfdisk ${DISK}

# Format the partitions
# /boot:
mkfs.fat -F 32 ${DISK}${PART_MRKR}1

# /:
mkfs.ext4 ${DISK}${PART_MRKR}2


BOOT_UUID=$(ls -l /dev/disk/by-uuid | grep $(echo $DISK | sed 's/\/dev\///')1 | awk '{print $9}')
ROOT_UUID=$(ls -l /dev/disk/by-uuid | grep $(echo $DISK | sed 's/\/dev\///')2 | awk '{print $9}')

echo "${BOOT_UUID}" > .boot_uuid
echo "${ROOT_UUID}" > .root_uuid
echo "BOOT_UUID : $BOOT_UUID"
echo "ROOT_UUID : $ROOT_UUID"

# Mount partitions
# Mount /
mount "/dev/disk/by-uuid/${ROOT_UUID}" /mnt
# Create mount points
mkdir -p /mnt/boot
mount "/dev/disk/by-uuid/${BOOT_UUID}" /mnt/boot

# Generate hardware-configuration.nix
nixos-generate-config --root /mnt

# Copy configuration to /etc/nixos
cp *.nix /mnt/etc/nixos/

# Create niv project and add lanzaboote source to /mnt/etc/nixos
COMEBACK=$(pwd)
cd /mnt/etc/nixos
niv init
niv add nix-community/lanzaboote -r v0.3.0 -v 0.3.0
cd $COMEBACK

# Generate Secure Boot Keys
echo -n "Generate Secure Boot Keys? (y/N)?: "
read SEC_BOOT
SEC_BOOT=$(echo "${SEC_BOOT}" | tr '[:upper:]' '[:lower:]')

if [[ "${SEC_BOOT}" = "y" ]] || [[ "${SEC_BOOT}" = "yes" ]]; then
  sbctl create-keys -d /mnt/etc/secureboot -e /mnt/etc/secureboot/keys
else
  echo "Not Generating Secure Boot Keys!!!!!!!"
  echo "Please ensure you have secure boot keys located at /mnt/etc/secureboot before continuing or this WILL FAIL!!!"
  CONTINUE="x"
  while [ ! "${CONTINUE}" = "y"] || [{ ! "${CONTINUE}" = "yes" ]; do
    echo -n "Continue? (Y/n)"
    read CONTINUE
    CONTINUE=$(echo "${CONTINUE}" | tr '[:upper:]' '[:lower:]')
  done
fi

# Install the system
nixos-install --root /mnt

Run The Install

Before we run the install, we need to make sure some things are done.

First, let's become root in a "proper environment".

nixos@nixos:~> sudo -i

Next we need to make sure we have git, sbctl, and niv installed.

nixos@nixos:~> nix-shell -p git niv sbctl

Now that we are in the right environment as root with all our utilities we need, let's set our DISK variable. This is not a partition, but a full disk.

nixos@nixos:~> DISK=/dev/sda

Now we can clone the repository with git:

nixos@nixos:~> git clone https://github.com/beardedtek/nixos-24.05-secureboot-installer

Cloning into 'nixos-24.05-secureboot-installer'...
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 17 (delta 6), reused 15 (delta 4), pack-reused 0
Receiving objects: 100% (17/17), 4.13 KiB | 4.13 MiB/s, done.
Resolving deltas: 100% (6/6), done.

[nix-shell:~]# cd nixos-24.05-secureboot-installer

[nix-shell:~/nixos-24.05-secureboot-installer]# ./install.sh

Double check these details:
HOSTNAME       : nixos
DISK           : /dev/sda
BOOT PARTITION : /dev/sda1
ROOT PARTITION : /dev/sda2
Checking that no-one is using this disk right now ... OK

Disk /dev/sda: 8 GiB, 8589934592 bytes, 16777216 sectors
Disk model: QEMU HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

>>> Script header accepted.
>>> Created a new GPT disklabel (GUID: 1B6DFD71-1D3A-4B2B-BA8A-22469990D704).
/dev/sda1: Created a new partition 1 of type 'EFI System' and of size 550 MiB.
/dev/sda2: Created a new partition 2 of type 'Linux filesystem' and of size 7.5 GiB.
/dev/sda3: Done.

New situation:
Disklabel type: gpt
Disk identifier: 1B6DFD71-1D3A-4B2B-BA8A-22469990D704

Device       Start      End  Sectors  Size Type
/dev/sda1     2048  1128447  1126400  550M EFI System
/dev/sda2  1128448 16775167 15646720  7.5G Linux filesystem

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
mkfs.fat 4.2 (2021-01-31)
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done                            
Creating filesystem with 1955840 4k blocks and 489600 inodes
Filesystem UUID: ac8762c1-67e0-4e16-8abe-372db31eb11c
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

BOOT_UUID : 12CE-A600
ROOT_UUID : ac8762c1-67e0-4e16-8abe-372db31eb11c
writing /mnt/etc/nixos/hardware-configuration.nix...
writing /mnt/etc/nixos/configuration.nix...
For more hardware-specific settings, see https://github.com/NixOS/nixos-hardware.
Initializing
  Creating nix/sources.nix
  Creating nix/sources.json
  Using known 'nixpkgs' ...
  Adding package nixpkgs
    Writing new sources file
  Done: Adding package nixpkgs
Done: Initializing
Adding package lanzaboote
  Writing new sources file
Done: Adding package lanzaboote
Generate Secure Boot Keys? (y/N)?: y
Created Owner UUID 56240e4f-5509-4559-be21-ee7585458107
Creating secure boot keys...✓ 
Secure boot keys created!
copying channel...
building the configuration in /mnt/etc/nixos/configuration.nix...
...

...
installing the boot loader...
setting up /etc...
Installing Lanzaboote to "/boot"...
Updating "/boot/EFI/BOOT/BOOTX64.EFI"...
Error reading file /boot/EFI/BOOT/BOOTX64.EFI: No such file or directory
Can't open image /boot/EFI/BOOT/BOOTX64.EFI
$"/boot/EFI/BOOT/BOOTX64.EFI" is not signed. Replacing it with a signed binary...
Updating "/boot/EFI/systemd/systemd-bootx64.efi"...
Error reading file /boot/EFI/systemd/systemd-bootx64.efi: No such file or directory
Can't open image /boot/EFI/systemd/systemd-bootx64.efi
$"/boot/EFI/systemd/systemd-bootx64.efi" is not signed. Replacing it with a signed binary...
Collecting garbage...
Successfully installed Lanzaboote.
setting up /etc...
setting up /etc...
setting root password...
New password:
Retype new password: 
passwd: password updated successfully
installation finished!

Now What?

If the installed finished without error, you now have secureboot enabled installation media!!!

Shut down the virtual machine, remove your USB drive, and plug it into your computer of choice you want to install on!

Enrolling Keys

Depending on your system, you may be able to just boot it up without enrolling any keys, or you might have to go into your UEFI Firmware settings to enroll them manually.

Each hardware manufacturer does this slightly differently, but with my MSI B550M PRO-VDH WIFI motherboard I was able to enroll the EFI image.  Check your motherboard / computer's manual for instructions on how to do this.

Where Can I Find More?

I'll be posting all of this to GitHub in the coming days.  If you have any questions, comments, bug fixes, etc, please get a hold of me there!

Creating an Image File

To use an image file in virt-manager instead of a physical drive, you must first create a blank image file.
nixos@nixos:~> fallocate -l 8GiB nixos-secureboot-installer.img
You can then use this image in place of the physical device.