Fixing Debian Kernel Updates with BTRFS Subvolumes
I’ve run into this issue with Debian 12 VMs using BTRFS subvolumes: you update the kernel, reboot, and get a kernel panic because the initramfs wasn’t correctly configured to include the BTRFS module, or because the standard post-install hook failed (e.g., due to a full /boot partition). This post documents the problem, the fix, and how to prevent it in the future.
The Problem
After a kernel update (e.g., from 6.1.0-42 to 6.1.0-44), the VM won’t boot:
kernel panic not syncing VFS unable to mount root fs on unknown block
The culprit: Debian’s default initramfs configuration sometimes fails to automatically include the btrfs module, which is required at boot time. While /etc/kernel/postinst.d/initramfs-tools is supposed to handle initramfs regeneration, it can fail silently if, for example, the /boot partition runs out of space.
Diagnosing the Issue
Check Boot Parameters
If you can still boot with an older kernel, verify the mismatch:
# Current boot parameters
cat /proc/cmdline
# What GRUB expects for the new kernel
sudo cat /boot/grub/grub.cfg | grep -A5 "6.1.0-44"
Red flags:
- Different parameters between old and new kernels
- Initramfs exists but is outdated (check modification times)
- BTRFS subvolume paths (
@rootfs) in the kernel parameters
Verify BTRFS Configuration
# List all subvolumes
sudo btrfs subvolume list /
# Check filesystem details
sudo btrfs filesystem show
# Confirm root mount
df -h /
mount | grep btrfs
Recovery Steps
1. Boot with the Old Kernel
- Start the VM
- Select Advanced options in the GRUB menu
- Choose the working older kernel (e.g., 6.1.0-42)
- Press Enter to boot
2. Regenerate Initramfs for the New Kernel
# Update module dependencies
sudo depmod -a 6.1.0-44-amd64
# Regenerate the initramfs (critical step)
sudo update-initramfs -u -k 6.1.0-44-amd64
# Update GRUB configuration
sudo update-grub
3. Test with the New Kernel
In GRUB, select the new kernel (6.1.0-44-amd64), should be default, and test.
# Reboot
sudo reboot
Preventing Future Boot Failures
To ensure BTRFS is always available at boot, you need to explicitly include the module in the initramfs configuration.
1. Add BTRFS to Initramfs Modules
Edit /etc/initramfs-tools/modules and add btrfs to the end of the file:
echo "btrfs" | sudo tee -a /etc/initramfs-tools/modules
2. Update Initramfs
Once added, regenerate the initramfs for all installed kernels to ensure the module is included:
sudo update-initramfs -u -k all
This configuration ensures that the initramfs creation process always includes the BTRFS driver, preventing boot failures after future kernel updates.
Troubleshooting
Initramfs Update Fails
# Run with verbose output
sudo update-initramfs -u -k 6.1.0-44-amd64 -v
# Check journal logs
sudo journalctl -xe | grep -i initramfs
GRUB Doesn’t Show New Kernel
# Update GRUB with verbose output
sudo update-grub --verbose
# Verify GRUB configuration
sudo cat /boot/grub/grub.cfg | grep vmlinuz
BTRFS Modules Missing from Initramfs
# Check if BTRFS module is loaded
sudo lsmod | grep btrfs
# Check if BTRFS is in kernel config
cat /boot/config-6.1.0-44-amd64 | grep CONFIG_BTRFS
# Verify BTRFS modules are in initramfs
lsinitramfs /boot/initrd.img-6.1.0-44-amd64 | grep btrfs
VM Checklist for Initial Setup
When setting up new Debian VMs with BTRFS:
- Add
btrfsto/etc/initramfs-tools/modulesto ensure it is always included at boot. - Ensure
/boothas sufficient space for kernel updates. - Keep an older working kernel as a recovery option.
Reference
Key files to know:
- Initramfs:
/boot/initrd.img-* - Kernel image:
/boot/vmlinuz-* - Kernel config:
/boot/config-* - GRUB configuration:
/boot/grub/grub.cfg - BTRFS subvolumes:
sudo btrfs subvolume list /
Essential commands:
update-initramfs -u -k all # Regenerate all initramfs
update-grub # Update GRUB configuration
btrfs subvolume list / # List BTRFS subvolumes
lsinitramfs /boot/initrd.img-* # Inspect initramfs contents