Programming · free software · the open webSubscribe via RSS

Rolling back NixOS does not roll back a database

NixOS generations let you select an earlier system configuration, while database files and other mutable data follow their own lifecycle and compatibility rules.

A generation selects a system configuration

A NixOS generation is a configuration that can be selected at boot or activated on a running system. The manual describes booting an earlier configuration from the GRUB menu when it has not been garbage-collected, and switching a running system to the previous configuration with nixos-rebuild switch --rollback. These mechanisms select system configurations and activate their services. [1]

That selection is powerful because system software is represented by immutable store paths. A previous system closure can still be available for activation without rebuilding every package. Yet activation does not imply that every file a service has touched is part of that closure. Configuration and executables can be generation-specific while the service’s working data remains in persistent locations.

Mutable state has a separate timeline

Consider a database service whose package is upgraded in one generation. On startup, it may migrate its data directory to a newer on-disk format. If the system is later switched back to an older generation, the older server executable may start against the already-migrated files and fail, reject the format, or behave incompatibly. The generation changed; the database directory did not rewind with it.

The same distinction applies to application uploads, user home directories, queues and files under /var. Their persistence is often intentional: a system upgrade should not erase user data. But persistence means a system rollback is not a general transaction rollback. The manual’s account of switching configuration versions provides a clear mechanism for configuration recovery; it does not describe restoring service data to an earlier snapshot. [1]

Compatibility is a relationship between versions

Before a migration, ask which versions of the service can read the resulting data. Some migrations are backward compatible, some are reversible, and some are one-way transformations. The answer depends on the application and its format, so it belongs in that service’s migration documentation and operational plan rather than in the system-generation mechanism.

A safe upgrade plan connects three things: the new executable, the migration it may perform, and a recovery path for the data. A backup is useful only if it is restorable and corresponds to a compatible software version. For a database, that can mean preserving a pre-migration snapshot and knowing which server version can consume it. For a queue, it can mean checking whether old code understands messages written by new code. The relevant unit of rollback is the pair of code and state, not code alone.

Trace the rollback boundary before relying on it

A useful thought experiment is to write down two columns before an upgrade. In the first, list items selected by the generation: system packages, service definitions and activation scripts. In the second, list state a running service can mutate: database files, generated caches, user data and external resources. After switching generations, the first column can point to an earlier version while the second remains at its latest state.

This trace shows where rollback helps and where it stops. If a new unit file prevents a service from starting, returning to an earlier configuration may restore the old activation behavior. If the new service has changed a database format, the earlier configuration cannot reconstruct old bytes. Plan backups, compatibility checks and migration sequencing around that boundary; use system rollback as configuration recovery rather than assuming it reverses application history. [1]

References

  1. NixOS manual: rolling back

Suggest a correction · All posts