Writing something about Systemd has been on my TODO list for a while, but given how easily discussions about it turn into flame wars, I wanted to tread carefully. People have lobbed a lot of criticism at Systemd. Some of it is built on either false beliefs about how Systemd works, or a superficial understanding of how something else works, but much of it is valid. Unfortunately, both varieties are too often bundled with personal attacks, which tend to kill off any productive discussion before it even begins. I want to be clear on one thing before I begin: this post isn’t about Lennart. It also isn’t about PulseAudio, or any of his other work, or Redhat, or Canonical, or any particular social dynamic in the free software community. It’s about a piece of software, its design, and its implications for the rest of the ecosystem it inhabits. Hopefully it contains some insights that are generalizable as well.
I’m highly critical of Systemd. I do think there’s some good stuff. Examples:
- It’s fast. Not just faster than the highly complex mess of symlinks that many distros wired together before Systemd (and other attempts at replacing it, like Upstart), but also faster than many of the lightweight KISS solutions in distros like Archlinux. Arch has since switched to Systemd, and my systems boot even faster than they did before.
- It actually works pretty well, for the most part. I’ve been using it for a while, both on my own systems and when doing work-related things, and haven’t really had any problems. Unlike a lot of highly complex systems, Systemd’s complexity doesn’t seem to have caused it to be a horribly bug-ridden mess, which is unfortunately more than I can say for some other bits of software to which people have similar reactions.
- It’s pretty easy to use from a sysadmin perspective, and writing unit files is a pretty decent experience in general.
- It’s actually usable across different distros, and not just within the Redhat ecosystem. You can write one service file, and it’ll work on RHEL, Fedora, Debian, Archlinux, heck even oddball stuff like NixOS.
Just as all software contains bugs, all software is legacy. Software development is about solving specific problems that exist now and not about solving none-specific or future problems. Every few years, once a decade perhaps, there is a paradigm shift. Given enough time parts of the technological bedrock will have changed enough to render previous patterns obsolete.
The important thing is that we learn from that and we make sure the future legacy is easy to understand. Write clean interfaces, good tests, descriptive commit messages.
I’d add something to this: Stay focused. Don’t do more than you have to. Do one thing and do it well. Keep your interfaces small. Maybe clean implies small, but let’s be explicit about this. People sometimes call parts of this “UNIX Philosophy,” I just call it common sense or, good design. UNIX doesn’t have a monopoly on simplicity.
Oh, and one more big thing: make it easy to delete.
Systemd is not going to be easy to delete, for a few different reasons. Rich Felker, the author of Musl, has written some criticism of Systemd, and he makes one point about this that is highly relevant:
For 30+ years, the choice of init system used has been completely irrelevant to everybody but system integrators and administrators. User applications have had no reason to know or care whether you use sysvinit with runlevels, upstart, my minimal init with a hard-coded rc script or a more elaborate process-supervision system, or even /bin/sh. Ironically, this sort of modularity and interchangibility is what made systemd possible; if we were starting from the kind of monolithic, API-lock-in-oriented product systemd aims to be, swapping out the init system for something new and innovative would not even be an option.
I do want to spend a bit of time justifying the claim that Systemd is in fact going to be hard to delete. There are a few reasons.
Firstly, it’s monolithic, and non-modular. Lennart has argued that that isn’t true, but I don’t buy it.
The post claims you can disable lots of functionality, “Not totally unlike …
the Linux kernel.” I actually downloaded the tarball and played around with
this. Even passing every single
--disable- flag, it still built
about 60% of the repo. That’s nothing like the Linux kernel. Besides,
the first thing anybody learns about the design of the Linux kernel is
that it is monolithic. Lennart draws a distinction between monolithic
and non-modular, but I don’t see one.
In fairness, I did this a few months ago, while his post is from 2013. Maybe his claim was more accurate at the time, I don’t know. However, if that’s true it supports my next point:
Even if it is currently designed such that you can easily swap around components, nobody does this with Systemd, and nobody really suggests doing it. A pertinent quote that I unfortunately can’t find the original source for: “That which is untested is broken.” If folks aren’t exercising those code paths, they’ll stop working. If Systemd isn’t a monolith now, it will be.
Secondly, its interfaces are far from simple. Take
sd_notify as an
example. As Rich points out, this and dbus are basically
the only ways of correctly handling notifying the init system that a
service is up and running. They both require specific knowledge of
sd_notify requires making a call to a C library, which comes
with a whole mess of problems. For anything that’s not actually written
in C, you now have to deal with the FFI. For Go, this means giving upon
some of the really nice deployability that it otherwise has. For shell
scripts, this means having to write wrapper programs in C (So Lennart’s
myth #4 is no myth). For anything, if you want to cross compile you now
need a C cross compiler, regardless of what language you’re actually
using for the rest of your program. This is for one function call, that
really shouldn’t need an FFI. Rich suggests an alternative (just write
to a file descriptor and close it) that solves all this.
sd_notify family of functions also does a lot more than this
simple “I’m ready” notification. More stuff to get stuck to.
In fairness, there really just isn’t a way to achieve this without the daemons knowing at least a little bit about the init system, but the interface could be dead simple, and it isn’t anywhere near that.
Lastly, Systemd encompasses way more than just the system start-up and process monitoring that people think of when they think of an init system. Stuff like Journald has nothing to do with what Systemd is supposed to be doing. I haven’t looked at how separable that is from the rest of the system, but I argued earlier that it isn’t likely to stay that way.
Systemd bugs me because I worry about what it (and software like it) will do to the future of our ecosystem. It’s a popular target, but by no means the only problematic program. I wish we in the free software community were more concious of this kind of thing. I also wish those of us who are could argue with more civility.