Recently I came to a hard realization: I am never going to get Sandstorm off the ground in a real way.
I’ve been part of the Sandstorm community since it was run by a for-profit startup, initially as a user and app developer. For the past three years, I’ve been the most active contributor to Sandstorm itself, and in that time I’ve fixed some bugs, added a few features, but ultimately most of the energy I’ve put into Sandstorm core has been:
- Putting out fires, mostly in the form of dealing with breakage from upstream dependencies
- Attempting to add more ambitious features, and bouncing off of structural problems in Sandstorm’s internals that make said feature difficult to implement
It feels like I’m just treading water. Nobody’s paying me, and hacking on Sandstorm is not fun. For a project that is now heavily dependent on volunteer labor, not being fun is fatal.
But the reasons Sandstorm exists are as relevant as ever, and I’m not willing to wait until Sandstorm is no longer runnable and then go back to not having anything like it.
So I’m starting over. I’ve started writing Tempest (working name), a platform that can run existing Sandstorm apps, but with a slightly different focus, and a tech stack that I (and some others I’ve talked to) can get excited about, and can actually maintain.
There are a few ways in which the focus is different.
Since Sandstorm-the-business shut down, the community has wanted to focus more on the self-hosted use case, especially smaller deployments – so Tempest is designed with that in mind from the get-go. Some concrete examples of this:
- I’m using SQLite for the database, instead of Mongo. Sandstorm is currently stuck on an ancient version of Mongo because we’d have to fully automate updating to a new version of a DBMS that was written with the assumption that the user has sysadmins. And if our automation has bugs, we can’t assume anyone will be around who has a hope of troubleshooting it. With SQLite, this problem just goes away.
- There’s no shell/backend split. Sandstorm was built to accommodate an alternative distributed/scaling backend (Blackrock), and multiple application servers. Ironically, Sandstorm’s Oasis hosting service never actually reached the scale where one giant server wouldn’t have been enough. With Tempest, everything is in one process.
Sandstorm was also never good at dealing with apps that want to talk to the outside world. With all the recent excitement about the Fediverse, I really, really want Sandstorm to be a good solution for hosting ActivityPub-enabled apps. But the platform features we’d need to make that happen are too difficult to build with the current tech stack. The details of why this is hard in Sandstorm are a topic for another post, but the ability to actually deliver these features was a major consideration in selecting technologies to build on.
I’ll have more to say about the whys of Tempest’s tech stack in future blog posts; I spent a fair amount of time evaluating different options and I want to give that process the space it deserves. The TL;DR:
- Go on the backend and on the frontend (via WASM). There’s a tiny amount of C for Sandbox setup, but everything else is Go.
- Cap’n Proto for IPC and network communication everywhere we can, including between the browser and the server.
- No big web frameworks; smaller focused libraries.
I plan on writing blog posts talking about progress at least once a month; stay tuned.
Ok, so Sandstorm’s biggest contributor has given up on the project 😬. What does that mean for Sandstorm?
In the short to medium term, not much, actually. I’ll probably be more frank about the fact that we aren’t really shipping platform features, and I will waste less time trying to do ambitious things only to hit dead-ends. But I’m still using Sandstorm, and so I’ll keep putting out fires and keep it afloat until I’m actually ready to jump ship; Sandstorm will remain fit for purpose until I have something better for my purposes.
I’ll also keep working on app packaging and development; Tempest’s sandbox aims to be fully compatible with Sandstorm’s, so efforts here won’t be wasted. If you’re thinking of working on apps, don’t let this deter you either.
…and of course, I’m not the only contributor to Sandstorm; Kenton has managed to keep the dependencies up to date continuously since the business shut down, so there’s no reason to believe Sandstorm will just collapse in my absence when the time comes.
I’ll also continue to hang out in #sandstorm on libera.chat; feel free
to talk to me about either Sandstorm or Tempest there.