Zenhack.net

The Trouble With Library Side Projects

04 Sep 2016

It’s really hard to keep momentum going on a project that doesn’t have any users. Wanting to use it yourself can often be enough, and with any project where you are a potential user, dogfooding it as soon as possible is beneficial to its development.

Over the years I’ve started a lot of projects that are libraries or other “programming infrastructure” things. Doing this basically requires having two side projects going concurrently, and in the past when I haven’t had another project with an immediate use for one of these infrastructure projects, the project has died fairly quickly. In cases where I have had another project, the infrastructure project doesn’t tend to outlive the dependent project.

There’s also another problem: if the application is the thing driving development, it’s easy for the library to stay in a perpetual state of being work-in-progress. Since I as the maintainer have the ability to add functionality to the library just as my specific application needs it, there’s a danger of large parts of functionality that obviously belongs in the library being indefinitely missing, or certain obvious bugs just never being fixed.

As an example: my current project, IRC Idler is dependent on a library I’m building for talking directly to Sandstorm’s Cap'N Proto APIs. One of the things the library provides is some helpers for gluing go’s net/http.Handler interface to Sandstorm’s HTTP interfaces. It’s currently usable for IRC Idler’s purposes, but it’s woefully incomplete. Requests with HTTP verbs other than GET and POST don’t get delivered. Cookies and some other headers just get dropped. If sandstorm decides to do a “Streaming POST” as opposed to “here’s all the data now”, the library will return an error. None of these are an issue for IRC Idler yet, because I don’t use that functionality, and when I do need it I can just add it. I have push access to the repo, so there’s not even the overhead of having to send somebody else a patch.

IRC Idler is one of the handful of projects I’ve actually managed to stick with. I’m currently dogfooding it, and while it’s pretty janky, it’s improving. Pretty soon I’ll need to start focusing on making it useful to folks other than myself, and my plan is to try to find people to alpha test it, since I don’t think I can realistically get it to 1.0 without feedback.

I’m still mulling over how to upgrade a library project to something “first class.”