Things are still settling down, but I think I’ve finally completed the first step of my migration of Asymptomatic into this new server.
While I’m sad to leave Habari as the software that operates this site after so much good that it did for me and others, I think it’s time for this site to serve the two functions it always has: Allow me to write and let me experiment with the technology that is of most interest to me at the time.
While my PHP days are not over, I have moved on to other languages. I’ve met folks over the years who have called themselves “polyglots” for having learned HTML, CSS, and javascript, and while I suppose that’s true, I’m really headed for a more rounded language knowledge. As will come to no surprise to people who I interact with regularly on tech, this site software is written in Elixir.
I found a project called jelly_shot on Github that implements some features of a blog engine that I find desirable:
- Written in some fine Elixir to run as a standalone server (No Apache/Nginx)
- Uses markdown files instead of a database, enabling a git workflow
- Does not rebuild everything after every edit
- Serves pages fast
jelly_shot itself is fine, but is very experimental. It loads a whole directory worth of markdown files into memory, rendering them into HTML, then holds them there. When a request is made, it embeds the rendered post data into a template and serves it. It is quite fast.
The original project uses an Agent, which is a simple implementation of an Elixir GenServer, to hold all of the post data in a single list. The Agent is its own Elixir process, and, to simplify the explanation, using this process allows the server to handle concurrent requests deftly – this is the underpinning reason of why Elixir is so good at high-concurrency.
Using the Agent works well, but the List that jelly_shot uses is suspect. A List is fine if you have 20 posts, but not if you have 3000+ like this site. In an effort to make the application perform better, I have converted the List to use ETS (Erlang Term Storage), which is a kind of simple in-memory key-value store. I store each post under a unique key, and then create other ETS tables that point at those posts based on certain criteria. For example, there is a tags table that would index many posts for a particular tag.
Using ETS makes the whole thing perform much faster than the O(n) lookup in a List. It also has future implications, in that by converting from ETS to DETS, I could store that data structure on disk periodically, and reload it, instead of having to build it from the markdown files and re-render each time. Currently, re-render and indexing time of all 3000+ posts on this site takes about 7 seconds.
There are a handful of things I will want to update even as the site comes back online. One of the first things is that the theme isn’t working fully the way it was originally. I think this is because Habari was serving pre-compiled CSS files from CDN, and jelly_shot is using webpack to combine all of those files together. It’s having some weird side-effects that I haven’t figured out yet.
I have set up jelly_shot behind Haproxy, and am using Lets Encrypt to create a certificate that is used by Haproxy. Eventually I may move the handling of the certificate into the Elixir app, and set up the app so that it automatically renews the cert, instead of the complex web of cron job and shell scripts that are currently being used. A fully self-contained app would be ideal.
I’d like to update the app so that I can supply an environment variable to it to tell it where my git checkout of markdown posts is. I could use a similar setting for theme files, as well, so that they could be included in the git checkout. The complication here is twofold. For one, Elixir apps seem not fond of runtime configuration, as currently the location used is quite dictated by the directory structure of Phoenix (the Elixir web framework). For two, specifying templates this way for the theme would require doing some crafty things with EEX files, which are embedded Elixir templates. One thing that makes Elixir so quick to serve pages is that the HTML templates are compiled into the application. To do this at runtime would require that, at runtime, it load the template file from the specified directory, compile it, and replace what’s currently in place. It may be complex to add additional templates beyond what is initially defined. I’ll have to investigate.
On the plus side of all of this, if this post authoring works out, it’ll just show up on the blog without much fanfare. If the Elixir I’ve written is stable, the site will be reasonably unhackable. I’ve already seen in the logs the numerous requests from hacking scripts trying to get into unsecured WordPress instances (which haven’t worked with this site for a long, long time), which all necessarily fail. I have yet to switch over to production mode for the app, while I’m still debugging some things, and that will likely make the app even more secure.
Here’s hoping for a nice return to regular writing.
