I put off cleaning up cruft on my server for too long and then put off writing anything about it while I waited to see what fell off in the wash. It turns out, things were pretty painless.
The truth of the matter is, this server was getting a little long in the tooth, dating back several years and running an out of date version of Ubuntu (12.04), which has since gone end-of-life. I was also keen to try out a few "big" changes that were unavailable or difficult to upgrade to.
Since I was tearing down the old VPS entirely, I decided to shop around some. I had, for several years been hosting my server on Digital Ocean on their cheapest plan, but in the intervening years a number of compelling alternatives have arisen. I have ended up switching to Vultr, where I'm able to maintain the same specs on the server as Digital Ocean, but at half the cost (now down to $2.50 a month). For reference, this server is now running Debian 9 on:
Which is usually over-provisioned for the kind of load I typically use, running Nginx for this site, a basic home-page, and the occasional web-app.
The good news in migrating servers was that I had previously written some basic provisioning scripts with Ansible to ease the setup. The bad news was, I hadn't quite maintained them well enough, coupled with the fact that a good deal of my previous server setup was largely undocumented meant things weren't quite as painless as I might like.
It is probably a cheap and valuable lesson though, to consider "infrastructure as code" is as susceptible to bit-rot as anything else; it has also led me to keep things almost painfully spartan on the new server. Partly for my own recollection a list of those things I ended up installing is as follows:
sudoThis ended up being the easiest thing in the world, I just rsynced the
directory tree from one server to the other and updated the Nginx config files.
In the process of migrating glue-boy from the old server to the new, I noted a few annoying details that finally did me in with respect to the old configuration. Firstly, I never bothered to setup an init script on the 12.04 machine, so any server restarts would not gracefully bring back this stupidly-simple pastebin. Secondly, the application was a basic Flask app, running on gunicorn, reverse-proxied behind Nginx. I couldn't exactly work out why but gunicorn was continually resulting in a few zombie processes over time, which was just messy enough to be annoying.
So I gave in and took the opportunity to
re-write the application (a third time), using
Twisted, which I've been
experimenting with recently. I ended
up using Klein, which gets all the
asynchronous benefits of Twisted while remaining familiar to Flask users (as
they both utilize Werkzeug).
The result is a SQLite-backed
Python application that required less than half the memory of the previous
pre-fork worker-model implementation due to the lack of worker processes (or
their resulting zombies). As a nice bonus, I got to try out Twisted's
integration with async and await.
systemdThe last piece to solve my process management woes was to write a service file
for systemd (something new on the freshly created VPS). This ended up being
almost pleasant, though this may be colored by the amount of criticism
systemd receives online, I was expecting much worse. The one tricky piece,
which is really a gotcha from Twisted, was to specify the PYTHONPATH variable
in the service file (through the ENVIRONMENT configuration):
[Unit]
Description=glue boy
[Service]
Environment=PYTHONPATH=/home/nolan/async-glue-boy
ExecStart=/home/nolan/venvs/async-glue-boy/bin/twist web -n --class=glue_boy.resource
WorkingDirectory=/home/nolan/async-glue-boy
Restart=always
[Install]
WantedBy=multi-user.target
The result is automatic restarts, standardized logging (albeit through
syslog) and a real control interface through systemctl.
I'm not sure if it is entirely down to a clean work-space, with the new server, or some of it is experience, but setting up was much easier than in the past and I'm almost looking forward to easier upgrades in the future. Async in Python is quickly winning me over, Twisted, though "old", has a number of niceties that I'm excited to try out - like re-implementing a spam activity monitor as a log observer