How I Use Dokku

As I've started to explore cloud and microservices-based projects, I've turned to Dokku to host and manage my running projects. It's become more or less my one-stop-shop for hosting all the projects I've worked on. A lot of my projects have different requirements, though, so I wanted to share the techniques and setup I use to keep everything running smoothly. This isn't an exhaustive list of all the projects I host, but it chronicles the difficulties I've had over time.

This isn't going to be a How-To guide for Dokku--there are plenty of those already. It's more of a collection of tips and tricks, and an explanation of which features I find most useful.

Link to this section What is Dokku?

With the growth in popularity of microservices, there has also been a growth in "Platform-as-a-Service" providers like Heroku. These platforms abstract away the details of running a Linux VM and installing dependencies, which drastically simplifies the process of deploying apps. However, these services can be expensive--for instance, once you burn through Heroku's free tier, you'll be charged $7 per month per container. This might be a fair price for businesses who need solid reliability and high performance, but it's prohibitively expensive for a tinkerer who just wants to try out a new technology or work on a side project every so often.

Dokku is a self-hosted platform-as-a-service, and it can be installed on any Linux machine you have access to. It's limited to only one machine, so all of your apps will need to fight over the system's CPU/ram/etc, and there's no easy way to scale across multiple servers. However, renting a single server is much cheaper than running many different containers on a retail PaaS. At time of writing, I'm running about 15 different projects, which would cost about $7×15 = **$105** every month. That's a lot more expensive than a $5 VPS.

Link to this section The Server

My Dokku instance is currently being rented from Azure with credits I've been getting for free (it's a long story). I run Ubuntu LTS because it's stable, popular, and what I'm familiar with. I just used Dokku's bootstrap.sh script to get started. Their docs have a good installation guide.

I use Dokku's letsencrypt plugin to manage HTTPS for all the apps. Handling this at the platform layer instead of the application layer makes applications easier to develop--I don't have to worry about encryption for every single app, I can just configure it once in Dokku. Encryption is a necessity for me, as my website runs on a .dev domain and is thus on the HSTS preload list by default. (I'm glad that Google did this, and I'm happy to be part of the push for HTTPS adoption everywhere, but boy is working around it frustrating sometimes.)

Link to this section Snowflake

I've written about Snowflake here before, but here's the TL;DR: It's a service for generating unique, time-ordered, 64-bit ID numbers. It's built with Python and Flask.

Notably, it's segmented into two distinct parts: "Snowflake" generates the IDs based on its instance number, and "Snowcloud" assigns the instance numbers to the Snowflake instances. This is for scalability reasons: it's possible to scale the Snowflake app to many different instances, but as long as each has a unique instance number, the IDs will not conflict. Additionally, Redis is used to keep track of the in-use instance numbers.

Deploying Flask apps to Dokku is easy--no special buildpacks are required, and the Procfile is simply web: gunicorn app:app. Adding Redis is also pretty straightforward using Dokku's official Redis plugin. When you create a Redis instance and link it to your app, Dokku will set the REDIS_URL environment variable in your app to point to that Redis instance. For testing locally, I just put export REDIS_URL=redis://localhost:6379 in my .env so that my test and production environments are similar.

The Snowcloud app will only assign instance IDs to processes with the proper API key. Thus, this key needs to be set in both the Snowcloud and Snowflake app. The dokku config command is your friend here: I just set them as environment variables using dokku config:set SNOWCLOUD-KEY={key}. Being able to set secrets like this in environment variables is really handy--it's great to keep them out of the repo (especially because I like to share my code on my GitHub), and it's a lot easier than trying to store them in a file or something.

Link to this section AutoRedditor

AutoRedditor is a project I made to quickly return random Reddit posts on demand. It has two main parts: a worker thread to retrieve Reddit posts and store them in Redis, and a web thread to retrieve posts from Redis when a request is received. It's built with Python, but using asyncio, mostly because I wanted an excuse to learn the technology.

Running the worker thread was as simple as adding worker: python3 worker.py to the Procfile. However, running the web thread was a bit more difficult, as I'm using Quart instead of Flask here. Typically, deploying a Quart app is just as simple as replacing Gunicorn with Hypercorn, but it's a bit trickier here. Dokku apps need to respect the PORT environment variable and make their web interface available on that port. The solution is to explicitly specify the port variable in the command: web: hypercorn -b 0.0.0.0:${PORT} app:app.

Link to this section Cards

Cards is a service to generate image or HTML-based cards for embedding into websites. It uses Pyppeteer, a Python port of the popular Puppeteer JS library used for automating actions in a headless web browser. Specifically, Pyppeteer is used to render and screenshot HTML-based cards in order to produce images. (I experimented with using Selenium, but I found Pyppeteer was easier to install and use.) Because Pyppeteer is based on asyncio, I decided to go with Quart (+ Hypercorn) for this project as well. Getting Pyppeteer to run in an app container isn't particularly straightforward, as it requires a Chromium install to use for the browser.

Thankfully, Heroku has published a Google Chrome buildpack that can be used to install Chrome into the app. Running dokku buildpacks:add cards https://github.com/heroku/heroku-buildpack-google-chrome and dokku buildpacks:add cards https://github.com/heroku/heroku-buildpack-python will configure the app to install both Chrome and a Python runtime. Heroku's buildpack sets the GOOGLE_CHROME_SHIM environment variable, and this just has to be passed to Pyppeteer's launch(executablePath) function. For local testing, leave this variable unset, and Pyppeteer will just use your local Chrome install.

Cards will also cache the screenshots so that it doesn't have to run Pyppeteer for every request. To do this, I needed to mount some sort of persistent storage to the running container. The dokku docker-options command was perfect for this: I just needed to add -v /home/breq/cards:/storage to the deploy options.

Link to this section Emoji

I made a simple emoji keyboard at emoji.breq.dev because I was frustrated that I couldn't send emoji from my computer with Google Voice. I used Jekyll, which is overkill for a single-page site, but I wanted to use my existing website theme and avoid repeating the same code over and over for every single emoji. I wanted to keep my built _site folder out of the repo to avoid cluttering things up, which complicated Dokku deployment a bit.

I needed to find a way to make Dokku build the site when I deployed. I found some buildpacks that worked: Heroku's nginx pack and inket's Jekyll pack. These were surprisingly painless--the Jekyll pack tells Dokku how to install Ruby, run Jekyll to build the site, and point nginx to the _site folder.

Link to this section Breqbot

Breqbot is a Discord bot I built. It uses the traditional Gateway API instead of the newer Interactions one, so most functions are handled by a worker thread that connects to Discord over a WebSocket. However, some information is available over a REST API, so there's a web thread that runs as well.

The most difficult part about getting Breqbot to work was the voice features. Breqbot includes a soundboard feature to play sound in a Discord voice channel. Handling these audio codecs requires installing quite a few packages: I ended up using Heroku's Apt buildpack to install libffi-dev, libnacl-dev, and libopus-dev, and I used jonathanong's ffmpeg buildpack for, well, ffmpeg.

Link to this section Minecraft

I wanted to host a Minecraft server on my cloud VPS as well, so I decided to try to use Dokku to manage it. So far, it's been working out pretty well, and it's nice to have everything managed in one place. However, getting Minecraft to work initially in a containerized setup wasn't straightforward.

I found itzg's Docker Minecraft server image and set about making it work with Dokku. Dokku does support deploying from a Docker image, and although the process isn't particularly straightforward, it is at least well documented.

Next, I set the docker options:

Finally, I needed a way to access the server console to run commands while the server is running. I found mesacarlos' WebConsole plugin, which can provide a password-protected console over the Internet. To expose this console, I used dokku proxy to proxy ports 80 and 443 on the host to port 8080 inside the container. I'm currently hosting the web interface in a separate Dokku app. I just made sure to set the WebConsole port to 443 in the interface to connect to the container using HTTPS.

Link to this section Syncthing

I run a Syncthing instance to sync files between my desktop and laptop, so I decided to try to run this through Dokku as well. Syncthing provides an official Docker image, so I didn't have to use a third-party one, and they have good documentation as well. The only change I made from the guide was to proxy the web GUI through Dokku with dokku proxy:ports-add syncthing https:443:8384 instead of exposing it directly to the Internet.

Link to this section Wireguard

I also decided to run a VPN, and I chose Wireguard because it seemed simple, well-supported, and lightweight. The LinuxServer.io team maintains a Wireguard image, so I just needed to deploy it to a Dokku app. Using the documentation as a reference, I set these docker options:

Link to this section Conclusion

Running everything all in one place has really simplified things a lot. Being able to deploy apps quickly is really nice for prototyping ideas. Overall, I'm really glad I decided to start using Dokku to manage all the services I'm hosting in the cloud.