About 2 years ago I started a bit of an odyssee to find a convenient, affordable and production ready way to deploy my web application stack. My goal was to make it viable by the above criteria, especially to have a stack/product in a segment that would normally end up on a shared webhost with clients that are used to shared webhosting pricing. First I discovered sloppy.io and started to use their lowest tier. About two months in their entire infrastructure went down for 24 hours and I was left with angry clients, took responsibility and found a better way: now.sh. That seemed like the best of both worlds. Almost zero configuration, regional deployments, integrated DNS. It seemed to good to be true - easy and affordable? Count me in!
Turns out it was too good to be true. 4 months in, ZEIT (now known as VERCEL) decided that Docker is over and serverless is the future and basically deprecated their Docker deployment options overnight. At that point I lost all confidence in the service and tried my luck again with sloppy: it simply was the only comparable service with the kind of pricing I was looking for and comparable tooling.
And again: 8 months later, right around Christmas 2019, I get an email from Sloppy that they will shut down their service with a few month's notice. Apparently their investors decided to pull out or something like that. For fuck's sake...
At this point I decided to turn my back on all the shiny stuff and to go "homebrew" for my infrastructure. For better or worse, I had to build my own solution and luckily, I found Dokku.
On a cold winter night, some time between Christmas and New Year, I decided to set up Dokku on a Hetzner Cloud instance and rewrote my GitLab pipeline. I ended up with, in essence, a similar solution like all of the above, deplyoing in a "herokuish" manner on my own infrastructure. At a fraction of the cost and knowing that if things go down, I can at least do someting about it. I was actually surprised how easy all of that was. Within 2 days, I moved all my projects to Dokku.
Before I started using Dokku, I checked out Ranger and pondered wether I should start investing my time into Kubernetes as my infrastructure. But it just felt, let's be honest, way too overengineered for a couple of websites with low traffic. Dokku nicely fits into that gap between static, shared webhosting and managed solutions on AWS or Gcloud that cost a metric shit ton per month for each single project.
Hindsight is 20:20 – I fully acknowledge that I might have been a bit too eager in wanting to use the shiny new thing and I payed the price for my naivité. Switching providers costs time, introduces new learning curves each time and might cause downtime for client projects. But with a modern web stack based on NodeJS, Express and Docker, there's no common infrastructure pattern like there is with LAMP. Everything is a bit more hands-on, technical and sometimes frustrating.
At the same time, this is something I like about the whole thing. Sounds masochistic, I know. But in a way, using the same thing I used for the past 20 years seems to be so utterly boring that I happily accept the potential frustration that comes with being a bit more "on the edge".
Over the past 4 months, I refined the deployment and build processes and learned a few things along the way.
Here's the full .gitlab-pipeline.yml. Some explanations are included as comments below.
Dokku plugins used:
1 2 3 4 5 6
dokku plugin:install https://github.com/mbreit/dokku-monit.git dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git dokku plugin:install https://github.com/dokku/dokku-http-auth.git dokku plugin:install https://github.com/dokku/dokku-redirect.git dokku plugin:install https://github.com/ribot/dokku-slack.git dokku plugin:install https://github.com/josegonzalez/dokku-docker-direct.git
.gitlab-pipeline.yml (updated: 2020/05/12)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172
# GitLab Variables on Group level # DOKKU_USER - usually the default user, "dokku" # DOKKU_DOCKER_PORT - the default internal port of the containers # DOKKU_PRIVATE_KEY_FOR_DOKKUHOST - private key for all SSH tasks run as DOKKU_USER # and DOKKU_CI_USER # DOKKU_DOMAIN - the default domain of the dokku host variables: APP_NAME: "myspecialappcom" # A slug that will be used in creating the Dokku app and tags APP_DOMAIN: "myspecialapp.com" # Primary domain that will be linked to the Dokku app APP_DOMAIN_ALTERNATIVE: "www.myspecialapp.com" # An optional, secondary domain DOKKU_HOST: "dokkuhost.mydomain.io" # The Dokku host we're deploying to DOKKU_MEMORY_LIMIT: "256MB" # Limiting the Dokku app's memory on small cloudlet instances STAGING_LOGIN: "staging staging" # HTTPAUTH for the staging step SSH_PRIVATE_KEY: $DOKKU_PRIVATE_KEY_FOR_DOKKUHOST # SSH private key USE_LETSENCRYPT: "true" # Wether or not Dokku should add SSL certs during the deployment USE_APP_DOMAIN: "false" # Wether or not Dokku should set up domains other than the automatically created subdomains # My workflow includes an E2E stage using Cypress. Production can only be deployed if # staging and e2e finished without errors. stages: - staging - e2e - production .init_functions: &init_functions | function init() { export SSH_BINARY=$(which ssh) export SSH="$SSH_BINARY -qtt $DOKKU_USER@$DOKKU_HOST --" export APP="$APP_NAME-$CI_ENVIRONMENT_SLUG" export DOKKU_APP_EXISTS="true" } # Preparing openSSH client function ssh_setup() { eval $(ssh-agent -s) ssh-add <(echo "$SSH_PRIVATE_KEY") mkdir -p ~/.ssh && touch ~/.ssh/known_hosts ssh-keyscan $DOKKU_HOST >> ~/.ssh/known_hosts chmod 644 ~/.ssh && chmod 644 ~/.ssh/known_hosts } # Isolated LE handling with error catcher function letsencrypt() { init if [[ $USE_LETSENCRYPT == "false" ]]; then echo -e "Skipping SSL certificate provisioning" exit 0 fi echo -e "Provisioning SSL certificates" # Letsencrypt errors don't exit the pipeline properly, we need to parse the output # for errors manually and "exit 1" to trigger a failed deployment LE_RESULT=$($SSH letsencrypt $APP 2>&1) echo -e "$LE_RESULT" if [[ $LE_RESULT == *"ERROR"* ]]; then echo "ERROR: Could not generate SSL certs" exit 1 fi } # Dokku App config if app does not exist function create_basic_app_if_not_exists() { $SSH apps:exists $APP || export DOKKU_APP_EXISTS="false" if [[ $DOKKU_APP_EXISTS == "false" ]]; then cfcli --activate false -t CNAME add $APP.$DOKKU_DOMAIN $DOKKU_HOST || true $SSH apps:create $APP $SSH slack:set $APP $DOKKU_SLACK_WEBHOOK $SSH resource:limit --memory $DOKKU_MEMORY_LIMIT $APP $SSH resource:reserve --memory $DOKKU_MEMORY_LIMIT $APP $SSH config:set --no-restart $APP NODE_ENV=production $SSH config:set --no-restart $APP ENVIRONMENT=$CI_ENVIRONMENT_SLUG $SSH proxy:ports-add $APP http:80:$DOKKU_DOCKER_PORT fi } function build_container_and_push_to_gitlab() { init docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA:latest || true docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA --tag $CI_REGISTRY_IMAGE:latest . docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA } function pull_container_from_dokkuserver() { init $SSH docker-direct login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY $SSH docker-direct pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $SSH docker-direct tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA dokku/$APP:latest $SSH docker-direct image rm $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA } function deploy_to_staging() { init create_basic_app_if_not_exists $SSH tags:create $APP latest $SSH tags:deploy $APP latest letsencrypt $SSH http-auth:on $APP $STAGING_LOGIN } function deploy_to_production() { init create_basic_app_if_not_exists if [[ $DOKKU_APP_EXISTS == "false" ]]; then if [[ $USE_APP_DOMAIN == "true" ]]; then $SSH domains:add $APP "$APP_DOMAIN" "$APP_DOMAIN_ALTERNATIVE" fi fi $SSH tags:create $APP latest $SSH tags:deploy $APP latest letsencrypt $SSH monit:enable $APP } function cleanup() { $SSH docker-direct container prune -f $SSH docker-direct image prune -af } before_script: - *init_functions staging: stage: staging image: zweitakt/nextjs-gitlab-runner:latest only: - master environment: name: staging url: https://$CI_ENVIRONMENT_SLUG.$DOKKU_DOMAIN/ script: - ssh_setup - build_container_and_push_to_gitlab - pull_container_from_dokkuserver - deploy_to_staging # E2E tests are testing against deployed instances, # we don't need to worry about checking out code. e2e: stage: e2e image: cypress/base:13.6.0 only: - master environment: name: e2e variables: CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress" cache: key: ${CI_COMMIT_REF_SLUG} paths: - cache/Cypress script: - yarn install - yarn workspace @zweitakt/e2e run test-staging production: stage: production image: zweitakt/nextjs-gitlab-runner:latest when: manual only: - master environment: name: production url: https://$APP_DOMAIN/ script: - ssh_setup - pull_container_from_dokkuserver - deploy_to_production - cleanup
If the stages run successfully, the apps will be available on "myspecialapp-{PIPELINE_NAME}.mydokkuhost.com" and the configured domain for the production container respectively. DNS needs to be managed manually for now. I use Cloudflare - but anything goes, really. Cloudflare is nice because it supports CNAME flattening at the root ("@") level. It also might come in handy for DDoS attack situations or when a quick solution has to be found for a high-traffic/load situation. But I'll cross that bridge when I reach it.
Of course this is all very specific to my needs. Your needs might be different. The included shell functions are universally applicable though, to a lot of build and deployment contexts with Dokku.
Things I want to look into when the need arises:
In terms of scaling - well, I yet have to find that out. So far all my projects have been tugging along nicely and didn't encounter any load issues. If a project reaches a certain limit, I just scale up the Hetzer cloudlet or extract it to it's own cloudlet/Dokku instance. That will be good enough for 99% of all my projects.