Things Ian Says

Using Traefik Proxy with Docker Compose and LetsEncrypt

Traefik Proxy is one of the newer reverse proxies available (compared to more established applications such as nginx and Apache httpd). The thing which differentiates traefik is that it was created in a post-Docker world and integrates with Docker to reduce the manual configuration needed. It also supports let’s encrypt to provide SSL encryption, with minimal extra effort.

This article looks at how we can use traefik as a reverse proxy across a docker-compose managed suite of containers and then use let’s encrypt to add SSL certificates for https access.

Background

Back in the earlier days of web hosting, we typically used a single application (like Apache httpd, then more recently nginx) to host all our sites (on a specific server). These applications took care of both serving up the websites, and routing traffic to the correct website. Although this worked perfectly well, for me, it never felt aligned with the Unix tooling principle of do one thing well.

When Docker containers first became a production-ready technology, the first thing we did was to package up the technology we were already using into containers (for example creating Docker containers with multiple web sites running in nginx). This made testing and deployment much simpler, but was clearly just a first step. Once we got a bit more sophisticated, and started getting tooling to orchestrate multiple containers (from Docker Compose to Docker Swarm to Kubernetes), we could start looking at optimisations, like creating much smaller containers (for example, I previously described how switching from nginx to busybox httpd could reduce our container size from 100M to 1M).

This meant that I could break down the way I hosted content, into separate, self-contained, independently deployable units. However, I still needed a way to route traffic to the correct one. Initially, I used nginx for this, but I was looking around for suitable alternatives, and came across Traefik. This looked very promising, since it has some great features — I can define the routing, just by adding labels to my container (I don’t need special configuration), and it would also handle generating certificates from Let’s Encrypt.

This article describes how I use Traefik to manage my hosting, enabling me to serve websites from three different containers, with a proxy handling routing and https handoff:

Traefik Configuration File

A small amount of configuration needs to be written for Traefik. Different configuration file formats are supported, and I chose to use toml, which is kind of a cross between YAML and basic property files. My file is called traefik.toml and I put it in my base directory.

The start of this file just sets up log level, and the fact that I will allow http and https:

logLevel = "INFO"
defaultEntryPoints = ["http", "https"]

The remainder of the configuration file is split into sections. The toml format specifies sections by enclosing the section name in square brackets.

The first section I created was concerned with_Traefik_’s ability to surface information about its configuration and use via an API, which can then be presented using a dashboard (which I show later). This is configured in the api section of the config file. I will label the _entrypoint_ for this as api, which allows me to specify a different configuration than the one I use for other requests (http/https). I will also configure want the dashboard to be enabled, but without any debug-level information coming through. That looks like this:

[api]
entrypoint = "api"
dashboard = true
debug = false

My next section concerns the entryPoints — how Traefik responds to calls. First thing is that I don’t want anybody being able to connect over a plain http connection. So I set up my http port to redirect to https. I also set up the port for https connections too:

[entryPoints]

  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"

  [entryPoints.https]
  address = ":443"

  [entryPoints.https.tls]

I also set up my entry point for the API (dashboard) connection I defined earlier. I set this up to listen on port 8080 (internally), but I want to password protect it (so that it’s not available to the entire World). I do that by adding a subsection for Basic HTTP Auth. There are multiple ways I can add my username and password, but for the purposes of this example, I will just add them in the configuration file:

  [entrypoints.api]
  address = ":8080"
    [entrypoints.api.auth.basic]
    users = ["secret:$apr1$XdZOrEEp$xu/mDVqCVZaILlngqUm/e1"]

There are different ways to create the user’s username and password, but I find the htpasswd utility does the trick. You supply the username, then it prompts you for a password and generates the appropriate result. If you supply the -n flag to the command, it displays the result as an output. Here’s an example where I am using the word secret as both my username and password:

$ htpasswd -n secret
New password:
Re-type new password:
secret:$apr1$XdZOrEEp$xu/mDVqCVZaILlngqUm/e1

My next section is to support Docker. It specifies the domain to use for my containers, and I also set the watch flag, which means Traefik will reload my configuration file whenever it detects a change to my setup:

[docker]
domain = "example.uk"
watch = true

Finally, I set up my automated free certificates with Let’s Encrypt. This section is called acme, because that is the name of the protocol used for communication with Let’s Encrypyt (the Automatic Certificate Management Environment). As part of this configuration, I need to specify the challenge that Let’s Encrypt will use to verify that I own the host I am requesting for the certificate. I use the http challenge, which means that Let’s Encrypt will specify a web page it expects to find on my server, and I need to set up a positive response to a request for that page. Fortunately, Traefik handles all that for me, so I don’t need to worry about the details of that process.

Here’s that final section of configuration:

[acme]
email = "letsencrypt@example.uk"
storage = "/etc/traefik/acme/acme.json"
entryPoint = "https"
onDemand = false
OnHostRule = true
acmeLogging = true

[acme.httpChallenge]
entryPoint = "http"

The two lines, onDemand and onHostRule, specify under what circumstances Traefik requests certificates from letsencrypt. The onDemand option will cause Traefik to request certificates whenever a web request is received for a domain which does not already have a certificate. This can potentially be used for denial of service attacks (by requesting many different domain names from the server) so I have set this to false. The onHostRule only requests new certificates for domain names listed in Traefik’s frontend rules in the docker-compose file. This reduces the chances of a denial of service attack, since we have a small list of valid domains to choose from. I have therefore set this to true.

That completes the Traefik configuration changes we need. For completeness, here is the whole configuration:

logLevel = "INFO"
defaultEntryPoints = ["http", "https"]

[api]
entrypoint = "api"
dashboard = true
debug = false

[entryPoints]

  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"

  [entryPoints.https]
  address = ":443"

  [entryPoints.https.tls]

  [entrypoints.api]
  address = ":8080"
    [entrypoints.api.auth.basic]
    users = ["secret:$apr1$XdZOrEEp$xu/mDVqCVZaILlngqUm/e1"]

[docker]
domain = "example.uk"
watch = true

[acme]
email = "letsencrypt@example.uk"
storage = "/etc/traefik/acme/acme.json"
entryPoint = "https"
onDemand = false
OnHostRule = true
acmeLogging = true

[acme.httpChallenge]
entryPoint = "http"

Docker Compose

I am using Docker Compose to orchestrate my Docker containers. Let’s start by looking at the basic docker-compose.yaml file, which spins up my containers:

version: "2"
services:

  traefik:
    image: traefik:1.7-alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - $PWD/run/acme:/etc/traefik/acme
      - $PWD/traefik.toml:/etc/traefik/traefik.toml

  website:
    image: my-website
    environment:
      - GA_WEBSITE

  blog:
    image: my-blog
    environment:
      - GA_BLOG

  holdingPage:
    image: my-holding-page

Just running through those containers from the top, I use the alpine version of the Traefik Docker image, since alpine usually gives smaller containers. I expose ports 80 and 443 (for http and https) to allow standard web access. I also need to map some files into the container. Traefik needs to read information from the Docker daemon, so I need to map in docker.sock to allow that communication. I need to be able to persist the certificate details from Let's Encrypt — that is /etc/traefik/acme inside the container, and I am using /run/acme on my host. Finally, I need to map in the traefik.toml file I created earlier.

The other three containers (website, blog and holdingPage) are just run as basic invocations, with the addition of environmental variables which define my Google Analytics tracking token.

If I were using a conventional proxy server (like nginx), I would now need to set up a set of rules in nginx configuration, specifying how to proxy all the services correctly. However, the magic of Traefik is that I can now just add labels to this configuration file, and Traefik will use them to set up the proxying automatically.

Let’s start with my website. Traefik needs two parts to proxy traffic — the backend (the container to connect to) and the frontend (the rule for when to send traffic to this container). For my website, I want to be able to reach it when someone types in the web address https://example.uk or https://example.co.uk (plus the variations with www on the front). So that is the rule I put in the frontend. The backend just needs the name website:

website:
  image: my-website
  environment:
    - GA_WEBSITE
  labels:
    - "traefik.backend=website"
    - "traefik.frontend.rule=Host:example.uk, www.example.uk, example.co.uk, www.example.co.uk"

I do a similar thing for my blog:

blog:
  image: my-blog
  environment:
    - GA_BLOG
  labels:
    - "traefik.backend=blog"
    - "traefik.frontend.rule=Host: blog.example.org.uk, www.blog.example.org.uk"
    - "traefik.frontend.rule=Host: example.com, www.example.com"

Notice that I have split the frontend into two separate rules. This doesn’t affect the behaviour of Traefik as a proxy, but it does influence the certificates we get from Let’s Encrypt. With the above configuration, I will get two certificates — one for blog.example.org.uk and www.blog.example.org.uk, and another one for example.com and www.example.com. If I put all the URLs in the same frontend rule, I would get a single certificate covering all these domains.

For completeness, here are the labels I add to my holding page container:

holdingPage:
  image: my-holding-page
  labels:
    - "traefik.backend=holding"
    - "traefik.frontend.rule=Host:example.org.uk, www.example.org.uk"

Finally, I need to add labels to the Traefik container itself. This is to allow me to access the dashboard pages. The backend rule is just like the other containers, and I also need to add a traefik.enable label (it’s switchable, since not every setup will want to expose the Traefik dashboard). Because the standard ports on this container are being used as the public access to the hosting, I need to use a traefik.port label to specify a different port for this backend (port 8080). Finally, the frontend rule is slightly more involved, because I want to use the same domain as one of my other containers (example.org.uk), but with /traefik/ added to the end of the web address. So, I specify the hosts the same as before, but also add a PathPrefixStrip term too — this will only match when the URL ends with /traefik/, but it will strip it off the URL before it passes it to the backend. Putting it all together, it looks like this:

traefik:
  image: traefik:1.7-alpine
  labels:
    - "traefik.backend=traefik"
    - "traefik.enable=true"
    - "traefik.port=8080"
    - "traefik.frontend.rule=Host:example.org.uk, www.example.org.uk; PathPrefixStrip:/traefik/"
  ports:
    - "80:80"
    - "443:443"
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - $PWD/run/acme:/etc/traefik/acme
    - $PWD/traefik.toml:/etc/traefik/traefik.toml

And here is the entire Docker compose file:

version: "2"
services:

  traefik:
    image: traefik:1.7-alpine
    labels:
      - "traefik.backend=traefik"
      - "traefik.enable=true"
      - "traefik.port=8080"
      - "traefik.frontend.rule=Host:example.org.uk, www.example.org.uk; PathPrefixStrip:/traefik/"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - $PWD/run/acme:/etc/traefik/acme
      - $PWD/traefik.toml:/etc/traefik/traefik.toml

  website:
    image: my-website
    environment:
      - GA_WEBSITE
    labels:
      - "traefik.backend=website"
      - "traefik.frontend.rule=Host:example.uk, www.example.uk, example.co.uk, www.example.co.uk"

  blog:
    image: my-blog
    environment:
      - GA_BLOG
    labels:
      - "traefik.backend=blog"
      - "traefik.frontend.rule=Host: blog.example.org.uk, www.blog.example.org.uk"
      - "traefik.frontend.rule=Host: example.com, www.example.com"

  holdingPage:
    image: my-holding-page
    labels:
      - "traefik.backend=holding"
      - "traefik.frontend.rule=Host:example.org.uk, www.example.org.uk"

I can now start up my servers (using docker-compose up) and I will be able to access all my backend containers via Traefik as a proxy. I also have a useful dashboard (which is password protected) which will let me see information about my hosting. Here is the screen which shows my frontends and backends:

And here is the screen which shows me the health of my hosting (e.g. uptime, response time, etc):

Now I have this setup, adding additional web servers is as simple as adding a few new lines for those servers in the docker-compose file. I don’t need to change the Traefik configuration in any way to accommodate the server and it will automatically be added to the routes and get new SSL certificates generated and deployed (and renewed) for the domains we define.

Using Let’s Encrypt to get Certificates

The best part about using Traefik is that I don’t actually need to write anything in this section. The configuration and Docker compose file I gave above takes care of fetching the Let’s Encrypt certificates if they don’t exist, and also renews them when they are due to expire. It also sorts out responding to the http challenge when setting up those certificates. So I don’t need to do anything extra to get my free https certificates.

Summary

As you can see, there is only a small amount of configuration needed for both the domain-based routing and the https connections to be supported. And once we have created the inital configuration, adding in a new container just requires a couple of extra labels on top of the normal docker-compose.yaml content. What’s more, use of Let’s Encrypt is free, so there’s really no excuse not to have https on your website now.