Pocket Readings

个人阅读清单记录博客

0%

Best Practices Around Production Ready Web Apps with Docker Compose

Posted on Edited on In
Best Practices Around Production Ready Web Apps with Docker Compose

Here’s a few patterns I’ve picked up based on using Docker since 2014. I’ve extracted these from doing a bunch of freelance work.



Tags: docker, engineering



via Pocket https://ift.tt/3yWJGWi original site



June 04, 2021 at 01:22PM

Comments

from: github-actions[bot] on: 6/4/2021

Best Practices Around Production Ready Web Apps with Docker Compose by @nickjanetakis

Updated on June 1st, 2021 in #docker, #elixir, #flask, #javascript, #rails

Best Practices Around Production Ready Web Apps with Docker Compose

blog/cards/best-practices-around-production-ready-web-apps-with-docker-compose.jpg

Here’s a few patterns I’ve picked up based on using Docker since 2014. I’ve extracted these from doing a bunch of freelance work.

Quick Jump: Example Web Apps Using Docker / Docker Compose | Docker Compose | Your Web App | Dockerfile | Git and Docker Ignore Files | Closing Thoughts

On May 27th, 2021 I gave a live demo for DockerCon 21. It was a 29 minute talk where I covered a bunch of Docker Compose and Dockerfile patterns that I’ve been using and tweaking for years while developing and deploying web applications.

It’s pretty much 1 big live demo where we look at these patterns applied to a multi-service Flask application but I also reference a few other example apps written in different languages using different web frameworks (more on this soon).

If you prefer video instead of reading, here’s the video on YouTube with timestamps. This is the director’s cut which has 4 extra minutes of content that had to be cut out from DockerCon due to a lack of time.

This post is a written form of the video. The talk goes into more detail on some topics, but I’ve occasionally expanded on certain topics here in written form because even the director’s cut version was pressed for time at the time I recorded it.

As a disclaimer, these are all personal opinions. I’m not trying to say everything I’m doing is perfect but I will say that all of these things have worked very nicely so far in both development and production for both my personal and client’s projects.

Example Web Apps Using Docker / Docker Compose

A majority of the patterns are applied exactly the same with any language and web framework and I just want to quickly mention that I’m in the process of putting together example apps for a bunch of languages and frameworks.

All of them pull together a few common services like running a web app, background worker (if applicable), PostgreSQL, Redis and Webpack.

A few ready to go example web apps using Docker:

As for the Play example, I want to give a huge shout out to Lexie.

She’s a software engineer who primarily works with Scala and by sheer luck we ended up getting in contact about something unrelated to Docker. Long story short, after a few pair programming sessions it was ready to go. There’s no way that Play example app could have existed without her help and expertise.

Docker Compose

Let’s start off with a few patterns, tips and best practices around using Docker Compose in both development and production.

Dropping the version property at the top of the file

The Docker Compose spec mentions that the version property is deprecated and it’s only being defined in the spec for backwards compatibility. It’s informative only.

Prior to this it was common to define version: "3.8" or whatever API version you wanted to target because it controlled which properties were available. With Docker Compose v1.27+ you can drop it all together, yay for deleting code!

It seems like we’ve gone full circle back to the days when Docker Compose used to be called Fig and version 1 had no version definition.

Avoiding 2 Compose Files for Dev and Prod with an Override File

On the topic of development / production parity I like using the same docker-compose.yml in all environments. But this gets interesting when you want to run certain containers in development but not in production.

For example you might want to run a Webpack watcher in development but only serve your bundled assets in production. Or perhaps you want to use a managed PostgreSQL database in production but run PostgreSQL locally in a container for development.

You can solve these types of problems with a docker-compose.override.yml file.

The basic idea is you could create that file and add something like this to it:

services:
  webpack:
    build:
      context: "."
      target: "webpack"
      args:
        - "NODE_ENV=${NODE_ENV:-production}"
    command: "yarn run watch"
    env_file:
      - ".env"
    volumes:
      - ".:/app"

It’s a standard Docker Compose file, and by default when you run a docker-compose up then Docker Compose will merge both your docker-compose.yml file and docker-compose.override.yml into 1 unit that gets run. This happens automatically.

Then you could add this override file to your .gitignore file so when you push your code to production (let’s say a VPS that you’ve set up) it won’t be there and voila, you’ve just created a pattern that lets you run something in dev but not in prod without having to duplicate a bunch of services and create a docker-compose-dev.yml + docker-compose-prod.yml file.

For developer convenience you can also add a docker-compose.override.yml.example to your repo that isn’t ignored from version control and now all you have to do is cp docker-compose.override.yml.example docker-compose.override.yml to use the real override file when cloning down the project. This is handy in both dev and CI.

Reducing Service Duplication with Aliases and Anchors

This can be done using YAML’s aliases and anchors feature along with extension fields from Docker Compose. I’ve written about this in detail in Docker Tip #82.

But here’s the basic idea, you can define this in your docker-compose.yml file:

x-app: &default-app
  build:
    context: "."
    target: "app"
    args:
      - "FLASK_ENV=${FLASK_ENV:-production}"
      - "NODE_ENV=${NODE_ENV:-production}"
  depends_on:
    - "postgres"
    - "redis"
  env_file:
    - ".env"
  restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
  stop_grace_period: "3s"
  tty: true
  volumes:
    - "${DOCKER_WEB_VOLUME:-./public:/app/public}"

And then in your Docker Compose services, you can use it like this:

web:
  <<: *default-app
  ports:
    - "${DOCKER_WEB_PORT_FORWARD:-127.0.0.1:8000}:8000"

worker:
  <<: *default-app
  command: celery -A "hello.app.celery_app" worker -l "${CELERY_LOG_LEVEL:-info}"

That’s going to apply all of the first code snippet of properties to both the web and worker services. This avoids having to duplicate those ~15 lines of properties.

You can also override an aliased property in a specific service which lets you customize it. In the above example you could set stop_grace_period: "10s" for just the worker service if you wanted to. It’ll take precedence over what’s in the alias.

This pattern is especially handy in cases like this where 2 services might use the same Dockerfile and code base but have other minor differences.

As an aside, I’m going to be showing relevant lines of code for each topic so what you see in each section isn’t everything included. You can check out the GitHub repos for the full code examples.

Defining your HEALTHCHECK in Docker Compose not your Dockerfile

Overall I try not to make assumptions about where I might deploy my apps to. It could be on a single VPS using Docker Compose, a Kubernetes cluster or maybe even Heroku.

In all 3 cases I’ll be using Docker but how they run is drastically different.

That means I prefer defining my health check in the docker-compose.yml file instead of a Dockerfile. Technically Kubernetes will disable a HEALTHCHECK if it finds one in your Dockerfile because it has its own readiness checks but the takeaway here is if we can avoid potential issues then we should. We shouldn’t depend on other tools disabling things.

Here’s what a health check looks like when defining it in a docker-compose.yml file:

web:
  <<: *default-app
  healthcheck:
    test: "${DOCKER_WEB_HEALTHCHECK_TEST:-curl localhost:8000/up}"
    interval: "60s"
    timeout: "3s"
    start_period: "5s"
    retries: 3

What’s neat about this pattern is it allows us to adjust our health check in development vs production since the health check gets set at runtime.

This is done using environment variables which we’ll talk more about soon but the takeaway for now is in development we can define a health check which does /bin/true instead of the default curl localhost:8000/up health check.

That means in dev we won’t get barraged by log output related to the health check firing every minute. Instead /bin/true will run which is pretty much a no-op. It’s a very fast running command that returns exit code 0 which will make the health check pass.

Making the most of environment variables

Before we get into this, one common pattern here is we’ll have an .env file in our code repo that’s ignored from version control. This file will have a combination of secrets along with anything that might change between development and production.

We’ll also include an .env.example file that is commit to version control which has non-secret environment variables so that in development and CI it’s very easy to get up and running by copying this file to .env with cp .env.example .env.

Here’s a snippet from an example env file:

# Which environment is running? These should be "development" or "production".
#export FLASK_ENV=production
#export NODE_ENV=production
export FLASK_ENV=development
export NODE_ENV=development

For documentation I like commenting out what the default value is. This way when it’s being overwritten we know exactly what’s it’s being changed to.

Speaking of defaults, I try to stick to using what I want the values to be in production. This reduces human mistakes because it means in production you only need to set a few environment variables (secrets, a handful of others, etc.).

In development it doesn’t matter how many we override because that can all be set up and configured in the example file beforehand.

All in all when you combine environment variables with Docker Compose and build args with your Dockerfile you can use the same code in all environments while only changing a few env variables.

Going back to our theme of dev / prod parity we can really take advantage of environment variables in our docker-compose.yml file.

You can define environment variables in this file by setting something like ${FLASK_ENV}. By default Docker Compose will look for an .env file in the same location as your docker-compose.yml file to find and use that env var’s value.

It’s also a good idea to set a default value in case it’s not defined which you can do with ${FLASK_ENV:-production}. It uses the same syntax as shell scripting, except it’s more limited than shell scripting since you can’t nest variables as another variable’s default value.

Here’s a few common and useful ways to take advantage of environment variables.

Controlling which health check to use:
web:
  healthcheck:
    test: "${DOCKER_WEB_HEALTHCHECK_TEST:-curl localhost:8000/up}"
    interval: "60s"
    timeout: "3s"
    start_period: "5s"
    retries: 3

We covered this one before.

By default it runs the curl command but in our .env file we can set export DOCKER_WEB_HEALTHCHECK_TEST=/bin/true in development.

If you’re wondering why I use export ... in all of my .env files it’s so that I can source .env in other scripts which comes in handy when creating project specific shell scripts. I’ve created a separate video on that topic. Docker Compose 1.26+ is compatible with export.

Publishing ports more securely in production:
web:
  ports:
    - "${DOCKER_WEB_PORT_FORWARD:-127.0.0.1:8000}:8000"

In the past I’ve written about how I like running nginx outside of Docker directly on the Docker host and using this pattern ensures that the web’s port won’t be accessible to anyone on the public internet.

By default it’s restricted to only allow connections from localhost, which is where nginx would be running on a single server deploy. That prevents folks on the internet from accessing example.com:8000 without needing to set up a cloud firewall to block what’s been set by Docker in your iptables rules.

Even if you do set up a cloud firewall with restricted ports I would still do this. It’s another layer of security and security is all about layers.

And in dev you can set export DOCKER_WEB_PORT_FORWARD=8000 in the .env file to allow connections from anywhere. That’s handy if you’re running Docker in a self managed VM instead of using Docker Desktop or in cases where you want to access your site on multiple devices (laptop, iPad, etc.) on your local network.

Taking advantage of Docker’s restart policies:
web:
  restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"

Using unless-stopped in production will make sure that your containers will come up after rebooting your box or if they crash in such a way that they can be recovered by restarting the process / container.

But in development it would be a bit crazy if you rebooted your dev box and every project you ever created in your entire life came up so we can set export DOCKER_RESTART_POLICY=no to prevent them from starting automatically.

Switching up your bind mounts depending on your environment:
web:
  volumes:
    - "${DOCKER_WEB_VOLUME:-./public:/app/public}"

If you plan to access your static files (css, js, images, etc.) from nginx that’s not running in a container then a bind mount is a reasonable choice.

This way we only volume mount in our public/ directory which is where those files would be located. That location might be different depending on which web framework you use and in most of the example apps I try to use public/ when I can.

This mount itself could be read-only or a read-write mount based on whether or not you plan to support uploading files directly to disk in your app.

But in development you can set export DOCKER_WEB_VOLUME=.:/app so you can benefit from having code updates without having to rebuild your image(s).

Limiting CPU and memory resources of your containers:
web:
  deploy:
    resources:
      limits:
        cpus: "${DOCKER_WEB_CPUS:-0}"
        memory: "${DOCKER_WEB_MEMORY:-0}"

If you set 0 then your services will use as many resources as they need which is effectively the same as not defining these properties. On single server deploys you could probably get by without setting these but with some tech stacks it could be important to set, such as if you use Elixir. That’s because the BEAM (Erlang VM) will gobble up as many resources as it can which could interfere with other services you have running, such as your DB and more.

Although even for single server deploys with any tech stack it’s useful to know what resources your services require because it can help you pick the correct hardware specs of your server to help eliminate overpaying or under-provisioning your server.

Also, you’ll be in much better shape to deploy your app into Kubernetes or other container orchestration platforms. That’s because if Kubernetes knows your app uses 75mb of memory it knows it can fit 10 copies of it on a server with 1 GB of memory available.

Without knowing this information, you may end up wasting resources on your cluster.

Your Web App

We’ve covered a lot of ground around Docker Compose but now let’s switch gears and talk about configuring your application.

Your web server’s config file

With Flask and other Python based web frameworks you might use gunicorn or uwsgi for your app server. With Rails you might use Puma. Regardless of your tech stack there’s a few things you’ll likely want to configure for your app server.

I’ll be showing examples from a gunicorn.py file from the Flask example app but you can apply all or most of these anywhere.

Your bind host and port
bind = f"0.0.0.0:{os.getenv('PORT', '8000')}"

We’ll bind to 0.0.0.0 so that you’ll be able to connect to your container from outside of the container. Lots of app servers default to localhost which is a gotcha when working with Docker because it’ll block you from being able to connect from your browser on your dev box. That’s why this value is hard coded, it’s not going to change.

When it comes to the port, I like making this configurable and even more importantly I chose to use PORT as the name because it’s what Heroku uses. Whenever possible I try to make decisions that make my apps able to be hosted on a wide range of services. In this case it’s an easy win.

Workers and threads:
workers = int(os.getenv("WEB_CONCURRENCY", multiprocessing.cpu_count() * 2))
threads = int(os.getenv("PYTHON_MAX_THREADS", 1))

With Python, Ruby and some other languages your worker and thread count control how many requests per second your app server can serve. The more you have, the more concurrency you can handle at the cost of using more memory and CPU resources.

Similar to the PORT, the naming convention for both env vars are based on Heroku’s names.

In the case of the workers, it defaults to twice as many vCPUs you have on the host. This is nice because it means if you upgrade servers later on you don’t need to worry about updating any configuration, not even an env variable.

But it’s still configurable with an env variable if you want to override that value.

In development I set both of these values to 1 in the .env.example because it’s easier to debug an app that doesn’t fork under the hood. You’ll see both are set to 1 in the example apps that have app servers which support these options.

Code reloading or no?
from distutils.util import strtobool

reload = bool(strtobool(os.getenv("WEB_RELOAD", "false")))

Certain web frameworks and app servers handle code reloading differently. With gunicorn you’ll need to explicitly configure gunicorn to do code reloading or not.

This is prime pickings for an environment variable. Here we can default to false for production but then for our dev environment in our .env file we can set it to true.

Log to standard out (stdout):
accesslog = "-"

It’s a good idea to log to stdout instead of a file on disk when working with Docker because if you log to disk it’ll disappear as soon as you stop and remove your container.

Instead, if you log to stdout you can configure Docker to persist your logs however you see fit. You could log to journald and then explore your logs with journalctl (great for single server deploys) or have your logs get sent to CloudWatch on AWS or any 3rd party service.

The takeaway here is all of your apps can log to stdout and then you can handle logging at the Docker daemon level in 1 spot.

Configuring your database

pg_user = os.getenv("POSTGRES_USER", "hello")
pg_pass = os.getenv("POSTGRES_PASSWORD", "password")
pg_host = os.getenv("POSTGRES_HOST", "postgres")
pg_port = os.getenv("POSTGRES_PORT", "5432")
pg_db = os.getenv("POSTGRES_DB", pg_user)

db = f"postgresql://{pg_user}:{pg_pass}@ {pg_host}:{pg_port}/{pg_db}"
SQLALCHEMY_DATABASE_URI = os.getenv("DATABASE_URL", db)

The above is specific to configuring SQLAlchemy but this same concept applies when using Rails, Django, Phoenix or any other web framework too.

The idea is to support using POSTGRES_* env variables that match up with what the official PostgreSQL Docker image expects us to set, however the last line is interesting because it lets us pass in a DATABASE_URL which will get used instead of the individual env vars.

Now, I’m sure you know that the PostgreSQL Docker image expects us to set at least POSTGRES_USER and POSTGRES_PASSWORD in order to work but the above pattern lets us use a managed database outside of Docker in production and a locally running PostgreSQL container in development.

We can combine this with the override file pattern as well and now we get the best of both worlds. Local development with a local copy of PostgreSQL running in Docker and a managed database of your choosing in production. I went with DATABASE_URL as the name because it’s a convention that a lot of hosting providers use.

Now configuring your database is as easy as changing an env variable in your .env file.

In a similar fashion you could also define:
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")

Setting up a health check URL endpoint

@ page.get("/up")
def up():
    redis.ping()
    db.engine.execute("SELECT 1")
    return ""

A healthy application is a happy application, but seriously having a health check endpoint for your application is a wonderful idea.

It allows for you to hook up automated tools to visit this endpoint on a set interval and notify you if something abnormal happens, such as not getting an HTTP status code 200 or even notify you if it takes a really long time to get a response.

The above health check returns a 200 if it’s successful but it also makes sure the app can connect to PostgreSQL and Redis. The PostgreSQL check is nice because it’s proof your DB is up, your app can login with the correct user / password and you have at least read access to the DB. Likewise with Redis, that’s a basic connection test.

End to end that entire endpoint will likely respond in less than 1ms with most web frameworks so it won’t be a burden on your server.

I really like having a dedicated health check endpoint because you can do the most minimal work possible to get the results you want. For example, you could hit the home page of your app but if your home page performs 8 database queries and renders 50kb of HTML, that’s kind of wasteful if your health check is going to access that page every minute.

With the dedicated health check in place now you can use it with Docker Compose, Kubernetes or an external monitoring service like Uptime Robot.

I went with /up as the URL because it’s short and descriptive. In the past I’ve used /healthy but switched to /up after hearing DHH (the creator of Rails) mention that name once. He’s really good at coming up with great names!

Dockerfile

Now let’s switch gears and talk a little bit about your Dockerfile. All of concepts we’ll talk about will apply to just about any web framework. It’s really interesting at how similar most of the example apps are in terms of Dockerfile configuration.

Using Multi-stage builds to optimize image size

FROM node:14.15.5-buster-slim AS webpack

#

FROM python:3.9.2-slim-buster AS app

The Dockerfile has (2) FROM instructions in it. That’s because there’s 2 different stages. Each stage has a name, which is webpack and app.

COPY --chown=python:python --from=webpack /app/public /public

In the Node build stage we build our bundled assets into an /app/public directory, but the above line is being used in the Python build stage which copies those files to /public.

This lets us take the final assets and make them a part of the Python image without having to install Node, Webpack and 500mb+ worth of packages. We only end up with a few CSS, JS and image files. Then we can volume mount out those files so nginx can read them.

This is really nice and comes full circle with the override file pattern. Now in production we don’t need to run Webpack because our fully built assets are built into the Python image.

This is only 1 example of how you can make use of multi-stage builds.

Both the Play and Phoenix example apps demonstrate how you can make optimized production images because both tech stacks let you create jars and releases which only have the bare minimum necessary to run your app.

In the end it means we’ll have smaller images to pull and run in production.

Running your container as a non-root user

From a security perspective, it’s a good idea to not run your containers as the root user but there’s also other perks like if you happen to use volumes in development with native Linux (or WSL 2 on Windows) you can have your files owned by your dev box’s user without having to do anything too special in your Dockerfile.

I’m only going to include relevant lines to what we’re talking about btw, check out the example apps for a complete Dockerfile reference.

# These lines are important but I've commented them out to focus going over the other 3 lines.
# FROM node:14.15.5-buster-slim AS webpack
# WORKDIR /app/assets
# RUN mkdir -p /node_modules && chown node:node -R /node_modules /app

USER node

COPY --chown=node:node assets/package.json assets/*yarn* ./

RUN yarn install

The above is a snippet from the Webpack build stage of the Flask example app.

By default the official Node image creates a node user for you but it’s not switched to it by default. Using USER node will switch to it and now every instruction after that will be executed as the node user assuming the instruction supports the idea of a user.

There’s a gotcha here around using COPY too. Even if you set the user, you need to explicitly --chown the files being copied to that user. If you didn’t do that step they’ll still be owned as root:root.

What about volume mounted files in development?

Good question! You might think that volume mounted files will be owned by the node:node user and group on your dev box which likely won’t exist, so you’ll end up with errors.

But fortunately it doesn’t work exactly like that.

The node user in the Docker image has a uid:gid (user id and group id) of 1000:1000. If you’re running native Linux or are using WSL 2 on Windows, chances are your user account also has 1000:1000 as its uid:gid because that is a standard Unix convention since it’s the first user account created on the system. You can check by running id from your terminal.

Basically what this means is even though the files are owned by node:node in the image, when they’re bind mounted back to your dev box they will be owned by your dev box’s user because the 1000:1000 matches on both sides of the mount.

If you’re using macOS with Docker Desktop it’ll work too. Technically your uid:gid probably isn’t 1000:1000 on macOS but Docker Desktop will make sure the permissions work correctly. This also works if you happen to use WSL 1 along with Docker Desktop.

In production on self managed servers it also works because you’re in full control over your deploy server (likely native Linux). On fully managed servers or container orchestration platforms typically you wouldn’t be using bind mounts so it’s a non-issue.

The only place this typically doesn’t work is on CI servers because you can’t control the uid:gid of the CI user. But in practice this doesn’t end up being an issue because you can disable the mounts in CI. All of the example apps come configured with GitHub Actions and solve this problem.

If for whatever reason your set up is unique and 1000:1000 won’t work for you you can get around this by making UID and GID build arguments and pass their values into the useradd command (discussed below). We’ll talk more about build args soon.

What about other Docker images besides Node?

Some official images create a user for you, others do not. For example, in the Dockerfile for the Flask example the Python image does not create a user for you.

So I’ve created a user with useradd --create-home python.

I chose python because one pattern I detected is that most official images that create a user for you will name the user based on the image name. If the Python image ever decides to create a user in the future, it means all we would have to do is remove our useradd.

Customizing where package dependencies get installed

RUN mkdir -p /node_modules && chown node:node -R /node_modules /app

In the above case, in my .yarnrc file I’ve customized where Node packages will get installed to by adding --modules-folder /node_modules to that file.

I like this pattern a lot because it means if you yarn install something you won’t end up with a node_modules/ directory in your app’s WORKDIR, instead dependencies will be installed to /node_modules in the root of the Docker image which isn’t volume mounted. That means you won’t end up with a billion Node dependencies volume mounted out in development.

You also don’t need to worry about volume mounts potentially clobbering your installed dependencies in an image (I’ve seen this happen a number of times doing client work).

The chown node:node is important there because without it our custom /node_modules directory won’t be writeable as the node user. We also do the same for the /app directory because otherwise we’ll get permission errors when we start copying files between multi-stage builds.

This pattern isn’t limited to yarn too. You can do it with mix in Elixir and composer in PHP. For Python you can install dependencies into your user’s home directory and Ruby installs them on your system path so the same problem gets solved in a different but similar way.

Taking advantage of layer caching

COPY --chown=node:node assets/package.json assets/*yarn* ./

RUN yarn install

COPY --chown=node:node assets .

This is Docker 101 stuff but the basic idea is to copy in our package management file (package.json file in this case), install our dependencies and then copy in the rest of our files. We’re triple dipping our first COPY by copying in the yarn.lock and .yarnc files too, since they all go in the same spot and are related to installing our packages.

This lets Docker cache our dependencies into its own layer so that if we ever change our source code later but not our dependencies we don’t need to re-run yarn install and wait forever while they’re all installed again.

This pattern works with a bunch of different languages and all of the example apps do this.

Leveraging tools like PostCSS

COPY --chown=node:node hello /app/hello

This is in the Webpack stage of all of my example apps and it’s very specific to using PostCSS. If you happen to use TailwindCSS this is really important to set. The hello directory in this case is the Flask app’s name.

We need to copy the main web app’s source code into this stage so that PurgeCSS can find our HTML / JS templates so it knows what to purge and keep in the final CSS bundle.

This doesn’t bloat anything in the end because only the final assets get copied over in another build stage. Yay for multi-stage builds!

Using build arguments

ARG NODE_ENV="production"
ENV NODE_ENV="${NODE_ENV}" \
    USER="node"

RUN if [ "${NODE_ENV}" != "development" ]; then \
  yarn run build; else mkdir -p /app/public; fi

Build arguments let you do neat things like being able to run something specific during build time but only if a certain build argument value is set. They can also let you set environment variables in your image without hard coding their value.

In the above case NODE_ENV is being set as a build argument, then an env variable is being set with the value of that build arg, and finally production assets are being built in the image only when the NODE_ENV is not development.

This allows us to generate Webpack bundles in production mode but in development mode a more light weight task will run which is to mkdir that public directory.

This isn’t the only thing you can use build args for but it’s something I do in most projects. The same pattern is being used in all of the example apps.

x-app: &default-app
  build:
    context: "."
    target: "app"
    args:
      - "FLASK_ENV=${FLASK_ENV:-production}"
      - "NODE_ENV=${NODE_ENV:-production}"

The build arguments themselves are defined in the docker-compose.yml file under the build property and we’re using variable substitution to read in the values from the .env file.

This means we have a single source of truth (.env file) for these values and we never have to change the Dockerfile or docker-compose.yml file to change their value.

# Which environment is running? These should be "development" or "production".
#export FLASK_ENV=production
#export NODE_ENV=production
export FLASK_ENV=development
export NODE_ENV=development

You just update your .env file and rebuild the image.

But with great power comes great responsibility. I try to keep my Dockerfiles set up to build the same image in all environments to keep things predictable but I think for this specific use case of only generating production assets it’s a good spot to use this pattern.

However with that said, going back to file permissions if you did need to customize the UID and GID, using build arguments is a reasonable thing to do. This way you can use different values in whatever environment needs them, and you can have them both default to 1000.

Setting environment variables

ARG FLASK_ENV="production"
ENV FLASK_ENV="${FLASK_ENV}" \
    FLASK_APP="hello.app" \
    FLASK_SKIP_DOTENV="true" \
    PYTHONUNBUFFERED="true" \
    PYTHONPATH="." \
    PATH="${PATH}:/home/python/.local/bin" \
    USER="python"

If you have env variables that you know won’t change you might as well include them in your Dockerfile to eliminate the possibility of forgetting to set them in your .env file.

Typically you’ll always be setting at least the FLASK_ENV or whatever env var your web framework uses to differentiate dev vs prod mode so you’re not really taking a layer hit here since we can add multiple env vars in 1 layer.

Python specific things to be aware of:

Setting PYTHONUNBUFFERED=true is useful so that your logs will not get buffered or sent out of order. If you ever found yourself not being able to see your server’s logs in Docker Compose it’s because you likely need to set this or the equivalent var in your language of choice.

I’ve only ever needed to set it for Python. Ruby, Elixir, Node, Scala and PHP work fine without it. That’s all I tried but it might be necessary with other languages.

Setting PYTHONPATH="." is useful too. Certain packages may expect this to be set, and using . will set it to the WORKDIR which is almost always what you’d want to make your app work.

Updating your PATH:

Setting the PATH is a reasonable idea too because if you start running your containers as a non-root user and install your packages in your user’s home directory or a custom location you won’t be able to access binaries directly.

For example without setting that, we wouldn’t be able to run gunicorn without supplying the full path to where it exists which would be /home/python/.local/bin/gunicorn in our case.

Packages end up in the home directory of the python user because in the Dockerfile when I pip3 install the packages it’s being done with the --user flag. Check out the Flask or Django example app to see how that’s done.

Unix conventions:

RUN useradd --create-home python

ENV USER="python"

Certain Unix tools may expect a home directory to exist as well as having the USER environment variable set.

They are freebies for us so we might as well set them. The alternative is to wake up one day battling some crazy edge case because a tool you decided to install tries to install something to your user’s home directory which doesn’t exist. No thanks!

Setting EXPOSE for informational purposes:

EXPOSE 8000

This is in the web app stage for each example app. Technically you don’t need to set this but it’s considered a best practice because it lets you know which port your process is running on inside of the container.

You can see this port when you run docker container ls:

CONTAINER ID   IMAGE                  PORTS                                    
5f7e00e36b8e   helloflask_web         0.0.0.0:8000->8000/tcp, :::8000->8000/tcp

Since we’re publishing a port we can see both the published and exposed port, but if we didn’t publish the port in our docker-compose.yml file the PORTS column would be totally empty if we didn’t set EXPOSE 8000 in our Dockerfile.

I’ve written a more detailed post on expose vs publish in Docker Tip #59.

Array (Exec) vs String (Shell) CMD syntax:

# Good (preferred).
CMD ["gunicorn", "-c", "python:config.gunicorn", "hello.app:create_app()"]

# Bad.
CMD gunicorn -c "python:config.gunicorn" "hello.app:create_app()"

Both options will run gunicorn but there’s a pretty big difference between the 2 variants.

The first one is the array (exec) syntax and it will directly run gunicorn as PID 1 inside of your container. The second one is the string (shell) syntax and it will run your shell as PID 1 (such as /bin/sh -c "...") in your container.

Running ps in your container to check its PID:
# The output of `ps` when you use the array (exec) variant:
PID   USER     COMMAND
  1   python   /usr/local/bin/python gunicorn -c python:config.gunicorn hello.app:create_app()

# The output of `ps` when you use the string (shell) variant:
PID   USER     COMMAND
  1   python   /bin/sh -c gunicorn -c "python:config.gunicorn" "hello.app:create_app()"

I’ve truncated the gunicorn paths so it fits on 1 line. Normally you would see the full path being listed out, such as /home/python/.local/bin/gunicorn instead of gunicorn.

But notice what’s running as PID 1.

The array version is preferred and even recommended by Docker. It’s better because the shell variant will not pass Unix signals such as SIGTERM, etc. back to your process (gunicorn) correctly. The preferred array variant also avoids having to run a shell process.

Of course that comes with the downside of not being able to use shell scripting in your CMD such as wanting to use && but that’s not a big deal in the end. If you’re doing complicated shell scripting in your CMD you should likely reach for using an ENTRYPOINT script instead.

It’s also worth pointing out that when you set the command property in docker-compose.yml it will automatically convert the string syntax into the array syntax.

ENTRYPOINT script

COPY --chown=python:python bin/ ./bin

ENTRYPOINT ["/app/bin/docker-entrypoint-web"]

Before we can execute our ENTRYPOINT script we need to copy it in. This is taken from the Flask example app but this pattern is used in every example app.

You can technically COPY this script anywhere but I like to keep it in a bin/ directory.

#!/bin/bash

set -e

# This will clean up old md5 digested files since they are volume persisted.
# If you want to persist older versions of any of these files to avoid breaking
# external links outside of your domain then feel free remove this line.
rm -rf public/css public/js public/fonts public/images

# Always keep this here as it ensures the built and digested assets get copied
# into the correct location. This avoids them getting clobbered by any volumes.
cp -a /public /app

exec "$@ "

This is the ENTRYPOINT script itself.

This takes care of copying the bundled assets from /public to /app/public, but unlike building an image this runs every time the container starts.

The basic idea of how all of this comes together:
  • We build and bundle our assets in the Webpack stage (done at build time)
  • They get copied to /public in the Python build stage (done at build time)
  • When the container starts it copies /public to a volume mounted directory (done here)

That volume mounted directory is /app/public and it’s what ends up being set as the nginx root. This song and dance lets us persist our assets to disk and even if we decided to save user uploaded files to disk we wouldn’t lose them when the container stops.

The first part of the script cleans up old md5 tagged assets. This idea of pre-compiling or digesting assets is common in Flask, Django, Rails, Phoenix, Laravel and other frameworks. The first command will delete the old assets, but of course if you wanted to keep them around you could comment out that line.

Taking a peek at the docker-compose.yml file again:
worker:
  <<: *default-app
  command: celery -A "hello.app.celery_app" worker -l "${CELERY_LOG_LEVEL:-info}"
  entrypoint: []

Since the worker service isn’t running a web server it doesn’t make sense to do this copy operation twice so we set an empty entrypoint.

Without setting that I was getting a race condition error in the cp command because it was trying to copy files from 2 different sources very quickly because both the web and worker services are sharing that volume.

What else can you use ENTRYPOINT scripts for?

This isn’t the only thing you can use ENTRYPOINT scripts for. Basically if you want something to run every time your container starts then using an ENTRYPOINT script is the way to go, but you should think carefully about using one.

For example, if it’s a deterministic task you may want to consider putting it into a RUN instruction in your Dockerfile so it happens at build time (such as installing dependencies). This will help with creating portable and repeatable Docker images too.

The above ENTRYPOINT script runs in a few hundred milliseconds and it’s only necessary because volume mounts aren’t something you do at build time.

Git and Docker Ignore Files

# .gitignore
.webpack_cache/
public/*
!public/.keep

.env*
!.env.example
docker-compose.override.yml

# There's more files associated to Python / Node / etc, ommitting for brevity.

When it comes to committing code to version control, if you’re using Webpack or another build tool then chances are you’ll want to ignore your public/ directory or whatever your destination directory is.

However, if you follow the example apps we’ll want to make sure the public/ directory always exists so that’s why there’s a .keep file. This is necessary to ensure that the directory ends up being owned by the correct user. If we didn’t do this then Docker would end up creating the directory and it would be owned by root:root and we’d get permission errors.

Also, as we went over before we’ll ignore our real override file so we can control which containers get run in production.

As for the .env file, we’ll ignore all of them except for our example file. The reason I ignore all of them is because depending on how you do configuration management you might scp a .env.prod file over to your production server (as .env on the server). That means you’ll end up having multiple .env.X files in your repo, all of which should be ignored.

# .dockerignore
.git/
.pytest_cache/
.webpack_cache/
__pycache__/
assets/node_modules/
public/

.coverage
.dockerignore
.env*
!.env.example
celerybeat-schedule
docker-compose.override.yml

It’s a good idea to keep your images small and tidy and a .dockerignore file helps us to do that. For example, we don’t need to copy in our entire .git/ directory so let’s ignore that.

This file will vary depending on what tech stack you use and all of the example apps have both files ready to go.

But one general takeaway is to remove unnecessary files and don’t forget to ignore all of your .env files because you wouldn’t want to copy sensitive files into your image because now if you pushed your image to a Docker registry now that registry has access to your sensitive information since it’s in your image.

Instead it will be expected you transfer your .env file over to your server and let Docker Compose make it available with the env_file property. How you get this .env file onto your server is up to you, that could be scp or maybe using something like Ansible.

Closing Thoughts

I think in general best practices are a moving target and they will change over time. I fully expect discovering new things over time and tweaking my set up as Docker introduces new features and I improve my skills.

By the way, I have a new course coming out focused on deploying web apps with Terraform, Ansible and Docker Compose. You can learn more about it here.

So that’s about it. If you made it to the end thanks a lot for reading it!

What are some of your Docker best practices and tips? Let us know below.