Protecting your keys in public Docker containers
Last updated
Last updated
Access to the mirrors and caches in StableBuild are limited through API keys which are listed directly in your Dockerfile. For example, here your API key is both used in FROM and set as an ARG which is used by sb-apt.sh to authenticate with the package registry.
It's important to keep these keys private, as these keys are used to bill your account for traffic. If you see weird traffic spikes, check the raw usage logs to see if a key might be leaked. You can rotate keys via Dashboard > Keys.
The approach to list keys in your Dockerfile is fine if you build Docker containers for internal use, but problematic if you want to publish your Docker containers publicly (in Docker Hub, or another public registry). Anyone pulling the container can see the layers that make up the container, and your API key will be listed in there. In addition, the build can write files to your container containing your API key. For instance: in the example above the key can be seen both when you inspect the layers, and its persisted on file system.
To view this, first build the container above via:
And then inspect the layers via dive to see that the key is present:
You can also find the key on the file system. Run the container with an interactive shell via:
And you can find your key persisted on disk via:
Let's protect these keys. First, our base image. The FROM line:
Is not leaked to any layers (visible via dive) or on the file system; but is in the image metadata. To fix this you can create a multi-stage Docker image instead:
Next, let's tackle the key leaking through the layers:
Instead of hard-coding the key you can use Docker build secrets. Secrets are mounted in at build time, and are not included in the final image. Here you can use:
If you build this container via:
And then inspect the image again via dive:
The hard-coded key is gone:
To avoid leaking the keys through the file system, you'll need to clean up any file where the key is persisted in the same step as where you use the key - otherwise someone can inspect the file system at a specific layer and recover your key that way.
From inspecting the file system earlier we found that the key is listed in /etc/apt/sources.list (written by sb-apt.sh) and that the key is present in the file names of files at /var/lib/apt/lists/ (apt cache files). We can rewrite our Dockerfile to:
When you build this container via:
And then run the container and search the file system, there are no results anymore:
As a final check we can export the full image (including all layers and metadata) to disk, and do a final scan for our API key:
Great 🎉! Your container now no longer contains your key, and you can safely push this container to Docker Hub or another public registry.
The above works for the Ubuntu package registry, but if you use other mirrors or caches this will require some manual work in inspecting the layers and file system to ensure no keys are leaked. Both dive and just opening an interactive shell into the container should make this relatively painless though.
If you use Kaniko to build your containers then build secrets are not available. However, you can use the fact that the /kaniko folder on your file system is shared between the Kaniko container and the build process. So you can use this in your Dockerfile:
Here, the target /kaniko/sb-api-key.txt is on a shared file system, so you can just write the key to this location inside the Kaniko container before calling /kaniko/executor:
You can do this f.e. by setting the entrypoint of the gcr.io/kaniko-project/executor container to sh, and the arguments to [ "-c", "echo -n 'stablebuild-my-secret-api-key' > /kaniko/sb-api-key.txt && /kaniko/executor --dockerfile=./Dockerfile" ].
dive)
