A Definitive Guide on Docker Secrets

Every application needs a robust security mechanism to ensure the safety and proper handling of its sensitive data. Managing such data is challenging in modern applications, especially in containerized environments. Hard-coding sensitive data or using environment variables can create more vulnerabilities. Docker Secrets offer a more reliable and secure way to store and manage sensitive data, even when using a large number of containers.

This article explores the concept of Docker Secrets, their key features and benefits, how they work, and how to use them for effective data management in containerized applications.

1. What are Docker Secrets?

Docker Secrets is a feature providing robust ways to store and manage your sensitive app data securely in Docker containers in Docker swarm mode. It encrypts data both at rest and in transit and can only be accessed by authorized services. This option is much more secure than alternatives like embedding secrets directly into Docker images or storing them in environment variables. 

A docker secret command is used to manage secrets. By default, the sensitive information is saved as a file in the /run/secrets/ directory inside by default in Docker. They are typically managed as secret files, often created from a local file containing sensitive information, such as a text file with a password or API key. Here are a few types of secrets that Docker Secrets are designed to manage and protect: 

• Encryption keys used for encrypting sensitive files and data.
• API keys that provide access to external services and APIs. 
• Tokens used for authentication and authorization purposes. 
• Credentials like passwords and usernames. 
• Software licenses containing sensitive information. 
• Digital certificates for authentication and secure communication.

1.1 Key Features of Docker Secrets

Let us now briefly understand the features of Docker Secrets: 

• Secret Rotation: Secrets are identified by a unique identifier, allowing them to be rotated without any downtime. 
• Encryption: Docker Secrets or sensitive information, are encrypted both at rest and in transit. 
• Runtime Only Access: Docker containers mount secrets by default in the /run/secrets directory. When a container stops, the secrets are unmounted automatically. 
• Native Integrations: Docker Secrets supports native integration with Docker Swarm and Docker Compose.

1.2 Why Use Docker Secrets?

There are many reasons for using Secrets. Here are a few showcasing its importance: 

1. Improved Security

Docker Secrets are considered more secure than traditional methods. It stores sensitive information in a Docker-managed virtual filesystem, which is only accessible to containers explicitly granted access. Docker Secrets provide a secure way to manage sensitive data like passwords, API keys, and certificates within Docker environments. Additionally, they are encrypted both at rest and in transit. 

2. Ease of Management

Having a single storage location on the Docker host provides centralized control over sensitive information. You can easily track it and allow containers running on the host to access this information. So, even when using multiple containers with Docker Compose or Docker Swarm, managing secrets across all containers becomes straightforward. The Docker CLI also allows easier management of secrets. 

3. Reduced Exposure

Docker strategically reduces the risks of breaches and accidental leaks by not exposing secrets in logs or the environment. The exposure of Docker Secrets is kept to a minimum and limited only to explicitly authorized services.

4. Abstraction between Secrets and Containers

When using multiple environments, the containers typically require specific credentials for each environment. However, that is not the case when using Docker Secrets. Instead, they use a secret name consistently across all environments and abstract the underlying details. This abstraction provides the flexibility to deploy the same container image across multiple environments without making any changes.

1.3 How Do Docker Secrets Work?

Docker Secrets only work with Docker Swarm mode. They are not natively supported in standalone containers or basic Docker Compose setups without Swarm. While adding secrets in Docker Swarm, Docker ensures a secure connection to a Swarm manager using a mutual Transport Layer Security (TLS). It uses an encrypted log called Raft log to store the secrets. Docker Swarm secrets are a secure way to manage sensitive data in containerized applications deployed in a Swarm cluster.

The Raft log is replicated along with the secrets across every manager in the Swarm. So, if one manager fails, the secrets and management data remain available. When a service needs to use sensitive information, such as credentials or API keys, Docker retrieves the encrypted secret from the Raft log, decrypts it, and mounts it as a read-only file in the container’s memory. 

Only the container of the service can access this special file system. Linux containers place the secret at /run/secrets/<secret_name> by default, while Windows containers mount it at C:/ProgramData/Docker/secrets. You have the flexibility to choose a different location. When mounting secrets, the secret’s metadata and access permissions are tightly controlled and available to authorized services. When rotating secrets, the target option can be used to remount the new secret to the original path within the container.

Swarm runs the Docker containers, but its nodes will have access to encrypted secrets only if they are performing a task for the service and have been granted explicit access to those secrets or if they are Swarm managers. Docker’s built-in secrets management is not compatible with standalone containers created using plain ‘docker run’, so alternative methods are needed for those containers.

Any access to secrets is removed from the in-memory file system of the container as soon as the container stops running. Access is also cleared from the memory of the node. If a node running a task has access to a secret but loses connection with the Swarm, the task retains access but will be barred from receiving any updates until it reconnects with the Swarm.

2. Creating Docker Secrets

Using Secrets is possible as long as you are working with a collection of containers. Being an inherent feature of the Docker Swarm, it is easy to initialize Secrets in the Swarm. But you can also create secrets through the Docker CLI or Docker Compose files. Let’s go through each process one by one. 

2.1 Using Docker Secrets & Docker Swarm

While running containerized services, Docker Swarm and Docker Secrets work together to ensure the secure management of secrets. Secrets provides a secure mechanism to save and retrieve sensitive information from the system without exposing it. They uses a unique key to encrypt the data both at rest and in transit. 

Meanwhile, Docker Swarm is a robust tool capable of handling a cluster of nodes for distributed apps. It can easily deploy containerized apps at a large scale. Managing multiple nodes within a cluster becomes easy with Docker Swarm, which automatically distributes the workloads among the nodes, allowing resources to remain available even during peak usage times or unexpected traffic spikes. 

Together, these tools offer an effective way to handle your sensitive information amid continuously growing security requirements. Let’s see how you can create and manage a secret in the Docker Swarm using specific Docker commands.  

Before creating a service, we need to initialize the Docker Swarm:

docker swarm init

After initializing the service, create the secret using the

docker secret create

ssh-keygen -t rsa -b 4096 -N "" -f git_deploy_key
docker secret create git_ssh_private_key git_deploy_key
rm git_deploy_key

In the above code, we created an ssh private key using the ssh-keygen command. This private key, named git_deploy_key, can be used to access private git repositories in deployment workflows. Then we created a docker secret with the docker secret create command. Finally, we deleted the original key in order to prevent security vulnerabilities. 

Now, confirm whether the creation of a secret is successful by running the docker secret ls command, which displays a list of available secrets, including their IDs, names, and creation dates:

docker secret ls

You can also inspect a secret’s metadata without exposing its contents using the docker secret inspect command.

Now, this secret can be used in a Docker container. Start by passing it along with a –secret flag while creating a service.

docker service  create --name git-cloner --secret git_ssh_private_key alpine

Now, the above command creates a Docker Swarm service named git-cloner and securely mounts the git_ssh_private_key secret into the container.

When rotating a secret, you create a new secret and replace the old secret in the service configuration to ensure the container uses the updated version.

This secret can also be passed to the docker-compose.yml file.

version: '3.7'
services:
  gitagent:
    image: alpine
    secrets:
      - git_ssh_private_key
secrets:
  git_ssh_private_key:
    external: true

In the above example for the compose file, A secret named git_ssh_private_key is defined in the secrets section, which is required for the gitagent service. By default, Docker mounts it at /run/secrets/git_ssh_private_key in the container as a file.

2.2 Using Docker Compose

Docker Compose is widely utilized to define and run multi-container Docker-based apps. A docker-compose.yaml file helps define the stack, which allows you to configure the services that make up your application. It also includes volumes, networks, ports, and environment variables. Setting up an app in a single configuration file and quickly deploying it across multiple environments is easy with Docker Compose. 

However, using an environment variable to store sensitive data, such as passwords, is less secure than using Docker Secrets because environment variables are stored in plaintext and can be accidentally exposed. Docker Compose offers a solution to handle sensitive information like passwords and encryption keys securely. Secrets can be read from an external file, but make sure not to commit this file along with your code. 

The following example provides a step-by-step illustration of configuring Docker Compose secrets to securely set a root password for a MySQL container:

version: '3.7'
services:
  payment-service:
    image: myapp:latest
    secrets:
      - stripe_api_key
secrets:
  stripe_api_key:
    file: ./stripe_api_key.txt
t

This example demonstrates how to securely configure an API key to the payment-service container using Docker Compose. This is especially important when working with sensitive credentials like payment provider keys. When you run Docker Compose, secrets are mounted to the /run/secrets/ directory inside the container, making them more secure than environment variables.

Note: Make sure the secret file (stripe_api_key.txt) is added to .gitignore and also excluded from version control to prevent accidental exposure.

2.3 Using the Docker CLI to Manage Secrets

Secrets allows users to create secrets and control which services have access to them and when they can use those secrets. Let’s consider an example where you create a MySQL service consisting of a root password. Add their credentials as secrets. 

Docker Secrets is a built-in feature of Docker Swarm services and is available only in this mode. Therefore, to use secrets, you must first set up a Docker Swarm. The first step to take is to install Docker CLI and then initialize or join the swarm. 

All these commands are executed from the command line interface to manage secrets securely.

docker swarm init

Example output:

Swarm initialized: current node (abc123……) is now a manager.

This command turns your current machine into a manager node and gives you a join command with a token. 

To add another machine as a worker node, run the following (on the new machine):

docker swarm join --token <worker-token> <manager-ip>:2377

Here <worker-token> is a secure key that Docker gives you after running swarm init and <manager-ip> is the IP address of the manager node (your first machine).

To add the secret to Docker, create a text file in the location where you want to save the secret. Running the command given below would create a strapi-jwt.txt file containing the super-secret-strapi-jwt string.

$ echo "super-secret-strapi-jwt" > $HOME/strapi-jwt.txt

Now, use the file we created above and the Docker secret command to generate a new secret.

$ docker secret create strapi-jwt-secret $HOME/strapi-jwt.txt

l1m5jvgcox1l96i6bjbz4dvnt

The above-mentioned string doesn’t represent the secret itself but the secret ID. So, it might differ in your environment. There is another way to use the Docker secret create command. Set the last argument to delete the file. This represents the file to the minus symbol.

$ echo "another-strapi-jwt-secret" | docker secret create strapi-jwt-inline -

Running the above command will give the following output:

q3bjo7wx4xqr45ma20lk5j2g7

In this case, the secret is read directly from the terminal input, and no file is saved on disk, which is more secure in many automation or CI/CD workflows. 

After creating and listing secrets, you can inspect a secret’s metadata in JSON format using the ‘docker secret inspect’ command. This displays details and configuration without exposing the secret’s contents.

3. Docker Secrets Best Practices

Manage your sensitive data efficiently and securely by following the best practices when using Docker Secrets. 

• Use Secrets for Sensitive Data Only: Store only sensitive information like credentials, encryption keys, API keys, passwords, digital licenses, etc, as Docker Secrets. Using non-sensitive information increases unnecessary complexities. 
• Avoid Hard-coding Secrets: Hard-coding secrets in app configuration files or source code may seem convenient, but it’s a bad practice. Instead, use Docker Secrets to store sensitive information and reference them in the app. 
• Immutable Secrets: Treat all secrets as immutable. When you need to update a secret, create a new one and then update the service to use the new secret. Avoid updating the existing secrets. Creating new versions of secrets instead of updating the existing ones helps easily track the changes and revert to the previous versions whenever you want.  
• Never Store Secrets in Version Control Systems: Don’t save secrets in a version control system. Use secrets management systems like Docker Secrets or ensure secrets files like .gitignore to securely handle. 
• Regular Rotation of Secrets: Implement a routine to rotate your secrets regularly. Secrets doesn’t have any built-in automatic rotation, so to ensure security, you have to keep changing the secrets frequently. 
• Access Control Management: Limit access to secrets by implementing strict access controls. Make sure only essential services have access to secrets. When creating a service, define clearly whether it would need access to the secret and under what specific circumstances. 
• Monitoring Secrets Usage: Docker Secrets lacks built-in audit logging functionality. So, you need to monitor secrets at the service level using the following code:

4. Conclusion

Docker Secrets is a crucial feature that provides a secure and simple way to manage sensitive information. By understanding how it works, along with the do’s and don’ts of using it, your team can significantly enhance the security of your containerized application and its data.