Secret Management the Easy Way with Aqua CSP

Secrets management in the context of PaaS is one of the most important topics and a primary objective to guarantee. One use case leads us to answer the question: How to manage and securely provide secrets to each container at runtime? Before landing on this discussion, let us underline the problem encounter with secrets in PaaS firstly.   

Problem with secrets in PaaS  

If you are using containers, the probability that you already meet Docker or Kubernetes is higher. In effect, Kubernetes and Docker, besides other container solutions, provide mechanisms to pass a secret to a container. Reading the Docker documentation (https://docs.docker.com/engine/swarm/secrets/), you will find some methods propose by docker to manage your sensitive data using docker secrets. Moreover, Kubernetes offers a similar approach to deal with secrets using Kubernetes Secrets. Still, the problems with the Built-in Secrets Mechanism in Kubernetes are known; there are also some recommendations of Kubernetes here (https://kubernetes.io/docs/concepts/configuration/secret/#security-properties).   

One can write a lot about the problems encountered by secrets in PaaS. Aqua Security’ Amir Jerbi (https://blog.aquasec.com/managing-secrets-in-docker-containers) covers some Problem encountered with Secret Management. Here we provide some challenges by managing secrets in Containers. 

• Managing which containers have access to which secrets:

Secrets should only be accessible to the containers that need them – One can do this at the container level. In this case, one needs a mechanism that will deliver or map secrets to the containers that need the secrets and specify which container is using which secret. 

• When should you retrieve the secret from the Vault? Assume you use Vault

As underlined in Kubernetes or Docker’s documentation recommendations, putting the secret in the container images exposes it to many more users and processes than is needed. That can lead to misuse of the secret. So, find the right method to connect the container, and the secret is relevant. Generally, it is better for this when the container runs and not before. 

• How to store the secret and retrieve it?:

It is essential to keep in mind that the container must have access to the secret when running, but you don’t have to store the secret on disk or expose it at the host level. Following some security recommendations, one should retrieve the secret from memory giving access to the secret only to the appropriate container. Assure that if the container stopped, the secret disappears.   

• How to handle secret rotations?

Sometimes one can change the secret over time. So, considering an example of a database password, we need to change the password to avoid or reduce risk exposure. To accomplish that rotation, one needs a system that can update all containers currently using the new secret’ value securely and smoothly. 

Now we are aware of the secrets’ problems in PaaS. If you want to know more about the Built-in Secrets Mechanism issues in Kubernetes, Aqua Security’ Rani Osnat  https://blog.aquasec.com/managing-kubernetes-secrets#tools underline this aspect.  

As specified in the title of this document, there is a possibility to solve the problems that one can encounter by managing secrets and containers.  Some secrets management systems are already present on the market and propose some measures to deal with secrets and containers. Let’s consider the following problems. 

Problem Statement 

• A running container should be able to access precisely the secrets that belong to itself. 
• A container that starts in the PaaS environment should uniquely access the key/secret that it claims. 
• The rotation of a secret should not affect a running container 
• How to attribute a secret x easily to a specific container during the running time. 

These problems are similar to those presented precedent. One can also find a lot of countermeasures to address those problems. However, dealing, for example, with more than 2000 containers leads to limitations because of the manual work. At evoila GmbH, we used one of the most reliable container security platforms (https://www.aquasec.com/): Aqua CSP to address those problems (https://www.aquasec.com/). 

Aqua CSP 

Managing secrets with Aqua is relatively more straightforward. Aqua doesn’t replace your Secrets Keystore Management, although it can. Instead, Aqua CSP acts as Man-in-the-Middle (MitM) between your Secrets Keystore Management solution and your PaaS environment. The following picture illustrates the way Aqua operates with your KeyStore; Here HashiCorp Vault.  

One can integrate Aqua with secret management tools such as Amazon KMS, HashiCorp Vault, Azure Key Vault, and CyberArk Enterprise Password Vault. If you don’t have an existing secrets store, Aqua provides its encrypted database for storing secrets.   

Here we provide an example of HashiCorp Vault Integration. 

Aqua provides ways to improve the security and usability of secrets in Kubernetes (https://blog.aquasec.com/managing-kubernetes-secrets#tools) (PaaS) 

1. Encryption at Rest: Secrets are encrypted at rest through third-party storage. Access to Secrets is provided using access tokens or IAM roles of the current host. 

 2. No Persistence: Aqua securely delivers secrets to containers, encrypted at rest, loading them in memory with no persistence on disk. They are only visible to the container that needs them. 

3. Access Control: You can control user access to secrets (which may be integrated with Active Directory) and group together containers with similar security features. 

4. Write Only: Once you create a secret, it cannot be seen via the web interface or the API. You can update the content of the secret or delete it, but not read it. 

5. Rotation: When you modify a secret, the new value is updated in real-time on the running container.  You do not have to restart the container for the changes in secrets to propagate. 

6. Revocation: When the Vault revokes a secret, Aqua removes those secrets from the containers that use them, with no need to restart the containers. 

7. SSH Encryption: Aqua enforces mutually authenticated SSH encrypted communications within a Kubernetes cluster. The Enforcer uses its public key to verify the Gateway, and the Enforcer makes itself known via an authentication token. 

8. Administration and usage audit: Each administrative access event to a secret is logged, and you can see which containers had access to which secrets during runtime.   

The possibilities offered by Aqua CSP solves the problem underlined at the beginning 

Aqua CSP – Approach 

Aqua CSP offers different possibilities to solve the mentioned problems. 

• One can set a LABEL on each image created with docker and authorize that only image/container with that LABEL should start and access the secret. 
• Another way to solve the problem is to use Aqua’s services, create User Access Control, and access the secret through the resources. 
• The third way to approach the problem is to use LABELs. Aqua CSP gives us the ability to use LABELs to identify specific resources and link particular resources in our environment to a LABEL.   

In the third approach, we can manage more than 2000 Secrets easily by using LABEL. We are not going to do it per hand. Aqua CSP proposes a fully functional API that can be used to automate the Secrets’   attribution process. One of the important ways to underline here is the LABEL’s use that makes that process more straightforward. 

Here is an illustration of secrets attributions using labels prod. That means one can also change the group (depending on your environment) of a secret only through the Labels. 

Finally, attributing a secret to a running container is possible, and Aqua CSP covers quickly that tasks and more. 

Summary 

Based on the limitations encountered by the Built-in Secret’s management tools in PaaS, Aqua CSP found a way to solve the most used market solutions’ inherent imperfection. We illustrated that a smooth approach could lead to more efficient and robust solutions. If you are interested in more use cases to cover with Aqua CSP, have a look at our Container Security Website at evoila.com.