Detailed Guide to Securely Manage Secrets for Kubernetes Using Hashicorp Vault

August 22, 2022 | Comments(0) |

TABLE OF CONTENT

1. Overview
2. What is a Secret
3. What is a HashiCorp Vault?
4. Deployment Diagram
5. How Hashicorp Vault solves real-life problems
6. Prerequisites
7. Kubernetes Components
8. Step-by-Step Guide
9. Conclusion
10. About CloudThat
11. FAQs

 

Overview

This blog gives a detailed deep dive into Hashi-Corp Vault, its integration with the Kubernetes cluster, and its use cases.

HashiCorp Vault is the newest feature that HashiCorp launched to address the management issue and securely inject secrets into a Kubernetes cluster. It is a valuable tool, especially when dealing with entities like databases that need credentials inserted into the Frontend pod in the Kubernetes cluster.

The traditional way to secure the credentials inside Kubernetes is by creating a Kubernetes secret object and embedding a database or other credentials into it. These Secrets encoded in the Kubernetes secret object were in base64 format, which could be easily decoded by anyone using the base64 encryption data.

What is a Secret?

Secrets are the credentials we use to authenticate and authorize, which need to be secured.

What is a HashiCorp Vault?

HashiCorp Vault is a tool that manages secrets that are specifically constructed to control confidential data in any environment. It can be used to store sensitive values and, at the same time, dynamically generate access for specific services/applications on lease.

Deployment Diagram

Hashicorp Vault

How Hashicorp vault solves real-life problems

In Vault implementation, we use the Kubernetes service account.

You may wonder how the vault agent injects the secrets into the pod from the vault server.”?

There are two ways to inject the traffic into the pod.

Static approach:

Step 1: Admin will insert credentials in the vault manually in a key-value format

Step 2: A service account needs to be created and attached to the pod. The same service account needs to be defined in the Hashi-Corp vault.

Step 3: Vault will fetch the credentials and mount it into the pod Automatically

  • Dynamic approach:

Step 1: Vault will generate and fetch the temporary credentials from the database.

Step 2: The pod authenticates to the vault using the service account.

Step 3: The vault agent will mount the credentials into the pod automatically.

The pod can use this secret for the life cycle of the pod.

If the pod gets deleted, the vault will automatically go back to the database and clean up the temporary credentials it created.

Therefore, any operational team does not have to maintain the database user.

Prerequisites

  • An AWS account
  • Kubernetes knowledge
  • Helm chart
  • EKS cluster

Kubernetes Components

  • Deployment
  • Stateful set
  • Services
  • Secrets
  • Volumes

Step-by-Step Guide

Here we are going to Perform the Static approach to inject the secrets into pods.

Step 1: Create one EKS cluster in AWS, or you can create a Kubeadm cluster.

https://blog.cloudthat.com/deploying-wordpress-application-kubernetes-cluster-using-kubeadm-tool-aws-ec2-instances/#overview

Step 2: Install Helm.

1: Run the wget command with the tar file provided for the lab to install Helm.

2: Run the ls command to ensure the tar file is downloaded.

3: Run the tar -xzf, and ls -l commands to extract and view the linux-amd64 folder.

4: Run the mv command to move the helm executable folder to the /usr/local/bin/ directory.

Step 3: Deploy Vault Helm Chart with the below code.

Now type the below command to check whether the pods are provisioned or not.

Hashicorp Vault

Step 4:  Unseal the Vault.

1: Exec into the vault using the below command

Type the below command to initialize the vault. It gives the unseal keys and a token for login.

Hashicorp Vault

Use the unseal key to unseal the vault. Type the below command thrice and provide the different unseal keys each time.

Hashicorp Vault

Provide the vault token after typing the below command for logging into the vault.

Step 4:  Set a secret in the vault

Exec into the vault to set a secrets

Hashicorp Vault

Step 5: Enable the Kubernetes authentication method.

Hashicorp Vault

Step 6:  Create a secret at path internal/database/config with a username and password.

Hashicorp Vault

Check whether the data was inserted into the vault or not.

Hashicorp Vault

Step 6: Configure Kubernetes authentication

Vault accepts a service token from any client in the Kubernetes cluster. During authentication, the vault verifies that the service account token is valid by querying a token review Kubernetes endpoint.

Hashicorp Vault

Configure the Kubernetes authentication method to use the location of the Kubernetes API.

Hashicorp Vault

For the client, to read the secret data defined at path “internal/database/config”, read capability is required for the path “internal/data/database/config”. This is an example of a policy. A policy defines a set of capabilities.

Write out the ” internal-app ” policy that enables the “read” capability for secrets at path = internal/data/database/config.

Hashicorp Vault

Create a Kubernetes authentication role named internal-app

Hashicorp Vault

Step 7: Define a Kubernetes service account

The Vault Kubernetes authentication role defined a Kubernetes service account named internal-app.

Get all the service accounts in the default namespace.

Hashicorp Vault

Step 8: Launch a frontend application

  • Nano wordpress.yaml

  • Apply the file
  • Kubectl apply –f yaml

Hashicorp Vault

Verify that no secrets are written to the WordPress container in the website pod.

Hashicorp Vault

The Vault Agent Injector only modifies a deployment if it contains a specific set of annotations. An existing deployment may have its definition patched to include the necessary annotations.

  • vi pathwebsite.yaml

spec:

These annotations define a partial structure of the deployment schema and are prefixed with vault.hashicorp.com.

  • agent-inject : – It enables the vault agent injector service
  • role : – it is the Role created in the vault for vault Kubernetes authentication.
  • agent-inject-secret-FILEPATH prefixes the file’s path, database-config.txt written to the /vault/secrets The value is the path to the secret defined in the vault.
  • kubectl patch deployment website –patch “$(cat patchwebsite.yaml)”

Hashicorp Vault

Verify that secrets are written to the WordPress container in the website pod using the code below.

Next, you can verify that secrets are written to the WordPress container in the website pod using the code below.

Hashicorp Vault

The unformatted secret data is present on the container:

Step 9: Apply a template to the injected secrets

The structure of the injected secrets may need to be structured in a way for an application to use. Before writing the secrets to the file system a template can structure the data. To apply this template a new set of annotations need to be applied.

Display the annotations file that has a template definition.

  • vi patch.yaml

spec:

  • kubectl patch deployment website –patch “$(cat patch.yaml)”

Hashicorp Vault

Verify that secrets are written to the WordPress container in the website pod by using the below code, it will give output in a well-formatted way.

  • kubectl exec \

    $(kubectl get pod -l app=website  -o jsonpath=”{.items[0].metadata.name}”) \

    –container wordpress — cat /vault/secrets/database-config.txt

Hashicorp Vault

Now we got the secrets in a much more formatted way.

Successfully injected the PostgreSQL secrets from the vault to the website pods.

Conclusion

Nowadays, the Hashicorp vault is widely used in many environments as it is very secure in terms of login. Whenever the vault starts, it will be in a sealed state. We need to unseal it as a Shamir seal-type login access. Vault divides the login key into five parts. Out of the five keys, at least three are needed to unseal the vault.

About CloudThat

CloudThat is the official AWS (Amazon Web Services) Advanced Consulting Partner, Microsoft Gold Partner, Google Cloud Partner, and Training Partner helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Explore our consulting here.

If you have any queries regarding Hashicorp Vault, Kubernetes, or managing secrets, drop a line in the comments section. I will get back to you at the earliest. 

FAQs

1: Do Vault secrets bound to the namespaces?

Ans: Yes. The Vault secrets are bound to the namespaces.

Pods should run in a namespace defined in the Vault Kubernetes authentication role. In that case, the pod can access the secrets defined on that path.

In the above Process, we have given the default namespace in the vault Kubernetes authentication.

2: Why is the Secrets component in Kubernetes considered not very safe?

Ans: secrets encode data in base64 format. Anyone with the base64 encoded secret can easily decode it. As such the secrets can be considered as not very safe. Credentials might be leaked when it moves across the system.


Leave a Reply