first commit
This commit is contained in:
285
secrets/sealed-secrets/README.md
Normal file
285
secrets/sealed-secrets/README.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Introduction to Sealed Secrets
|
||||
|
||||
Checkout the [Sealed Secrets GitHub Repo](https://github.com/bitnami-labs/sealed-secrets) </br>
|
||||
|
||||
There are a number of use-cases where this is a really great concept. </br>
|
||||
|
||||
1) GitOps - Storing your YAML manifests in Git and using GitOps tools to sync the manifests to your clusters (For example Flux and ArgoCD!)
|
||||
|
||||
2) Giving a team access to secrets without revealing the secret material.
|
||||
|
||||
developer: "I want to confirm my deployed secret value is X in the cluster" </br>
|
||||
|
||||
developer can compare `sealedSecret` YAML in Git, with the `sealedSecret` in the cluster and confirm the value is the same. </br>
|
||||
|
||||
## Create a kubernetes cluster
|
||||
|
||||
In this guide we we'll need a Kubernetes cluster for testing. Let's create one using [kind](https://kind.sigs.k8s.io/) </br>
|
||||
|
||||
```
|
||||
kind create cluster --name sealedsecrets --image kindest/node:v1.23.5
|
||||
```
|
||||
|
||||
See cluster up and running:
|
||||
|
||||
```
|
||||
kubectl get nodes
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
sealedsecrets-control-plane Ready control-plane,master 2m12s v1.23.5
|
||||
```
|
||||
|
||||
## Run a container to work in
|
||||
|
||||
### run Alpine Linux:
|
||||
```
|
||||
docker run -it --rm -v ${HOME}:/root/ -v ${PWD}:/work -w /work --net host alpine sh
|
||||
```
|
||||
|
||||
### install kubectl
|
||||
|
||||
```
|
||||
apk add --no-cache curl
|
||||
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
|
||||
chmod +x ./kubectl
|
||||
mv ./kubectl /usr/local/bin/kubectl
|
||||
```
|
||||
|
||||
### install helm
|
||||
|
||||
```
|
||||
curl -o /tmp/helm.tar.gz -LO https://get.helm.sh/helm-v3.10.1-linux-amd64.tar.gz
|
||||
tar -C /tmp/ -zxvf /tmp/helm.tar.gz
|
||||
mv /tmp/linux-amd64/helm /usr/local/bin/helm
|
||||
chmod +x /usr/local/bin/helm
|
||||
```
|
||||
|
||||
### test cluster access:
|
||||
```
|
||||
/work # kubectl get nodes
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
sealedsecrets-control-plane Ready control-plane,master 3m26s v1.23.5
|
||||
```
|
||||
|
||||
## Install Sealed Secret Controller
|
||||
|
||||
### download the YAML
|
||||
|
||||
In this demo we'll use version [0.19.1](https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.19.1/controller.yaml) of the sealed secrets controller downloaded from the
|
||||
[Github releases](https://github.com/bitnami-labs/sealed-secrets/releases) page
|
||||
|
||||
```
|
||||
curl -L -o ./kubernetes/secrets/sealed-secrets/controller-v0.19.1.yaml https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.19.1/controller.yaml
|
||||
|
||||
```
|
||||
|
||||
### install using Helm
|
||||
|
||||
You can also install the controller using `helm`
|
||||
|
||||
```
|
||||
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
|
||||
helm search repo sealed-secrets --versions
|
||||
helm template sealed-secrets --version 2.7.0 -n kube-system sealed-secrets/sealed-secrets \
|
||||
> ./kubernetes/secrets/sealed-secrets/controller-helm-v0.19.1.yaml
|
||||
|
||||
```
|
||||
With `helm template` we can explore the YAML and then replace the `helm template` with `helm install`
|
||||
to install the chart
|
||||
|
||||
### install using YAML manifest
|
||||
|
||||
```
|
||||
kubectl apply -f kubernetes/secrets/sealed-secrets/controller-v0.19.1.yaml
|
||||
```
|
||||
|
||||
### Check the installation
|
||||
|
||||
The controller deploys to the `kube-system` namespace by default.
|
||||
|
||||
```
|
||||
kubectl -n kube-system get pods
|
||||
```
|
||||
|
||||
Check the logs of the sealed secret controller
|
||||
|
||||
```
|
||||
kubectl -n kube-system logs -l name=sealed-secrets-controller --tail -1
|
||||
```
|
||||
|
||||
From the logs we can see that it writes the encryption key its going to use as a kubernetes secret </br>
|
||||
Example log:
|
||||
|
||||
```
|
||||
2022/11/05 21:38:20 New key written to kube-system/sealed-secrets-keymwzn9
|
||||
```
|
||||
|
||||
## Encryption keys
|
||||
|
||||
```
|
||||
kubectl -n kube-system get secrets
|
||||
kubectl -n kube-system get secret sealed-secrets-keygxlvg -o yaml
|
||||
```
|
||||
|
||||
## Download KubeSeal
|
||||
|
||||
The same way we downloaded the sealed secrets controller from the [GitHub releases](https://github.com/bitnami-labs/sealed-secrets/releases) page,
|
||||
we'll want to download kubeseal from the assets section
|
||||
```
|
||||
|
||||
curl -L -o /tmp/kubeseal.tar.gz \
|
||||
https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.19.1/kubeseal-0.19.1-linux-amd64.tar.gz
|
||||
tar -xzf /tmp/kubeseal.tar.gz -C /tmp/
|
||||
chmod +x /tmp/kubeseal
|
||||
mv /tmp/kubeseal /usr/local/bin/
|
||||
```
|
||||
|
||||
We can now run `kubeseal --help`
|
||||
|
||||
## Sealing a basic Kubernetes Secret
|
||||
|
||||
Looks at our existing Kubernetes secret YAML
|
||||
|
||||
```
|
||||
cat kubernetes/secrets/secret.yaml
|
||||
```
|
||||
|
||||
If you run `kubeseal` you will see it pause and expect input from `stdin`. </br>
|
||||
You can paste your secret YAML and press CTRL+D to terminate `stdin`. </br>
|
||||
You will notice it writes a `sealedSecret` to `stdout`. </br>
|
||||
We can then automate this using `|` characters. </br>
|
||||
|
||||
Create a sealed secret using `stdin` :
|
||||
|
||||
```
|
||||
cat kubernetes/secrets/secret.yaml | kubeseal -o yaml > kubernetes/secrets/sealed-secrets/sealed-secret.yaml
|
||||
```
|
||||
|
||||
Create a sealed secret using a YAML file:
|
||||
|
||||
```
|
||||
kubeseal -f kubernetes/secrets/secret.yaml -o yaml > kubernetes/secrets/sealed-secrets/sealed-secret.yaml
|
||||
```
|
||||
|
||||
Deploy the sealed secret
|
||||
|
||||
```
|
||||
kubectl apply -f kubernetes/secrets/sealed-secrets/sealed-secret.yaml
|
||||
```
|
||||
|
||||
Now few seconds later, see the secret
|
||||
|
||||
```
|
||||
kubectl -n default get secret
|
||||
NAME TYPE DATA AGE
|
||||
mysecret Opaque 1 25s
|
||||
```
|
||||
|
||||
## How the encryption key is managed
|
||||
|
||||
By default the controller generates a key as we saw earlier and stores it in a Kubernetes secret. </br>
|
||||
By default, the controller will generate a new active key every 30 days.
|
||||
It keeps old keys so it can decrypt previous encrypted sealed secrets and will use the active key with new encryption. </br>
|
||||
|
||||
It's important to keep these keys secured. </br>
|
||||
When the controller starts it consumes all the secrets and will start using them </br>
|
||||
This means we can backup these keys in a Vault and use them to migrate our clusters if we wanted to. </br>
|
||||
|
||||
We can also override the renewal period to increase or decrease the value. `0` turns it off </br>
|
||||
|
||||
To showcase this I can set `--key-renew-period=<value>` to 5min to watch how it works. </br>
|
||||
|
||||
```
|
||||
apk add nano
|
||||
export KUBE_EDITOR=nano
|
||||
```
|
||||
Set the flag on the command like so to add a new key every 5 min for testing:
|
||||
|
||||
```
|
||||
spec:
|
||||
containers:
|
||||
- command:
|
||||
- controller
|
||||
- --key-renew-period=5m
|
||||
|
||||
kubectl edit deployment/sealed-secrets-controller --namespace=kube-system
|
||||
```
|
||||
|
||||
You should see a new key created under secrets in the `kube-system` namespace
|
||||
|
||||
```
|
||||
kubectl -n kube-system get secrets
|
||||
```
|
||||
|
||||
## Backup your encryption keys
|
||||
|
||||
To get your keys out for backup purpose, it's as simple as grabbing a secret by label using `kubectl` :
|
||||
|
||||
```
|
||||
kubectl get secret -n kube-system \
|
||||
-l sealedsecrets.bitnami.com/sealed-secrets-key \
|
||||
-o yaml \
|
||||
> kubernetes/secrets/sealed-secrets/sealed-secret-keys.key
|
||||
```
|
||||
This can be used when migrating from one cluster to another, or simply for keeping backups. </br>
|
||||
|
||||
## Migrate your encryption keys to a new cluster
|
||||
|
||||
To test this, lets delete our cluster and recreate it. </br>
|
||||
|
||||
```
|
||||
kind delete cluster --name sealedsecrets
|
||||
kind create cluster --name sealedsecrets --image kindest/node:v1.23.5
|
||||
|
||||
# check the cluster
|
||||
kubectl get nodes
|
||||
|
||||
# redeploy sealed-secrets controller
|
||||
kubectl apply -f kubernetes/secrets/sealed-secrets/controller-v0.19.1.yaml
|
||||
|
||||
kubectl -n kube-system get pods
|
||||
|
||||
```
|
||||
|
||||
### restore our encryption keys
|
||||
|
||||
```
|
||||
kubectl apply -f kubernetes/secrets/sealed-secrets/sealed-secret-keys.key
|
||||
```
|
||||
|
||||
### apply our old sealed secret
|
||||
|
||||
```
|
||||
kubectl apply -f kubernetes/secrets/sealed-secrets/sealed-secret.yaml
|
||||
```
|
||||
|
||||
### see sealed secret status
|
||||
|
||||
To troubleshoot the secret, you can use the popular `kubectl describe` command. </br>
|
||||
Note that we're unable to decrypt the secret. </br>
|
||||
Why is that ? </br>
|
||||
|
||||
We'll this is because the encryption key secrets are read when the controller starts. </br>
|
||||
So we will need to restart the controller to that it can read ingest the encryption keys:
|
||||
|
||||
```
|
||||
kubectl delete pod -n kube-system -l name=sealed-secrets-controller
|
||||
```
|
||||
|
||||
## Re-encrypting secrets with the latest key
|
||||
|
||||
We can also use `kubeseal --re-encrypt` to encrypt a secret again. </br>
|
||||
Let's say we want to encrypt with the latest key. </br>
|
||||
This will re-encrypt the sealed secret without having to pull the actual secret to the client </br>
|
||||
|
||||
```
|
||||
cat ./kubernetes/secrets/sealed-secrets/sealed-secret.yaml \
|
||||
| kubeseal --re-encrypt -o yaml
|
||||
```
|
||||
|
||||
I can then save this to override the original old local sealed secret file:
|
||||
|
||||
```
|
||||
cat ./kubernetes/secrets/sealed-secrets/sealed-secret.yaml \
|
||||
| kubeseal --re-encrypt -o yaml \
|
||||
> tmp.yaml && mv tmp.yaml ./kubernetes/secrets/sealed-secrets/sealed-secret.yaml
|
||||
```
|
||||
Reference in New Issue
Block a user