mirror of
https://github.com/marcel-dempers/docker-development-youtube-series.git
synced 2025-06-06 17:01:30 +00:00
182 lines
6.6 KiB
Markdown
182 lines
6.6 KiB
Markdown
# Github Actions Runner Controller on Kubernetes
|
|
|
|
Let's start with the github [actions-runner-controller](https://github.com/actions/actions-runner-controller) documentation
|
|
|
|
## Create a kubernetes cluster
|
|
|
|
In this guide we we''ll need a Kubernetes cluster for testing. Let's create one using [kind](https://kind.sigs.k8s.io/) </br>
|
|
|
|
```
|
|
cd github/actions/kubernetes
|
|
|
|
kind create cluster --name demo --image kindest/node:v1.28.0
|
|
```
|
|
|
|
Next up, I will be running a small container where I will be doing all the work from:
|
|
You can skip this part if you already have `kubectl` and `helm` on your machine.
|
|
|
|
```
|
|
docker run -it --rm --net host -v ${HOME}/.kube/:/root/.kube/ -v ${PWD}:/work -w /work alpine sh
|
|
```
|
|
|
|
Install `kubectl`
|
|
|
|
```
|
|
apk add --no-cache curl
|
|
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
|
|
chmod +x ./kubectl
|
|
mv ./kubectl /usr/local/bin/kubectl
|
|
```
|
|
|
|
Install `helm`
|
|
|
|
```
|
|
HELM_VERSION=3.14.1
|
|
curl -LO https://get.helm.sh/helm-v${HELM_VERSION}-linux-amd64.tar.gz
|
|
tar -C /tmp/ -zxvf helm-v${HELM_VERSION}-linux-amd64.tar.gz
|
|
rm helm-v${HELM_VERSION}-linux-amd64.tar.gz
|
|
mv /tmp/linux-amd64/helm /usr/local/bin/helm
|
|
chmod +x /usr/local/bin/helm
|
|
```
|
|
|
|
Now we have `helm` and `kubectl` and we can test our cluster access:
|
|
|
|
```
|
|
kubectl get nodes
|
|
|
|
NAME STATUS ROLES AGE VERSION
|
|
demo-control-plane Ready control-plane 4m12s v1.28.0
|
|
```
|
|
|
|
## Deploy the Github Action Runner Controller
|
|
|
|
The runner controller is the core controller that manages the entire runner ecosystem. </br>
|
|
Its uses the Kubernetes Operator pattern so you can deploy runners. </br>
|
|
|
|
Let's go to the [Quickstart Guide](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller) </br>
|
|
|
|
We can deploy the controller using `helm`
|
|
```
|
|
VERSION=0.8.2
|
|
NAMESPACE="github"
|
|
helm install arc \
|
|
--namespace "${NAMESPACE}" \
|
|
--create-namespace \
|
|
--version ${VERSION} \
|
|
oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller
|
|
|
|
```
|
|
|
|
Once chart is deployed, we can see the controller running in the `github` namespace
|
|
|
|
```
|
|
kubectl -n github get pods
|
|
NAME READY STATUS RESTARTS AGE
|
|
arc-gha-rs-controller-7bf474df55-v7vm9 1/1 Running 0 16s
|
|
```
|
|
|
|
Now this pod will not do anything, other than allowing us to deploy runner scale sets. </br>
|
|
We can check its logs:
|
|
```
|
|
kubectl -n github logs -l app.kubernetes.io/name=gha-rs-controller
|
|
```
|
|
|
|
## Deploy Github Action Runner scale sets
|
|
|
|
To make it useful, we need to add it to our repo or organisation by [Authenticating to Github](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api)
|
|
|
|
Create a new [Github App](https://docs.github.com/en/apps/using-github-apps/installing-your-own-github-app) for your account or organization.
|
|
|
|
Once we have created our Github App, we need to configure its authentication by creating a kubernetes secret with the authentication details of the app. </br>
|
|
|
|
```
|
|
kubectl create secret generic github-app-secret \
|
|
--namespace=github \
|
|
--from-literal=github_app_id=xxxxx \
|
|
--from-literal=github_app_installation_id=xxxxx \
|
|
--from-file=github_app_private_key='github.pem'
|
|
|
|
```
|
|
|
|
The [documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#configuring-the-runner-image) showcases how to customise the runner with the helm values file
|
|
|
|
This allows us to customer the pod template for our runner. <br/>
|
|
Let's create a basic one:
|
|
|
|
```
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: runner
|
|
image: "ghcr.io/actions/actions-runner:latest"
|
|
imagePullPolicy: Always
|
|
command: ["/home/runner/run.sh"]
|
|
```
|
|
|
|
We'll need to use the helm upgrade to apply the changes again. </br>
|
|
|
|
We can also set our runner name using the values file using the [advanced configuration options](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#using-advanced-configuration-options)
|
|
|
|
```
|
|
runnerScaleSetName: "marcels-runner"
|
|
```
|
|
|
|
Deploy the Github actions runner scaleset
|
|
|
|
```
|
|
INSTALLATION_NAME="arc-runner-set"
|
|
NAMESPACE="github"
|
|
GITHUB_CONFIG_URL="https://github.com/marcel-dempers/docker-development-youtube-series"
|
|
|
|
helm install "${INSTALLATION_NAME}" \
|
|
--namespace "${NAMESPACE}" \
|
|
--create-namespace \
|
|
--values scaleset-values.yaml \
|
|
--set githubConfigUrl="${GITHUB_CONFIG_URL}" \
|
|
oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set
|
|
```
|
|
|
|
Now this will deploy a listener which will listen to jobs for the organisation or github repo \ account :
|
|
|
|
```
|
|
kubectl -n github get pods
|
|
NAME READY STATUS RESTARTS AGE
|
|
arc-gha-rs-controller-7bf474df55-v7vm9 1/1 Running 0 14m
|
|
marcels-runner-9d8dc86f-listener 1/1 Running 0 10s
|
|
```
|
|
|
|
If we look at the logs again we can see the controller created a listener
|
|
|
|
```
|
|
kubectl -n github logs -l app.kubernetes.io/name=gha-rs-controller
|
|
kubectl -n github logs -l app.kubernetes.io/component=runner-scale-set-listener
|
|
```
|
|
|
|
It's important to check both logs to ensure all your authentication is setup correctly and there are no problems. </br>
|
|
|
|
### Customise the runner image
|
|
|
|
Now when we run our GitHub action, it fails because there is no `docker` command and no `docker` daemon running in the runner. </br>
|
|
To add docker there are a number of things we can do, we can setup docker in the container or make use of docker on the host or run docker as a sidecar container and leverage the `docker-in-docker` image. </br>
|
|
|
|
This approach is also explained in the [documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#using-docker-in-docker-mode)
|
|
|
|
```
|
|
template:
|
|
```
|
|
|
|
To update, we simply use the `helm upgrade` command
|
|
|
|
```
|
|
INSTALLATION_NAME="arc-runner-set"
|
|
NAMESPACE="github"
|
|
GITHUB_CONFIG_URL="https://github.com/marcel-dempers/docker-development-youtube-series"
|
|
|
|
helm upgrade "${INSTALLATION_NAME}" \
|
|
--namespace "${NAMESPACE}" \
|
|
--values scaleset-values.yaml \
|
|
--set githubConfigUrl="${GITHUB_CONFIG_URL}" \
|
|
oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set
|
|
```
|
|
|