How to deploy GitHub Actions Self-Hosted Runners on Kubernetes

Mahesh Khade

Cloud & DevOps

Tags:

kubernetes

Github

Github Actions

MySQL

minikube

k8s

GitHub Actions jobs are run in the cloud by default; however, sometimes we want to run jobs in our own customized/private environment where we have full control. That is where a self-hosted runner saves us from this problem.

To get a basic understanding of running self-hosted runners on the Kubernetes cluster, this blog is perfect for you.

We’ll be focusing on running GitHub Actions on a self-hosted runner on Kubernetes.

An example use case would be to create an automation in GitHub Actions to execute MySQL queries on MySQL Database running in a private network (i.e., MySQL DB, which is not accessible publicly).

A self-hosted runner requires the provisioning and configuration of a virtual machine instance; here, we are running it on Kubernetes. For running a self-hosted runner on a Kubernetes cluster, the action-runner-controller helps us to make that possible.

This blog aims to try out self-hosted runners on Kubernetes and covers:

Deploying MySQL Database on minikube, which is accessible only within Kubernetes Cluster.
Deploying self-hosted action runners on the minikube.
Running GitHub Action on minikube to execute MySQL queries on MySQL Database.

Steps for completing this tutorial:

Create a GitHub repository

Create a private repository on GitHub. I am creating it with the name velotio/action-runner-poc.

Setup a Kubernetes cluster using minikube

Install Docker.
Install Minikube.
Install Helm
Install kubectl

Install cert-manager on a Kubernetes cluster

By default, actions-runner-controller uses cert-manager for certificate management of admission webhook, so we have to make sure cert-manager is installed on Kubernetes before we install actions-runner-controller.

Run the below helm commands to install cert-manager on minikube.

Verify installation using “kubectl --namespace cert-manager get all”. If everything is okay, you will see an output as below:

Setting Up Authentication for Hosted Runners‍

There are two ways for actions-runner-controller to authenticate with the GitHub API (only 1 can be configured at a time, however):

Using a GitHub App (not supported for enterprise-level runners due to lack of support from GitHub.)
Using a PAT (personal access token)

To keep this blog simple, we are going with PAT.

To authenticate an action-runner-controller with the GitHub API, we can use a PAT with the action-runner-controller registers a self-hosted runner.

Go to account > Settings > Developers settings > Personal access token. Click on “Generate new token”. Under scopes, select “Full control of private repositories”.

Click on the “Generate token” button.

Copy the generated token and run the below commands to create a Kubernetes secret, which will be used by action-runner-controller deployment.

export GITHUB_TOKEN=XXXxxxXXXxxxxXYAVNa

view raw .sh hosted with ❤ by GitHub

kubectl create ns actions-runner-system

view raw .sh hosted with ❤ by GitHub

Create secret

	kubectl create secret generic controller-manager -n actions-runner-system
	--from-literal=github_token=${GITHUB_TOKEN}

view raw .sh hosted with ❤ by GitHub

Install action runner controller on the Kubernetes cluster

Run the below helm commands

	helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
	helm repo update
	helm upgrade --install --namespace actions-runner-system
	--create-namespace --wait actions-runner-controller
	actions-runner-controller/actions-runner-controller --set
	syncPeriod=1m

view raw .sh hosted with ❤ by GitHub

Verify that the action-runner-controller installed properly using below command

kubectl --namespace actions-runner-system get all

view raw .sh hosted with ❤ by GitHub

Create a Repository Runner

Create a RunnerDeployment Kubernetes object, which will create a self-hosted runner named k8s-action-runner for the GitHub repository velotio/action-runner-poc
Please Update Repo name from “velotio/action-runner-poc” to “<Your-repo-name>”
To create the RunnerDeployment object, create the file runner.yaml as follows:

	apiVersion: actions.summerwind.dev/v1alpha1
	kind: RunnerDeployment
	metadata:
	name: k8s-action-runner
	namespace: actions-runner-system
	spec:
	replicas: 2
	template:
	spec:
	repository: velotio/action-runner-poc

view raw runner.yaml hosted with ❤ by GitHub

To create, run this command:

kubectl create -f runner.yaml

view raw .sh hosted with ❤ by GitHub

Check that the pod is running using the below command:

kubectl get pod -n actions-runner-system | grep -i "k8s-action-runner"

view raw .sh hosted with ❤ by GitHub

‍

If everything goes well, you should see two action runners on the Kubernetes, and the same are registered on Github. Check under Settings > Actions > Runner of your repository.

Check the pod with kubectl get po -n actions-runner-system

Install a MySQL Database on the Kubernetes cluster

‍

Create PV and PVC for MySQL Database.
Create mysql-pv.yaml with the below content.

	apiVersion: v1
	kind: PersistentVolume
	metadata:
	name: mysql-pv-volume
	labels:
	type: local
	spec:
	capacity:
	storage: 2Gi
	accessModes:
	- ReadWriteOnce
	hostPath:
	path: "/mnt/data"
	---
	apiVersion: v1
	kind: PersistentVolumeClaim
	metadata:
	name: mysql-pv-claim
	spec:
	accessModes:
	- ReadWriteOnce
	resources:
	requests:
	storage: 2Gi

view raw mysql-pv.yaml hosted with ❤ by GitHub

Create mysql namespace

kubectl create ns mysql

view raw .sh hosted with ❤ by GitHub

Now apply mysql-pv.yaml to create PV and PVC

kubectl create -f mysql-pv.yaml -n mysql

view raw .sh hosted with ❤ by GitHub

Create the file mysql-svc-deploy.yaml and add the below content to mysql-svc-deploy.yaml

Here, we have used MYSQL_ROOT_PASSWORD as “password”.

	apiVersion: v1
	kind: Service
	metadata:
	name: mysql
	spec:
	ports:
	- port: 3306
	selector:
	app: mysql
	clusterIP: None
	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: mysql
	spec:
	selector:
	matchLabels:
	app: mysql
	strategy:
	type: Recreate
	template:
	metadata:
	labels:
	app: mysql
	spec:
	containers:
	- image: mysql:5.6
	name: mysql
	env:
	# Use secret in real usage
	- name: MYSQL_ROOT_PASSWORD
	value: password
	ports:
	- containerPort: 3306
	name: mysql
	volumeMounts:
	- name: mysql-persistent-storage
	mountPath: /var/lib/mysql
	volumes:
	- name: mysql-persistent-storage
	persistentVolumeClaim:
	claimName: mysql-pv-claim

view raw mysql-svc-deploy.yaml hosted with ❤ by GitHub

Create the service and deployment

kubectl create -f mysql-svc-deploy.yaml -n mysql

view raw .sh hosted with ❤ by GitHub

Verify that the MySQL database is running

kubectl get po -n mysql

view raw .sh hosted with ❤ by GitHub

‍

Create a GitHub repository secret to store MySQL password

As we will use MySQL password in the GitHub action workflow file as a good practice, we should not use it in plain text. So we will store MySQL password in GitHub secrets, and we will use this secret in our GitHub action workflow file.

Create a secret in the GitHub repository and give the name to the secret as “MYSQL_PASS”, and in the values, enter “password”.

Create a GitHub workflow file

YAML syntax is used to write GitHub workflows. For each workflow, we use a separate YAML file, which we store at .github/workflows/ directory. So, create a .github/workflows/ directory in your repository and create a file .github/workflows/mysql_workflow.yaml as follows.

	---
	name: Example 1
	on:
	push:
	branches: [ main ]
	jobs:
	build:
	name: Build-job
	runs-on: self-hosted
	steps:
	- name: Checkout
	uses: actions/checkout@v2

	- name: MySQLQuery
	env:
	PASS: ${{ secrets.MYSQL_PASS }}
	run: \|
	docker run -v ${GITHUB_WORKSPACE}:/var/lib/docker --rm mysql:5.6 sh -c "mysql -u root -p$PASS -hmysql.mysql.svc.cluster.local </var/lib/docker/test.sql"

view raw mysql_workflow.yaml hosted with ❤ by GitHub

If you check the docker run command in the mysql_workflow.yaml file, we are referring to the .sql file, i.e., test.sql. So, create a test.sql file in your repository as follows:

	use mysql;
	CREATE TABLE IF NOT EXISTS Persons (
	PersonID int,
	LastName varchar(255),
	FirstName varchar(255),
	Address varchar(255),
	City varchar(255)
	);

	SHOW TABLES;

view raw test.sql hosted with ❤ by GitHub

In test.sql, we are running MySQL queries like create tables.
Push changes to your repository main branch.
If everything is fine, you will be able to see that the GitHub action is getting executed in a self-hosted runner pod. You can check it under the “Actions” tab of your repository.

You can check the workflow logs to see the output of SHOW TABLES—a command we have used in the test.sql file—and check whether the persons tables is created.

References

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.

How to deploy GitHub Actions Self-Hosted Runners on Kubernetes

To get a basic understanding of running self-hosted runners on the Kubernetes cluster, this blog is perfect for you.

We’ll be focusing on running GitHub Actions on a self-hosted runner on Kubernetes.

This blog aims to try out self-hosted runners on Kubernetes and covers:

Deploying MySQL Database on minikube, which is accessible only within Kubernetes Cluster.
Deploying self-hosted action runners on the minikube.
Running GitHub Action on minikube to execute MySQL queries on MySQL Database.

Steps for completing this tutorial:

Create a GitHub repository

Create a private repository on GitHub. I am creating it with the name velotio/action-runner-poc.

Setup a Kubernetes cluster using minikube

Install Docker.
Install Minikube.
Install Helm
Install kubectl

Install cert-manager on a Kubernetes cluster

By default, actions-runner-controller uses cert-manager for certificate management of admission webhook, so we have to make sure cert-manager is installed on Kubernetes before we install actions-runner-controller.

Run the below helm commands to install cert-manager on minikube.

Verify installation using “kubectl --namespace cert-manager get all”. If everything is okay, you will see an output as below:

Setting Up Authentication for Hosted Runners‍

There are two ways for actions-runner-controller to authenticate with the GitHub API (only 1 can be configured at a time, however):

Using a GitHub App (not supported for enterprise-level runners due to lack of support from GitHub.)
Using a PAT (personal access token)

To keep this blog simple, we are going with PAT.

To authenticate an action-runner-controller with the GitHub API, we can use a PAT with the action-runner-controller registers a self-hosted runner.

Go to account > Settings > Developers settings > Personal access token. Click on “Generate new token”. Under scopes, select “Full control of private repositories”.

Click on the “Generate token” button.

Copy the generated token and run the below commands to create a Kubernetes secret, which will be used by action-runner-controller deployment.

export GITHUB_TOKEN=XXXxxxXXXxxxxXYAVNa

view raw .sh hosted with ❤ by GitHub

kubectl create ns actions-runner-system

view raw .sh hosted with ❤ by GitHub

Create secret

	kubectl create secret generic controller-manager -n actions-runner-system
	--from-literal=github_token=${GITHUB_TOKEN}

view raw .sh hosted with ❤ by GitHub

Install action runner controller on the Kubernetes cluster

Run the below helm commands

	helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
	helm repo update
	helm upgrade --install --namespace actions-runner-system
	--create-namespace --wait actions-runner-controller
	actions-runner-controller/actions-runner-controller --set
	syncPeriod=1m

view raw .sh hosted with ❤ by GitHub

Verify that the action-runner-controller installed properly using below command

kubectl --namespace actions-runner-system get all

view raw .sh hosted with ❤ by GitHub

Create a Repository Runner

Create a RunnerDeployment Kubernetes object, which will create a self-hosted runner named k8s-action-runner for the GitHub repository velotio/action-runner-poc
Please Update Repo name from “velotio/action-runner-poc” to “<Your-repo-name>”
To create the RunnerDeployment object, create the file runner.yaml as follows:

	apiVersion: actions.summerwind.dev/v1alpha1
	kind: RunnerDeployment
	metadata:
	name: k8s-action-runner
	namespace: actions-runner-system
	spec:
	replicas: 2
	template:
	spec:
	repository: velotio/action-runner-poc

view raw runner.yaml hosted with ❤ by GitHub

To create, run this command:

kubectl create -f runner.yaml

view raw .sh hosted with ❤ by GitHub

Check that the pod is running using the below command:

kubectl get pod -n actions-runner-system | grep -i "k8s-action-runner"

view raw .sh hosted with ❤ by GitHub

‍

If everything goes well, you should see two action runners on the Kubernetes, and the same are registered on Github. Check under Settings > Actions > Runner of your repository.

Check the pod with kubectl get po -n actions-runner-system

Install a MySQL Database on the Kubernetes cluster

‍

Create PV and PVC for MySQL Database.
Create mysql-pv.yaml with the below content.

	apiVersion: v1
	kind: PersistentVolume
	metadata:
	name: mysql-pv-volume
	labels:
	type: local
	spec:
	capacity:
	storage: 2Gi
	accessModes:
	- ReadWriteOnce
	hostPath:
	path: "/mnt/data"
	---
	apiVersion: v1
	kind: PersistentVolumeClaim
	metadata:
	name: mysql-pv-claim
	spec:
	accessModes:
	- ReadWriteOnce
	resources:
	requests:
	storage: 2Gi

view raw mysql-pv.yaml hosted with ❤ by GitHub

Create mysql namespace

kubectl create ns mysql

view raw .sh hosted with ❤ by GitHub

Now apply mysql-pv.yaml to create PV and PVC

kubectl create -f mysql-pv.yaml -n mysql

view raw .sh hosted with ❤ by GitHub

Create the file mysql-svc-deploy.yaml and add the below content to mysql-svc-deploy.yaml

Here, we have used MYSQL_ROOT_PASSWORD as “password”.

	apiVersion: v1
	kind: Service
	metadata:
	name: mysql
	spec:
	ports:
	- port: 3306
	selector:
	app: mysql
	clusterIP: None
	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: mysql
	spec:
	selector:
	matchLabels:
	app: mysql
	strategy:
	type: Recreate
	template:
	metadata:
	labels:
	app: mysql
	spec:
	containers:
	- image: mysql:5.6
	name: mysql
	env:
	# Use secret in real usage
	- name: MYSQL_ROOT_PASSWORD
	value: password
	ports:
	- containerPort: 3306
	name: mysql
	volumeMounts:
	- name: mysql-persistent-storage
	mountPath: /var/lib/mysql
	volumes:
	- name: mysql-persistent-storage
	persistentVolumeClaim:
	claimName: mysql-pv-claim

view raw mysql-svc-deploy.yaml hosted with ❤ by GitHub

Create the service and deployment

kubectl create -f mysql-svc-deploy.yaml -n mysql

view raw .sh hosted with ❤ by GitHub

Verify that the MySQL database is running

kubectl get po -n mysql

view raw .sh hosted with ❤ by GitHub

‍

Create a GitHub repository secret to store MySQL password

Create a secret in the GitHub repository and give the name to the secret as “MYSQL_PASS”, and in the values, enter “password”.

Create a GitHub workflow file

YAML syntax is used to write GitHub workflows. For each workflow, we use a separate YAML file, which we store at .github/workflows/ directory. So, create a .github/workflows/ directory in your repository and create a file .github/workflows/mysql_workflow.yaml as follows.

	---
	name: Example 1
	on:
	push:
	branches: [ main ]
	jobs:
	build:
	name: Build-job
	runs-on: self-hosted
	steps:
	- name: Checkout
	uses: actions/checkout@v2

	- name: MySQLQuery
	env:
	PASS: ${{ secrets.MYSQL_PASS }}
	run: \|
	docker run -v ${GITHUB_WORKSPACE}:/var/lib/docker --rm mysql:5.6 sh -c "mysql -u root -p$PASS -hmysql.mysql.svc.cluster.local </var/lib/docker/test.sql"

view raw mysql_workflow.yaml hosted with ❤ by GitHub

If you check the docker run command in the mysql_workflow.yaml file, we are referring to the .sql file, i.e., test.sql. So, create a test.sql file in your repository as follows:

	use mysql;
	CREATE TABLE IF NOT EXISTS Persons (
	PersonID int,
	LastName varchar(255),
	FirstName varchar(255),
	Address varchar(255),
	City varchar(255)
	);

	SHOW TABLES;

view raw test.sql hosted with ❤ by GitHub

In test.sql, we are running MySQL queries like create tables.
Push changes to your repository main branch.
If everything is fine, you will be able to see that the GitHub action is getting executed in a self-hosted runner pod. You can check it under the “Actions” tab of your repository.

You can check the workflow logs to see the output of SHOW TABLES—a command we have used in the test.sql file—and check whether the persons tables is created.

References

About the Author

Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.

We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.

Explore current openings

Subscribe to get the latest technology updates

How to deploy GitHub Actions Self-Hosted Runners on Kubernetes

Mahesh Khade

Steps for completing this tutorial:

Create a GitHub repository

Setup a Kubernetes cluster using minikube

Install cert-manager on a Kubernetes cluster

Setting Up Authentication for Hosted Runners‍

Install action runner controller on the Kubernetes cluster

Create a Repository Runner

Install a MySQL Database on the Kubernetes cluster

Create a GitHub repository secret to store MySQL password

Create a GitHub workflow file

References

MORE POSTS BY THIS AUTHOR

Mahesh Khade

You may also like

Linux Internals of Kubernetes Networking

Shiwam Jaiswal

Strategies for Cost Optimization Across Amazon EKS Clusters

Saurabh Taneja

Mastering Prow: A Guide to Developing Your Own Plugin for Kubernetes CI/CD Workflow

Bhavya Jain

How to deploy GitHub Actions Self-Hosted Runners on Kubernetes

Steps for completing this tutorial:

Create a GitHub repository

Setup a Kubernetes cluster using minikube

Install cert-manager on a Kubernetes cluster

Setting Up Authentication for Hosted Runners‍

Install action runner controller on the Kubernetes cluster

Create a Repository Runner

Install a MySQL Database on the Kubernetes cluster

Create a GitHub repository secret to store MySQL password

Create a GitHub workflow file

References

About the Author

Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.

We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.

About Velotio

Subscribe to get the latest technology updates

Related Posts

Linux Internals of Kubernetes Networking

Strategies for Cost Optimization Across Amazon EKS Clusters

Mastering Prow: A Guide to Developing Your Own Plugin for Kubernetes CI/CD Workflow

Simplifying MySQL Sharding with ProxySQL: A Step-by-Step Guide

Streamline Kubernetes Storage Upgrades

Unlocking Key Insights in NATS Development: My Journey from Novice to Expert - Part 1

Unveiling the Magic of Kubernetes: Exploring Pod Priority, Priority Classes, and Pod Preemption

How to Setup HashiCorp Vault HA Cluster with Integrated Storage (Raft)

How To Get Started With Logging On Kubernetes?

Create CI/CD Pipeline in GitLab in under 10 mins

Acquiring Temporary AWS Credentials with Browser Navigated Authentication

How to Avoid Screwing Up CI/CD: Best Practices for DevOps Team

How to Make Your Terminal More Productive with Z-Shell (ZSH)

Setting Up A Robust Authentication Environment For OpenSSH Using QR Code PAM

Hacking Your Way Around AWS IAM Roles

Monitoring a Docker Container with Elasticsearch, Kibana, and Metricbeat

Autoscaling in Kubernetes using HPA and VPA

Managing a TLS Certificate for Kubernetes Admission Webhook

Prow + Kubernetes - A Perfect Combination To Execute CI/CD At Scale

Building A Containerized Microservice in Golang: A Step-by-step Guide

Kubernetes Migration: How To Move Data Freely Across Clusters

OPA On Kubernetes: An Introduction For Beginners

To Go Serverless Or Not Is The Question

Ensure Continuous Delivery On Kubernetes With GitOps’ Argo CD

How To Implement Chaos Engineering For Microservices Using Istio

Helm 3: A More Secured and Simpler Kubernetes Package Manager

An Introduction To Cloudflare Workers And Cloudflare KV store

Getting Started With Kubernetes Operators (Golang Based) - Part 3

Getting Started With Kubernetes Operators (Ansible Based) - Part 2

Getting Started With Kubernetes Operators (Helm Based) - Part 1

How to Write Jenkinsfile for Angular and .Net Based Applications

Kubernetes CSI in Action: Explained with Features and Use Cases

A Comprehensive Tutorial to Implementing OpenTracing With Jaeger

The Ultimate Guide to Disaster Recovery for Your Kubernetes Clusters

Know Everything About Spinnaker & How to Deploy Using Kubernetes Engine

Mesosphere DC/OS Masterclass : Tips and Tricks to Make Life Easier

Managing Secrets Using AWS Systems Manager Parameter Store and IAM Roles

Taking Amazon's Elastic Kubernetes Service for a Spin

Extending Kubernetes APIs with Custom Resource Definitions (CRDs)