Introduction

In the first, getting started with Kubernetes operators (Helm based), and the second part, getting started with Kubernetes operators (Ansible based), of this Introduction to Kubernetes operators blog series we learned various concepts related to Kubernetes operators and created a Helm based operator and an Ansible based operator respectively. In this final part, we will build a Golang based operator. In case of Helm based operators, we were executing a helm chart when changes were made to the custom object type of our application, similarly in the case of an Ansible based operator we executed an Ansible role. In case of Golang based operator we write the code for the action we need to perform (reconcile logic) whenever the state of our custom object change, this makes the Golang based operators quite powerful and flexible, at the same time making them the most complex to build out of the 3 types.

What Will We Build?

The database server we deployed as part of our book store app in previous blogs didn’t have any persistent volume attached to it and we would lose data in case the pod restarts, to avoid this we will attach a persistent volume attached to the host (K8s worker nodes ) and run our database as an statefulset rather than a deployment. We will also add a feature to expand the persistent volume associated with the mongodb pod.

Building the Operator

1. Set up the project:  

operator-sdk new bookstore-operator --dep-manager=dep

CODE: https://gist.github.com/velotiotech/ba6958f9cf3fae4ca4e714afef84cefa.js

The above command creates the bookstore-operator folder in our $GOPATH/src, here we have set the --dep-manager as dep which signifies we want to use dep for managing dependencies, by default it uses go modules for managing dependencies. Similar to what we have seen earlier the operator sdk creates all the necessary folder structure for us inside the bookstore-operator folder.

2. Add the custom resource definition

operator-sdk add api --api-version=blog.velotio.com/v1alpha1 --kind=BookStore

The above command creates the CRD and CR for the BookStore type. It also creates the golang structs (pkg/apis/blog/v1alpha1/bookstore_types.go)  for BookStore types.  It also registers the custom type (pkg/apis/blog/v1alpha1/register.go) with schema and generates deep-copy methods as well. Here we can see that all the generic tasks are being done by the operator framework itself allowing us to focus on building and object and the controller. We will update the spec of our BookStore object later. We will update the spec of BookStore type to include two custom types BookApp and BookDB.

CODE: https://gist.github.com/velotiotech/66b51e30826738ee60d4583f18e911b9.js

Let’s also update the BookStore CR (blog.velotio.com_v1alpha1_bookstore_cr.yaml)

CODE: https://gist.github.com/velotiotech/bc9a851331519adfc4187bfd6be2ec5b.js

3. Add the bookstore controller

operator-sdk add controller --api-version=blog.velotio.com/v1alpha1 --kind=BookStore

CODE: https://gist.github.com/velotiotech/78768c07d2c18657a2ee2f6fe19bb5e6.js

The above command adds the bookstore controller (pkg/controller/bookstore/bookstore_controller.go) to the project and also adds it to the manager.

If we take a look at the add function in the bookstore_controller.go file we can see that a new controller is created here and added to the manager so that the manager can start the controller when it (manager) comes up,  the add(mgr manager.Manager, r reconcile.Reconciler) is called by the public function Add(mgr manager.Manager) which also creates a new reconciler objects and passes it to the add where the controller is associated with the reconciler, in the add function we also set the type of object (BookStore) which the controller will watch.

CODE: https://gist.github.com/velotiotech/61bf2d0c6e3d929d39525058f288ed5e.js

This ensures that for any events related to any object of BookStore type, a reconcile request (a namespace/name key) is sent to the Reconcile method associated with the reconciler object (ReconcileBookStore) here.

4. Build the reconcile logic

The reconcile logic is implemented inside the Reconcile method of the reconciler object of the custom type which implements the reconcile loop.

As a part of our reconcile logic we will do the following

1. Create the bookstore app deployment if it doesn’t exist.
2. Create the bookstore app service if it doesn’t exist.
3. Create the Mongodb statefulset if it doesn’t exist.
4. Create the Mongodb service if it doesn’t exist.
5. Ensure deployments and services match their desired configurations like the replica count, image tag, service port, size of the PV associated with the Mongodb statefulset etc.

 There are three possible events that can happen with the BookStore object

1. The object got created: Whenever an object of kind BookStore is created we create all the k8s resources we mentioned above
2. The object has been updated: When the object gets updated then we update all the k8s resources associated with it..
3. The object has been deleted: When the object gets deleted we don’t need to do anything as while creating the K8s objects we will set the `BookStore` type as its owner which will ensure that all the K8s objects associated with it gets automatically deleted when we delete the object.

On receiving the reconcile request the first step if to lookup for the object.

CODE: https://gist.github.com/velotiotech/6f956e0b5de0fd61321b87f2414f9c9b.js

If the object is not found, we assume that it got deleted and don’t requeue the request considering the reconcile to be successful.

If any error occurs while doing the reconcile then we return the error and whenever we return non nil error value then controller requeues the request.

In the reconcile logic we call the BookStore method which creates or updates all the k8s objects associated with the BookStore objects based on whether the object has been created or updated.

CODE: https://gist.github.com/velotiotech/ace211bab96b23aeabdcefa2980e6b25.js

The implementation of the above method is a bit hacky but gives an idea of the flow. In the above function, we can see that we are setting the BookStore type as an owner for all the resources controllerutil.SetControllerReference(c, bookStoreDep, r.scheme) as we had discussed earlier. If we look at the owner reference for these objects we would see something like this.

CODE: https://gist.github.com/velotiotech/1b03889e62c77dc12c6b8ab8fb0b66e4.js

5.  Deploy the operator and verify its working

The approach to deploy and verify the working of the bookstore application is similar to what we did in the previous two blogs the only difference being that now we have deployed the Mongodb as a stateful set and even if we restart the pod we will see that the information that we stored will still be available.

6. Verify volume expansion

For updating the volume associated with the mongodb instance we first need to update the size of the volume we specified while creating the bookstore object. In the example above I had set it to 2GB let’s update it to 3GB and update the bookstore object.

Once the bookstore object is updated if we describe the mongodb PVC we will see that it still has 2GB PV but the conditions we will see something like this.

CODE: https://gist.github.com/velotiotech/c9dc2bd02582a1a8251d4f35e2e517de.js

It is clear from the message that we need to restart the pod for resizing of volume to reflect. Once we delete the pod it will get restarted and the PVC will get updated to reflect the expanded volume size.

The complete code is available here.

Conclusion

Golang based operators are built mostly for stateful applications like databases. The operator can automate complex operational tasks allow us to run applications with ease. At the same time, building and maintaining it can be quite complex and we should build one only when we are fully convinced that our requirements can’t be met with any other type of operator. Operators are an interesting and emerging area in Kubernetes and I hope this blog series on getting started with it help the readers in learning the basics of it.