A couple of weeks back, we started working with the Kubernetes Python client to carry out basic operations on its components/ resources, and that’s when we realized how few resources there were (guides, docs) on the internet. So, we experimented and decided to share our findings with the community.

This article is targeted towards an audience that is familiar with Kubernetes, its usage, and its architecture. This is not a simple Kubernetes guide; it’s about Kubernetes using Python, so as we move further, we may shed light on a few things that are required, but a few will be left for self exploration.

Kubernetes Overview

Kubernetes is an open-source container orchestration tool, largely used to simplify the process of deployment, maintenance, etc. in application development. Kubernetes is built to offer highly available, scalable, and reliable applications.

Generally, kubectl commands are used to create, list, and delete the Kubernetes resources, but for this article, we put on a developer's hat and use the Python way of doing things. In this article, we learn how to create, manage, and interact with Kubernetes resources using the Kubernetes' Python library.

But why, you may ask?

Well, having an option of doing things programmatically creates potential of endless exciting innovations for developers. Using Python, we can:

• Create and manage Kubernetes resources dynamically

• Apply algorithms that change the state, amount of resources in our cluster

• Build a more robust application with solid alerting and monitoring features

So, let us begin:

Kubernetes achieves what it does with the help of its resources. These resources are the building blocks for developing a scalable, reliable application.

Let's briefly explore these resources to understand what they are and how exactly they work together in Kubernetes:

• Node: Simple server, a physical/virtual machine.

• Pod: Smallest unit of Kubernetes, provides abstraction over a container. Creates a running env/layer on top of the container. Usually runs only one application container but can run multiple as well.

• Service: Static IP address for the pod. Remains the same even after the pod dies. Also doubles as a load-balancer for multiple pods:
  a) External services are used to make the app accessible through external sources.
  b) Internal services are used when accessibility is to be restricted.

• Ingress: Additional layer of security and address translation for services. All the requests first go to ingress then forwarded to the service.

• ConfigMap: External configuration of your app like urls of database or other services.

• Secret: To store secret/sensitive data like db-credentials, etc., encoded in base_64 format.

• Volumes: Kubernetes does not manage any data persistence on its own. Volumes are used to persist data generated by pods. It attaches a physical storage to the pod that can be both local or remote like cloud or on-premise servers.

• Deployment: Defines blueprint of the pods and its replication factor. A layer of abstraction over pods makes the configuration convenient.

• StatefulSet: Applications that are stateful are created using these to avoid data inconsistency. Same as deployment.

These are a few of the basic Kubernetes resources. If you want to explore the rest of these resources, you can click here.

Apart from these resources, there are also namespaces in Kubernetes. You will come across them quite a few times in this article, so here are some noteworthy points for Kubernetes namespaces:

Kubernetes namespaces: Used to group/organize resources in the cluster. It is like a virtual cluster inside a cluster. Namespaces are used to:

1. Structure resources
   ‍
2. Avoid conflicts between teams
   ‍
3. Share services between different environments
   ‍
4. Access restrictions
   ‍
5. Limiting resources.
   

Setting up the system

The Kubernetes library comes to our aid with quite a few modules, the ones featured in this article are client and config modules from the package; we will be using these two heavily. So, let’s install the Kubernetes Python Client:

To install the Kubernetes Python client, we make use of Python’s standard package installer pip:

CODE: https://gist.github.com/velotiotech/16abd197c4f71979df08e5409bd4732e.js

For installation from the source, we can refer to this guide from the official Python client git repository. 

Now that we have the python-kubernetes package installed, we can import it as:

CODE: https://gist.github.com/velotiotech/a6ffb5c2d899bd43af2d80f4eb6ef993.js

Loading cluster configurations

To load our cluster configurations, we can use one of the following methods:

CODE: https://gist.github.com/velotiotech/d85ca7efefa7ad26cf2f6370f6baeaeb.js

Executing this will load the configurations for your clusters from your local or remote .kube/config file.

Interacting with Kubernetes Resources

Now that we have loaded the configurations, we can use the client module to interact with the resources.

Get Resources: kubectl get commands are used to list all kinds of resources in a cluster for eg:

- List nodes: To list all the nodes in the cluster, we fire following kubectl command:

CODE: https://gist.github.com/velotiotech/73ce1680b79ecb60fcff14c56b6b8e92.js

In Python, we instantiate CoreV1Api class from client module:

CODE: https://gist.github.com/velotiotech/ccf4da4686fb7ff2eeb331a0fa91d692.js

- List namespaces: To list all the namespaces in your cluster, by-default lists at least four:

CODE: https://gist.github.com/velotiotech/a868caa65993b6d30479cd448a0a92bb.js

In the Python client, we can achieve the same by:

CODE: https://gist.github.com/velotiotech/f4b7aec6da33ffb924dccc47358718f0.js

Similarly, we can list all the resources or resources in a particular namespace.

For example, to list pods in all namespaces:

CODE: https://gist.github.com/velotiotech/01645633ca95ef6d8032e430c1ba1e36.js

For all the resources that can be group within a given namespace, we can use:

CODE: https://gist.github.com/velotiotech/aa7009eeb222ac86d38ddd8446d4b32d.js

Creating Resources: The usual way to create resources in Kubernetes is to use a kubectl create command with required parameters (defaults if not specified) or to use kubectl apply command, which takes a YAML/JSON format configuration file as input. This file contains all the specifications and metadata for the component to be created. For example:

CODE: https://gist.github.com/velotiotech/00dbaab45c19b9e47c94c77e2afe367c.js

Where the contents of nginx_depl.yaml could be as follows:

CODE: https://gist.github.com/velotiotech/4ceff07647185431cbe1edcf9ca5a4e7.js

To create resources in Python, though, we use create functions from the same instance of CoreV1Api class:

# v1.create_namespaced_pod(<namespace>, <body>)
# v1.create_namespaced_persistent_volume_claim(<namespace>, <body>)

So, basically, we just need two things: a string type namespace in which we want our resource to be in and a body.

This body is the same as the config.yaml file that we saw earlier. But, how exactly do we create or use that in our code? We utilize the component specific classes that this library offers us for this.

Let us take an example, to create a pod we use V1Pod class from the Kubernetes.client.

An instance of this V1Pod contains all the params like kind, metadata, spec, etc., so all we need to pass them and then we are good. And while we are at it, let’s create metadata and spec as well using a couple more classes.

1. V1ObjectMeta: This takes all the fields that can be part of metadata as parameters, e.g.

CODE: https://gist.github.com/velotiotech/11b8c44676e15a62cb57af2a53039176.js

2. V1Container: If you recall the brief definition of Kubernetes pods given earlier, we realize that pods are just layers above containers, which means we will have to provide the container(s) that the pods abstracts over. The V1Container class from Kubernetes client does just what we need.

These containers run the specified image, with their name taken as a parameter by the object. Containers also have several other parameters like volume_mounts, ports that can also be passed while instantiation or could be set later using object reference.

We create a container using:

CODE: https://gist.github.com/velotiotech/b00322287d1fec6130f919314682b2e1.js

Kubernetes pods can have multiple containers running inside, hence the V1PodSpec class expects a list of those while we create a pod spec.

containers = [container1, container2...]

‍3. V1PodSpec: Depending on the component we are working on, the class for its spec and params change. For a pod, we can use V1PodSpec as:

CODE: https://gist.github.com/velotiotech/deb3d0813887026088d61c517003dbf4.js

Now that we have both metadata and spec, let’s construct the pod’s body:

CODE: https://gist.github.com/velotiotech/75e5e5e60b13e17fff8fbdbd9aac9e14.js

And then, finally we could pass these to create a pod:

CODE: https://gist.github.com/velotiotech/96438c9ab47b9c0a64c8cb4b16a0ed15.js

And there you have it, that's how you create a pod.

Similarly, we can create other resources, although not all resources take the same set of parameters, for example PersistentVolume (PV in short) does not come under namespaces, it is a cluster wide resource, so naturally it won’t be expecting a namespace parameter.

Fetching Logs:

When it comes to monitoring and debugging Kubernetes’ resources, logs play a major role. Using the Kubernetes Python client, we can fetch logs for resources. For example, to fetch logs for a pod:

Using kubectl:

CODE: https://gist.github.com/velotiotech/e841cee2eed34c054da64055f81304e3.js

Using Python:

CODE: https://gist.github.com/velotiotech/0cb36a2b2473e8153d2d81376e2d7868.js‍

Deleting Resources: For deletion, we will be following the same class that we have been using so far, i.e kubernetes.client.CoreV1Api.

There are functions that directly deal with deletion of that component, for example:

CODE: https://gist.github.com/velotiotech/f7a9249dcc8de47785c064f8e166d04b.js

Pass the required parameters and the deletion will take place as expected.

Complete Example for creating a Kubernetes Pod:

CODE: https://gist.github.com/velotiotech/0fead1d5f4540767cd79c48f6000ea82.js

Conclusion

There are quite a lot of ways this article could have been written, but as we conclude, it’s quite evident that we have barely scratched the surface. There are many more interesting, advanced things that we can do with this library, but those are beyond the scope of this article.

We can do almost all the operations with the Python client that we usually do with kubectl on Kubernetes resources. We hope that we managed to keep the content both interesting and informative. 

If you’re looking for a comprehensive guide on Kubernetes or something interesting to do with it, don't worry, we’ve got you covered. You can refer to a few of our other articles and might find just what you need:

1. Kubernetes CSI in Action: Explained with Features and Use Cases

2. Continuous Deployment with Azure Kubernetes Service, Azure Container Registry & Jenkins

3. Demystifying High Availability in Kubernetes Using Kubeadm

References

1. Official Kubernetes documentation: https://kubernetes.io/docs

2. Kubernetes Resources: https://kubernetes.io/docs/reference/glossary/?core-object=true

3. Kubernetes Python client: https://github.com/kubernetes-client/python

4. Kubclt: https://kubernetes.io/docs/reference/kubectl/overview/