.svg)
.svg)
A couple of weeks back, we started working with the Kubernetes Python client to carry out basic operations on its components/ resources, and that’s when we realized how few resources there were (guides, docs) on the internet. So, we experimented and decided to share our findings with the community.
This article is targeted towards an audience that is familiar with Kubernetes, its usage, and its architecture. This is not a simple Kubernetes guide; it’s about Kubernetes using Python, so as we move further, we may shed light on a few things that are required, but a few will be left for self exploration.
Kubernetes is an open-source container orchestration tool, largely used to simplify the process of deployment, maintenance, etc. in application development. Kubernetes is built to offer highly available, scalable, and reliable applications.
Generally, kubectl commands are used to create, list, and delete the Kubernetes resources, but for this article, we put on a developer's hat and use the Python way of doing things. In this article, we learn how to create, manage, and interact with Kubernetes resources using the Kubernetes' Python library.
But why, you may ask?
Well, having an option of doing things programmatically creates potential of endless exciting innovations for developers. Using Python, we can:
So, let us begin:
Kubernetes achieves what it does with the help of its resources. These resources are the building blocks for developing a scalable, reliable application.
.jpg)
Let's briefly explore these resources to understand what they are and how exactly they work together in Kubernetes:
These are a few of the basic Kubernetes resources. If you want to explore the rest of these resources, you can click here.
Apart from these resources, there are also namespaces in Kubernetes. You will come across them quite a few times in this article, so here are some noteworthy points for Kubernetes namespaces:
Kubernetes namespaces: Used to group/organize resources in the cluster. It is like a virtual cluster inside a cluster. Namespaces are used to:
The Kubernetes library comes to our aid with quite a few modules, the ones featured in this article are client and config modules from the package; we will be using these two heavily. So, let’s install the Kubernetes Python Client:
To install the Kubernetes Python client, we make use of Python’s standard package installer pip:
| pip install kubernetes |
For installation from the source, we can refer to this guide from the official Python client git repository.
Now that we have the python-kubernetes package installed, we can import it as:
| from kubernetes import client, config |
To load our cluster configurations, we can use one of the following methods:
| config.load_kube_config() # for local environment | |
| # or | |
| config.load_incluster_config() |
Executing this will load the configurations for your clusters from your local or remote .kube/config file.
Now that we have loaded the configurations, we can use the client module to interact with the resources.
Get Resources: kubectl get commands are used to list all kinds of resources in a cluster for eg:
- List nodes: To list all the nodes in the cluster, we fire following kubectl command:
| kubectl get nodes # lists all the nodes |
In Python, we instantiate CoreV1Api class from client module:
| v1 = client.CoreV1Api() | |
| v1.list_node() | |
| # returns a JSON with all the info like spec, metadata etc, for each node |
- List namespaces: To list all the namespaces in your cluster, by-default lists at least four:
| kubectl get namespaces | |
| # NAME STATUS AGE | |
| # default Active 94d | |
| # kube-public Active 94d | |
| # kube-system Active 94d |
In the Python client, we can achieve the same by:
| v1.list_namespace() | |
| """ | |
| returns a JSON with all the info like spec, metadata for each namespace | |
| For eg: | |
| {'api_version': 'v1', | |
| 'items': [{'api_version': None, | |
| 'kind': None, | |
| 'metadata': {'annotations': None, | |
| 'cluster_name': None, | |
| 'creation_timestamp': datetime.datetime(2021, 2, 11, 11, 29, 32, tzinfo=tzutc()), | |
| 'deletion_grace_period_seconds': None, | |
| 'deletion_timestamp': None, | |
| 'finalizers': None, | |
| 'generate_name': None, | |
| 'generation': None, | |
| 'labels': None, | |
| 'managed_fields': [{'api_version': 'v1', | |
| 'fields_type': 'FieldsV1', | |
| 'fields_v1': {'f:status': {'f:phase': {}}}, | |
| 'manager': 'kube-apiserver', | |
| 'operation': 'Update', | |
| 'time': datetime.datetime(2021, 2, 11, 11, 29, 32, tzinfo=tzutc())}], | |
| 'name': 'default', | |
| 'namespace': None, | |
| 'owner_references': None, | |
| 'resource_version': '199', | |
| 'self_link': None, | |
| 'uid': '3a362d64-437d-45b5-af19-4af9ae2c75fc'}, | |
| 'spec': {'finalizers': ['kubernetes']}, | |
| 'status': {'conditions': None, 'phase': 'Active'}}], | |
| 'kind': 'NamespaceList', | |
| 'metadata': {'_continue': None, | |
| 'remaining_item_count': None, | |
| 'resource_version': '69139', | |
| 'self_link': None}} | |
| """ |
Similarly, we can list all the resources or resources in a particular namespace.
For example, to list pods in all namespaces:
| v1.list_pod_for_all_namespaces() | |
| v1.list_persistent_volume_claim_for_all_namespaces() |
For all the resources that can be group within a given namespace, we can use:
| # v1.list_namespaced_pod(<namespace>) | |
| v1.list_namespaced_pod(namespace=’default’) | |
| # v1.list_namespaced_service(<namespace>) | |
| v1.list_namespaced_service(namespace=’default’) | |
| and so on. |
Creating Resources: The usual way to create resources in Kubernetes is to use a kubectl create command with required parameters (defaults if not specified) or to use kubectl apply command, which takes a YAML/JSON format configuration file as input. This file contains all the specifications and metadata for the component to be created. For example:
| kubectl create deployment my-nginx-depl --image=nginx | |
| kubectl apply -f nginx_depl.yaml |
Where the contents of nginx_depl.yaml could be as follows:
| apiVersion: apps/v1 | |
| kind: Deployment | |
| metadata: | |
| name: nginx-deployment | |
| labels: | |
| app: nginx | |
| spec: | |
| replicas: 3 | |
| selector: | |
| matchLabels: | |
| app: nginx | |
| template: | |
| metadata: | |
| labels: | |
| app: nginx | |
| spec: | |
| containers: | |
| - name: nginx | |
| image: nginx:1.14.2 | |
| ports: | |
| - containerPort: 80 |
To create resources in Python, though, we use create functions from the same instance of CoreV1Api class:
# v1.create_namespaced_pod(<namespace>, <body>)
# v1.create_namespaced_persistent_volume_claim(<namespace>, <body>)
So, basically, we just need two things: a string type namespace in which we want our resource to be in and a body.
This body is the same as the config.yaml file that we saw earlier. But, how exactly do we create or use that in our code? We utilize the component specific classes that this library offers us for this.
Let us take an example, to create a pod we use V1Pod class from the Kubernetes.client.
An instance of this V1Pod contains all the params like kind, metadata, spec, etc., so all we need to pass them and then we are good. And while we are at it, let’s create metadata and spec as well using a couple more classes.
1. V1ObjectMeta: This takes all the fields that can be part of metadata as parameters, e.g.
| metadata = client.V1ObjectMeta(name='md1') | |
| # We could also set fields by accessing them through instance like: | |
| metadata.name = 'md2' |
2. V1Container: If you recall the brief definition of Kubernetes pods given earlier, we realize that pods are just layers above containers, which means we will have to provide the container(s) that the pods abstracts over. The V1Container class from Kubernetes client does just what we need.
These containers run the specified image, with their name taken as a parameter by the object. Containers also have several other parameters like volume_mounts, ports that can also be passed while instantiation or could be set later using object reference.
We create a container using:
| # container1 = client.V1Container(<name>, <image>) e.g: | |
| container1 = client.V1Container(‘my_container’, ‘nginx’) |
Kubernetes pods can have multiple containers running inside, hence the V1PodSpec class expects a list of those while we create a pod spec.
containers = [container1, container2...]
3. V1PodSpec: Depending on the component we are working on, the class for its spec and params change. For a pod, we can use V1PodSpec as:
| # pod_spec = client.V1PodSpec(<containers_list>) | |
| pod_spec = client.V1PodSpec(containers=containers) |
Now that we have both metadata and spec, let’s construct the pod’s body:
| pod_body = client.V1Pod(metadata=metadata, spec=pod_spec, kind='Pod', api_version='v1') |
And then, finally we could pass these to create a pod:
| pod = v1.create_namespaced_pod(namespace=my-namespace, body=pod_body) |
And there you have it, that's how you create a pod.
Similarly, we can create other resources, although not all resources take the same set of parameters, for example PersistentVolume (PV in short) does not come under namespaces, it is a cluster wide resource, so naturally it won’t be expecting a namespace parameter.
When it comes to monitoring and debugging Kubernetes’ resources, logs play a major role. Using the Kubernetes Python client, we can fetch logs for resources. For example, to fetch logs for a pod:
Using kubectl:
| # kubectl logs pod_name | |
| kubectl logs my-pod |
Using Python:
| pod_logs = v1.read_namespaced_pod_log(<pod_name>, <namespace>) | |
| pod_logs = v1.read_namespaced_pod_log(name=’my-app’, namespace=’default’) |
Deleting Resources: For deletion, we will be following the same class that we have been using so far, i.e kubernetes.client.CoreV1Api.
There are functions that directly deal with deletion of that component, for example:
| #v1.delete_namespaced_pod(<pod_name>, <namespace>) | |
| v1.delete_namespaced_pod(name=’my-app’, namespace=’default’) |
Pass the required parameters and the deletion will take place as expected.
Complete Example for creating a Kubernetes Pod:
| from kubernetes import client, config | |
| config.load_kube_config() | |
| v1 = client.CoreV1Api() | |
| namespaces_list = v1.list_namespace() | |
| namespaces = [item.metadata.name for item in namespaces_list.items] | |
| pods_list = v1.list_namespaced_pod(namespace=’default’) | |
| pods = [item.metadata.name for item in pod_list.items] | |
| containers = [] | |
| container1 = client.V1Container(name=’my-nginx-container’, image=’nginx’) | |
| containers.append(container1) | |
| pod_spec = client.V1PodSpec(containers=containers) | |
| pod_metadata = client.V1ObjectMeta(name=’my-pod’, namespace=’default’) | |
| pod_body = client.V1Pod(api_version=’v1’, kind=’Pod’, metadata=pod_metadata, spec=pod_spec) | |
| v1.create_namespaced_pod(namespace=’default’, body=pod_body) | |
| pod_logs = v1.read_namespaced_pod_log(name=’my-pod’, namespace='default') | |
| v1.delete_namespaced_pod(namespace=’default’, name=’my-pod’) |
There are quite a lot of ways this article could have been written, but as we conclude, it’s quite evident that we have barely scratched the surface. There are many more interesting, advanced things that we can do with this library, but those are beyond the scope of this article.
We can do almost all the operations with the Python client that we usually do with kubectl on Kubernetes resources. We hope that we managed to keep the content both interesting and informative.
If you’re looking for a comprehensive guide on Kubernetes or something interesting to do with it, don't worry, we’ve got you covered. You can refer to a few of our other articles and might find just what you need:
1. Kubernetes CSI in Action: Explained with Features and Use Cases
2. Continuous Deployment with Azure Kubernetes Service, Azure Container Registry & Jenkins
3. Demystifying High Availability in Kubernetes Using Kubeadm
1. Official Kubernetes documentation: https://kubernetes.io/docs
2. Kubernetes Resources: https://kubernetes.io/docs/reference/glossary/?core-object=true
3. Kubernetes Python client: https://github.com/kubernetes-client/python
4. Kubclt: https://kubernetes.io/docs/reference/kubectl/overview/
Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.
We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.
A couple of weeks back, we started working with the Kubernetes Python client to carry out basic operations on its components/ resources, and that’s when we realized how few resources there were (guides, docs) on the internet. So, we experimented and decided to share our findings with the community.
This article is targeted towards an audience that is familiar with Kubernetes, its usage, and its architecture. This is not a simple Kubernetes guide; it’s about Kubernetes using Python, so as we move further, we may shed light on a few things that are required, but a few will be left for self exploration.
Kubernetes is an open-source container orchestration tool, largely used to simplify the process of deployment, maintenance, etc. in application development. Kubernetes is built to offer highly available, scalable, and reliable applications.
Generally, kubectl commands are used to create, list, and delete the Kubernetes resources, but for this article, we put on a developer's hat and use the Python way of doing things. In this article, we learn how to create, manage, and interact with Kubernetes resources using the Kubernetes' Python library.
But why, you may ask?
Well, having an option of doing things programmatically creates potential of endless exciting innovations for developers. Using Python, we can:
So, let us begin:
Kubernetes achieves what it does with the help of its resources. These resources are the building blocks for developing a scalable, reliable application.
Let's briefly explore these resources to understand what they are and how exactly they work together in Kubernetes:
These are a few of the basic Kubernetes resources. If you want to explore the rest of these resources, you can click here.
Apart from these resources, there are also namespaces in Kubernetes. You will come across them quite a few times in this article, so here are some noteworthy points for Kubernetes namespaces:
Kubernetes namespaces: Used to group/organize resources in the cluster. It is like a virtual cluster inside a cluster. Namespaces are used to:
The Kubernetes library comes to our aid with quite a few modules, the ones featured in this article are client and config modules from the package; we will be using these two heavily. So, let’s install the Kubernetes Python Client:
To install the Kubernetes Python client, we make use of Python’s standard package installer pip:
| pip install kubernetes |
For installation from the source, we can refer to this guide from the official Python client git repository.
Now that we have the python-kubernetes package installed, we can import it as:
| from kubernetes import client, config |
To load our cluster configurations, we can use one of the following methods:
| config.load_kube_config() # for local environment | |
| # or | |
| config.load_incluster_config() |
Executing this will load the configurations for your clusters from your local or remote .kube/config file.
Now that we have loaded the configurations, we can use the client module to interact with the resources.
Get Resources: kubectl get commands are used to list all kinds of resources in a cluster for eg:
- List nodes: To list all the nodes in the cluster, we fire following kubectl command:
| kubectl get nodes # lists all the nodes |
In Python, we instantiate CoreV1Api class from client module:
| v1 = client.CoreV1Api() | |
| v1.list_node() | |
| # returns a JSON with all the info like spec, metadata etc, for each node |
- List namespaces: To list all the namespaces in your cluster, by-default lists at least four:
| kubectl get namespaces | |
| # NAME STATUS AGE | |
| # default Active 94d | |
| # kube-public Active 94d | |
| # kube-system Active 94d |
In the Python client, we can achieve the same by:
| v1.list_namespace() | |
| """ | |
| returns a JSON with all the info like spec, metadata for each namespace | |
| For eg: | |
| {'api_version': 'v1', | |
| 'items': [{'api_version': None, | |
| 'kind': None, | |
| 'metadata': {'annotations': None, | |
| 'cluster_name': None, | |
| 'creation_timestamp': datetime.datetime(2021, 2, 11, 11, 29, 32, tzinfo=tzutc()), | |
| 'deletion_grace_period_seconds': None, | |
| 'deletion_timestamp': None, | |
| 'finalizers': None, | |
| 'generate_name': None, | |
| 'generation': None, | |
| 'labels': None, | |
| 'managed_fields': [{'api_version': 'v1', | |
| 'fields_type': 'FieldsV1', | |
| 'fields_v1': {'f:status': {'f:phase': {}}}, | |
| 'manager': 'kube-apiserver', | |
| 'operation': 'Update', | |
| 'time': datetime.datetime(2021, 2, 11, 11, 29, 32, tzinfo=tzutc())}], | |
| 'name': 'default', | |
| 'namespace': None, | |
| 'owner_references': None, | |
| 'resource_version': '199', | |
| 'self_link': None, | |
| 'uid': '3a362d64-437d-45b5-af19-4af9ae2c75fc'}, | |
| 'spec': {'finalizers': ['kubernetes']}, | |
| 'status': {'conditions': None, 'phase': 'Active'}}], | |
| 'kind': 'NamespaceList', | |
| 'metadata': {'_continue': None, | |
| 'remaining_item_count': None, | |
| 'resource_version': '69139', | |
| 'self_link': None}} | |
| """ |
Similarly, we can list all the resources or resources in a particular namespace.
For example, to list pods in all namespaces:
| v1.list_pod_for_all_namespaces() | |
| v1.list_persistent_volume_claim_for_all_namespaces() |
For all the resources that can be group within a given namespace, we can use:
| # v1.list_namespaced_pod(<namespace>) | |
| v1.list_namespaced_pod(namespace=’default’) | |
| # v1.list_namespaced_service(<namespace>) | |
| v1.list_namespaced_service(namespace=’default’) | |
| and so on. |
Creating Resources: The usual way to create resources in Kubernetes is to use a kubectl create command with required parameters (defaults if not specified) or to use kubectl apply command, which takes a YAML/JSON format configuration file as input. This file contains all the specifications and metadata for the component to be created. For example:
| kubectl create deployment my-nginx-depl --image=nginx | |
| kubectl apply -f nginx_depl.yaml |
Where the contents of nginx_depl.yaml could be as follows:
| apiVersion: apps/v1 | |
| kind: Deployment | |
| metadata: | |
| name: nginx-deployment | |
| labels: | |
| app: nginx | |
| spec: | |
| replicas: 3 | |
| selector: | |
| matchLabels: | |
| app: nginx | |
| template: | |
| metadata: | |
| labels: | |
| app: nginx | |
| spec: | |
| containers: | |
| - name: nginx | |
| image: nginx:1.14.2 | |
| ports: | |
| - containerPort: 80 |
To create resources in Python, though, we use create functions from the same instance of CoreV1Api class:
# v1.create_namespaced_pod(<namespace>, <body>)
# v1.create_namespaced_persistent_volume_claim(<namespace>, <body>)
So, basically, we just need two things: a string type namespace in which we want our resource to be in and a body.
This body is the same as the config.yaml file that we saw earlier. But, how exactly do we create or use that in our code? We utilize the component specific classes that this library offers us for this.
Let us take an example, to create a pod we use V1Pod class from the Kubernetes.client.
An instance of this V1Pod contains all the params like kind, metadata, spec, etc., so all we need to pass them and then we are good. And while we are at it, let’s create metadata and spec as well using a couple more classes.
1. V1ObjectMeta: This takes all the fields that can be part of metadata as parameters, e.g.
| metadata = client.V1ObjectMeta(name='md1') | |
| # We could also set fields by accessing them through instance like: | |
| metadata.name = 'md2' |
2. V1Container: If you recall the brief definition of Kubernetes pods given earlier, we realize that pods are just layers above containers, which means we will have to provide the container(s) that the pods abstracts over. The V1Container class from Kubernetes client does just what we need.
These containers run the specified image, with their name taken as a parameter by the object. Containers also have several other parameters like volume_mounts, ports that can also be passed while instantiation or could be set later using object reference.
We create a container using:
| # container1 = client.V1Container(<name>, <image>) e.g: | |
| container1 = client.V1Container(‘my_container’, ‘nginx’) |
Kubernetes pods can have multiple containers running inside, hence the V1PodSpec class expects a list of those while we create a pod spec.
containers = [container1, container2...]
3. V1PodSpec: Depending on the component we are working on, the class for its spec and params change. For a pod, we can use V1PodSpec as:
| # pod_spec = client.V1PodSpec(<containers_list>) | |
| pod_spec = client.V1PodSpec(containers=containers) |
Now that we have both metadata and spec, let’s construct the pod’s body:
| pod_body = client.V1Pod(metadata=metadata, spec=pod_spec, kind='Pod', api_version='v1') |
And then, finally we could pass these to create a pod:
| pod = v1.create_namespaced_pod(namespace=my-namespace, body=pod_body) |
And there you have it, that's how you create a pod.
Similarly, we can create other resources, although not all resources take the same set of parameters, for example PersistentVolume (PV in short) does not come under namespaces, it is a cluster wide resource, so naturally it won’t be expecting a namespace parameter.
When it comes to monitoring and debugging Kubernetes’ resources, logs play a major role. Using the Kubernetes Python client, we can fetch logs for resources. For example, to fetch logs for a pod:
Using kubectl:
| # kubectl logs pod_name | |
| kubectl logs my-pod |
Using Python:
| pod_logs = v1.read_namespaced_pod_log(<pod_name>, <namespace>) | |
| pod_logs = v1.read_namespaced_pod_log(name=’my-app’, namespace=’default’) |
Deleting Resources: For deletion, we will be following the same class that we have been using so far, i.e kubernetes.client.CoreV1Api.
There are functions that directly deal with deletion of that component, for example:
| #v1.delete_namespaced_pod(<pod_name>, <namespace>) | |
| v1.delete_namespaced_pod(name=’my-app’, namespace=’default’) |
Pass the required parameters and the deletion will take place as expected.
Complete Example for creating a Kubernetes Pod:
| from kubernetes import client, config | |
| config.load_kube_config() | |
| v1 = client.CoreV1Api() | |
| namespaces_list = v1.list_namespace() | |
| namespaces = [item.metadata.name for item in namespaces_list.items] | |
| pods_list = v1.list_namespaced_pod(namespace=’default’) | |
| pods = [item.metadata.name for item in pod_list.items] | |
| containers = [] | |
| container1 = client.V1Container(name=’my-nginx-container’, image=’nginx’) | |
| containers.append(container1) | |
| pod_spec = client.V1PodSpec(containers=containers) | |
| pod_metadata = client.V1ObjectMeta(name=’my-pod’, namespace=’default’) | |
| pod_body = client.V1Pod(api_version=’v1’, kind=’Pod’, metadata=pod_metadata, spec=pod_spec) | |
| v1.create_namespaced_pod(namespace=’default’, body=pod_body) | |
| pod_logs = v1.read_namespaced_pod_log(name=’my-pod’, namespace='default') | |
| v1.delete_namespaced_pod(namespace=’default’, name=’my-pod’) |
There are quite a lot of ways this article could have been written, but as we conclude, it’s quite evident that we have barely scratched the surface. There are many more interesting, advanced things that we can do with this library, but those are beyond the scope of this article.
We can do almost all the operations with the Python client that we usually do with kubectl on Kubernetes resources. We hope that we managed to keep the content both interesting and informative.
If you’re looking for a comprehensive guide on Kubernetes or something interesting to do with it, don't worry, we’ve got you covered. You can refer to a few of our other articles and might find just what you need:
1. Kubernetes CSI in Action: Explained with Features and Use Cases
2. Continuous Deployment with Azure Kubernetes Service, Azure Container Registry & Jenkins
3. Demystifying High Availability in Kubernetes Using Kubeadm
1. Official Kubernetes documentation: https://kubernetes.io/docs
2. Kubernetes Resources: https://kubernetes.io/docs/reference/glossary/?core-object=true
3. Kubernetes Python client: https://github.com/kubernetes-client/python
4. Kubclt: https://kubernetes.io/docs/reference/kubectl/overview/
.svg)
.svg)
.png)
