.svg)
.svg)
Continuous Integration and Continuous Delivery (CI/CD) pipelines are essential components of modern software development, especially in the world of Kubernetes and containerized applications. To facilitate these pipelines, many organizations use Prow, a CI/CD system built specifically for Kubernetes. While Prow offers a rich set of features out of the box, you may need to develop your own plugins to tailor the system to your organization’s requirements. In this guide, we’ll explore the world of Prow plugin development and show you how to get started.
Before diving into Prow plugin development, ensure you have the following prerequisites:
While Prow provides a wide range of built-in plugins, your organization’s Kubernetes workflow may have specific requirements that aren’t covered by these defaults. This is where developing custom Prow plugins comes into play. Custom plugins allow you to extend Prow’s functionality to cater to your needs. Whether automating workflows, integrating with other tools, or enforcing custom policies, developing your own Prow plugins gives you the power to tailor your CI/CD pipeline precisely.
Developing a custom Prow plugin may seem daunting, but with the right approach and tools, it can be a rewarding experience. Here’s a step-by-step guide to get you started:
Before diving into plugin development, you need to set up your development environment. You will need Git, Go, and access to a Kubernetes cluster for testing your plugins. Ensure you have the necessary permissions to make changes to your organization’s Prow setup.
Prow supports various plugin types, including postsubmits, presubmits, triggers, and utilities. Choose the type that best fits your use case.
Once you’ve chosen a plugin type, it’s time to create it. Below is an example of a simple Prow plugin written in Go, named comment-plugin.go. It will create a comment on a pull request each time an event is received.
This code sets up a basic HTTP server that listens for GitHub events and handles them by creating a comment using the GitHub API. Customize this code to fit your specific use case.
| package main | |
| import ( | |
| "encoding/json" | |
| "flag" | |
| "net/http" | |
| "os" | |
| "strconv" | |
| "time" | |
| "github.com/sirupsen/logrus" | |
| "k8s.io/test-infra/pkg/flagutil" | |
| "k8s.io/test-infra/prow/config" | |
| "k8s.io/test-infra/prow/config/secret" | |
| prowflagutil "k8s.io/test-infra/prow/flagutil" | |
| configflagutil "k8s.io/test-infra/prow/flagutil/config" | |
| "k8s.io/test-infra/prow/github" | |
| "k8s.io/test-infra/prow/interrupts" | |
| "k8s.io/test-infra/prow/logrusutil" | |
| "k8s.io/test-infra/prow/pjutil" | |
| "k8s.io/test-infra/prow/pluginhelp" | |
| "k8s.io/test-infra/prow/pluginhelp/externalplugins" | |
| ) | |
| const pluginName = "comment-plugin" | |
| type options struct { | |
| port int | |
| config configflagutil.ConfigOptions | |
| dryRun bool | |
| github prowflagutil.GitHubOptions | |
| instrumentationOptions prowflagutil.InstrumentationOptions | |
| webhookSecretFile string | |
| } | |
| type server struct { | |
| tokenGenerator func() []byte | |
| botUser *github.UserData | |
| email string | |
| ghc github.Client | |
| log *logrus.Entry | |
| repos []github.Repo | |
| } | |
| func helpProvider(_ []config.OrgRepo) (*pluginhelp.PluginHelp, error) { | |
| pluginHelp := &pluginhelp.PluginHelp{ | |
| Description: `The sample plugin`, | |
| } | |
| return pluginHelp, nil | |
| } | |
| func (o *options) Validate() error { | |
| return nil | |
| } | |
| func gatherOptions() options { | |
| o := options{config: configflagutil.ConfigOptions{ConfigPath: "./config.yaml"}} | |
| fs := flag.NewFlagSet(os.Args[0], flag.ExitOnError) | |
| fs.IntVar(&o.port, "port", 8888, "Port to listen on.") | |
| fs.BoolVar(&o.dryRun, "dry-run", false, "Dry run for testing. Uses API tokens but does not mutate.") | |
| fs.StringVar(&o.webhookSecretFile, "hmac-secret-file", "/etc/hmac", "Path to the file containing GitHub HMAC secret.") | |
| for _, group := range []flagutil.OptionGroup{&o.github} { | |
| group.AddFlags(fs) | |
| } | |
| fs.Parse(os.Args[1:]) | |
| return o | |
| } | |
| func main() { | |
| o := gatherOptions() | |
| if err := o.Validate(); err != nil { | |
| logrus.Fatalf("Invalid options: %v", err) | |
| } | |
| logrusutil.ComponentInit() | |
| log := logrus.StandardLogger().WithField("plugin", pluginName) | |
| if err := secret.Add(o.webhookSecretFile); err != nil { | |
| logrus.WithError(err).Fatal("Error starting secrets agent.") | |
| } | |
| gitHubClient, err := o.github.GitHubClient(o.dryRun) | |
| if err != nil { | |
| logrus.WithError(err).Fatal("Error getting GitHub client.") | |
| } | |
| email, err := gitHubClient.Email() | |
| if err != nil { | |
| log.WithError(err).Fatal("Error getting bot e-mail.") | |
| } | |
| botUser, err := gitHubClient.BotUser() | |
| if err != nil { | |
| logrus.WithError(err).Fatal("Error getting bot name.") | |
| } | |
| repos, err := gitHubClient.GetRepos(botUser.Login, true) | |
| if err != nil { | |
| log.WithError(err).Fatal("Error listing bot repositories.") | |
| } | |
| serv := &server{ | |
| tokenGenerator: secret.GetTokenGenerator(o.webhookSecretFile), | |
| botUser: botUser, | |
| email: email, | |
| ghc: gitHubClient, | |
| log: log, | |
| repos: repos, | |
| } | |
| health := pjutil.NewHealthOnPort(o.instrumentationOptions.HealthPort) | |
| health.ServeReady() | |
| mux := http.NewServeMux() | |
| mux.Handle("/", serv) | |
| externalplugins.ServeExternalPluginHelp(mux, log, helpProvider) | |
| logrus.Info("starting server " + strconv.Itoa(o.port)) | |
| httpServer := &http.Server{Addr: ":" + strconv.Itoa(o.port), Handler: mux} | |
| defer interrupts.WaitForGracefulShutdown() | |
| interrupts.ListenAndServe(httpServer, 5*time.Second) | |
| } | |
| func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) { | |
| logrus.Info("inside http server") | |
| _, _, payload, ok, _ := github.ValidateWebhook(w, r, s.tokenGenerator) | |
| logrus.Info(string(payload)) | |
| if !ok { | |
| return | |
| } | |
| logrus.Info(w, "Event received. Have a nice day.") | |
| if err := s.handleEvent(payload); err != nil { | |
| logrus.WithError(err).Error("Error parsing event.") | |
| } | |
| } | |
| func (s *server) handleEvent(payload []byte) error { | |
| logrus.Info("inside handler") | |
| var pr github.PullRequestEvent | |
| if err := json.Unmarshal(payload, &pr); err != nil { | |
| return err | |
| } | |
| logrus.Info(pr.Number) | |
| if err := s.ghc.CreateComment(pr.PullRequest.Base.Repo.Owner.Login, pr.PullRequest.Base.Repo.Name, pr.Number, "comment from smaple-plugin"); err != nil { | |
| return err | |
| } | |
| return nil | |
| } |
To deploy your custom Prow plugin, you will need to create a Docker image and deploy it into your Prow cluster.
| FROM golang as app-builder | |
| WORKDIR /app | |
| RUN apt update | |
| RUN apt-get install git | |
| COPY . . | |
| RUN CGO_ENABLED=0 go build -o main | |
| FROM alpine:3.9 | |
| RUN apk add ca-certificates git | |
| COPY --from=app-builder /app/main /app/custom-plugin | |
| ENTRYPOINT ["/app/custom-plugin"] |
docker build -t jainbhavya65/custom-plugin:v1 .
docker push jainbhavya65/custom-plugin:v1
| apiVersion: apps/v1 | |
| kind: Deployment | |
| metadata: | |
| name: comment-plugin | |
| spec: | |
| progressDeadlineSeconds: 600 | |
| replicas: 1 | |
| revisionHistoryLimit: 10 | |
| selector: | |
| matchLabels: | |
| app: comment-plugin | |
| strategy: | |
| rollingUpdate: | |
| maxSurge: 25% | |
| maxUnavailable: 25% | |
| type: RollingUpdate | |
| template: | |
| metadata: | |
| creationTimestamp: null | |
| labels: | |
| app: comment-plugin | |
| spec: | |
| containers: | |
| - args: | |
| - --github-token-path=/etc/github/oauth | |
| - --hmac-secret-file=/etc/hmac-token/hmac | |
| - --port=80 | |
| image: <IMAGE> | |
| imagePullPolicy: Always | |
| name: comment-plugin | |
| ports: | |
| - containerPort: 80 | |
| protocol: TCP | |
| volumeMounts: | |
| - mountPath: /etc/github | |
| name: oauth | |
| readOnly: true | |
| - mountPath: /etc/hmac-token | |
| name: hmac | |
| readOnly: true | |
| volumes: | |
| - name: oauth | |
| secret: | |
| defaultMode: 420 | |
| secretName: oauth-token | |
| - name: hmac | |
| secret: | |
| defaultMode: 420 | |
| secretName: hmac-token |
| apiVersion: v1 | |
| kind: Service | |
| metadata: | |
| name: comment-plugin | |
| spec: | |
| ports: | |
| - port: 80 | |
| protocol: TCP | |
| targetPort: 80 | |
| selector: | |
| app: comment-plugin | |
| sessionAffinity: None | |
| type: ClusterIP |
After creating the deployment and service, integrate it into your organization’s Prow configuration. This involves updating your Prow plugin.yaml files to include your plugin and specify when it should run.
| external_plugins: | |
| - name: comment-plugin | |
| # No endpoint specified implies "http://{{name}}". // as we deploy plugin into same cluster | |
| # if plugin is not deployed in same cluster then you can give endpoint | |
| events: | |
| # only pull request and issue comment events are send to our plugin | |
| - pull_request | |
| - issue_comment |
Mastering Prow plugin development opens up a world of possibilities for tailoring your Kubernetes CI/CD workflow to meet your organization’s needs. While the initial learning curve may be steep, the benefits of custom plugins in terms of automation, efficiency, and control are well worth the effort.
Remember that the key to successful Prow plugin development lies in clear documentation, thorough testing, and collaboration with your team to ensure that your custom plugins enhance your CI/CD pipeline’s functionality and reliability. As Kubernetes and containerized applications continue to evolve, Prow will remain a valuable tool for managing your CI/CD processes, and your custom plugins will be the secret sauce that sets your workflow apart from the rest.
Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.
We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.
Continuous Integration and Continuous Delivery (CI/CD) pipelines are essential components of modern software development, especially in the world of Kubernetes and containerized applications. To facilitate these pipelines, many organizations use Prow, a CI/CD system built specifically for Kubernetes. While Prow offers a rich set of features out of the box, you may need to develop your own plugins to tailor the system to your organization’s requirements. In this guide, we’ll explore the world of Prow plugin development and show you how to get started.
Before diving into Prow plugin development, ensure you have the following prerequisites:
While Prow provides a wide range of built-in plugins, your organization’s Kubernetes workflow may have specific requirements that aren’t covered by these defaults. This is where developing custom Prow plugins comes into play. Custom plugins allow you to extend Prow’s functionality to cater to your needs. Whether automating workflows, integrating with other tools, or enforcing custom policies, developing your own Prow plugins gives you the power to tailor your CI/CD pipeline precisely.
Developing a custom Prow plugin may seem daunting, but with the right approach and tools, it can be a rewarding experience. Here’s a step-by-step guide to get you started:
Before diving into plugin development, you need to set up your development environment. You will need Git, Go, and access to a Kubernetes cluster for testing your plugins. Ensure you have the necessary permissions to make changes to your organization’s Prow setup.
Prow supports various plugin types, including postsubmits, presubmits, triggers, and utilities. Choose the type that best fits your use case.
Once you’ve chosen a plugin type, it’s time to create it. Below is an example of a simple Prow plugin written in Go, named comment-plugin.go. It will create a comment on a pull request each time an event is received.
This code sets up a basic HTTP server that listens for GitHub events and handles them by creating a comment using the GitHub API. Customize this code to fit your specific use case.
| package main | |
| import ( | |
| "encoding/json" | |
| "flag" | |
| "net/http" | |
| "os" | |
| "strconv" | |
| "time" | |
| "github.com/sirupsen/logrus" | |
| "k8s.io/test-infra/pkg/flagutil" | |
| "k8s.io/test-infra/prow/config" | |
| "k8s.io/test-infra/prow/config/secret" | |
| prowflagutil "k8s.io/test-infra/prow/flagutil" | |
| configflagutil "k8s.io/test-infra/prow/flagutil/config" | |
| "k8s.io/test-infra/prow/github" | |
| "k8s.io/test-infra/prow/interrupts" | |
| "k8s.io/test-infra/prow/logrusutil" | |
| "k8s.io/test-infra/prow/pjutil" | |
| "k8s.io/test-infra/prow/pluginhelp" | |
| "k8s.io/test-infra/prow/pluginhelp/externalplugins" | |
| ) | |
| const pluginName = "comment-plugin" | |
| type options struct { | |
| port int | |
| config configflagutil.ConfigOptions | |
| dryRun bool | |
| github prowflagutil.GitHubOptions | |
| instrumentationOptions prowflagutil.InstrumentationOptions | |
| webhookSecretFile string | |
| } | |
| type server struct { | |
| tokenGenerator func() []byte | |
| botUser *github.UserData | |
| email string | |
| ghc github.Client | |
| log *logrus.Entry | |
| repos []github.Repo | |
| } | |
| func helpProvider(_ []config.OrgRepo) (*pluginhelp.PluginHelp, error) { | |
| pluginHelp := &pluginhelp.PluginHelp{ | |
| Description: `The sample plugin`, | |
| } | |
| return pluginHelp, nil | |
| } | |
| func (o *options) Validate() error { | |
| return nil | |
| } | |
| func gatherOptions() options { | |
| o := options{config: configflagutil.ConfigOptions{ConfigPath: "./config.yaml"}} | |
| fs := flag.NewFlagSet(os.Args[0], flag.ExitOnError) | |
| fs.IntVar(&o.port, "port", 8888, "Port to listen on.") | |
| fs.BoolVar(&o.dryRun, "dry-run", false, "Dry run for testing. Uses API tokens but does not mutate.") | |
| fs.StringVar(&o.webhookSecretFile, "hmac-secret-file", "/etc/hmac", "Path to the file containing GitHub HMAC secret.") | |
| for _, group := range []flagutil.OptionGroup{&o.github} { | |
| group.AddFlags(fs) | |
| } | |
| fs.Parse(os.Args[1:]) | |
| return o | |
| } | |
| func main() { | |
| o := gatherOptions() | |
| if err := o.Validate(); err != nil { | |
| logrus.Fatalf("Invalid options: %v", err) | |
| } | |
| logrusutil.ComponentInit() | |
| log := logrus.StandardLogger().WithField("plugin", pluginName) | |
| if err := secret.Add(o.webhookSecretFile); err != nil { | |
| logrus.WithError(err).Fatal("Error starting secrets agent.") | |
| } | |
| gitHubClient, err := o.github.GitHubClient(o.dryRun) | |
| if err != nil { | |
| logrus.WithError(err).Fatal("Error getting GitHub client.") | |
| } | |
| email, err := gitHubClient.Email() | |
| if err != nil { | |
| log.WithError(err).Fatal("Error getting bot e-mail.") | |
| } | |
| botUser, err := gitHubClient.BotUser() | |
| if err != nil { | |
| logrus.WithError(err).Fatal("Error getting bot name.") | |
| } | |
| repos, err := gitHubClient.GetRepos(botUser.Login, true) | |
| if err != nil { | |
| log.WithError(err).Fatal("Error listing bot repositories.") | |
| } | |
| serv := &server{ | |
| tokenGenerator: secret.GetTokenGenerator(o.webhookSecretFile), | |
| botUser: botUser, | |
| email: email, | |
| ghc: gitHubClient, | |
| log: log, | |
| repos: repos, | |
| } | |
| health := pjutil.NewHealthOnPort(o.instrumentationOptions.HealthPort) | |
| health.ServeReady() | |
| mux := http.NewServeMux() | |
| mux.Handle("/", serv) | |
| externalplugins.ServeExternalPluginHelp(mux, log, helpProvider) | |
| logrus.Info("starting server " + strconv.Itoa(o.port)) | |
| httpServer := &http.Server{Addr: ":" + strconv.Itoa(o.port), Handler: mux} | |
| defer interrupts.WaitForGracefulShutdown() | |
| interrupts.ListenAndServe(httpServer, 5*time.Second) | |
| } | |
| func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) { | |
| logrus.Info("inside http server") | |
| _, _, payload, ok, _ := github.ValidateWebhook(w, r, s.tokenGenerator) | |
| logrus.Info(string(payload)) | |
| if !ok { | |
| return | |
| } | |
| logrus.Info(w, "Event received. Have a nice day.") | |
| if err := s.handleEvent(payload); err != nil { | |
| logrus.WithError(err).Error("Error parsing event.") | |
| } | |
| } | |
| func (s *server) handleEvent(payload []byte) error { | |
| logrus.Info("inside handler") | |
| var pr github.PullRequestEvent | |
| if err := json.Unmarshal(payload, &pr); err != nil { | |
| return err | |
| } | |
| logrus.Info(pr.Number) | |
| if err := s.ghc.CreateComment(pr.PullRequest.Base.Repo.Owner.Login, pr.PullRequest.Base.Repo.Name, pr.Number, "comment from smaple-plugin"); err != nil { | |
| return err | |
| } | |
| return nil | |
| } |
To deploy your custom Prow plugin, you will need to create a Docker image and deploy it into your Prow cluster.
| FROM golang as app-builder | |
| WORKDIR /app | |
| RUN apt update | |
| RUN apt-get install git | |
| COPY . . | |
| RUN CGO_ENABLED=0 go build -o main | |
| FROM alpine:3.9 | |
| RUN apk add ca-certificates git | |
| COPY --from=app-builder /app/main /app/custom-plugin | |
| ENTRYPOINT ["/app/custom-plugin"] |
docker build -t jainbhavya65/custom-plugin:v1 .
docker push jainbhavya65/custom-plugin:v1
| apiVersion: apps/v1 | |
| kind: Deployment | |
| metadata: | |
| name: comment-plugin | |
| spec: | |
| progressDeadlineSeconds: 600 | |
| replicas: 1 | |
| revisionHistoryLimit: 10 | |
| selector: | |
| matchLabels: | |
| app: comment-plugin | |
| strategy: | |
| rollingUpdate: | |
| maxSurge: 25% | |
| maxUnavailable: 25% | |
| type: RollingUpdate | |
| template: | |
| metadata: | |
| creationTimestamp: null | |
| labels: | |
| app: comment-plugin | |
| spec: | |
| containers: | |
| - args: | |
| - --github-token-path=/etc/github/oauth | |
| - --hmac-secret-file=/etc/hmac-token/hmac | |
| - --port=80 | |
| image: <IMAGE> | |
| imagePullPolicy: Always | |
| name: comment-plugin | |
| ports: | |
| - containerPort: 80 | |
| protocol: TCP | |
| volumeMounts: | |
| - mountPath: /etc/github | |
| name: oauth | |
| readOnly: true | |
| - mountPath: /etc/hmac-token | |
| name: hmac | |
| readOnly: true | |
| volumes: | |
| - name: oauth | |
| secret: | |
| defaultMode: 420 | |
| secretName: oauth-token | |
| - name: hmac | |
| secret: | |
| defaultMode: 420 | |
| secretName: hmac-token |
| apiVersion: v1 | |
| kind: Service | |
| metadata: | |
| name: comment-plugin | |
| spec: | |
| ports: | |
| - port: 80 | |
| protocol: TCP | |
| targetPort: 80 | |
| selector: | |
| app: comment-plugin | |
| sessionAffinity: None | |
| type: ClusterIP |
After creating the deployment and service, integrate it into your organization’s Prow configuration. This involves updating your Prow plugin.yaml files to include your plugin and specify when it should run.
| external_plugins: | |
| - name: comment-plugin | |
| # No endpoint specified implies "http://{{name}}". // as we deploy plugin into same cluster | |
| # if plugin is not deployed in same cluster then you can give endpoint | |
| events: | |
| # only pull request and issue comment events are send to our plugin | |
| - pull_request | |
| - issue_comment |
Mastering Prow plugin development opens up a world of possibilities for tailoring your Kubernetes CI/CD workflow to meet your organization’s needs. While the initial learning curve may be steep, the benefits of custom plugins in terms of automation, efficiency, and control are well worth the effort.
Remember that the key to successful Prow plugin development lies in clear documentation, thorough testing, and collaboration with your team to ensure that your custom plugins enhance your CI/CD pipeline’s functionality and reliability. As Kubernetes and containerized applications continue to evolve, Prow will remain a valuable tool for managing your CI/CD processes, and your custom plugins will be the secret sauce that sets your workflow apart from the rest.
.svg)
.svg)
