This blog post will demonstrate what a Kubernetes mutating webhook is and how to create a mutating webhook to inject some environment variable or configmap to all pods based on some condition.
Mutating webhook
So, what is a mutating webhook? According to the Kubernetes official documentation Admission, webhooks are HTTP callbacks that receive admission requests and do something with them. You can define two types of admission webhooks, validating admission webhook and mutating admission webhook. Mutating admission webhooks are invoked first and can modify objects sent to the API server to enforce custom defaults. Please refer to the below diagram to understand the workflow.
In this example, we will create an environment variable “username = demo-user” to all the pods (which contains the label “hello = test”) automatically through a Mutating admission webhook, so that every time we send a request to the API to create a Pod, the Pod spec is mutated before being saved to storage. Then when the Kubelet creates our Pod on worker nodes, it should have the environment variable included automatically.
Code repository
The complete code required to run this sample project is uploaded to my Github repository The repository also contains all the Kubernetes manifest files for deployment. I recommend you to pull this code to your machine as I will refer file names from this repository.
Pre-requsites
A working Kubernetes cluster is required for this tutorial. You can set up a Kubernetes cluster on VirtualBox with the help of vagrant, click here for more details.
Create k8s resources
Configmap
The first Kubernetes resource is configmap (filename = configmap.yaml), in this configmap we will create the environment variable “username = demo-user”. I have created a namespace called ns-webhook to deploy all our required resources.
kubectl create ns ns-webhookapiVersion: v1
kind: ConfigMap
metadata:
name: hello-configmap
namespace: ns-webhook
data:
username: demo-userkubectl apply -f configmap.yamlWebhook code
Next, we need a webhook that we will create using a simple Go API server. To create the webhook docker image, go to the webhook folder from the Git repository and execute the below command.
vagrant@master:~/webhook-repo/webhook$ sudo docker build .If everything goes fine you will get a final docker image and the below message.
Step 12/13 : COPY --from=builder /app/main ./main
---> cacddf9d08d2
Step 13/13 : CMD ["./main"]
---> Running in 9ba1feef8baa
Removing intermediate container 9ba1feef8baa
---> cf6395ca261d
Successfully built cf6395ca261dTag the image and upload it to your docker hub repository, later we will use this image for Kubernetes deployment.
sudo docker tag cf6395ca261d techiescorner/webhook:latest
sudo docker login
sudo docker push techiescorner/webhook:latest
TLS certificates
To secure the communication between the webhook and API server we need to deploy a certificate resource. This “Certificate” (filename = cert-manager.yaml) resource provides the required certificates for our webhook. To generate a certificate first we need to install cert-manager in our kubernetes cluster.
Execute the below command to install Cert-manger.
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install cert-manager bitnami/cert-manager --namespace cert-manager --set installCRDs=trueapiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: self-signer
namespace: ns-webhook
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my-webhook-certificate
namespace: ns-webhook
spec:
secretName: my-webhook-certificate-secret
dnsNames:
# - my-webhook
# - my-webhook.temp-webhook
- hello-webhook-service.ns-webhook.svc
issuerRef:
name: self-signerwebhook API server deployment
We have already created a webhook API server docker image and uploaded it to the Docker hub repository. Next, we have to deploy it to the Kubernetes cluster as a deployment, so create a deployment manifest file (file name = deployment.yaml). [Refer to the file from my Github repository].
Cluster IP service for the deployment
We have to create a Cluster IP service for our webhook API server deployment. The service will use port 443.
apiVersion: v1
kind: Service
metadata:
name: hello-webhook-service
namespace: ns-webhook
spec:
type: ClusterIP
selector:
app: hello-webhook
ports:
- protocol: TCP
port: 443
targetPort: 8000Next, create a manifest file (file name = webhook.yaml) to register our mutating webhook configuration with the Kubernetes API server.
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: "www.hello-webhook.com"
namespace: ns-webhook
annotations:
cert-manager.io/inject-ca-from: ns-webhook/my-webhook-certificate
webhooks:
- name: "www.hello-webhook.com"
objectSelector:
matchLabels:
hello: "true"
rules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE"]
resources: ["pods"]
scope: "Namespaced"
clientConfig:
service:
namespace: "ns-webhook"
name: "hello-webhook-service"
path: /mutate
admissionReviewVersions: ["v1", "v1beta1"]
sideEffects: None
timeoutSeconds: 10Deploy all above manifest files to the Kubernetes cluster and make sure that it is running without any errors.
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
kubectl apply -f cert-manager.yaml
issuer.cert-manager.io/self-signer created
certificate.cert-manager.io/my-webhook-certificate created
kubectl apply -f webhook.yaml
mutatingwebhookconfiguration.admissionregistration.k8s.io/www.hello-webhook.com created$ kubectl get all -n ns-webhook
NAME READY STATUS RESTARTS AGE
pod/hello-webhook-deployment-69c844d9c4-zhpzj 1/1 Running 0 13s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-webhook-service ClusterIP 10.105.251.50 <none> 443/TCP 13s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/hello-webhook-deployment 1/1 1 1 13s
NAME DESIRED CURRENT READY AGE
replicaset.apps/hello-webhook-deployment-69c844d9c4 1 1 1 13sTesting
We have completed all the Kubernetes deployment and next is testing. To test the mutating webhook deployment I will deploy 2 “busybox” containers or pods, one with the label “hello = test” and the other without the label “hello = test”. The pod with the label “hello=test” return the username and the other should return an error message. Execute below two commands.
kubectl run busybox-1 -n ns-webhook --image=busybox --restart=Never -l=app=busybox,hello=test -- sleep 3600
kubectl exec busybox-1 -it -n ns-webhook -- sh -c "cat /etc/config/username"
demo-userkubectl run busybox-2 -n ns-webhook --image=busybox --restart=Never -l=app=busybox,hello=false -- sleep 3600
pod/busybox-2 created
kubectl exec busybox-2 -it -n ns-webhook -- sh -c "cat /etc/config/username"
cat: can't open '/etc/config/username': No such file or directory
command terminated with exit code 1From the above outputs, we can confirm that the mutating webhook, which we installed is working as expected.
