In this blog post, I will demonstrate the two best methods to protect Kubernetes ingress from the internet. As you all know when you deploy a service to the Kubernetes cluster with the help of NGINX ingress public IP address then by default it can be accessed from the internet. But we can restrict the public domain which is pointed to the ingress by NGINX ingress annotation as well as Kubernetes network policy.
Here I will demonstrate the two methods with the help of an example. Before we start, I hope that you have a working Kubernetes cluster ready. For this demo purpose, I will use Azure Kubernetes Service (AKS) but you can use EKS or GKE. Before I deep dive into the two methods, here are my cluster setup and installed application details.
Two-node AKS cluster. If you don’t have any cloud Kubernetes service you can set up a Kubernetes cluster with the help of Vagrant click here to know more about it.
$ kubectl get nodes NAME STATUS ROLES AGE VERSION aks-agentpool-29580704-vmss000001 Ready agent 4d3h v1.23.12 aks-agentpool-29580704-vmss000002 Ready agent 4d3h v1.23.12
Installed NGINX ingress controller in the ingress-nginx Namespace using Helm chart. Execute below commands to install the NGINX ingress.
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx kubectl create ns ingress-nginx helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE ingress-nginx-controller-5cc4f7674-v52gg 1/1 Running 0 2m18s
The NGINX ingress controller installation has been completed. Next, install a sample application in another namespace and make sure that it uses the NGINX for access from the internet.
apiVersion: v1 kind: Service metadata: name: hello-one namespace: techies-ns spec: type: ClusterIP ports: - port: 80 targetPort: 80 selector: app: hello-one --- apiVersion: apps/v1 kind: Deployment metadata: name: hello-one namespace: techies-ns spec: replicas: 3 selector: matchLabels: app: hello-one template: metadata: labels: app: hello-one spec: containers: - name: hello-ingress image: nginxdemos/hello ports: - containerPort: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: techies-ingress namespace: techies-ns annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: techies-ingress-test.com http: paths: - pathType: Prefix path: "/" backend: service: name: hello-one port: number: 80
The above manifest will install a Deployment, Service and Ingress. Here I used a fake domain name (techies-ingress-test.com) to access it from the browser. Apply the above file using kubectl.
kubectl create ns techies-ns kubectl apply -f hello-one.yaml service/hello-one created deployment.apps/hello-one created ingress.networking.k8s.io/techies-ingress created
From the above output, you can see that the service, deployment and ingress are created successfully. Execute the below command to get the external IP address and this IP address should be added to the /etc/hosts file to access the application from the browser.
$ kubectl get ingress -n techies-ns NAME CLASS HOSTS ADDRESS PORTS AGE techies-ingress <none> techies-ingress-test.com 22.214.171.124 80 144m
Add the IP address (126.96.36.199) and hostname (techies-ingress-test.com) in the /etc/host to resolve the domain from the local machine. Once you added the entry open a browser and access the domain.
Yes, our application is accessible and it is accessible to the public, we need to restrict access from a limited number of IP addresses, say only from Organization’s IP address.
Ingress protection using Network policy
Network policy is a Kubernetes object (kind) which can be used as a firewall to restrict the communication between Namespace, Pod and Ingress etc. The network policy will work on OSI layer 3 or 4 so compare to NGINX annotation, network policy will provide an additional layer of protection.
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: ingress-access namespace: ingress-nginx spec: podSelector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/component: controller app.kubernetes.io/instance: ingress-nginx policyTypes: - Ingress ingress: - from: - ipBlock: cidr: 188.8.131.52/32 ports: - protocol: TCP port: http - protocol: TCP port: https
A network policy specification consists of four elements:
- podSelector: the pods that will be subject to this policy (the policy target) – mandatory
- policyTypes: specifies which types of policies are included in this policy, ingress and/or egress – this is optional but recommended to always specify it explicitly.
- ingress: allowed inbound traffic to the target pods – optional
- egress: allowed outbound traffic from the target pods – optional
The above policy will allow ingress access only from the 184.108.40.206/32 (my local ISP address) address. The policy will target the NGINX ingress pods which we deployed in the ingress-nginx namespace. Multiple IP CIDR ranges cab added using ipBlock values.
- from: - ipBlock: cidr: 220.127.116.11/32 - ipBlock: cidr: 18.104.22.168/16
For testing, whitelist any random IP address and try to access the application from the browser or any from another server using the curl command.
curl http://techies-ingress-test.com curl: (52) Empty reply from server
This is how the ingress can be protected with the help of Network policy. Next, we will see how we can protect the ingress with the help of NGINX ingress annotation.
Ingress protection using NGINX annotation
To protect an ingress from the internet, the IP whitelisting will be done in the ingress manifest file. This can be achieved with the help of the “nginx.ingress.kubernetes.io/whitelist-source-range “ annotation. value is a comma-separated list of CIDRs, e.g. 10.0.0.0/24, 22.214.171.124/24. To configure this setting globally for all Ingress rules, the “whitelist-source-range” value may be set in the NGINX ConfigMap. Note that adding an annotation to an Ingress rule overrides any global restriction.
Modify the above ingress manifest file to include the NGINX annotation, example ingress is given below.
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: techies-ingress namespace: techies-ns annotations: kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/whitelist-source-range: "126.96.36.199/32" spec: rules: - host: techies-ingress-test.com http: paths: - pathType: Prefix path: "/" backend: service: name: hello-one port: number: 80
Before deploying the updated ingress manifest file, delete the existing network policy from the cluster.
kubectl get networkpolicy -n ingress-nginx NAME POD-SELECTOR AGE ingress-access app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx 1d kubectl delete networkpolicy ingress-access -n ingress-nginx
Apply the updated manifest file.
kubectl apply -f hello-one.yaml service/hello-one unchanged deployment.apps/hello-one unchanged ingress.networking.k8s.io/techies-ingress configured
Try to access the domain http://techies-ingress-test.com/ from the browser and you can see it is loading because we have whitelisted our IP address. Now change the IP address to some other IP address range and try to access the domain again, you will get a Forbidden error.
As we have learned the best approach to protect our Kubernetes ingress from the internet.