How to protect Kubernetes Ingress from the internet

In this blog post, I will demonstrate the two best methods to protect Kubernetes ingress from the internet. As you all know when you deploy a service to the Kubernetes cluster with the help of NGINX ingress public IP address then by default it can be accessed from the internet. But we can restrict the public domain which is pointed to the ingress by NGINX ingress annotation as well as Kubernetes network policy.

Here I will demonstrate the two methods with the help of an example. Before we start, I hope that you have a working Kubernetes cluster ready. For this demo purpose, I will use Azure Kubernetes Service (AKS) but you can use EKS or GKE. Before I deep dive into the two methods, here are my cluster setup and installed application details.

Cluster setup

Two-node AKS cluster. If you don’t have any cloud Kubernetes service you can set up a Kubernetes cluster with the help of Vagrant click here to know more about it.

$ kubectl get nodes

NAME                                STATUS   ROLES   AGE    VERSION

aks-agentpool-29580704-vmss000001   Ready    agent   4d3h   v1.23.12
aks-agentpool-29580704-vmss000002   Ready    agent   4d3h   v1.23.12

Installed NGINX ingress controller in the ingress-nginx Namespace using Helm chart. Execute below commands to install the NGINX ingress.

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

kubectl create ns ingress-nginx

helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx

kubectl get pods -n ingress-nginx

NAME                                       READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-5cc4f7674-v52gg   1/1     Running   0          2m18s

The NGINX ingress controller installation has been completed. Next, install a sample application in another namespace and make sure that it uses the NGINX for access from the internet.

apiVersion: v1
kind: Service
metadata:
  name: hello-one
  namespace: techies-ns
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: hello-one
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-one
  namespace: techies-ns
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-one
  template:
    metadata:
      labels:
        app: hello-one
    spec:
      containers:
      - name: hello-ingress
        image: nginxdemos/hello
        ports:
        - containerPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: techies-ingress
  namespace: techies-ns
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: techies-ingress-test.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-one
            port:
              number: 80

The above manifest will install a Deployment, Service and Ingress. Here I used a fake domain name (techies-ingress-test.com) to access it from the browser. Apply the above file using kubectl.

kubectl create ns techies-ns

kubectl apply -f hello-one.yaml 
service/hello-one created
deployment.apps/hello-one created
ingress.networking.k8s.io/techies-ingress created

From the above output, you can see that the service, deployment and ingress are created successfully. Execute the below command to get the external IP address and this IP address should be added to the /etc/hosts file to access the application from the browser.

$ kubectl get ingress -n techies-ns
NAME              CLASS    HOSTS                      ADDRESS           PORTS   AGE
techies-ingress   <none>   techies-ingress-test.com   13.14.15.16      80      144m

Add the IP address (13.14.15.16) and hostname (techies-ingress-test.com) in the /etc/host to resolve the domain from the local machine. Once you added the entry open a browser and access the domain.

Yes, our application is accessible and it is accessible to the public, we need to restrict access from a limited number of IP addresses, say only from Organization’s IP address.

Ingress protection using Network policy

Network policy is a Kubernetes object (kind) which can be used as a firewall to restrict the communication between Namespace, Pod and Ingress etc. The network policy will work on OSI layer 3 or 4 so compare to NGINX annotation, network policy will provide an additional layer of protection.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ingress-access
  namespace: ingress-nginx
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
  policyTypes:
    - Ingress
  ingress:
  - from:
    - ipBlock:
        cidr: 9.10.11.12/32
    ports:
      - protocol: TCP
        port: http
      - protocol: TCP
        port: https

A network policy specification consists of four elements:

1. podSelector: the pods that will be subject to this policy (the policy target) – mandatory
2. policyTypes: specifies which types of policies are included in this policy, ingress and/or egress – this is optional but recommended to always specify it explicitly.
3. ingress: allowed inbound traffic to the target pods – optional
4. egress: allowed outbound traffic from the target pods – optional

The above policy will allow ingress access only from the 9.10.11.12/32 (my local ISP address) address. The policy will target the NGINX ingress pods which we deployed in the ingress-nginx namespace. Multiple IP CIDR ranges cab added using ipBlock values.

- from:
    - ipBlock:
        cidr: 9.10.11.12/32
    - ipBlock:
        cidr: 1.1.1.1/16

For testing, whitelist any random IP address and try to access the application from the browser or any from another server using the curl command.

curl http://techies-ingress-test.com
curl: (52) Empty reply from server

This is how the ingress can be protected with the help of Network policy. Next, we will see how we can protect the ingress with the help of NGINX ingress annotation.

Ingress protection using NGINX annotation

To protect an ingress from the internet, the IP whitelisting will be done in the ingress manifest file. This can be achieved with the help of the “nginx.ingress.kubernetes.io/whitelist-source-range “ annotation. value is a comma-separated list of CIDRs, e.g. 10.0.0.0/24, 172.10.0.1/24. To configure this setting globally for all Ingress rules, the “whitelist-source-range” value may be set in the NGINX ConfigMap. Note that adding an annotation to an Ingress rule overrides any global restriction.

Modify the above ingress manifest file to include the NGINX annotation, example ingress is given below.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: techies-ingress
  namespace: techies-ns
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/whitelist-source-range: "9.10.11.12/32"
spec:
  rules:
  - host: techies-ingress-test.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-one
            port:
              number: 80

Before deploying the updated ingress manifest file, delete the existing network policy from the cluster.

kubectl get networkpolicy -n ingress-nginx
NAME             POD-SELECTOR                                                                                                           AGE
ingress-access   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx   1d

kubectl delete networkpolicy ingress-access -n ingress-nginx

Apply the updated manifest file.

kubectl apply -f hello-one.yaml 
service/hello-one unchanged
deployment.apps/hello-one unchanged
ingress.networking.k8s.io/techies-ingress configured

Try to access the domain http://techies-ingress-test.com/ from the browser and you can see it is loading because we have whitelisted our IP address. Now change the IP address to some other IP address range and try to access the domain again, you will get a Forbidden error.

As we have learned the best approach to protect our Kubernetes ingress from the internet.