Introduction
In this post, I would like to share how we can configure SSO authentication for Grafana with the help of Azure Active Directory (AD). This configuration helps you authenticate Grafana without a username and password, hence it becomes more secure.
In my previous blog post, I wrote about configuring an email service for Grafana alerting. To read this blog click here.
Pre-requisite.
- Kubernetes cluster (Version 1.25)
- Helm
- SealedSecret
- Azure cloud account
- NGINX ingress controller
- Cert Manager
Installation steps
Grafana helm chart will be installed on the Kubernetes cluster. The Grafana URL will be exposed using the Nginx ingress controller. Azure Cloud is used to create an App registration so Azure AD users can log in to Grafana. The app-registration secret key has to be copied to the Grafana configuration file. To hide this secret from the public, SealedSecret will be used.
List the cluster details.
In this tutorial, we will be using a 3-worker node cluster running Kubernetes version 1.25
kubectl get nodes
NAME STATUS ROLES AGE VERSION
worker-node1 Ready <none> 96s v1.25.4
worker-node2 Ready <none> 97s v1.25.4
worker-node3 Ready <none> 109s v1.25.4Sealed-Secret installation
Sealed-Secret is an application that can seal your secrets like username and password, SSL/TLS certificate, token etc and can store even in public repositories like GitHub, Gitlab, Bitbucket etc. To understand it’s working and configuration, I recommend you read their Github pages.
kubectl create namespace sealed
namespace/sealed created
helm install sealed-secret oci://registry-1.docker.io/bitnamicharts/sealed-secrets --namespace=sealed
NAME: sealed-secret
LAST DEPLOYED: Wed May 17 20:01:54 2023
NAMESPACE: sealed
STATUS: deployed
REVISION: 1
TEST SUITE: None
The sealed-Secret installation has been completed and is ready to use. Next, we will install the NGINX ingress controller.
NGINX ingress installation
The NGINX ingress controller will be used to expose Grafana to the public. Commands to install the NGINX ingress controller are given below.
helm repo update
helm upgrade --install \
ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--set controller.service.type=LoadBalancer \
--create-namespaceTo confirm the above installation is working, execute the below command. Please wait for a few seconds to get the external IP address.
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.128.145.244 10.20.26.15 80:30400/TCP,443:30888/TCP 3m11s
ingress-nginx-controller-admission ClusterIP 10.128.32.108 <none> 443/TCP 3m11sCert-Manager installation
Cert-Manger is a plugin which can be used to generate SSL certificates for a domain in the Kubernetes cluster. SSL certificate is required because Azure app registration is supported only by the HTTP request. To install cert-manger execute the below commands.
helm repo add jetstack https://charts.jetstack.io
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.11.0 \
--set installCRDs=true
Verify that the cert-manager has started and the status of the pods is Running.
kubectl get pods -n cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-59bf757d77-58w5m 1/1 Running 0 44s
cert-manager-cainjector-547c9b8f95-2v89f 1/1 Running 0 44s
cert-manager-webhook-6787f645b9-r2q9b 1/1 Running 0 44sOnce the cert-manager is running, we need to create the ClusterIssuer resource for Let’s Encrypt. Create a file named cluster-issuer-prod.yaml and add the following, remember to replace your-email@addresss.com with your E-mail address:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: your-email@address.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource used to store the account's private key.
name: letsencrypt-prod
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginxkubectl apply -f cluster-issuer-prod.yaml
issuer.cert-manager.io/letsencrypt-prod createdCert-manager installation has been completed. Next, install the Grafana, for installation execute the below command.
Grafana installation
kubectl create namespace grafana
helm install grafana-release grafana/grafana -n grafana
sha256:2212f07d582cbff59ee89fd3b70a28bf21b7f43804052365b857e2d3ea6878fd
NAME: grafana-release
LAST DEPLOYED: Thu May 18 20:09:56 2023
NAMESPACE: grafana
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: grafana
CHART VERSION: 8.4.2
APP VERSION: 9.5.2Make sure that the Grafana pods are running.
kubectl get pods -n grafana
NAME READY STATUS RESTARTS AGE
grafana-release-86bbc8f7f9-w472d 1/1 Running 0 74sApp-Registration for SSO
Log in to the Azure cloud portal and search for App-registration in the search field. Click on New Registration to create a new App-registration In the new pop-up page give a name for the app (demo-grafana) and choose web under the Redirect URI (optional) field, Add the following redirect URLs https://demo-grafana.ai/login/azuread then click Register. The app’s Overview page opens..
In the new pop-up page give a name for the app (demo-grafana) and choose web under the Redirect URL(optional) file, Add the following redirect URL https://demo-grafana.ai/login/genericauth then click Register.
From the overview page take note of the Application ID.
Click the “endpoint” to get the authorization endpoints. Note the OAuth 2.0 authorization endpoint (v2) URL and OAuth 2.0 token endpoint (v2).
Next, click on the “Certificate & Secret” from the left side panel and add a new client secret. Set a name and expiry for the secret. Please keep the secret safe.
Seal the secret.
We are going to use the App-registration secret key in the Grafana configuration, but it is in plain text and not secure. To secure this secret we will use the sealed secret which we installed earlier. Execute the below command to install the “kubeseal” client. This client package is required for generating sealed secrets.
brew install kubesealThe Sealed-Secred application seals a secret with the help of its controller certificates, so first download the certificate to your local machine by executing the below command.
kubectl get svc -n sealed
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
sealed-secret-sealed-secrets ClusterIP 10.128.63.174 <none> 8080/TCP 4d20h
kubeseal --controller-name=sealed-secret-sealed-secrets-7d95f99856-74b85 --controller-namespace=sealed --fetch-cert > sealing-cert.pem
kubectl create secret generic grafana-secret -n grafana --dry-run=client \
--from-literal=client_secret=BHR33add2XTR51jcw -o yaml | \
kubeseal --cert sealing-cert.pem -o yaml > grafana-sealed-secret.yaml
The generated grafana-sealed-secret.yaml is a sealed secret and we store it on any public repositories like Github. Deploy this secret to the cluster so it will be available for Grafana. There will be two resources in the cluster one is secret and the other one is sealed secret.
kubectl apply -f grafana-sealed-secret.yaml
sealedsecret.bitnami.com/grafana-secret created
kubectl get secret -n grafana
NAME TYPE DATA AGE
default-token-wwjcq kubernetes.io/service-account-token 3 4d20h
grafana-secret Opaque 1 9sGrafana SSO configuration
We have created an App-registration and created a sealed secret. We have all the configuration details which are required for grafana sso configuration. The next, step is to add all these details to the values.yaml and upgrade the Grafana Helm installation. Execute the below commands.
persistence:
enabled: true
storageClassName: cloud-block-storage
ingress:
enabled: true
annotations:
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
hosts:
- "demo-grafana.ai"
ingressClassName: "nginx"
tls:
- hosts:
- demo-grafana.ai
secretName: armada-grafana-tls
grafana.ini:
paths:
data: /var/lib/grafana/
logs: /var/log/grafana
plugins: /var/lib/grafana/plugins
provisioning: /etc/grafana/provisioning
analytics:
check_for_updates: true
log:
mode: console
grafana_net:
url: https://grafana.net
server:
domain: "{{ if (and .Values.ingress.enabled .Values.ingress.hosts) }}{{ .Values.ingress.hosts | first }}{{ else }}''{{ end }}"
server:
root_url: https://demo-grafana.ai/
auth.generic_oauth:
enabled: true
icon: signin
name: SSO
allow_sign_up: true
auto_login: true
scopes: "openid email profile"
auth_url: https://login.microsoftonline.com/111-11240-11111/oauth2/v2.0/authorize
token_url: https://login.microsoftonline.com/111-11240-11111/oauth2/v2.0/token
client_id: xxxxxxx-xxxxx-xxx-1111
client_secret: $__file{/etc/secrets/grafana-secret/client_secret}
extraSecretMounts:
- name: grafana-secret
secretName: grafana-secret
defaultMode: 0440
mountPath: /etc/secrets/grafana-secret
readOnly: trueIn my values.yaml file I have added a storage class, please make sure that you are using your cloud-provided class name here. If you don’t know it please check with your cloud provider. I have sealed only the App-registration client_secret but you can add other secrets like client_id to the sealed secret file and can create a new sealed secret file for deployment.
To upgrade the helm installation execute the below commands.
helm upgrade grafana-release grafana/grafana -n grafana -f values.yaml Make sure that the Grafana pods are running
kubectl get pods -n grafana
NAME READY STATUS RESTARTS AGE
grafana-release-787b4fbbbb-5zlbg 1/1 Running 0 11m
Access the domain https://demo-grafana.ai on the browser and you can see there is a new button to log in using SSO or Azure AD.
Click the “Sign in with SSO” button and enter your organization email ID (If you created the App-registration in your organization) and you will be logged in to Grafana.
We have successfully configured Grafana with SSO login.
