Monitoring Keycloak using Prometheus Operator - Kubernetes & Helm Charts
Tags: kubernetes, devops, security, tutorial
Published: 2022-06-25 09:55:30

## Introduction

Just like how a surveillance system over a compound wall enhances the security of our homes, Keycloak too requires continuous surveillance to better safeguard its application.

This tutorial assumes that the reader has basic understanding on [Keycloak](https://www.keycloak.org/) and it helps the readers to set up [Prometheus Operator](https://prometheus-operator.dev/) to monitor their Keycloak. 

##Setup

Before we begin, let us first, visualize what we will be doing to achieve our target, to monitor Keycloak using Prometheus Operator.

![Solution Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cm8esmmbjtuxfnnju3b4.png)

Here, let us assume that Keycloak is already installed on Keycloak Namepace.
 
We will then: 

1. Install Prometheus Operator on Monitoring Namespace which will automatically deploy Prometheus pod.
 
2. Enable Keycloak to publish its metrics at _keycloak-metrics_ Service.

3. Create a Service Monitor (CRD provided by Operator) which points to the _keycloak-metrics_ Service and attach it to our Prometheus Pod.

For this tutorial, I have used Keycloak [helm chart](https://github.com/bitnami/charts/tree/master/bitnami/keycloak) from Bitnami. Feel free to use other helm charts and adjust the parameters accordingly.

## Installing Prometheus Operator
We will install the Prometheus Operator using [Helm charts](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) from prometheus-community. Feel free to use any other helm chart and change the parameters accordingly.

### Adding Helm Repository

```
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
```

### Install Prometheus-Operator

**Note:** Adjust other parameters accordingly based on requirements.

_custom-values.yaml_
```
      ...
      prometheus:
        prometheusSpec:
          serviceMonitorSelector:
            matchLabels:
              <Service Monitor Label Key>: <Service Monitor Label Value>
      ...

```
We are intimating Prometheus to look out for a **Service Monitor** by adding labels into serviceMonitorSelector. 
Remember the labels, as we will have to use the same labels while creating Service Monitor later in this tutorial.

Once _custom-values.yaml_ are modified according to the requirement, install Prometheus-Operator using below command

```
helm install -f custom-values.yaml --namespace=[MONITORING_NAMESPACE] [RELEASE_NAME] prometheus-community/kube-prometheus-stack
```

## Exposing Keycloak Metrics

Ultimately, our aim is to monitor Keycloak's metrics. Keycloak provides options to make it's metrics available for scraping and monitoring. To enable it, make sure to adjust this parameter to  Keycloak helm's _values.yaml_

_values.yaml_
```
      ...
      metrics:
        enabled: true
      ...
```
**Note:**
By default, [keycloak-metrics](https://github.com/bitnami/charts/blob/master/bitnami/keycloak/templates/metrics-service.yaml) Service will include http-management port, but the metrics for scraping are enabled at **http port** (80). So, make sure you **patch** the _keycloak-metrics_ Service to include **http port** in its ports specifications.

## Service Monitors

Now that we have Prometheus-Operator running and Keycloak making it's metrics available at http port, how do we point the Prometheus to scrape at the http port where Keycloak furnishes it's metrics?

The answer is Service Monitors. Service Monitors are used by Prometheus to automatically detect it's target service to scrape data. Refer [here](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/getting-started.md) for more information.

Since during our installation, we had notified the Prometheus to look out for a Service Monitor by providing labels in serviceMonitorSelector, we will now create that service monitor with the information of our _keycloak-metrics_ service.

Deploy the below manifest.
**Note**: This manifest should be deployed only after the Prometheus-Operator is installed. As we are attaching this Service Monitor to Prometheus, therefore it must be there in the namespace where this Service Monitor is deployed.

_Service Monitor Manifest_
```
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: <Insert Appropriate Name for Service Monitor>
  Namespace: <MONITORING_NAMESPACE>
  labels:
    <Service Monitor Label Key>: <Service Monitor Label Value>
spec:
  endpoints:
    - ports: http
      path: /auth/realms/master/metrics
      interval: <Select Appropriate Intervals at which the Metrics will be scraped automatically>
  selector:
    matchLabels:
      app.kubernetes.io/component: metrics
  namespaceSelector:
    matchNames:
      - <Insert Namespace where Keycloak is installed>
```
**Note**: Make sure that the **metadata.labels** have the same key/values as the ones we have provided in serviceMonitorSelector during Prometheus-Operator installation.

I will now breakdown the manifest for better understanding.


```
  endpoints:
    - ports: http
      path: /auth/realms/master/metrics
      interval: <Select Appropriate Intervals at which the Metrics will be scraped automatically>
```
In _endpoints_ of Service Monitor specification, we mention at _**which port**_ to scrape, _**what path**_ to scrape, and _**when (interval)**_ to scrape. 

**Note**: _/auth/realms/master/metrics_ will publish the data for all the realms.


```
  selector:
    matchLabels:
      app.kubernetes.io/component: metrics
  namespaceSelector:
    matchNames:
      - <Insert Namespace where Keycloak is installed>
```
The above two selectors provide the service monitor with information about the service which will publish the data. So make sure you provide appropriate _selector labels_ for _keycloak-metrics service_ and _namespace_ where _Keycloak is installed_.


And that's it! Prometheus will now start scraping the Keycloak metrics automatically at given intervals. To visualize them, you now can open the dashboard and view the metrics!

## Summary

By going through this tutorial, we will now be able to monitor and analyze Keycloak metrics and view them on customizable Dashboards of Prometheus/Grafana.

Thanks for checking out this post!

Cover Image credits: Image by [Lorenzo Cafaro](https://pixabay.com/users/3844328-3844328/?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=1863880) from Pixabay