Monitoring spring boot services using micrometer , prometheus, Grafana
By: Gopalkrushna Pattanaik
Tags: Monitoring, Metrics And Analytics, Microservices, Prometheus, Observability

Observability is a key factor of modern application, traditionally majority of java applications depends on the robust logging. But visualising have been done from logs using some popular enterprise tools like Splunk, App-dynamics.

Now in market from past few years we have many open source tools like ElasticSearch,LogStash & Kibana [ELK] which provides very convenient way of depicting and visualising the observability of softwares.

With the release of SpringBoot2 , It has been very easy to monitor the health and other metrics of the application using Micrometer, Prometheus, Grafana.

Spring Boot Actuator : Exposes the application metrics as REST endpoints

: Exposes the application metrics as REST endpoints Micrometer : As Actuator endpoints are not vendor neutral, so to overcome of this we can use Micrometer which exposes Actuator metrics to external monitoring systems such as Prometheus, AWS Cloudwatch, and many more , in a specific format that vendors can understand. We can consider Micrometer as a facade for metrics for various vendor types.

: As Actuator endpoints are not vendor neutral, so to overcome of this we can use Micrometer which exposes Actuator metrics to external monitoring systems such as Prometheus, AWS Cloudwatch, and many more , in a specific format that vendors can understand. We can consider Micrometer as a facade for metrics for various vendor types. Prometheus : Time series DB to Store our metric data , it scrapes the endpoints and will store the metrics.

: Time series DB to Store our metric data , it scrapes the endpoints and will store the metrics. Grafana: Visualises our data in graphs , backed by Prometheus as Datasource.

In this Project We have added the below dependencies , which will expose the actuator endpoints using micrometer which Prometheus understands.

compile('org.springframework.boot:spring-boot-starter-web')

compile group: 'org.springframework.boot', name: 'spring-boot starter-actuator', version: '2.1.6.RELEASE'

compile('io.micrometer:micrometer-registry-prometheus')

After running the spring boot application with above dependencies, @ default port , spring-boot-actuator will automatically expose the system metrics @ http://localhost:8080/actuator/

{

“_links”: {

“self”: {

“href”: “http://localhost:8080/actuator",

“templated”: false

},

“auditevents”: {

“href”: “http://localhost:8080/actuator/auditevents",

“templated”: false

},

“beans”: {

“href”: “http://localhost:8080/actuator/beans",

“templated”: false

},

“caches-cache”: {

“href”: “http://localhost:8080/actuator/caches/{cache}",

“templated”: true

},

“caches”: {

“href”: “http://localhost:8080/actuator/caches",

“templated”: false

},

“health”: {

“href”: “http://localhost:8080/actuator/health",

“templated”: false

},

“health-component”: {

“href”: “http://localhost:8080/actuator/health/{component}",

“templated”: true

},

“health-component-instance”: {

“href”: “http://localhost:8080/actuator/health/{component}/{instance}",

“templated”: true

},

“conditions”: {

“href”: “http://localhost:8080/actuator/conditions",

“templated”: false

},

“configprops”: {

“href”: “http://localhost:8080/actuator/configprops",

“templated”: false

},

“env”: {

“href”: “http://localhost:8080/actuator/env",

“templated”: false

},

“env-toMatch”: {

“href”: “http://localhost:8080/actuator/env/{toMatch}",

“templated”: true

},

“info”: {

“href”: “http://localhost:8080/actuator/info",

“templated”: false

},

“loggers-name”: {

“href”: “http://localhost:8080/actuator/loggers/{name}",

“templated”: true

},

“loggers”: {

“href”: “http://localhost:8080/actuator/loggers",

“templated”: false

},

“heapdump”: {

“href”: “http://localhost:8080/actuator/heapdump",

“templated”: false

},

“threaddump”: {

“href”: “http://localhost:8080/actuator/threaddump",

“templated”: false

},

“prometheus”: {

“href”: “http://localhost:8080/actuator/prometheus",

“templated”: false

},

“metrics-requiredMetricName”: {

“href”: “http://localhost:8080/actuator/metrics/{requiredMetricName}",

“templated”: true

},

“metrics”: {

“href”: “http://localhost:8080/actuator/metrics",

“templated”: false

},

“scheduledtasks”: {

“href”: “http://localhost:8080/actuator/scheduledtasks",

“templated”: false

},

“httptrace”: {

“href”: “http://localhost:8080/actuator/httptrace",

“templated”: false

},

“mappings”: {

“href”: “http://localhost:8080/actuator/mappings",

“templated”: false

}

}

}

Here in the above response we can see that prometheus has its own endpoint exposed which it can use to get the metrics @http://localhost:8080/actuator/prometheus.

Now we will Install and Configure Prometheus & Grafana to integrate with our application.

Prometheus

Prometheus is a time-series database that stores our metric data by pulling it (using a built-in data scraper) periodically over HTTP. The intervals between pulls can be configured. we have to provide the URL to pull from. It also has a simple user interface where we can visualise/query on all of the collected metrics.

Install and Configure Prometheus

We can use docker compose image to run prometheus , here we are using Homebrew to install

brew update

brew install prometheus

vi prometheus.yml

Create the Prometheus config file [ prometheus.yml ], where we have to provide mention the target URL from where prometheus server will scrape the metrics and form the graphs to visualise them.

global:

scrape_interval: 10s



scrape_configs:

- job_name: 'spring_micrometer'

metrics_path: '/actuator/prometheus'

scrape_interval: 5s

static_configs:

- targets: ['localhost:8080']

We have provided the details to Prometheus server, now we can start the server using this config file , so that the metrics can be scrapped.

prometheus --config.file=prometheus.yml

prometheus will start @ default port 9090 http://localhost:9090/

We can see the prometheus server is up .

Home page

lets check what target endpoint it is scrapping by navigating @http://localhost:9090/targets

Target

We can see the job defined before in the config file spring_micrometer is shown along with the endpoint configured.

Now lets quickly check few of the system metrics using the graphs.

System CPU usage

JVM Memory Usage

Grafana

While Prometheus does provide some crude visualisation, Grafana offers a rich UI where you can build up custom graphs quickly and create a dashboard out of many graphs in no time.

There are many built in dashboards readily available @ https://grafana.com/grafana/dashboards which we can use directly.

Grafana dashboards are backed by any given data source (https://grafana.com/docs/grafana/latest/features/datasources/#supported-data-sources/) from where it pulls details , few of the examples are

AWS CloudWatch

Azure Monitor

Elasticsearch

Google Stack driver

Graphite

InfluxDB

Loki

Microsoft SQL Server (MSSQL)

Mixed

MySQL

OpenTSDB

PostgreSQL

Prometheus

Testdata

We will be using Prometheus as our datasource for our grafana setup.

Install and Configure Grafana

I am using Homebrew again to install grafana for this proj , Docker compose is recommended for general usage.

brew update

brew install grafana

brew tap homebrew/services

brew services start grafana

Grafana will start at its default address @ http://localhost:3000/

and default username/password :- admin/admin

First step is to add a data source to start with any dashboard, we will add a data source for Prometheus

Add DataSource

Add the Endpoint to Prometheus

Now we can create a dashboard to monitor the application, we will use a very popular JVM dashboard available @ https://grafana.com/grafana/dashboards/4701

Once added we can monitor our system metrics as given below with rich UI experience.

Prometheus & Grafana has different types of logging metrics available like

Counter to check number at cumulative level that only ever goes up . It is useful for data like number of sessions, number of events consumed ,number of Requests served , Error occurred and so on.

Gauge is a number that can go up or down with time based on some factors. it is useful for data like JVM Memory Usage , Sessions in your website.

Histogram & Summary are sample observations, typically request durations or response sizes. They track the number of observations and the sum of the observed values, allowing you to calculate the average of the observed values.

Thats all as of now. Thank you & Happy learning :)