Metrics Visualization With Collectd And Grafana
By: Maximilien Belinga
Tags: Visualization, Metrics, Collectd, Grafana

Environment

In this post, we use :

Ubuntu 16.04

Collectd 5.5.1

Grafana 4.0.2

Cyanite 0.5.1

Graphite-Api 1.1.2

Apache Cassandra 3.9

Install Cassandra

Download Cassandra, decompress it into your home directory, and launch it.

$ wget http://www-eu.apache.org/dist/cassandra/3.9/apache-cassandra-3.9-bin.tar.gz $ tar –xzvf apache-cassandra-3.9-bin.tar.gz $ cd apache-cassandra-3.9 && bin/cassandra

Install And Configure Cyanite

Clone the Cyanite Git repository in your home folder and import the database initial script.

$ git clone https://github.com/pyr/cyanite.git $ cd cyanite $ /home/user/apache-cassandra-3.9/bin/cqlsh –f doc/schema.cql

Now Install Leiningen. This is the build system tool used by the Cyanite project. Its very friendly seeming and installs locally into your home directory. This allows you to build JARs and other distributable versions of the code.

$ wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein $ sudo mv lein /usr/bin/ $ sudo chmod a+x /usr/bin/lein $ lein

We need to distribute code as Debian packages for Ubuntu. Fortunately, we have a target to build just that.

$ cd /home/user/cyanite $ lein fatdeb

This produces artifacts in the target/ directory.

$ cd /home/user/cyanite/target $ sudo dpkg –i cyanite_0.5.1_all.deb

The cyanite daemon runs now at port 8080, and you need to configure it before using it correctly. If you are not familiar with cyanite configuration, it can be a bit challenging to understand everything. Fortunately, I already configured my own file, so you can download the it: cyanite.yaml, and put it under /etc folder. You can also change the configuration if needed.

Install And Configure Graphite-api

Download and install graphite-api:

$ wget https://github.com/brutasse/graphite-api/releases/download/1.1.2/graphite-api_1.1.2-1447943655-debian8_amd64.deb $ sudo dpkg –i graphite-api_1.1.2-1447943655-debian8_amd64.deb

Once installed, Graphite-api runs as a service on port 8888.

Now download this file: graphite-api.yaml, and put it under /etc folder, you can change the configuration if needed, and restart all your services.

$ sudo service graphite-api restart $ sudo service cyanite restart

Install and Configure Collectd

Now we need to Download, install and configure Collectd on our host.

Download and install Collectd.

$ sudo apt-get update $ sudo apt-get install collectd collectd-utils

Download this file: collectd.conf, and put it under /etc/collectd folder. Now restart collectd.

$ sudo service collectd restart

Our data are fed to Cassandra database, we can start to visualize them.

Install Grafana and visualize data

To install Grafana, just download it and decompress the deb file, then start the service. It runs on port 3000.

$ wget https://grafanarel.s3.amazonaws.com/builds/grafana_4.0.2-1481203731_amd64.deb $ sudo dpkg –i grafana_4.0.2-1481203731_amd64.deb $ sudo service grafana restart

Now go to http://localhost:3000

Login with the following credentials if you did not change them. username: admin, password: admin (default credentials)

Create a data source

Create a Data Source in Grafana

To be able to receive data from Collectd, we need to connect our data source to graphite-api, which is running under port 8888. So, in the data source configuration put this as url: http://localhost:8888.

Data source configuration

Now, create a new dashboard

Create a New Dashboard

Then, Choose a graph

Select the type of metric to visualize

Add a single metric to see how it works. In the example below, we can can monitor the CPU memory usage in our server.

CPU Memory Usage

You can also add many metrics as necessary for you monitoring.

CPU/Buffered/Free Memory metrics

Now you have an overview of how you can monitor your sever’s critical resources using open source tools. It enables you to take critical decisions and, using big data potential, gives you great opportunities.