fiware-prometheus-publisher 0.0.40
Ceilometer Publisher for the FIWARE infrastructure
Requires Python: <3
Keywords: monitoring ceilometer FIWARE metrics

# FIWARE Prometheus Publisher
This software (**fiprom**) allows to publish metrics generated by Ceilometer in Prometheus. It allows to modify/enrich metrics before publishing them.

This project started from the need of collecting metrics generated by the FIWARE node located in Vicenza that is running an old version of Ceilometer (v 2015.1.1). Newer version of Ceilometer have a Prometheus publisher distributed along with Ceilometer itself and, therefore, there is no need for this custom publisher (excepting for customizing metrics metadata).


## Installation
Fiprom works as a Ceilometer plugin loaded at runtime. It must be installed as library in the same environment where Ceilometer services are installed. It can be done simply:

```bash
pip install fiware-prometheus-publisher
```

An alternative way is to install the component in a Python virtualenv. In this case, also Ceilometer must be installed and executed from the virtualenv.

```bash
export PBR_VERSION=0.10.0
pip install -U 'pbr==0.11.0' testrepository setuptools

wget https://github.com/openstack/ceilometer/archive/2015.1.1.zip && unzip 2015.1.1.zip
RUN cd ceilometer-2015.1.1 && pip install .

pip install fiware-prometheus-publisher
```


Alternatively, the Docker image [fiware-prometheus-publisher](https://cloud.docker.com/repository/docker/gabrielegiammatteo/fiware-prometheus-publisher) can be used. The image runs a Ceilometer Collector service configured to use fiprom as storage backend.

## Configuration

Fiprom load its configuration through the `ceilometer.conf` file in the `[fiprom]` section. These are the supported keys and their default values:

```ini

[fiprom]

# URL of the Push Gateway where metrics created will be publihsed
push_gateway = http://localhost:9091/metrics/job/fiprom/infrastructure/fiware

# Configuration file for the conversion of Ceilometer samples to Prometheus metrics
converter_conf_file =

# Customize the location of the labels cache
cache_file=/tmp/fiprom.cache

# Mapping file for names of VMs, hosts and users
names_file =

# Mapping file for tenant groups
tenant_group_file =

# If set, logs samples received and metrics created in this file
log_file =

# If true, do not  make the call to the Push Gateway
dryrun = False
```

In addition to this configuration, the following has to be added:

- to use as Ceilomter pipeline publisher, add the following in your `pipeline.yaml` :
  ```yaml
  - publishers:
    - fiprom://
  ```
- to use as backend for the Ceilometer Collector, add the following to your `ceilometer.conf`

    ```ini
    [database]
    metering_connection=fiprom://
    # also the key connection can be used
    ```


## Further configuration files 
The basic configuration (see [Configuration](##Configuration)) activate fiprom, but to be useful it needs further configuration on how to convert metrics from Ceilometer to Prometheus.

### Converter Conf file

This file (`converter_conf_file`) tells fiprom which metrics should be converted and how labels of each metric should be mapped. It is in YAML format and must follow this schema:
```yaml
enabled:
  - '<sample_name_pattern_1>'
  - '<sample_name_pattern_2>'
  - ...
labels:
  - <sample_name_pattern_x>:
    - <label_name_1>: '<python_expression>'
    - <label_name_2>: '<python_expression>'
    - ...
  - <sample_name_pattern_y>:
  	- ...

```

A working configuration with detailed comments can be found at [fiprom_converter_conf.yaml](conf/firpom_converter_conf.yaml) file.

### Names file

This file (`names_file`) contains the mapping between IDs of OpenStack objects and their names. Since most of the metrics created by Ceilometer only contain IDs of objects, this mapping file is required to add label with names (more human readable) to Prometheus metrics.

The file must contain one mapping for each line in the format:
```ini
<id> <delimiter> <name>
```
'=' or ':' characters are used as delimiter if found in the row, otherwise space, tab or comma characters are used. All of these are valid lines:
```ini
acede503-78e4-49cf-adbc-bd87390d4f6a helloworld
4c9e370ae557689709a64e9b543d50fd0f1538d0c5a22ac30c5afb41,  host1.eng.it
f7eb45f96d62423a805bd2ee5734d961: admin
f00a0cba70174563b6deef58dffebda2  = This is a demo, project 
```

### Tenant Group file
This file (`tenant_group_file`) is used to assign a group to each tenant. Fiprom will add a `tenant_group` label when a mapping is found. The file is in the format:
```ini
<tenant_id_or_name> <delimiter> <group>
```

Same rules on delimiters applies of the names file.