firestealer 0.5.0
A command line tool for parsing Prometheus metrics
Keywords: prometheus influxdb metrics charm cli

firestealer
===========

Parse and filter metrics from `Prometheus <https://prometheus.io/>`_ endpoints
so that they can be visualized, stored into
`InfluxDB <https://www.influxdata.com/time-series-platform/influxdb/>`_, or used
as `Juju charm <https://jujucharms.com/>`_ metrics.

Installation
------------

Run `pip install firestealer`. This will make the `fsteal` command available.

CLI
---
Print JSON formatted samples from a Prometehus endpoint::

    $ fsteal http://localhost:8000/metrics

Filter metrics using a regular expression against the sample name::

    $ fsteal http://localhost:8000/metrics -m '^myservice(Downloads|Uploads)'

Store samples into InfluxDB::

    $ fsteal localhost:8000/metrics influxdb://user:passwd@10.0.0.1:8086/mydb

Store a subset of samples into InfluxDB::

    $ fsteal localhost:8000/metrics influxdb://user:passwd@10.0.0.1:8086/mydb -m '^myservice'

In the example above the InfluxDB connection info is provided using a
`influxdb://$user:$passwd@$host:$port/$database` connection string.

Only print specific metrics values (useful when executing fsteal in order to
retrieve charm metrics)::

    $ fsteal localhost:8000/metrics a-single-specific-key --format values-only

API Usage
---------

An API is exposed so that firestealer can be used as a library from Python
applications. For instance, a charm could propagate metrics from Prometheus
to Juju with the following snippet placed in the
`collect-metrics <https://jujucharms.com/docs/2.3/reference-charm-hooks#collect-metrics>`_
hook::

    from firestealer import (
        add_metrics,
        retrieve_metrics,
    )

    url = 'https://localhost:8000/metrics'
    with open('metrics.yaml') as f:
        metrics = yaml.safe_load(f)
    samples = retrieve_metrics(url, metrics, noverify=True)
    add_metrics(samples)

Similarly, to write relevant samples to InfluxDB, the following snippet can be
used::

    from firestealer import (
        retrieve_metrics,
        write_to_influx,
    )

    url = 'https://localhost:8000/metrics'
    with open('metrics.yaml') as f:
        metrics = yaml.safe_load(f)
    samples = retrieve_metrics(url, metrics)
    write_to_influx('user:passwd@1.2.3.4/dbname', samples)

API Reference
-------------

**add_metrics(samples)**
    Add the given samples as metrics to Juju.

    This function assumes an add-metric executable to be present in the PATH.

**retrieve_metrics(url, metrics, noverify=False)**
    Retrieve samples for the given metrics from the given Prometheus URL.

    Raise a firestealer.PrometheusError if the samples cannot be retrieved.

    Only return samples whose name is included in the given metrics object,
    which is a decoded metrics.yaml content. Samples are renamed based on the
    metrics provided.
    If noverify is True, then do not validate URL certificates.

**retrieve_samples(url, regex='', prefix='', noverify=False)**
    Retrieve and return samples from the given Prometheus URL.

    Raise a firestealer.PrometheusError if the samples cannot be retrieved.

    Only return samples whose name matches the given regex.
    If a prefix is provided, include that in the resultimg sample names.
    If noverify is True, then do not validate URL certificates.

**samples_to_points(samples)**
    Convert samples to InfluxDB points.

    Only samples with an actual value are included in the resulting list.

**text_to_samples(text, regex='', prefix='')**
    Parse the given metrics text and return a sequence of samples.

    Only return samples whose name matches the given regex.
    If a prefix is provided, include that in the resultimg sample names.

**write_to_influx(conn_string, samples)**
    Write the given samples to the InfluxDB at the given connection string.

    The connection string is in "$user:$passwd@$host:$port/$database" form.
    Raise a firestealer.InfluxError if the connection string is not well formed
    or when it is not possible to communicate with InfluxDB.

**Sample** = namedtuple('Sample', ['name', 'tags', 'value'])
    A sample represents a sigle Prometheus key/value measurement.