Package: aioprometheus-api-client
aioprometheus-api-clientA Python wrapper for the Prometheus http api and some tools for metrics processing.InstallationTo install the latest release:pip install aioprometheus-api-clientTo install directly from this branch:pip install git+https://github.com/trisongz/aioprometheus-api-clientLinksSlackGoogle ChatDocumentationGetting StartedUsagePrometheus, a Cloud Native Computing Foundation project, is a systems and service monitoring system. It collects metrics (time series data) from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true. The raw time series data obtained from a Prometheus host can sometimes be hard to interpret. To help better understand these metrics we have created a Python wrapper for the Prometheus http api for easier metrics processing and analysis.Theprometheus-api-clientlibrary consists of multiple modules which assist in connecting to a Prometheus host, fetching the required metrics and performing various aggregation operations on the time series data.Connecting and Collecting Metrics from a Prometheus hostThePrometheusConnectmodule of the library can be used to connect to a Prometheus host. This module is essentially a class created for the collection of metrics from a Prometheus host. It stores the following connection parameters:url- (str) url for the prometheus hostheaders– (dict) A dictionary of http headers to be used to communicate with the host. Example: {“Authorization”: “bearer my_oauth_token_to_the_host”}disable_ssl– (bool) If set to True, will disable ssl certificate verification for the http requests made to the prometheus hostfromprometheus_api_clientimportPrometheusConnectprom=PrometheusConnect(url="<prometheus-host>",disable_ssl=True)# Get the list of all the metrics that the Prometheus host scrapesprom.all_metrics()You can also fetch the time series data for a specific metric using custom queries as follows:prom=PrometheusConnect()my_label_config={'cluster':'my_cluster_id','label_2':'label_2_value'}prom.get_current_metric_value(metric_name='up',label_config=my_label_config)# Here, we are fetching the values of a particular metric nameprom.custom_query(query="prometheus_http_requests_total")# Now, lets try to fetch the `sum` of the metricsprom.custom_query(query="sum(prometheus_http_requests_total)")We can also use custom queries for fetching the metric data in a specific time interval. For example, let's try to fetch the past 2 days of data for a particular metric in chunks of 1 day:# Import the required datetime functionsfromprometheus_api_client.utilsimportparse_datetimefromdatetimeimporttimedeltastart_time=parse_datetime("2d")end_time=parse_datetime("now")chunk_size=timedelta(days=1)metric_data=prom.get_metric_range_data("up{cluster='my_cluster_id'}",# this is the metric name and label configstart_time=start_time,end_time=end_time,chunk_size=chunk_size,)For more functions included in thePrometheusConnectmodule, refer to thisdocumentation.Understanding the Metrics Data FetchedTheMetricsListmodule initializes a list of Metric objects for the metrics fetched from a Prometheus host as a result of a promql query.# Import the MetricsList and Metric modulesfromprometheus_api_clientimportPrometheusConnect,MetricsList,Metricprom=PrometheusConnect()my_label_config={'cluster':'my_cluster_id','label_2':'label_2_value'}metric_data=prom.get_metric_range_data(metric_name='up',label_config=my_label_config)metric_object_list=MetricsList(metric_data)# metric_object_list will be initialized as# a list of Metric objects for all the# metrics downloaded using get_metric query# We can see what each of the metric objects look likeforiteminmetric_object_list:print(item.metric_name,item.label_config,"\n")Each of the items in themetric_object_listare initialized as aMetricclass object. Let's look at one of the metrics from themetric_object_listto learn more about theMetricclass:my_metric_object=metric_object_list[1]# one of the metrics from the listprint(my_metric_object)For more functions included in theMetricsListandMetricsmodule, refer to thisdocumentation.Additional Metric FunctionsTheMetricclass also supports multiple functions such as adding, equating and plotting various metric objects.Adding MetricsYou can add add two metric objects for the same time-series as follows:metric_1=Metric(metric_data_1)metric_2=Metric(metric_data_2)metric_12=metric_1+metric_2# will add the data in ``metric_2`` to ``metric_1``# so if any other parameters are set in ``metric_1``# will also be set in ``metric_12``# (like ``oldest_data_datetime``)Equating MetricsOverloading operator =, to check whether two metrics are the same (are the same time-series regardless of their data)metric_1=Metric(metric_data_1)metric_2=Metric(metric_data_2)print(metric_1==metric_2)# will print True if they belong to the same time-seriesPlotting Metric ObjectsPlot a very simple line graph for the metric time series:fromprometheus_api_clientimportPrometheusConnect,MetricsList,Metricprom=PrometheusConnect()my_label_config={'cluster':'my_cluster_id','label_2':'label_2_value'}metric_data=prom.get_metric_range_data(metric_name='up',label_config=my_label_config)metric_object_list=MetricsList(metric_data)my_metric_object=metric_object_list[1]# one of the metrics from the listmy_metric_object.plot()Getting Metrics Data as pandas DataFramesTo perform data analysis and manipulation, it is often helpful to have the data represented using apandas DataFrame. There are two modules in this library that can be used to process the raw metrics fetched into a DataFrame.TheMetricSnapshotDataFramemodule converts "current metric value" data to a DataFrame representation, and theMetricRangeDataFrameconverts "metric range values" data to a DataFrame representation. Example usage of these classes can be seen below:importdatetimeasdtfromprometheus_api_clientimportPrometheusConnect,MetricSnapshotDataFrame,MetricRangeDataFrameprom=PrometheusConnect()my_label_config={'cluster':'my_cluster_id','label_2':'label_2_value'}# metric current valuesmetric_data=prom.get_current_metric_value(metric_name='up',label_config=my_label_config,)metric_df=MetricSnapshotDataFrame(metric_data)metric_df.head()""" Output:+-------------------------+-----------------+------------+-------+| __name__ | cluster      | label_2         | timestamp  | value |+==========+==============+=================+============+=======+| up       | cluster_id_0 | label_2_value_2 | 1577836800 | 0     |+-------------------------+-----------------+------------+-------+| up       | cluster_id_1 | label_2_value_3 | 1577836800 | 1     |+-------------------------+-----------------+------------+-------+"""# metric values for a range of timestampsmetric_data=prom.get_metric_range_data(metric_name='up',label_config=my_label_config,start_time=(dt.datetime.now()-dt.timedelta(minutes=30)),end_time=dt.datetime.now(),)metric_df=MetricRangeDataFrame(metric_data)metric_df.head()""" Output:+------------+------------+-----------------+--------------------+-------+|            |  __name__  | cluster         | label_2            | value |+-------------------------+-----------------+--------------------+-------+| timestamp  |            |                 |                    |       |+============+============+=================+====================+=======+| 1577836800 | up         | cluster_id_0    | label_2_value_2    | 0     |+-------------------------+-----------------+--------------------+-------+| 1577836801 | up         | cluster_id_1    | label_2_value_3    | 1     |+-------------------------+-----------------+------------=-------+-------+"""For more functions included in theprometheus-api-clientlibrary, please refer to thisdocumentation.Running testsPROM_URL="http://demo.robustperception.io:9090/" pytestCode Styling and LintingPrometheus Api client usespre-commitframework to maintain the code linting and python code styling.The AICoE-CI would run the pre-commit check on each pull request.We encourage our contributors to follow the same pattern, while contributing to the code.we would like to keep the same standard and maintain the code for better quality and readability.The pre-commit configuration file is present in the repository.pre-commit-config.yamlIt contains the different code styling and linting guide which we use for the application.we just need to runpre-commitbefore raising a Pull Request.Following command can be used to run the pre-commit:pre-commit run --all-filesIf pre-commit is not installed in your system, it can be install with :pip install pre-commit