Package: django-elasticsearch-metrics
django-elasticsearch-metricsDjango app for storing time-series metrics in Elasticsearch.Pre-requisitesPython 2.7 or >=3.6Django 1.11 or 2.0Elasticsearch 6Installpip install django-elasticsearch-metricsQuickstartAdd"elasticseach_metrics"toINSTALLED_APPS.INSTALLED_APPS+=["elasticsearch_metrics"]Define theELASTICSEARCH_DSLsetting.ELASTICSEARCH_DSL={"default":{"hosts":"localhost:9200"}}This setting is passed toelasticsearch_dsl.connections.configureso
it takes the same parameters.In one of your apps, define a new metric inmetrics.py.AMetricis a subclass ofelasticsearch_dsl.Document.# myapp/metrics.pyfromelasticsearch_metricsimportmetricsclassPageView(metrics.Metric):user_id=metrics.Integer(index=True,doc_values=True)Use thesync_metricsmanagement command to ensure that theindex templatefor your metric is created in Elasticsearch.# This will create an index template called myapp_pageviewpythonmanage.pysync_metricsNow add some data:frommyapp.metricsimportPageViewuser=User.objects.latest()# By default we create an index for each day.# Therefore, this will persist the document# to an index called, e.g. "myapp_pageview_2020.02.04"PageView.record(user_id=user.id)Go forth and search!# perform a search across all page viewsPageView.search()Per-month or per-year indicesBy default, an index is created for every day that a metric is saved.
You can change this to create an index per month or per year by changing
theELASTICSEARCH_METRICS_DATE_FORMATsetting.# settings.py# Monthly:ELASTICSEARCH_METRICS_DATE_FORMAT="%Y.%m"# Yearly:ELASTICSEARCH_METRICS_DATE_FORMAT="%Y"Index settingsYou can configure the index template settings by settingMetric.Index.settings.classPageView(metrics.Metric):user_id=metrics.Integer()classIndex:settings={"number_of_shards":2,"refresh_interval":"5s"}Index templatesEachMetricwill have its ownindex template.
The index template name and glob pattern are computed from the app label
for the containing app and the class's name. For example, aPageViewclass defined inmyapp/metrics.pywill have an index template with the
namemyapp_pageviewand a template glob pattern ofmyapp_pageview_*.If you declare aMetricoutside of an app, you will need to setapp_label.classPageView(metrics.Metric):classMeta:app_label="myapp"Alternatively, you can settemplate_nameand/ortemplateexplicitly.classPageView(metrics.Metric):user_id=metrics.Integer()classMeta:template_name="myapp_pviews"template="myapp_pviews_*"Abstract metricsfromelasticsearch_metricsimportmetricsclassMyBaseMetric(metrics.Metric):user_id=metrics.Integer()classMeta:abstract=TrueclassPageView(MyBaseMetric):classMeta:app_label="myapp"Optional factory_boy integrationimportfactoryfromelasticsearch_metrics.factoryimportMetricFactoryfrom..myapp.metricsimportMyMetricclassMyMetricFactory(MetricFactory):my_int=factory.Faker("pyint")classMeta:model=MyMetricdeftest_something():metric=MyMetricFactory()# index metric in ESassertisinstance(metric.my_int,int)ConfigurationELASTICSEARCH_DSL: Required. Connection settings passed toelasticsearch_dsl.connections.configure.ELASTICSEARCH_METRICS_DATE_FORMAT: Date format to use when creating
indexes. Default:%Y.%m.%d(same date format Elasticsearch uses fordate math)Management commandssync_metrics: Ensure that index templates have been created for
your metrics.show_metrics: Pretty-print a listing of all registered metrics.check_metrics: Check if index templates are in sync. Exits
with an error code if any metrics are out of sync.SignalsSignals are located in theelasticsearch_metrics.signalsmodule.pre_index_template_create(Metric, index_template, using): Sent beforePUTting a new index
template into Elasticsearch.post_index_template_create(Metric, index_template, using): Sent afterPUTting a new index
template into Elasticsearch.pre_save(Metric, instance, using, index): Sent at the beginning of a
Metric'ssave()method.post_save(Metric, instance, using, index): Sent at the end of a
Metric'ssave()method.Caveats_sourceis disabled by default on metric indices in order to save
disk space. For most metrics use cases, Users will not need to retrieve the source
JSON documents. Be sure to understand the consequences of
this:https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#_disabling_source.
To enable_source, you can override it inclass Meta.classMyMetric(metrics.Metric):classMeta:source=metrics.MetaField(enabled=True)ResourcesElasticsearch as a Time Series Data StorePythonic Analytics with ElasticsearchIn Search of Agile Time Series DatabaseLicenseMIT Licensed.