Package: apptuit
Python client for Apptuit.AIInstallationpip install apptuit --upgradeDependenciesRequirementsrequests,pyformance- installed automatically if you usepipto install apptuitpandas- not installed by default, you should install it manually if you intend to use thequeryAPI and
create dataframes using theto_df()method (seeQuerying for Data)UsageContentsIntroductionWorking with Apptuit ClientWorking with Apptuit Pyformance ReporterConfigurationSending DataSending data using Apptuit pyformance reporterError Handling in ApptuitReporterSending Tags/MetadataAbout Host tagRestrictions on TagsMeta MetricsPython Process MetricsSending Data usingsend()APISending Data usingsend_timeseries()APIQuerying for DataIntroductionThis package provides functionality to send timeseries data to Apptuit and also to query it.
There are two main componentsThe Apptuit client - provides core functionality to query and send dataApptuit pyformance reporter - provides a high level abstraction on top of the client
to make it easy for you to report metrics from your applications to Apptuit.
It is based on Coda Hale's metrics design and provides primitives likemeter,gauge,counterto accumulate and report data.
It usesPyformanceunderneath.Working with Apptuit Client:The Apptuit client object can be created as simply as the following line:fromapptuitimportApptuitclient=Apptuit(token=my_apptuit_token,global_tags={"service":"order-service"},sanitize_mode="prometheus")token: should be your apptuit tokenglobal_tags: should be the set of default tags you want to apply on all your data. It is an optional parametersanitize_mode: Is a string value which specifies the sanitization mode to be used
for metric names and tag keys.
You can setsanitize_modeto three values:None: disables sanitization.apptuit: set the sanitize mode to apptuit, which will replace
all the invalid characters with_. Valid characters in this mode are all
ASCII letters, digits,/,-,.,_and Unicode letters.
Anyhing else is invalid character.prometheus: set the sanitize mode to prometheus, which will replace
all the invalid characters with_. Valid characters in this mode are ASCII letters, digits
and_, anything else is considered invalid.Apart from these, the Apptuit constructor takes a couple of more optional parameters explained below:api_endpoint: This should be the http endpoint for calling Apptuit apis. Normally you don't need to specify this and the default value is set tohttps://api.apptuit.ai.ignore_environ_tags: This is False by default. It tells the client whether to look up for
the global tags in environment variables or not. Global tags are tags which are applied to all the
datapoints sent through the client. We will have more to say on this in the configuration section.The client provides two methods,queryandsend, which are described in theQuerying for DataandSending data using send()sections respectively.Working with Apptuit Pyformance ReporterThe apptuit pyformance reporter is an abstraction based on Code Hale's metrics. It provides
high level primitives to accumulate data in the form of metrics such asmeter,timer,gaugeetc. and send to Apptuit. These things are described in more
detail in thereporter section,
here we will see how to create a reporter and various parameters it supports.fromapptuit.pyformanceimportApptuitReporterfrompyformanceimportMetricsRegistryreporter_tags={"service":"order-service"}registry=MetricsRegistry()reporter=ApptuitReporter(token="my_apptuit_token",registry=registry,reporting_interval=60,tags=reporter_tags,collect_process_metrics=True,sanitize_mode="prometheus")Here:token: Is your Apptuit tokenregistry: Is an instance of MetricsRegistry (explained more inreporter section)reporting_interval: Number of seconds to wait before reporing againtags: A dictionary of tag keys and values.
These tags apply to all the metrics reported through this reporter.collect_process_metrics: Is a boolean value which will enable or disable collection
of various metrics related to the Python process (CPU, memory, GC, and threads). By default
it is disabled, set this parameter toTrueto enable it.sanitize_mode: This is same as thesanitize_modeparameter for the
client (see above in client usage example).ConfigurationAs we saw above, we need to pass the token and global tags as parameter to the
Apptuit client when instantiating it. Alternatively we can set these as
environment variables, so that we don't need to hard-code them in our code.
These environment variables are described below.APPTUIT_API_TOKEN: If the Apptuit client and the ApptuitReporter are not passed a token parameter they look for the token in this variable. If this variable is also not set, the client will raiseApptuitExceptionto indicate about the missing tokenAPPTUIT_TAGS: This is an alternative for theglobal_tagsparameter for the Apptuit client. If the Apptuit client does not receive a value forglobal_tagsparameter it checks this environment variable. Both theglobal_tagsparameter
andAPPTUIT_TAGSenvironment variable are strictly optional. If present, the Apptuit client adds those tags to every
point it is sending.The format of the value of this variable is as follows:exportAPPTUIT_TAGS="tag_key1: tag_val1, tag_key2: tag_val2, tag_key3: tag_val3"The spaces after the comma and colon are optional.TheAPPTUIT_TAGSvariable is also read by theApptuitReporter, which combines them with its reporter tags.
In case of a conflict of same tag keys in both sets of tags, the reporter tag take preference.Note: Support for these variable was added in the version1.0.0of apptuit-py and is not available
in any of the earlier released versions.Sending dataThere are two ways of sending the data to Apptuit. First is to use theApptuitReporter, and
the second options is to use thesend()method of the Apptuit client.
We will show how to use both of the options below.Sending data using Apptuit pyformance reporterimportsocketfrompyformanceimportMetricsRegistryfromapptuit.pyformance.apptuit_reporterimportApptuitReporterclassOrderService:def__init__(self,apptuit_token):self.registry=MetricsRegistry()self.init_reporter(apptuit_token,self.registry)definit_reporter(self,token,registry):hostname=socket.gethostname()global_tags={"host":hostname,"env":"dev","service":"order-service"}self.reporter=ApptuitReporter(registry=registry,reporting_interval=60,# data reported every 1 minutetoken=token,tags=global_tags,retry=2#this will retry in case of 500 response or connection errors occur.)# reporter.start() will start reporting the data asynchronously based on the reporting_interval set.self.reporter.start()defhandle_order(self,order):order_counter=self.registry.counter("order_count")# order handling related codeorder_counter.inc()defshutdown(self):# you can stop the reporter when you no longer wish to send data or when shutting downself.reporter.stop()One thing worth pointing out in the above example:Inhandle_orderwe create a new counterorder_counterwith the metric nameorder_count. The first
time this method is called a new counter object will be created and registered with the registry. For
subsequent calls, that counter will get reused since internally the registry will already have a
counter with that name.MetricsRegistryMetricsRegistry is the container for all the metrics in our application. We can use it to register and create
various kinds of metrics (meter, gauge, counter etc.). For example:frompyformanceimportMetricsRegistryregistry=MetricsRegistry()counter=registry.counter("order_count")meter=registry.meter("order_requests_rate")timer=registry.timer("order_requests_processing_time")Now, let's take a look at the different types of metrics and how to use them.MeterA meter measures the the rate of events, such as requests per second. Meter maintains the mean rate, and
1-, 5-, 15- minute moving averages.frompyformanceimportMetricsRegistryregistry=MetricsRegistry()metric_name="order_requests_rate"requests_meter=registry.meter(metric_name)defhandle_request(request):requests_meter.mark()# handle requestGaugeA gauge is an instantaneous measurement of a value. For example, number of pending jobs in a queue.fromqueueimportQueuefrompyformanceimportMetricsRegistryfrompyformance.meters.gaugeimportCallbackGaugeclassQueueManager:def__init__(self,registry,name):self.q=Queue()jobs_metric=registry.add(name,CallbackGauge(self.get_queue_size))defget_queue_size(self):returnself.q.size()The reporter will call theget_queue_sizefunction at its scheduled frequency and report
the size of the queue.CounterA counter can be used to simply count some data. It provides two methodsinc()to increment
its value anddec()to decrement it.frompyformanceimportMetricsRegistryregistry=MetricsRegistry()jobs_counter=registry.counter('pending_jobs')defadd_job(self,job):jobs_counter.inc(1)self.q.put(job)deftake_job(self):jobs_counter.dec(1)self.q.get()TimerA timer aggregates timing durations and provides duration statistics, as well as throughput statistics.frompyformanceimportMetricsRegistryregistry=MetricsRegistry()timer=registry.timer("response_time")defhandle_request(request):withtimer.time():return"OK"The above example will use the timer to report the time taken to serve each request.HistogramA histogram measures the statistical distribution of values in a stream of data. It provides aggregate data
such as the min, max, mean, sum, and count.frompyformanceimportMetricsRegistryregistry=MetricsRegistry()response_sizes=registry.histogram('response_size')defhandle_request(request):response=do_query(request)# process the queryresponse_sizes.add(response.size())Error Handling in ApptuitReporterThe ApptuitReporter sends data asynchronously (unless we are explicitly using it in synchronous mode
by not calling thestart()method). In asynchronous
mode it is very difficult to know if the reporter is working properly or not. To make this easier theApptuitReportertakes anerror_handlerargument.error_handleris expected to be a function reference
which takes 4 arguments. The signature of the function and the arguments are explained below:deferror_handler(status_code,successful_points_count,failed_points_count,errors):passstatus_code: The HTTP status code of the POST API call to Apptuitsuccessful_points_count: Number of points successfully processedfailed_points_count: Number of points which could not be processed due to errorserrors: List of error messages describing the reason of failure of each of the failed pointsBy default, theApptuitReporterregisters adefault_error_handler, which writes the errors tostderr.
To override that you can pass your own error handler implementation, or if you don't wish to do anything for errors
you can passNonefor theerror_handlerargument.Reporter with default error handlerimportlogging#reporter with default error handler (writes to stderr)reporter=ApptuitReporter(token=my_apptuit_token,registry=registry,reporting_interval=60,tags=reporter_tags)Reporter with No error handlerreporter_with_no_error_handler=ApptuitReporter(token=my_apptuit_token,registry=registry,reporting_interval=60,tags=reporter_tags,error_handler=None)The error handler function by definition takes only four arguments.
If you wish to pass extra arguments to the error handler you can use
closures or partial functions to get around the limitation.Passing extra argument using Partialimportloggingfromfunctoolsimportpartialdefcustom_error_handler(logger,status_code,successful,failed,errors):logger.error("ApptuitReporter failed to send%dpoints, due to errors:%s"%(failed,str(errors)))logger=logging.getLogger("logger key")apptuit_custom_error_handler=partial(custom_error_handler,logger)reporter=ApptuitReporter(token=my_apptuit_token,registry=registry,reporting_interval=60,tags=reporter_tags,error_handler=apptuit_custom_error_handler)Passing extra argument using closure...importloggingfromapptuitimportApptuitSendExceptionfromapptuit.pyformance.apptuit_reporterimportApptuitReporter...classOrderService:def__init__(self,apptuit_token):...self.logger=logging.getLogger("OrderService")...definit_reporter(self,token,registry):...defapptuit_error_handler(status_code,successful,failed,errors):logger=self.loggerlogger.error(str(ApptuitSendException(status_code,successful,failed,errors)))self.reporter=ApptuitReporter(...,error_handler=apptuit_error_handler)...Tags/MetadataWhen creating the ApptuitReporter, you can provide a set of tags (referred as reporter tags from now on)
which will be part of all the metrics reported by that reporter. However, in order to provide tags
specific to each metric you need to provide them when registering the metric with the registry. For example:fromapptuitimporttimeseriesfrompyformanceimportMetricsRegistryregistry=MetricsRegistry()metric_name="node_cpu"tags={"type":"idle","host":"node-foo","service":"order-service"}metric=timeseries.encode_metric(metric_name,tags)meter=registry.meter(metric)Here we provided the metric specific tags by callingtimeseries.encode_metricand
providing the metric name and the tags as parameters. When registering the metric we provide this
encoded name to the registry instead of the plain metric name.To decode an encoded metric name use thedecode_metric()function fromtimeseriesmodule.fromapptuitimporttimeseriesencoded_metric=timeseries.encode_metric("node.cpu",{"type":"idle"})metric_name,tags=timeseries.decode_metric(encoded_metric)Arecommended practiseis to maintain a local cache of the created metrics and reuse them, rather than
creating them every time:importsocketimporttimefromapptuitimporttimeseriesfromapptuit.pyformanceimportApptuitReporterfrompyformanceimportMetricsRegistryclassOrderService:def__init__(self,apptuit_token):self.registry=MetricsRegistry()self.init_reporter(apptuit_token,self.registry)self.order_counters={}definit_reporter(self,token,registry):hostname=socket.gethostname()global_tags={"host":hostname,"env":"dev","service":"order-service"}self.reporter=ApptuitReporter(registry=registry,reporting_interval=60,# data reported every 1 minutetoken=token,tags=global_tags)# reporter.start() will start reporting the data asynchronously based on the reporting_interval set.self.reporter.start()defget_order_counter(self,city_code):# We have counters for every city codeifcity_codenotinself.order_counters:tags={"city-code":city_code}metric=timeseries.encode_metric("order_count",tags=tags)self.order_counters[city_code]=self.registry.counter(metric)returnself.order_counters[city_code]defhandle_order(self,order):order_counter=self.get_order_counter(order.city_code)order_counter.inc()self.process_order(order)defshutdown(self):# you can stop the reporter when you no longer wish to send data or when shutting downself.reporter.stop()defprocess_order(self,order):time.sleep(5)Here we have a methodget_order_counterwhich takes thecity_codeas a parameter. There
is a local cache of counters keyed by the encoded metric names. This avoids the unnecessary overhead
of encoding the metric name and tags every time, if we already have created a counter for that city.
It also ensures that we will report separate time-series for order-counts of different city codes.About Host TagThe reporter will add ahosttag key with host name as its value (obtained by callingsocket.gethostname()).
This is helpful in order to group the metrics by host if the reporter is being run on multiple servers. The value
of thehosttag key can be overridden by passing our ownhosttag in thetagsparameter to the reporter or
by setting ahosttag in the global environment variable for tagsIf we don't wish for thehosttag to be set by default we can disable it by setting thedisable_host_tagparameter of the reporter toTrue. Alternatively we can set the environment
variableAPPTUIT_DISABLE_HOST_TAGtoTrueto disable it.Restrictions on Tags and Metric namesAllowed characters in tag keys and metric names- Tag keys and metric names can have any unicode
etters (as defined by unicode specification) and the following special characters:.,-,_,/.
However, if we are looking to follow Prometheus compliant naming
([see specification])(https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels)
we should restrict them to ASCII letters, digits and  underscores only and it must match the
regex[a-zA-Z_][a-zA-Z0-9_]*. No such restriction is applicable on tag values.Maximum number of tags- Apptuit currently allows upto 25 tag key-value pairs per datapointMeta MetricsTheApptuitReporteralso reports a set of meta metrics which can be a useful indicator if the reporter is
working as expected or not, as well as to get a sense of how many points are being sent and the latency of
the Apptuit API. These meta metrics are described below.apptuit_reporter_send_total- Total number of points sentapptuit_reporter_send_successful- Number of points which were succssfully processedapptuit_reporter_send_failed- Number of points which failedapptuit_reporter_send_time- Timing stats of of the send APIPython Process MetricsTheApptutiReportercan also be configured to report various metrics of
the Python process it is running in. By default it is disabled but we can enable it by
passing setting the parametercollect_process_metricstoTruewhen creating the
reporter object. The reporter will collect metrics related to the system resource usage
by the process (cpu, memory, IPC etc.) as well as metrics related to garbage collection
and threads. The complete list of all the metrics collected is provided below:python_cpu_time_used_seconds- Total time spent by the process in user mode and system mode.python_memory_usage_bytes- Total amount of memory used by the process.python_page_faults- Total number of page faults received by the process.*python_process_swaps- Total number of times the process was swapped-out of the main memory.python_block_operations- Total number of block input and output operations.python_ipc_messages- Total number of inter-process messages sent and received by the process.*python_system_signals- Total number of signals received by the process.python_context_switches- Total number of context switches of the process.python_thread- Count of active, demon and dummy threads.python_gc_collection- Count of objects collected in gc for each generation.python_gc_threshold- Value of garbage collector threshold for each generation.Note- Metrics marked with*are zero on Linux because it does not support themGlobal tags, reporter tags and metric tagsWhen using the reporter we have three sets of tags, it's better to clarify a few things about them.ApptuitReportertakes a set of tags as parameter. It adds these tags to all the metrics it is reporting.If the environment variableAPPTUIT_TAGSis set, the reporter takes those into account as well, however
the tags passed to it take preference in case of a conflict because of common tag keys.Each metric being reported by the reporter might also have some tags attached, in case of a conflict
because of common tag keys, the metric tags take preference over reporter or global tags.Sending data using send() APIApart from using the Pyformance reporter, you can also use the low levelsend()API from the apptuit
client to directly send the data. If you want tags while sending you can use the global_tags
parameter of Apptuit class. If global_tags are set then environmental tags will not be used.fromapptuitimportApptuit,DataPointimporttimeimportrandomimportsockettoken="mytoken"client=Apptuit(token=token)metrics=["proc.cpu.percent","node.memory.bytes","network.send.bytes","network.receive.bytes","node.load.avg"]tags={"host":socket.gethostname()}curtime=int(time.time())dps=[]whileTrue:curtime=int(time.time())formetricinmetrics:dps.append(DataPoint(metric,tags,curtime,random.random()))iflen(dps)==100:client.send(dps,retry_count=3#this will retry in case of 500 response or connection errors occur.)dps=[]time.sleep(60)Sending data using send_timeseries() APIThesendAPI works with a list of DataPoint objects. Creating each DataPoint object involves validating the metric name and
the tags. If we are creating thousands of DataPoint objects with the metric name and tags, it can quickly get very expensive.
In order to avoid that overhead, there is an alternativesend_timeseriesAPI as well, which accepts a list ofTimeSeriesobjects. This is much more convenient as we need to create aTimeSeriesobject with a metric name and tags. Thereafter
we can add points to that timeseries object by calling itsadd_point()method. This avoids creation of DataPoint objects
and the overhead of the tag validation.Following is an example from a scraper we run internally. We call an HTTP API, get a JSON response and send it to us in the
form of timeseries. In order to avoid too many API calls to Apptuit we callsend_timeserieswhenever we have accumulated
50000 or more points. Once we make asend_timeseriescall we reset theseries_listobject to contain just the latestTimeSeriesobject (all the earlier series would have been sent to Apptuit).fromapptuitimportApptuit,TimeSeriesseries_list=[]points_count=0token="mytoken"client=Apptuit(token=token)response_data=make_request()forresultinresponse_data["results"]:metric_name=result["metric"]tags=result["tags"]series=TimeSeries(metric_name,tags)series_list.append(series)fortimestamp,valueinresult["values"]:series.add_point(timestamp,value)points_count+=1ifpoints_count>=50000:client.send_timeseries(series_list)points_count=0series_list=[TimeSeries(metric_name,tags)]ifpoints_count>0:client.send_timeseries(series_list)Querying for datafromapptuitimportApptuitimporttimetoken='my_token'# replace with your Apptuit tokenapptuit=Apptuit(token=token)start_time=int(time.time())-3600# let's query for data going back 1 hour from nowquery_res=apptuit.query("fetch('proc.cpu.percent').downsample('1m', 'avg')",start=start_timeretry_count=3#this will retry in case of 500 response or connection errors occur.)# we can create a Pandas dataframe from the result object by calling to_df()df=query_res[0].to_df()# Another way of creating the DF is accessing by the metric name in the queryanother_df=query_res['proc.cpu.percent'].to_df()It should be noted that using theto_df()method requires that you havepandasinstalled.
We don't installpandasby default as part of the requirements because not every user of the library
would want to query or create dataframes (many users just use thesendAPI or the reporter functionality)