Package: timeseer
Timeseer.AI ClientThe Timeseer.AI Client is a Python SDK to access the functionality ofTimeseer.AI.
Built onApache Arrow,
the SDK integrates natively with thePandasorPolarsecosystems.InstallingThe Timeseer.AI Client is available onPyPI.(venv) $ pip install timeseerConnectingThe Timeseer.AI Client relies onApache Arrow Flightfor highly efficient data transfers.Communications are protected by an API key.
An API key can be generated within Timeseer underConfigure > API keys.
Each API key has a name and a secret value that is shown only once.The API key is used to create a connection to a Timeseer instance running at a specific host and port:>>> from timeseer_client import *
>>> api_key=('<api-key-name>', '<api-key>')
>>> client = Client(api_key, host='localhost', port=8081)Functionality OverviewIn Timeseer,
time series data is available through two concepts:Sources contain a varying number of time series that are constantly updated with new data.Data Sets contain a fixed number of time series in a specific time range.Sources are typically used for continuous monitoring of data,
while Data Sets are the starting point for a data science project.Time series data from Sources and Data Sets is processed by Flows.
Flows analyze data or create derived Data Sets.Insights and data that is generated by Flows are made available through Data Services.The Timeseer.AI Client represents each of these concepts as a separate class that exposes the functionality that is specific to that concept.
Each concept class is created by passing theClientto the constructor.Full documentation is available in the code by running:>>> import timeseer_client
>>> help(timeseer_client)UsageThis usage sample generates a sine wave in using Pandas and numpy.
Values below 0 of the sine wave are assumed to be the result of a faulty sensor reading.
It shows how Timeseer can be used to analyze this and how it automatically creates a derived data set.First install Pandas:(venv) $ pip install pandasGenerate the sine wave data:>>> import numpy as np
>>> import pandas as pd
>>> ts = pd.date_range("2022-01-01T00:00:00Z", "2022-02-01T00:00:00Z", freq="H")
>>> values = np.round(10 * np.sin(2 * np.pi * ((ts.astype(np.int64) // 10**9) - ts[0].timestamp()) / (24*60*60)), decimals=2)
>>> df = pd.DataFrame(dict(ts=ts, value=values))
>>> df.head(20)
                          ts  value
0  2022-01-01 00:00:00+00:00   0.00
1  2022-01-01 01:00:00+00:00   2.59
2  2022-01-01 02:00:00+00:00   5.00
3  2022-01-01 03:00:00+00:00   7.07
4  2022-01-01 04:00:00+00:00   8.66
5  2022-01-01 05:00:00+00:00   9.66
6  2022-01-01 06:00:00+00:00  10.00
7  2022-01-01 07:00:00+00:00   9.66
8  2022-01-01 08:00:00+00:00   8.66
9  2022-01-01 09:00:00+00:00   7.07
10 2022-01-01 10:00:00+00:00   5.00
11 2022-01-01 11:00:00+00:00   2.59
12 2022-01-01 12:00:00+00:00  -0.00
13 2022-01-01 13:00:00+00:00  -2.59
14 2022-01-01 14:00:00+00:00  -5.00
15 2022-01-01 15:00:00+00:00  -7.07
16 2022-01-01 16:00:00+00:00  -8.66
17 2022-01-01 17:00:00+00:00  -9.66
18 2022-01-01 18:00:00+00:00 -10.00
19 2022-01-01 19:00:00+00:00  -9.66Define a Timeseer API key inConfigure > API keysand use it to create aClient:>>> from timeseer_client import *
>>> client = Client(("<api key name>", "<api key>"), host='timeseer.example.org', port=8081)Timeseer uses metadata to automatically profile a time series.
In this case, only thephysical lower limitof the sensor that measured the time series is known,
which is0.>>> from timeseer_client.metadata import fields
>>> series = SeriesSelector("Sines", {"function": "sine", "amplitude": "10"})
>>> metadata = Metadata(series, {fields.LimitLowPhysical: 0})Each time series in Timeseer is identified by aSeriesSelector.
EachSeriesSelectorhas a source ("Sines"),
which will become the data set name,
andtags and a field.
This time series has the"function"and"amplitude"tags and the (default)"value"field.For time series where additional structure is not available,
aSeriesSelectorcan also be created using a single"series name"tag:>>> SeriesSelector("Sines", "sine-10") == SeriesSelector("Sines", {"series name": "sine-10"})Profiling this time series can be done using theprofileconvenience function:>>> profile(client, "Sines", [(metadata, df)])
[{'type': 'flow', 'name': 'Sines'}, {'type': 'data service', 'name': 'Sines'}, {'type': 'data set', 'name': 'Sines'}]Theprofilefunction creates a Data Set, a Data Service and a Flow with the given name,
in this case"Sines".
It also evaluates the flow.Data should be provided as apyarrow.Tableor a PandasDataFrame.A Data Service summarizes the profiling results as Statistics and Event Frames.Event Frames define a time range where something interesting has been detected.>>> data_services = DataServices(client)
>>> data_service = DataServiceSelector('Sines', 'Sines')
>>> event_frames = data_services.get_event_frames(data_service)
>>> event_frames.to_pandas()['type'].value_counts()
compression - linear undercompression    61
Out of bounds (lower, physical)          31
Values below zero                        31
Upper limit is present                    1
Interpolation type is present             1
Compression - flat archival rate          1
Description is present                    1
Unit is present                           1
Name: type, dtype: int64Not all profiling results are issues.
In this case we can safely ignore the 'linear undercompression' events.
The 'Out of bounds (lower, physical)' event frames cannot be ignored though,
as was mentioned earlier.Statistics can be used to gain high-level insight into the data and explain the Event Frames:>>> data_services.get_statistics(data_service, series)
[... Statistic(name='Value statistics', data_type='table', result=[['Min', -10.0], ['Max', 10.0], ['Mean', 4.775152794086695e-18], ['Median', 0], ['Std', 7.073308943835715]]) ...]It is clear (and expected based on the data generation) that theOut of bounds (lower, physical)Event Frames occur because the minimum value is-10.0.Timeseer can automatically correct the data to be within bounds using various strategies.
To create derived data in periods where an Event Frame is detected,
a "filter" Block in a Flow on that event frame type needs to be inserted.The derived data can be stored in a few ways.
It is possible to create another Data Set, for example.
Storing them in a Data Service instead will allow verification that the problem has been resolved,
as data is stored there alongside quality indicators.There is no shorthand for data cleaning,
as each case will require different action.
The most readable way to define the Flow that will create the derived data is in YAML.Createsine-derive.yml:----type:data servicename:Derived sine resultskpiSet:Data quality fundamentalsrange:start:"2022-01-01T00:00:00Z"end:"2022-02-01T00:00:00Z"-type:flowname:Create derived sinedataSet:Sinesblocks:-name:Analyze time seriestype:analysis-name:Hold last value when out of boundstype:filteraugmentationStrategy:hold last valuefilters:-type:univariatefilter:"Outofbounds(lower,physical)"series:ALL-name:Analyze derived time seriestype:analysis-name:Keep results for derived series in Derived sine results data servicetype:data_service_contributedataServiceName:Derived sine resultscontributionBlockNames:[Analyze derived time series]TheResourcesandFlowsclasses allow creating resources and evaluating flows respectively.>>> resources = Resources(client)
>>> resources.create(path="sine-derive.yml")
>>> flows = Flows(client)
>>> flows.evaluate("Create derived sine")The derived data has been profiled by the Flow.
Profiling results are available in the"Derived sine results"Data Service:>>> derived_data_service = DataServiceSelector('Derived sine results', 'Sines')
>>> event_frames = data_services.get_event_frames(derived_data_service)
>>> event_frames.to_pandas()['type'].value_counts()
compression - linear undercompression    31
Compression - flat archival rate          1
Interpolation type is present             1
Unit is present                           1
Description is present                    1
Upper limit is present                    1
Name: type, dtype: int64The derived data does no longer contain values below 0:>>> derived_data = data_services.get_data(derived_data_service, series)
>>> derived_data.to_pandas().head(26)
                           value
ts
2022-01-01 00:00:00+00:00   0.00
2022-01-01 01:00:00+00:00   2.59
2022-01-01 02:00:00+00:00   5.00
2022-01-01 03:00:00+00:00   7.07
2022-01-01 04:00:00+00:00   8.66
2022-01-01 05:00:00+00:00   9.66
2022-01-01 06:00:00+00:00  10.00
2022-01-01 07:00:00+00:00   9.66
2022-01-01 08:00:00+00:00   8.66
2022-01-01 09:00:00+00:00   7.07
2022-01-01 10:00:00+00:00   5.00
2022-01-01 11:00:00+00:00   2.59
2022-01-01 12:00:00+00:00  -0.00
2022-01-01 13:00:00+00:00  -0.00
2022-01-01 14:00:00+00:00  -0.00
2022-01-01 15:00:00+00:00  -0.00
2022-01-01 16:00:00+00:00  -0.00
2022-01-01 17:00:00+00:00  -0.00
2022-01-01 18:00:00+00:00  -0.00
2022-01-01 19:00:00+00:00  -0.00
2022-01-01 20:00:00+00:00  -0.00
2022-01-01 21:00:00+00:00  -0.00
2022-01-01 22:00:00+00:00  -0.00
2022-01-01 23:00:00+00:00  -0.00
2022-01-02 00:00:00+00:00   0.00
2022-01-02 01:00:00+00:00   2.59This only scratches the surface of the functionality in Timeseer.
Learn more in theHelpmenu in the user interface.
All resources, blocks in Flows and event frame types are thoroughly documented.