Package: timeseriesql
TimeSeriesQLA Pythonic query language for time series dataTable of ContentsAbout the ProjectBuilt WithGetting StartedPrerequisitesInstallationUsageCSV BackendTimeSeriesPlotting LibsRoadmapContributingLicenseContactAcknowledgementsAbout The ProjectThere are many time series databases and each have their own query language.  Each platform takes time to invest in
learning the structure and keywords of that language and often the skills learned don't translate to other
platforms.  The goal of this project is to create a time series specific library that can be used across many different time series databases as well as easy to learn because it uses Python syntax.Built WithNumpyGetting StartedTo get a local copy up and running follow these simple steps.PrerequisitesThe requirements are in therequirements.txtfile.InstallationpippipinstalltimeseriesqlmanualClone the timeseriesqlgitclonehttps:://github.com/mbeale/timeseriesql.gitInstall librarycdtimeseriesql
pythonsetup.pyinstallUsageThe way this project works is to provide a general framework for querying a time series with pluggable
backends that communicate with specific time series databases.  The queries are created using Python
generators, a formatt familiar to Pythonistas.data=Query(xforxin"metric.name"ifx.some_label="some_value").by("a_label")[start:end:resolution]The return value is aTimeSeriesobject that uses a Numpy array as backend.  That object can haveufuncsand other numpy functions applied against it.  More examples to come.There are defaults for start and resolution that are controlled by environment variables.  That helps avoid
fetching all measurements from the beginning of time by accident.DEFAULT_START_OFFSET#defaults to 3600 secondsDEFAULT_RESOLUTION#defaults to 60 secondsCSV Backend UsageOften time series data is loaded from a CSV file.  The backend expects the first column to be the time index in
either a numerical timestamp or strings in ISO 8601 date or datetime format.  The filters are applied to the
headers of the CSV.  If labels are not in the CSV and are supplied as part of the query, then filters will not
be applied.If any columns are empty or don't contain a numeric value, the value becomes anp.nan.Basic CSV Usagefromtimeseriesql.backends.csv_backendimportCSVBackenddata=CSVBackend(xforxin"path/to.csv")[:]Basic CSV FilteringFor CSV files the labels are the column headers.  If there are columns that are not needed, they can be filtered out.fromtimeseriesql.backends.csv_backendimportCSVBackenddata=CSVBackend(xforxin"path/to.csv"ifx.label=="A")[:]data=CSVBackend(xforxin"path/to.csv"ifx.label!="B")[:]data=CSVBackend(xforxin"path/to.csv"ifx.labelin["B","C","G"])[:]data=CSVBackend(xforxin"path/to.csv"ifx.labelnotin["B","C","G"])[:]Set the Labelsfromtimeseriesql.backends.csv_backendimportCSVBackenddata=CSVBackend(xforxin"path/to.csv").labels([{"label":"one"},{"label":"two"},{"label":"three"},{"label":"four"},{"label":"five"},{"label":"six"},{"label":"seven"},])[:]TimeSeries UsageTheTimeSeriesobject is allows for manipulation of the time series data after the it's been queried from the
backend.In the following examples, the variables starting withts_are assumed to be queried data from a backend.TimeSeries Operations# Basic mathematical operations (+, -, /, *)ts_1+5# will return a new seriests_1+=5#will perform operation in placets_1+=ts_2#add together two TimeSeriesTimeSeries Time IndexThe time index is a array of floats but there is a built in method to convert the floats intonp.datetime64.ts_1.time# array of floatsts_1.time.dt#array of np.datetime64TimeSeries MergingTimeSeriesobjects can be combined but the ending time indexes must be the same.  This may require empty values
to be created where the indexes don't align.new_t=ts_1.merge([ts_2,ts_3])TimeSeries Grouping/ReducingIf there are multiple streams, they can be grouped and merged by the labels.reduced=ts_1.group(["hostname","name"]).add()reduced=ts_1.group("env").mean()reduced=ts_1.group("env").mean(axis=None)#setting the access to None will get the mean of the entire objectTimeSeries Special Indexingimportnumpyasnpbeg=np.datetime64('2019-02-25T03:00')end=np.datetime64('2019-02-25T04:00')ts_1[beg:end]# set a time rangets_1[beg:np.timedelta64(3,"m")]# fetch from beginning + 3 minutests_1[np.timedelta64(3,"m"):]#start from beginning + 3 minutests_1[:np.timedelta64(3,"m")]#end at the end - 3 minutests_1[{"hostname":"host2"}]# by labelsTimeSeries Rolling WindowsTherolling_windowmethod assumes that the data is filled and at a fixed resolution.  Number of periods is
an integer and not a time range.rolling_cum_sum=ts_1.rolling_window(12).add()#rolling cumsumrolling_mean=ts_1.rolling_window(12).mean()# rolling meanrolling=ts_1.rolling_window(12).median()#rolling medianTimeSeries ResampleTheresamplemethod allows a smaller period to be aggregated into a larger period.resampled=ts_1.resample(300).mean()#resamples to 5 minutes and takes the meanTimeSeries to PandasThe conversion returns 2 pandas DataFrames, one for the labels and the other for the data.data,labels=ts_1.to_pandas()Plotting LibsAvailablematplotlibCreating a custom backendStart by extending thePlotclass.fromtimeseries.plotimportPlotclassNewPlottingLib(Plot):passThere is a list of functions that can be extended for as different plots.  Also there are functions that generate titles, xlabel, ylabel, and legend labels.  Use those to grab default information.  They can be overridden to provide more custom logic around the those fields.RoadmapSee theopen issuesfor a list of proposed features (and known issues).ContributingContributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make aregreatly appreciated.Fork the ProjectCreate your Feature Branch (git checkout -b feature/AmazingFeature)Commit your Changes (git commit -m 'Add some AmazingFeature')Push to the Branch (git push origin feature/AmazingFeature)Open a Pull RequestLicenseDistributed under the MIT License. SeeLICENSEfor more information.ContactMichael Beale -michael.beale@gmail.comProject Link:https://github.com/mbeale/timeseriesql