Package: kairos
Version:0.10.1Download:http://pypi.python.org/pypi/kairosSource:https://github.com/agoragames/kairosKeywords:python, redis, mongo, sql, mysql, sqlite, postgresql, cassandra, timeseries, rrd, gevent, statisticsOverviewUsageConstructorStorage EnginesRedis (redis://)Mongo (mongodb://)SQL (*sql*://)Cassandra (cassandra://, cql://)Inserting DatainsertRedisbulk_insertRedisMongoSQLCassandraMeta DatalistpropertiesReading DatagetseriesiterateCustomized ReadsGeneralfetchprocess_rowcondensejoin_rowscollapsetransformRedisMongoSQLCassandraDeleting Datadeletedelete_allexpireDragons!InstallationTestsRoadmapLicenseOverviewKairos provides time series storage using Redis, Mongo, SQL or Cassandra
backends. Kairos is intended to replace RRD and Whisper in situations where
the scale and flexibility of other data stores is required. It works withgeventand is the library on whichtorusis built.Recommended for python 2.7 and later, it can work with previous versions if you
installOrderedDict.Kairos provides a consistent API for a variety of timeseries types and the
storage engines they’re implemented in. Each timestamp is resolved to a
consistent bucket identifier (“interval”) based on the number of whole seconds
since epoch, or a number corresponding to the Gregorian date associated with
the relative intervals[daily, weekly, monthly, yearly](e.g19991231).
Within that, data can optionally be stored at resolutions (e.g. “daily,
in 1 hour increments”). Multiple intervals can be tracked within a timeseries,
each with its own resolution and optional TTL (e.g. 60 days of daily data,
lifetime monthly data).In data stores that support it, TTLs can be set for automatically deleting
data after a set number of intervals; other data stores expose anexpire()method for deleting data programmatically.Within each interval (or resolution), data is stored according to the type of
the timeseries and what each backend supports. The values tracked in each
timeseries can be loosely typed for backends that support it, else the type
will be whatever is set in the timeseries constructor. Even when loosely typed,
it should be assumed that the value should be a string or number.seriesAll data will be stored in the order in which it arrives. Uses data store
list types where supported, else it will be timestamped records that
come as close as possible to the order in which they were written. Queries
will return list objects.histogramA hash of unique values to the number of its occurrences within an interval.
Uses data store dictionaries where supported, else it will be separate
records for each unique value and timestamp. Queries will return dictionary
objects. Built-in transforms assume that keys are real numbers.countA simple counter will be maintained for each interval, starting at 0 for
an interval. Queries will return an integer.gaugeStores the last-written value for each interval. Queries will return whatever
the value type was.setStores all the unique values within an interval. Uses data store sets where
supported, else it will be separate records for each unique value. Queries
will return set objects.UsageKairos supports all storage engines using the same API. The constructor will
return a Timeseries instance tailored for the type of data and the storage
engine, and the API for updating and querying the timeseries is consistent
for all combinations of data type and storage engine.ConstructorThe first argument is a handle to a supported storage engine or a URL (see
below), and the rest of the keyword arguments configure the timeseries. The
keyword arguments supported by all storage engines are:typeRequired, defines the type of the timeseries. Raises aNotImplementedErrorif the backend does not support the type.read_funcOptional, is a function applied to all values read back from the
database. Without it, values will be strings for Redis, whateverwrite_funcdefined for Mongo. Must accept a string value for Redis
(empty string for no data) and can return anything.write_funcOptional, is a function applied to all values when writing. Can be
used for histogram resolution, converting an object into an id, etc.
Must accept whatever can be inserted into a timeseries and return an
object which can be saved according to the rules of the storage engine.intervalsRequired, a dictionary of interval configurations in the form of:{
  # interval name, used in keys and should conform to best
  # practices according to the storage engine.
  minute: {

    # Required. The number of seconds that the interval will cover,
    # or a supported Gregorian interval.
    step: 60,

    # Optional. The maximum number of intervals to maintain. If supplied,
    # will use Redis and Mongo expiration to delete old intervals, else
    # intervals exist in perpetuity. If the storage engine doesn't support
    # expiry, will be used to implement the expire() call.
    steps: 240,

    # Optional. Defines the resolution of the data, i.e. the number of
    # seconds in which data is assumed to have occurred "at the same time".
    # So if you're tracking a month-long time series, you may only need
    # resolution down to the day, or resolution=86400. Defaults to same
    # value as "step". Can also be a Gregorian interval.
    resolution: 60,
  }
}In addition to specifyingstepandresolutionin terms of seconds,
kairos also supports a simplified format for larger time intervals. For
hours (h), days (d), weeks (w), months (m) and years (y), you can use
the format30dto represent 30 days, for example.As of0.3.0, kairos also supports the Gregorian calendar forstepandresolution. Either or both parameters can use the terms[daily, weekly, monthly, yearly]to describe an interval. You can also mix these
terms betweenstepandresolution(e.g.dailyin1hresolutions). The expiration time for Gregorian dates is still defined
in terms of seconds and may not match the varying month lengths, leap years,
etc. Gregorian dates are translated intostrptime- andstrftime-compatible
keys (as integers) and so may be easier to use in raw form or with any
external tools. Thedurationparameter to transforms run on gregorian
series will be seconds in whole number of days (where a day is 86400 seconds).Storage EnginesEach of the supported storage engines also supports a set of keyword arguments
to configure their behavior. When intializing with a URL, the keyword argumentclient_configcan optionally be a dictionary which will be passed as
keyword arguments to the constructor for the client associated with the URL.
If kairos implements any custom keyword arguments fromclient_configthose
are documented below.Redis (redis://)An example timeseries stored in Redis:from kairos import Timeseries
import redis

client = redis.Redis('localhost', 6379)
t = Timeseries(client, type='histogram', read_func=int, intervals={
  'minute':{
    'step':60,            # 60 seconds
    'steps':120,          # last 2 hours
  }
})

t.insert('example', 3.14159)
t.insert('example', 2.71828)
print t.get('example', 'minute')Additional keyword arguments are:prefix
  Optional, is a prefix for all keys in this timeseries. If
  supplied and it doesn't end with ":", it will be automatically appended.Supported URLformats:redis://localhost
redis://localhost/3Allsupportedconfiguration options can be passed inclient_config.Mongo (mongodb://)An example timeseries stored in Mongo:from kairos import Timeseries
import pymongo

client = pymongo.MongoClient('localhost')
t = Timeseries(client, type='histogram', read_func=float, intervals={
  'minute':{
    'step':60,            # 60 seconds
    'steps':120,          # last 2 hours
  }
})

t.insert('example', 3.14159)
t.insert('example', 2.71828)
print t.get('example', 'minute')Additional keyword arguments are:escape_character
  Optional, defines the character used to escape periods. Defaults to the
  unicode character "U+FFFF".Supported URLformats:mongodb://localhost
mongodb://localhost:27018/timeseries
mongodb://guest:host@localhost/authed_dbAllsupportedconfiguration arguments can be passed inclient_config, in addition to:database
  The name of the database to use. Defaults to 'kairos'. Required if using
  an auth database. Overrides any database provided in the URL.SQL (*sql*://)An example timeseries stored in a SQLite memory store:from kairos import Timeseries
from sqlalchemy import create_engine

client = create_engine('sqlite:///:memory:')
t = Timeseries(client, type='histogram', read_func=int, intervals={
  'minute':{
    'step':60,            # 60 seconds
    'steps':120,          # last 2 hours
  }
})

t.insert('example', 3.14159)
t.insert('example', 2.71828)
print t.get('example', 'minute')Additional keyword arguments are:string_length
  Optional, configures the length of strings (VARCHARs). Defaults to 255.
  All tables have at least 2 string columns, and the size of these columns
  may impact usability of the SQL storage engine.

text_length
  Optional, configures the length of TEXT and BLOB columns. Defaults to
  32Kbytes. Only matters if value_type is a text or blob.

table_name
  Optional, overrides the default table name for a timeseries type.

value_type
  Optional, defines the type of value to be stored in the timeseries.
  Defaults to float. Can be a string, a Python type or a SQLAlchemy type
  or instance.

  'blob'
  'bool'
  <type 'bool'>
  'boolean'
  'clob'
  'date'
  <type 'datetime.date'>
  'datetime'
  <type 'datetime.datetime'>
  'decimal'
  <class 'decimal.Decimal'>
  'float'
  <type 'float'>
  'int'
  'int64'
  'integer'
  <type 'int'>
  'long'
  <type 'long'>
  'str'
  'string'
  <type 'str'>
  'text'
  'time'
  <type 'datetime.time'>
  'unicode'
  <type 'unicode'>Supported URLformatsare many and varied. A few examples:sqlite:///:memory:
postgresql://scott:tiger@localhost/mydatabase
mysql+mysqldb://scott:tiger@localhost/foo
oracle://scott:tiger@127.0.0.1:1521/sidnameAllsupportedconstructor arguments can be used inclient_config.Cassandra (cassandra://, cql://)An example timeseries stored in Cassandra:from kairos import Timeseries
import cql

client = cql.connect('localhost', 9160, 'keyspace', cql_version='3.0.0')
t = Timeseries(client, type='histogram', read_func=int, intervals={
  'minute':{
    'step':60,            # 60 seconds
    'steps':120,          # last 2 hours
  }
})

t.insert('example', 3.14159)
t.insert('example', 2.71828)
print t.get('example', 'minute')Additional keyword arguments are:table_name
  Optional, overrides the default table name for a timeseries type.

pool_size
  Optional, set a cap on the pool size. Defines the maximum number of
  connections to maintain in the pool. Defaults to 0 for no maximum.

value_type
  Optional, defines the type of value to be stored in the timeseries.
  Defaults to float. Can be a string or a Python type.

  <type 'unicode'>
  string
  decimal
  <type 'long'>
  int
  double
  unicode
  float
  long
  <type 'bool'>
  <type 'float'>
  boolean
  int64
  str
  text
  blob
  clob
  integer
  bool
  <type 'str'>
  <type 'int'>
  inetSupported URL formats are:cql://
cassandra://localhost:9160
cassandra://localhost/databaseAllsupportedconstructor arguments can be used inclient_config.kairos requirescqlas it supportsCQL3and gevent. This
requires that the keyspace be created before the connection, and the keyword
argumentcql_version='3.0.0'must be used.A notable downside of this library is that it does not support a list of
endpoints to connect to, so is missing key High Availability features.It is likely that future versions of kairos will requirecassandra-driverwhen it
is ready.Cassandra counters can only store integers, and cannot be used for a
running total of floating point numbers.Kairos implements a connection pooling mechanism on top ofcql. The pool
is a simple soft-cap on the number of connections maintained in the pool,
but not necessarily the total number of connections at a time. An optional
hard cap may be implemented in a future release.Inserting DataThere are two methods to insert data,Timeseries.insertandTimeseries.bulk_insert.insertnameThe name of the statisticvalueThe value of the statistic (optional for count timeseries), or a list of valuestimestamp(optional)The timestamp of the statistic, defaults totime.time()if not suppliedintervals(optional)The number of time intervals before (<0) or after (>0)timestampto copy the data**kwargs(optional)Any additional keyword arguments supported by a backend, see belowForseriesandhistogramtimeseries types,valuecan be whatever
you’d like, optionally processed through thewrite_funcmethod before being
written to storage. Depending on your needs,value(or the output ofwrite_func) does not have to be a number, and can be used to track such
things as unique occurances of a string or references to other objects, such
as MongoDB ObjectIds. Note that many of the aggregate functions inhistogramexpect the data to be real numbers.For thecounttype,valueis optional and should be a float or integer
representing the amount by which to increment or decrementname; it defaults
to1.For thegaugetype,valuecan be anything and it will be stored as-is.For all timeseries types, ifvalueis one of(list,tuple,set), will
callbulk_insert.Theintervalsoption allows the caller to simulate the value appearing in
time periods before or after thetimestamp. This is useful for creating
fast trending (e.g. “count over last seven days”). It is important to note
that, because the time periods are simulated, resolution is lost for the
the simulated timestamps.RedisRedis supports an additional keyword argument,pipeline, to give the caller
control over batches of commands. Ifpipelineis supplied, theexecutemethod will not be called and it is up to the caller to do so.bulk_insertinsertsThe structure of inserts (see below)intervals(optional)The number of time intervals before (<0) or after (>0)timestampto copy the data**kwargs(optional)Any additional keyword arguments supported by a backend, see belowTheinsertsfield must take the following form:{
  timestamp : {
    name: [ value, ... ],
    ...
  },
  ...
}The meaning oftimestamp,nameandvalueare identical to those
parameters ininsert. The caller can insert any number of timestamps,
statistic names and values, and the backend will optimize the insert where
possible. See details on the different backends below. Where a backend does
not support an optimized bulk insert, the data structure will be processed
such that each value will be passed toinsert.Theinsertsstructure can be adictorOrderedDict. If you need
the insert order preserved, such as when inserting into aseriesorgauge, you should useOrderedDict.Iftimestampis unknown, useNonefor the key and it will be set to
the current value oftime.time(). Note that this may alter ordering ifinsertsis anOrderedDict.NOTEbulk inserts will increase memory usage of the client process.RedisRedis bulk inserts are implemented by using a single pipeline (without
transactions) and committing the pipeline after all bulk inserts have been
executed. The bulk insert also supports thepipelineargument, with the
same rules asinsert.MongoMongo bulk inserts are implemented by joining all of the data together into
a condensed set of queries and updates. As the configuration of a timeseries
may result in multiple timestamps resolving to the same record (e.g. per-day
data), this could result in significant performance gains when the timeseries
is acount,histogramorgauge.SQLThere is no optimization for bulk inserts in SQL due to the lack of
native update-or-insert support. The generic SQL implementation requires an
attempted update to be committed before kairos can determine if an insert is
required. Future versions may have optimized implementations for specific
SQL servers which support such a feature, at which time bulk inserts may be
optimized for those specific backends.CassandraThecqllibrary has no support for transactions, grouping, etc.Meta DataThere are two methods to query meta data about a Timeseries.listThere are no arguments. Returns a list of all of the stat names stored
in the Timeseries.propertiesTakes a single argument, the name of the timeseries. Returns a dictionary
with the following fields:{ interval : { 'first' : timestamp, 'last' : timestamp } }intervalwill be the named interval, such as “minute”. For each interval,
there is a dictionary of properties.firstis the timestamp of the first
data point in the timeseries, andlastis the last data point in the
timeseries.Reading DataThere are three methods to read data,Timeseries.get,Timeseries.seriesandTimeseries.iterate.getwill return data from a single bucket,
andserieswill return data from several buckets.iteratewill use
theTimeseries.propertiesmethod to determine the date range of the data,
and return a generator that callsgetfor every possible interval in
the date range.getSupports the following parameters. All optional parameters are keyword arguments.nameThe name of the statistic, or a list of names whose data will be joined together.intervalThe named interval to read fromtimestamp(optional)The timestamp to read, defaults totime.time()condensed(optional)DEPRECATEDUsecondenseinstead. Support for this will be removed entirely in a future release.transform(optional)Optionally process each row of data. Supports[mean, count, min, max, sum], or any callable that accepts datapoints according to the type of series (e.g histograms are dictionaries, counts are integers, etc). Transforms are called afterread_funchas cast the data type and after resolution data is optionally condensed. Iftransformis one of(list,tuple,set), will load the data once and run all the transforms on that data set. Iftransformis adictof the form{ transform_name : transform_func }, will run all of the transform functions on the data set. SeeCustomized Readsfor more on custom transforms.fetch(optional)Function to use instead of the built-in implementations for fetching data. SeeCustomized Reads.process_row(optional)Can be a callable to implementCustomized Reads.condense(optional)If using resolutions,Truewill collapse the resolution data into a single row. Can be a callable to implementCustomized Reads.join_rows(optional)Can be a callable to implementCustomized Reads.Returns a dictionary of{ timestamp : data }, wheretimestampis a Unix timestamp
anddatais a data structure corresponding to the type of series, or whatevertransformreturns.  If not using resolutions orcondense=True, the length
of the dictionary is 1, else it will be the number of resolution buckets within
the interval that contained data. Iftransformis a list,datawill be a
dictionary of{ transform_func : transformed_data }. Iftransformis adict,datawill be a dictionary of{ transform_name : transformed_data }.seriesAlmost identical toget, supports the following parameters. All optional parameters are keyword arguments.nameThe name of the statistic, or a list of names whose data will be joined together.intervalThe named interval to read fromstart(optional)The timestamp which should be in the first interval of the returned data.end(optional)The timestamp which should be in the last interval of the returned data.steps(optional)The number of steps in the interval to read, defaults to eitherstepsin the configuration or 1. Ignored if bothstartandendare defined. If eitherstartorendare defined,stepsis inclusive of whatever interval that timestamp falls into.condensed(optional)DEPRECATEDUsecondenseinstead. Support for this will be removed entirely in a future release.transform(optional)Optionally process each row of data. Supports[mean, count, min, max, sum], or any callable that accepts a list of datapoints according to the type of series (e.g histograms are dictionaries, counts are integers, etc). Transforms are called afterread_funchas cast the data type and after resolution data is optionally condensed. Iftransformis one of(list,tuple,set), will load the data once and run all the transforms on that data set. Iftransformis adictof the form{ transform_name : transform_func }, will run all of the transform functions on the data set. SeeCustomized Readsfor more on custom transforms.fetch(optional)Function to use instead of the built-in implementations for fetching data. SeeCustomized Reads.process_row(optional)Can be a callable to implementCustomized Reads.condense(optional)If using resolutions,Truewill collapse the resolution data into a single row. Can be a callable to implementCustomized Reads.join_rows(optional)Can be a callable to implementCustomized Reads.collapse(optional)Truewill collapse all of the data in the date range into a single result. Can be a callable to implementCustomized Reads.Returns an ordered dictionary of{ interval_timestamp : { resolution_timestamp: data } },
whereinterval_timestampandresolution_timestampare Unix timestamps
anddatais a data structure corresponding to the type of series, or whatevertransformreturns.  If not using resolutions orcondense=True, the dictionary
will be of the form{ interval_timestamp : data }.All variations oftransformand the resulting format ofdataare the same
as inget.If bothstartandendare defined, the returned data will start and end
on intervals including those timestamps. If onlystartis defined, then the
return data will start with an interval that includes that timestamp, with the
total number of intervals returned defined bysteps. If onlyendis
defined, then the return data will end with an interval that includes that
timestamp, with the total number of intervals preceeding it defined bysteps.It is important to note that the interval timestamps in the returned data will
not necessarily matchstartorend. This is because of the consistent
hashing scheme that kairos uses, such thatstartandendwill be
translated into the bucket in which it can be found.iterateAlmost identical togetexcept it does not accept atimestampargument.nameThe name of the statistic, or a list of names whose data will be joined together.intervalThe named interval to read fromtransform(optional)Optionally process each row of data. Supports[mean, count, min, max, sum], or any callable that accepts datapoints according to the type of series (e.g histograms are dictionaries, counts are integers, etc). Transforms are called afterread_funchas cast the data type and after resolution data is optionally condensed. Iftransformis one of(list,tuple,set), will load the data once and run all the transforms on that data set. Iftransformis adictof the form{ transform_name : transform_func }, will run all of the transform functions on the data set. SeeCustomized Readsfor more on custom transforms.fetch(optional)Function to use instead of the built-in implementations for fetching data. SeeCustomized Reads.process_row(optional)Can be a callable to implementCustomized Reads.condense(optional)If using resolutions,Truewill collapse the resolution data into a single row. Can be a callable to implementCustomized Reads.join_rows(optional)Can be a callable to implementCustomized Reads.Returns a generator which iterates over( timestamp, data )tuples, wheretimestampis a Unix timestamp anddatacorresponds to the rules
documented inget. Yields a tuple for each potential timestamp in the
entire date range of the timeseries, even if there is no data.Customized ReadsALPHAThis feature is still being explored and the API may change significantly.There are times when the data in a timeseries requires processing to
be pushed onto the datastore.There are times when one needs custom control over the reading and processing
of data in a timeseries. As there is no good way to do this generically,
thegetandseriesAPI supports several keyword arguments to customize
access to the data. Common use cases are to handle large sets of data that
can be processed in the datastore, and situations where one wants to implement
cutom analysis of the dataset such as calculating variance.GeneralThe following functions can be overloaded with keyword parameters togetandseries(collapsebeing only used for a series).fetchA customized database read function. The usage varies depending on the backends
which are described in detail below.IMPORTANTYou are welcome to change the type of the return value, but be
wary that transforms, condense and collapse functionality may not work
properly with the changed data types.process_rowThe function which handles the type casting of the data read from the backend
and also calling theread_funcif it has been defined for the time series.
It is required that you define this function if you overloadfetchsuch
that the returned data type is not the same as the time series’ native format
(dictfor histogram,listfor series, etc).The function must be in the form ofprocess_row(data), where:dataThe row data generated by the native orfetchimplementation, not
including any time stamps.The function may return any data type, but if it’s not the native format of the
time series, additional downstream functions may have to be overloaded.condenseIf thecondenseargument is a callable, the caller can override how resolution
data is collapsed (reduced) into a single interval. The argument will always be
in the form of:{
  'resolution_t0' : <data_t0>,
  'resolution_t1' : <data_t1>,
  ...
  'resolution_tN' : <data_tN>,
}Where<data_tN>is the data returned from the native orfetchimplementation and passed through the native or customprocess_rowimplementation.The function should return a single value, optionally in the same format as<data_tN>, but this method could also be used for calculating such
things as rate of change or variance within a time interval.join_rowsIf thejoin_rowsargument is a callable and thenameparameter togetorseriesis one of(list,tuple,set), this method will be called to join
the data from several named timeseries into a single result. The argument will
always be in the form of:[
  <data_series0>,
  <data_series1>,
  ...
  <data_seriesN>
]Where<data_series0>will be the data within a single timestamp window in
the series’ native format or whatever was generated by custom implementations
offetch,process_rowand/orcondense. It is important to note
that not every series will contain data points within a given time interval.In addition to reducing multiple time series’ worth of data within an interval
into a single result, this method could be used to implement cross-series
analytics such as unions, intersections and differentials.collapseIf thecollapseargument is a callable, the caller can override how interval
data is collapsed (reduced) into a single result. The native implementation is to
call thecondensefunction implemented by a time series. The arguments are
the same as a customcondensefunction, as-is the expected return value.It’s important to note that ifcollapseis defined, the series will
automatically be condensed as well, so iffetchis overloaded to return a
custom data type, thencondensemust also be defined. IfcollapseisTrue, the customcondensefunction will be used if defined.In addition to collapsing the result of a time series into a single data set,
this method could also be used to calculate data across a time series, such as
variance.transformAs noted previously,transformcan be any callable, list of names or callables,
or a named map of transform names or callables. The transforms will be processed
after all previous native or custom read functions, includingcollapse.Transforms must accept 2 parameters, the data and the time interval in seconds
over which that data was captured. Thedataparameter will be of a type
corresponding to the timeseries type, or whatever was generated by a customfetch. For example:def custom_rate_for_counts(data, duration):
  return float(data) / float(duration)For gregorian timeseries,durationwill seconds in terms of the whole
number of days over which the data was captured, where a day is 86400 seconds.RedisThe function must be in the form offetch(handle, key), where:handleEither a Redis client or pipeline instancekeyThe key for the timeseries dataThe return value should correspond to the data type of timeseries, e.g.dictfor a histogram. One should always assume thathandleis both a pipelineanda client, andfetchshould return the result of, e.g.handle.hlen(...), but that it cannot be used to return a literal, such
aslambda: h,k: { 'foo' : h.hlen(k) }MongoThe function must be in the form offetch(handle, **kwargs), where:handleA PyMongoCollectionspecThe (suggested) query specificationsortThe (suggested) sort definition for the querymethodThe suggested method to use on thehandleThe required return value depends on the value ofmethod.find_oneShould return a hash in the form{ value : <data> }, where<data>should correspond to the data type of the timeseries, e.g.listfor a series. May directly return a result frompymongo.collection.find_one.findShould return an iterable in the form[ { value: <data> }, ... ],
where<data>follows the same rules asfind_one.Re-implementing the default functionality would look like:def mongo_fetch(handle, spec, sort, method):
  if method=='find':
    return handle.find( spec=spec, sort=sort )
  elif method=='find_one':
    return handle.find_one( spec )SQLThe function must be in the formfetch(connection, table, name, interval, i_start, i_end), where:connectionA SQLAlchemyConnectiontableA SQLAlchemyTablenameThe name of the stat to fetchintervalThe interval of the stat to fetchi_startThe interval timestamp (starting) keyi_end(optional) For a series, the ending timestamp keyThe return value should be in the form of{
  'interval_t0' : {
    'resolution_t0t0' : <data_t0t0>,
    'resolution_t0t1' : <data_t0t1>,
    ...
    'resolution_t0tN' : <data_t0tN>
  },
  'interval_t1' : { ... },
  ...
  'interval_tN' : { ... },
}If the series doesn’t use resolutions, thenresolution_tNtNshould beNone, and so each interval will be in the form{ 'interval_tN: { None : <data_tN> } }. This is inherent in the way that
data is stored within the tables.Ifi_endis supplied, the query should be over the rangei_time >= i_start AND i_time <= i_end, else the query should be for
the intervali_time = i_start.CassandraThe function must be in the formfetch(connection, table, name, interval, i_start, i_end), where:cursorAcqlConnectiontableThe name of the tablenameThe name of the stat to fetchintervalThe interval of the stat to fetchintervalsThe list of interval timestampsThe return value should be in the form of{
  'interval_t0' : {
    'resolution_t0t0' : <data_t0t0>,
    'resolution_t0t1' : <data_t0t1>,
    ...
    'resolution_t0tN' : <data_t0tN>
  },
  'interval_t1' : { ... },
  ...
  'interval_tN' : { ... },
}If the series doesn’t use resolutions, thenresolution_tNtNshould beNone, and so each interval will be in the form{ 'interval_tN: { None : <data_tN> } }and can be determined when a row
has anr_timeof-1.Ifintervalsis a list of 1, it’s effectively agetquery wherei_time = intervals[0], else it’si_time >= intervals[0] AND i_time <=intervals[-1]. The full list of intervals is supplied to workaround
Cassandra’s lack of grouping support in situations where an aggregate peri_timeis desired.Deleting DatadeleteTakes a single argument, the name of the timeseries. Will delete all data for
that timeseries in all intervals.delete_allDeletes every timeseries for all intervals. This method may be fast in data
stores that support optimized deletes, else it will have to delete for each
timeseries returned inlist.expireTakes a single argument, the name of the timeseries. For storage engines that
do not support expiry, such as SQL, will delete expired data from intervals
for whichstepsis defined. All other storage engines will raise theNotImplementedErrorexception.Dragons!Kairos achieves its efficiency by using TTLs and data structures
in combination with a key naming scheme that generates consistent keys based on
any timestamp relative to epoch. However, just likeRRDtool, changing any attribute of the
timeseries means that new data will be stored differently than old data. For
this reason it’s best to completely delete all data in an old time series
before creating or querying using a new configuration.If you want to migrate data, there are tools intorusthat can help.InstallationKairos is available onpypiand can be
installed usingpippip install kairosIf installing from source:with development requirements (e.g. testing frameworks)pip install -r development.pipwithout development requirementspip install -r requirements.pipNote that kairos does not install packages for any of the supported backends,
and that you must do this yourself.TestsUsenoseto run the test suite.$ nosetestsThe test suite can be controlled through several environment variables, all
defaulting totrue.TEST_REDIStrueTEST_MONGOtrueTEST_SQLtrueTEST_CASSANDRAtrueTEST_SERIEStrueTEST_HISTOGRAMtrueTEST_COUNTtrueTEST_GAUGEtrueTEST_SETtrueSQL_HOSTsqlite:///:memory:CASSANDRA_KEYSPACEkairosRoadmapRound-robbin intervals for datastores without TTLsRound-robbin databases: memcache (and compatible, e.g. ElastiCache), Riak,
DynamoDB, SimpleDB, GDBM, Berkeley DB, and moreRedis optimizationsCapped collection support for mongoPython 3 supportInfluxDB supportBloom filtersJoined series populate a data structure at query timeJoined series support concurrency “runner”LicenseThis software is licensed under theNew BSD License. See theLICENSE.txtfile in the top distribution directory for the full license text.