Time series functions
Time series functions Time series functions are aggregate functions that operate on sequences of data values measured at points in time. The following sections describe some of the time series functions available in different time series packages. Transforms Transforms are functions that are applied on a time series resulting in another time series. The time series library supports various types of transforms, including provided transforms (by using from tspy.functions import transformers) as well as user defined transforms. The following sample shows some provided transforms: Interpolation >>> ts = tspy.time_series([1.0, 2.0, 3.0, 4.0, 5.0, 6.0]) >>> periodicity = 2 >>> interp = interpolators.nearest(0.0) >>> interp_ts = ts.resample(periodicity, interp) >>> interp_ts.print() TimeStamp: 0 Value: 1.0 TimeStamp: 2 Value: 3.0 TimeStamp: 4 Value: 5.0 Fillna >>> shift_ts = ts.shift(2) print("shifted ts to add nulls") print(shift_ts) print("nfilled ts to make nulls 0s") null_filled_ts = shift_ts.fillna(interpolators.fill(0.0)) print(null_filled_ts) shifted ts to add nulls TimeStamp: 0 Value: null TimeStamp: 1 Value: null TimeStamp: 2 Value: 1.0 TimeStamp: 3 Value: 2.0 TimeStamp: 4 Value: 3.0 TimeStamp: 5 Value: 4.0 filled ts to make nulls 0s TimeStamp: 0 Value: 0.0 TimeStamp: 1 Value: 0.0 TimeStamp: 2 Value: 1.0 TimeStamp: 3 Value: 2.0 TimeStamp: 4 Value: 3.0 TimeStamp: 5 Value: 4.0 Additive White Gaussian Noise (AWGN) >>> noise_ts = ts.transform(transformers.awgn(mean=0.0,sd=.03)) >>> print(noise_ts) TimeStamp: 0 Value: 0.9962378841388397 TimeStamp: 1 Value: 1.9681980879378596 TimeStamp: 2 Value: 3.0289374962174405 TimeStamp: 3 Value: 3.990728648807705 TimeStamp: 4 Value: 4.935338359740761 TimeStamp: 5 Value: 6.03395072999318 Segmentation Segmentation or windowing is the process of splitting a time series into multiple segments. The time series library supports various forms of segmentation and allows creating user-defined segments as well. * Window based segmentation This type of segmentation of a time series is based on user specified segment sizes. The segments can be record based or time based. There are options that allow for creating tumbling as well as sliding window based segments. >>> import tspy >>> ts_orig = tspy.builder() .add(tspy.observation(1,1.0)) .add(tspy.observation(2,2.0)) .add(tspy.observation(6,6.0)) .result().to_time_series() >>> ts_orig timestamp: 1 Value: 1.0 timestamp: 2 Value: 2.0 timestamp: 6 Value: 6.0 >>> ts = ts_orig.segment_by_time(3,1) >>> ts timestamp: 1 Value: original bounds: (1,3) actual bounds: (1,2) observations: [(1,1.0),(2,2.0)] timestamp: 2 Value: original bounds: (2,4) actual bounds: (2,2) observations: [(2,2.0)] timestamp: 3 Value: this segment is empty timestamp: 4 Value: original bounds: (4,6) actual bounds: (6,6) observations: [(6,6.0)] * Anchor based segmentation Anchor based segmentation is a very important type of segmentation that creates a segment by anchoring on a specific lambda, which can be a simple value. An example is looking at events that preceded a 500 error or examining values after observing an anomaly. Variants of anchor based segmentation include providing a range with multiple markers. >>> import tspy >>> ts_orig = tspy.time_series([1.0, 2.0, 3.0, 4.0, 5.0]) >>> ts_orig timestamp: 0 Value: 1.0 timestamp: 1 Value: 2.0 timestamp: 2 Value: 3.0 timestamp: 3 Value: 4.0 timestamp: 4 Value: 5.0 >>> ts = ts_orig.segment_by_anchor(lambda x: x % 2 == 0, 1, 2) >>> ts timestamp: 1 Value: original bounds: (0,3) actual bounds: (0,3) observations: [(0,1.0),(1,2.0),(2,3.0),(3,4.0)] timestamp: 3 Value: original bounds: (2,5) actual bounds: (2,4) observations: [(2,3.0),(3,4.0),(4,5.0)] * Segmenters There are several specialized segmenters provided out of the box by importing the segmenters package (using from tspy.functions import segmenters). An example segmenter is one that uses regression to segment a time series: >>> ts = tspy.time_series([1.0,2.0,3.0,4.0,5.0,2.0,1.0,-1.0,50.0,53.0,56.0]) >>> max_error = .5 >>> skip = 1 >>> reg_sts = ts.to_segments(segmenters.regression(max_error,skip,use_relative=True)) >>> reg_sts timestamp: 0 Value: range: (0, 4) outliers: {} timestamp: 5 Value: range: (5, 7) outliers: {} timestamp: 8 Value: range: (8, 10) outliers: {} Reducers A reducer is a function that is applied to the values across a set of time series to produce a single value. The time series reducer functions are similar to the reducer concept used by Hadoop/Spark. This single value can be a collection, but more generally is a single object. An example of a reducer function is averaging the values in a time series. Several reducer functions are supported, including: * Distance reducers Distance reducers are a class of reducers that compute the distance between two time series. The library supports numeric as well as categorical distance functions on sequences. These include time warping distance measurements such as Itakura Parallelogram, Sakoe-Chiba Band, DTW non-constrained and DTW non-time warped contraints. Distribution distances such as Hungarian distance and Earth-Movers distance are also available. For categorical time series distance measurements, you can use Damerau Levenshtein and Jaro-Winkler distance measures. >>> from tspy.functions import >>> ts = tspy.time_series([1.0, 2.0, 3.0, 4.0, 5.0, 6.0]) >>> ts2 = ts.transform(transformers.awgn(sd=.3)) >>> dtw_distance = ts.reduce(ts2,reducers.dtw(lambda obs1, obs2: abs(obs1.value - obs2.value))) >>> print(dtw_distance) 1.8557981638880405 * Math reducers Several convenient math reducers for numeric time series are provided. These include basic ones such as average, sum, standard deviation, and moments. Entropy, kurtosis, FFT and variants of it, various correlations, and histogram are