Package: tsdownsample
tsdownsampleExtremely fasttime series downsampling 📈for visualization, written in Rust.Features ✨Fast: written in rust with PyO3 bindingsleverages optimizedargminmax- which is SIMD accelerated with runtime feature detectionscales linearly with the number of data pointsmultithreaded with Rayon (in Rust)Why we do not use Python multiprocessingCiting thePyO3 docs on parallelism:CPython has the infamous Global Interpreter Lock, which prevents several threads from executing Python bytecode in parallel. This makes threading in Python a bad fit for CPU-bound tasks and often forces developers to accept the overhead of multiprocessing.In Rust - which is a compiled language - there is no GIL, so CPU-bound tasks can be parallelized (withRayon) with little to no overhead.Efficient: memory efficientworks on views of the data (no copies)no intermediate data structures are createdFlexible: works on any type of datasupported datatypes areforx:f32,f64,i16,i32,i64,u16,u32,u64,datetime64,timedelta64fory:f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64,datetime64,timedelta64,bool!! 🚀f16argminmaxis 200-300x faster than numpyIn contrast with all other data types above,f16is *not* hardware supported (i.e., no instructions for f16) by most modern CPUs!!🐌 Programming languages facilitate support for this datatype by either (i) upcasting to f32 or (ii) using a software implementation.💡 As for argminmax, only comparisons are needed - and thus no arithmetic operations - creating a symmetrical ordinal mapping fromf16toi16is sufficient. This mapping allows to use the hardware supported scalar and SIMDi16instructions - while not producing any memory overhead 🎉More details are described inargminmax PR #1.Easy to use: simple & flexible APIInstallpipinstalltsdownsampleUsagefromtsdownsampleimportMinMaxLTTBDownsamplerimportnumpyasnp# Create a time seriesy=np.random.randn(10_000_000)x=np.arange(len(y))# Downsample to 1000 points (assuming constant sampling rate)s_ds=MinMaxLTTBDownsampler().downsample(y,n_out=1000)# Select downsampled datadownsampled_y=y[s_ds]# Downsample to 1000 points using the (possible irregularly spaced) x-datas_ds=MinMaxLTTBDownsampler().downsample(x,y,n_out=1000)# Select downsampled datadownsampled_x=x[s_ds]downsampled_y=y[s_ds]Downsampling algorithms & APIDownsampling API 📑Each downsampling algorithm is implemented as a class that implements adownsamplemethod.
The signature of thedownsamplemethod:downsample([x], y, n_out, **kwargs) -> ndarray[uint64]Arguments:xis optionalxandyare both positional argumentsn_outis a mandatory keyword argument that defines the number of output values***kwargsare optional keyword arguments(seetable below):parallel: whether to use multi-threading (default:False)❗ The max number of threads can be configured with theTSDOWNSAMPLE_MAX_THREADSENV var (e.g.os.environ["TSDOWNSAMPLE_MAX_THREADS"] = "4")...Returns: andarray[uint64]of indices that can be used to index the original data.*When there are gaps in the time series, fewer thann_outindices may be returned.Downsampling algorithms 📈The following downsampling algorithms (classes) are implemented:DownsamplerDescription**kwargsMinMaxDownsamplerselects themin and maxvalue in each binparallelM4Downsamplerselects themin, max, first and lastvalue in each binparallelLTTBDownsamplerperforms theLargest Triangle Three BucketsalgorithmparallelMinMaxLTTBDownsampler(new two-step algorithm 🎉) first selectsn_out*minmax_ratiomin and maxvalues, then further reduces these ton_outvalues using theLargest Triangle Three Bucketsalgorithmparallel,minmax_ratio**Default value forminmax_ratiois 4, which is empirically proven to be a good default. More details here:https://arxiv.org/abs/2305.00332Handling NaNsThis library supports twoNaN-policies:OmitNaNs (NaNs are ignored during downsampling).Return index of firstNaNonce there is at least one present in the bin of the considered data.OmitNaNsReturnNaNsMinMaxDownsamplerNaNMinMaxDownsamplerM4DownsamplerNaNM4DownsamplerMinMaxLTTBDownsamplerNaNMinMaxLTTBDownsamplerLTTBDownsamplerNote that NaNs are not supported forx-data.Limitations & assumptions 🚨Assumes;x-data is (non-strictly) monotonic increasing (i.e., sorted)noNaNs inx-data👤Jeroen Van Der Donckt