Package: wotan
Wōtan......offers free and open source algorithms to automagically remove trends from time-series data.In Germanic mythology, Odin (/ˈoːðinː/ Old High German: Wōtan) is a widely revered god. He gave one of his eyes to Mimir in return for wisdom. Thus, in order to achieve a goal, one sometimes has to turn a blind eye. In Richard Wagner's "Der Ring des Nibelungen", Wotan is the King of the Gods (god of light, air, and wind) and a bass-baritone. According to Wagner, he is the "pinnacle of intelligence".Example usagefrom wotan import flatten
flatten_lc, trend_lc = flatten(time, flux, window_length=0.5, method='biweight', return_trend=True)For more details, have a look at theinteractive playground, thedocumentation.
We also have examples and tutorials available, such as the📑Example: Basic wotan functionalityAvailable detrending algorithmsTime-windowed sliders with location estimates:(📑Example: Comparison of sliders)biweightRobustM-estimatorusingTukey's biweight(📑Example)huberRobustM-estimatorfromHuber (1981)(iterative)huber_psiRobustM-estimatorbased onHuber's ψ(one-step)hampelRobustM-estimatorbased onHampel (1972), 3-part descending, known as (a,b,c), 17A, 25AandrewsinewaveRobustM-estimatorusingAndrew's sine wavewelschRobustM-estimatorfromWelsch-LeclercramsayRobust M-estimator fromRamsay (1977), known as Ramsay's EatauRobust τ estimator fromYohai & Zamar (1986)hodgesRank-based robust R-estimatorHodges-Lehmann-SenmedianThe most robust (but least efficient)medfiltA cadence-based median filter (nottime-windowed) for comparisonmeanThe least robust (but most efficient for white noise)trim_meanTrimmed mean (outliers are removed)winsorizeTrimmed mean (outliers arewinsorizedto a specified percentile)hampelfiltTrimmed mean (outliers arereplacedwith the median)Splines:(📑Example)rsplineSpline with iterative sigma-clippinghsplineSpline with a robust Huber estimator (Huber 1981)psplinePenalized spline to automatically select the knot distance(Eilers 1996), with iterative sigma-clippingPolynomials and sines:(📑Example)cofiamCosine Filtering with Autocorrelation Minimization (Kipping et al. 2013)cosineSum of sines and cosines, with option for iterative sigma-clippingsavgolSliding segments are fit with polynomials (Savitzky & Golay 1964), cadence-basedRegressions:(📑Example)lowessLocally weighted scatterplot smoothing (Cleveland 1979)supersmootherFriedman's (1984)Super-Smoother, a local linear regression with adaptive bandwidthFitting a model that is a sum of Gaussian bases:(📑Example)ridgeRidge regression (L2 loss,Tikhonov regularization)lassoLASSO regression
(L1 loss,Least Absolute Shrinkage Selector Operator,Tibshirani (1996))elasticnetLinear regression modelwith 50% L1 and 50% L2 norm regularizationgpGaussian Processes offering:(📑Example: GP Standard vs. robust)squared_expSquared-exponential kernel, with option for iterative sigma-clippingmaternMatern 3/2 kernel, with option for iterative sigma-clippingperiodicPeriodic kernel informed by a user-specified period(📑Example)periodic_autoPeriodic kernel informed by a Lomb-Scargle periodogram pre-searchAvailable featureswindow_lengthThe length of the filter window in units oftime(usually days).break_toleranceIf there are large gaps in time, especially with corresponding flux level offsets, the detrending is much improved when splitting the data into several sub-lightcurves and applying the filter to each individually. Comes with an empirical default and is fully adjustable.edge_cutoffTrends near edges are less robust. Depending on the data, it may be beneficial to remove edges.cvalTuning parameter for the robust estimators (seedocumentation)return_trendIfTrue, the method will return a tuple of two elements (flattened_flux,trend_flux) wheretrend_fluxis the removed trend. Otherwise, it will only returnflattened_flux.transit_maskMask known transits during detrending(📑Example)What method to choose?It depends on your data and what you like to achieve (relevant xkcd). If possible, try it out! Use wotan with a selection of methods, iterate over their parameter space, and choose what gives the best results for your research.If that is too much effort, you should first examine your data.Is it mostly white (Gaussian) noise? Use a time-windowed sliding mean. This is the most efficient method for white noise.With prominent outliers (such as transits or flares), use a robust time-windowed method such as thebiweight. This is usually superior to themedianor trimmed methods.Are there (semi-) periodic trends? In addition to a time-windowed biweight, try a spline-based method. Experimenting with periodic GPs is worthwhile.InstallationTo install the released version, type$ pip install wotanwhich automatically installsnumpy,numbaandscipyif not present. Depending on the algorithm, additional dependencies exist:huber,ramsay, andhampeldepend onstatsmodelshsplineandgpdepend onsklearnpsplinedepends onpygamsupersmootherdepends onsupersmootherTo install all additional dependencies, type$ pip install statsmodels sklearn supersmoother pygam.OriginalityAs all scientific work, wōtan isstanding on the shoulders of giants. Particularly, many detrending methods are wrapped from existing packages. Original contributions include:A time-windowed detrending master module with edge treatments and segmentation optionsRobust location estimates using Newton-Raphson iteration to a precision threshold for Tukey's biweight, Andrew's sine wave, and the Welsch-Leclerc. This is probably a "first", which reduces jitter in the location estimate by ~10 ppmRobustified (iterative sigma-clipping) penalized splines for automatic knot distance determination and outlier resistanceRobustified (iterative sigma-clipping) Gaussian processesGP with a periodic kernel informed by a Lomb-Scargle periodogram pre-searchBringing together many methods in one place in a common interface, with sensible defaultsProviding documentation, tutorials, and apaperwhich compares and benchmarks the methodsAttributionPlease citeHippke et al. (2019, AJ, 158, 143)if you find this code useful in your research. The BibTeX entry for the paper is:@ARTICLE{2019AJ....158..143H,
       author = {{Hippke}, Michael and {David}, Trevor J. and {Mulders}, Gijs D. and
         {Heller}, Ren{\'e}},
        title = "{W{\={o}}tan: Comprehensive Time-series Detrending in Python}",
      journal = {\aj},
     keywords = {eclipses, methods: data analysis, methods: statistical, planetary systems, planets and satellites: detection, Astrophysics - Earth and Planetary Astrophysics, Astrophysics - Instrumentation and Methods for Astrophysics},
         year = "2019",
        month = "Oct",
       volume = {158},
       number = {4},
          eid = {143},
        pages = {143},
          doi = {10.3847/1538-3881/ab3984},
archivePrefix = {arXiv},
       eprint = {1906.00966},
 primaryClass = {astro-ph.EP},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2019AJ....158..143H},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}