Package: light-curve
light-curveprocessing toolbox for PythonThe Python wrapper for Rustlight-curve-featureandlight-curve-dmdtpackages which gives a collection of high-performant time-series feature extractors.Installationpython3-mpipinstall'light-curve[full]'fullextras would install the package with all optional Python dependencies required by experimental features.
We also providelight-curve-pythonpackage which is just an "alias" to the mainlight-curve[full]package.Minimum supported Python version is 3.7.
We provide binary wheels viaPyPifor a number of platforms and architectures, both for CPython and PyPy.
We also provide binary wheels for stable CPython ABI, so the package is guaranteed to work with all future CPython3 versions.Support matrixArch \ OSLinux glibcLinux muslmacOSWindowshttps://github.com/light-curve/light-curve-python/issues/186x86-64wheel (MKL)wheel (MKL)wheelwheel (no Ceres, no GSL)i686srcsrc—not testedaarch64wheelwheelsrchttps://github.com/light-curve/light-curve-python/issues/5not testedppc64lewheelnot tested (no Rust toolchain)——"wheel": binary wheel is available on pypi.org, local building is not required for the platform, the only pre-requirement is a recentpipversion. For Linux x86-64 we provide binary wheels built with Intel MKL for better periodogram performance, which is not a default build option. For Windows x86-64 we provide wheel with no Ceres and no GSL support, which is not a default build option."src": the package is confirmed to be built and pass unit tests locally, but testing and package building is not supported by CI. It is required to have theGNU scientific library (GSL)v2.1+ and theRust toolchainv1.67+ to install it viapip install.ceres-solverandfftwmay be installed locally or built from source, in the later case you would also need C/C++ compiler andcmake."not tested": building from the source code is not tested, please report us building status via issue/PR/email.We build aarch64 macOS 12.0+ Python 3.8+ wheels locally and submit them running this command inlight-curvedirectory:rm -rf ./wheelhouse
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain stable -y
brew install gsl
CIBW_BUILD='cp3*-macosx_arm64' CIBW_ENVIRONMENT="MATURIN_PEP517_ARGS='--locked --no-default-features --features ceres-source,fftw-source,gsl'" CIBW_BEFORE_ALL='' python3 -mcibuildwheel --platform macos
CIBW_BUILD='cp310-macosx_arm64' CIBW_ENVIRONMENT="MATURIN_PEP517_ARGS='--locked --no-default-features --features ceres-source,fftw-source,gsl,abi3'" CIBW_BEFORE_ALL='' python3 -mcibuildwheel --platform macos
twine upload wheelhouse/*.whlFeature evaluatorsMost of the classes implement various feature evaluators useful for light-curve based
astrophysical source classification and characterisation.importlight_curveaslcimportnumpyasnp# Time values can be non-evenly separated but must be an ascending arrayn=101t=np.linspace(0.0,1.0,n)perfect_m=1e3*t+1e2err=np.sqrt(perfect_m)m=perfect_m+np.random.normal(0,err)# Half-amplitude of magnitudeamplitude=lc.Amplitude()# Fraction of points beyond standard deviations from meanbeyond_std=lc.BeyondNStd(nstd=1)# Slope, its error and reduced chi^2 of linear fitlinear_fit=lc.LinearFit()# Feature extractor, it will evaluate all features in more efficient wayextractor=lc.Extractor(amplitude,beyond_std,linear_fit)# Array with all 5 extracted featuresresult=extractor(t,m,err,sorted=True,check=False)print('\n'.join(f"{name}={value:.2f}"forname,valueinzip(extractor.names,result)))# Run in parallel for multiple light curves:results=amplitude.many([(t[:i],m[:i],err[:i])foriinrange(n//2,n)],n_jobs=-1,sorted=True,check=False,)print("Amplitude of amplitude is{:.2f}".format(np.ptp(results)))If you're confident in your inputs you could usesorted = True(tis in ascending order)
andcheck = False(no NaNs in inputs, no infs intorm) for better performance.
Note that if your inputs are not valid and are not validated bysorted=Noneandcheck=True(default values) then all kind of bad things could happen.Print feature classes listimportlight_curveaslcprint([xforxindir(lc)ifhasattr(getattr(lc,x),"names")])Read feature docsimportlight_curveaslchelp(lc.BazinFit)Available featuresSee the complete list of available feature evaluators and documentation inlight-curve-featureRust crate docs.Feature nameDescriptionMin data pointsFeatures numberAmplitudeHalf amplitude of magnitude:$\displaystyle \frac{\max (m)-\min (m)}{2}$11AndersonDarlingNormalUnbiased Anderson–Darling normality test statistic:$\displaystyle \left( 1+\frac{4}{N} -\frac{25}{N^{2}}\right) \times$$\times \left( -N-\frac{1}{N}\sum\limits_{i=0}^{N-1} (2i+1)\ln \Phi _{i} +(2(N-i)-1)\ln (1-\Phi _{i} )\right) ,$where $\Phi _{i\ } \equiv \Phi (( m_{i} \ -\ \langle m\rangle ) /\sigma _{m})$ is the commutative distribution function of the standard normal distribution, $N-$ the number of observations, $\langle m\rangle -$ mean magnitude and $\sigma _{m} =\sqrt{\sum\limits_{i=0}^{N-1}( m_{i} -\langle m\rangle )^{2} /( N-1) \ }$ is the magnitude standard deviation41BazinFitFive fit parameters and goodness of fit (reduced $\chi ^{2}$ of the Bazin function developed for core-collapsed supernovae:$\displaystyle f(t)=A\frac{\mathrm{e}^{-(t-t_{0} )/\tau _{fall}}}{1+\mathrm{e}^{-(t-t_{0} )/\tau _{rise}}} +B,$where $f(t)-$ flux observation61BeyondNStdFraction of observations beyond $n\sigma _{m}$ from the mean magnitude $\langle m\rangle $:$\displaystyle \frac{\sum _{i} I_{|m-\langle m\rangle | >n\sigma _{m}} (m_{i} )}{N},$where $I-$ an indicator function21CusumA range of cumulative sums:$\displaystyle \max(S) -\min(S),$where $S_{j} \equiv \frac{1}{N\sigma _{m}}\sum\limits _{i=0}^{j} (m_{i} -\langle m\rangle )$, $j\in \{1..N-1\}$21EtaVon Neummann $\eta $:$\displaystyle \eta \equiv \frac{1}{(N-1)\sigma _{m}^{2}}\sum\limits _{i=0}^{N-2} (m_{i+1} -m_{i} )^{2}$21EtaEModernisation ofEtafor unevenly time series:$\displaystyle \eta ^{e} \equiv \frac{(t_{N-1} -t_{0} )^{2}}{(N-1)^{3}}\frac{\sum\limits_{i=0}^{N-2}\left(\frac{m_{i+1} -m_{i}}{t_{i+1} -t_{i}}\right)^{2}}{\sigma _{m}^{2}}$21ExcessVarianceMeasure of the variability amplitude:$\displaystyle \frac{\sigma _{m}^{2} -\langle \delta ^{2} \rangle }{\langle m\rangle ^{2}},$where $\langle \delta ^{2} \rangle -$ mean squared error21InterPercentileRange$\displaystyle Q(1-p)-Q(p),$where $Q(n)$ and $Q(d)-$ $n$-th and $d$-th quantile of magnitude sample11KurtosisExcess kurtosis of magnitude:$\displaystyle \frac{N(N+1)}{(N-1)(N-2)(N-3)}\frac{\sum _{i} (m_{i} -\langle m\rangle )^{4}}{\sigma _{m}^{2}} -3\frac{(N+1)^{2}}{(N-2)(N-3)}$41LinearFitThe slope, its error and reduced $\chi ^{2}$ of the light curve in the linear fit of a magnitude light curve with respect to the observation error $\{\delta _{i}\}$:$\displaystyle m_{i} \ =\ c\ +\ \text{slope} \ t_{i} \ +\ \delta _{i} \varepsilon _{i} ,$where $c$ is a constant, $\{\varepsilon _{i}\}$ are standard distributed random variables33LinearTrendThe slope and its error of the light curve in the linear fit of a magnitude light curve without respect to the observation error $\{\delta _{i}\}$:$\displaystyle m_{i} \ =\ c\ +\ \text{slope} \ t_{i} \ +\ \Sigma \varepsilon _{i} ,$where $c$ and $\Sigma$ are constants, $\{\varepsilon _{i}\}$  are standard distributed random variables.22MagnitudePercentageRatioMagnitude percentage ratio:$\displaystyle \frac{Q(1-n)-Q(n)}{Q(1-d)-Q(d)}$11MaximumSlopeMaximum slope between two sub-sequential observations:$\displaystyle \max_{i=0\dotsc N-2}\left| \frac{m_{i+1} -m_{i}}{t_{i+1} -t_{i}}\right|$21MeanMean magnitude:$\displaystyle \langle m\rangle =\frac{1}{N}\sum\limits _{i} m_{i}$11MeanVarianceStandard deviation to mean ratio:$\displaystyle \frac{\sigma _{m}}{\langle m\rangle }$21MedianMedian magnitude11MedianAbsoluteDeviationMedian of the absolute value of the difference between magnitude and its median:$\displaystyle \mathrm{Median} (|m_{i} -\mathrm{Median} (m)|)$11MedianBufferRangePercentage$\displaystyle \mathrm{Median} (m)\pm q\times (\max (m)-\min (m))/2$11OtsuSplitDifference of subset means, standard deviation of the lower subset, standard deviation of the upper
subset and lower-to-all observation count ratio for two subsets of magnitudes obtained by Otsu's method split.Otsu's method is used to perform automatic thresholding. The algorithm returns a single threshold that separate values into two classes. This threshold is determined by minimizing intra-class intensity variance $\sigma^2_{W}=w_0\sigma^2_0+w_1\sigma^2_1$, or equivalently, by maximizing inter-class variance $\sigma^2_{B}=w_0 w_1 (\mu_1-\mu_0)^2$. There can be more than one extremum. In this case, the algorithm returns the minimum threshold.24PercentAmplitudeMaximum deviation of magnitude from its median:$\displaystyle \max_{i} |m_{i} \ -\ \text{Median}( m) |$11PercentDifferenceMagnitudePercentileRatio of $p$-th inter-percentile range to the median:$\displaystyle \frac{Q( 1-p) -Q( p)}{\text{Median}( m)}$11RainbowFitSeven fit parameters and goodness of fit (reduced $\chi ^{2}$). The Rainbow method is developed and detailed here : https://arxiv.org/abs/2310.02916). This implementation is suited for transient objects. It is based on a Bazin bolometric flux and a sigmoid temperature evolution:$\displaystyle F_{\nu}(t, \nu) = \frac{\pi\,B\left(T(t),\nu\right)}{\sigma_\mathrm{SB}\,T(t)^{4}} \times F_\mathrm{bol}(t),$where $F_{\nu}(t, \nu)-$ flux observation at a given wavelength61RainbowRisingFitSix fit parameters and goodness of fit (reduced $\chi ^{2}$). The Rainbow method is developed and detailed here : https://arxiv.org/abs/2310.02916). This implementation is suited for rising transient objects. It is based on a rising sigmoid bolometric flux and a sigmoid temperature evolution:$\displaystyle F_{\nu}(t, \nu) = \frac{\pi\,B\left(T(t),\nu\right)}{\sigma_\mathrm{SB}\,T(t)^{4}} \times F_\mathrm{bol}(t),$where $F_{\nu}(t, \nu)-$ flux observation at a given wavelength61ReducedChi2Reduced $\chi ^{2}$ of magnitude measurements:$\displaystyle \frac{1}{N-1}\sum _{i}\left(\frac{m_{i} -\overline{m}}{\delta _{i}}\right)^{2} ,$where $\overline{m} -$ weighted mean magnitude21SkewSkewness of magnitude:$\displaystyle \frac{N}{(N-1)(N-2)}\frac{\sum _{i} (m_{i} -\langle m\rangle )^{3}}{\sigma _{m}^{3}}$31StandardDeviationStandard deviation of magnitude:$\displaystyle \sigma _{m} \equiv \sqrt{\sum _{i} (m_{i} -\langle m\rangle )^{2} /(N-1)}$21StetsonKStetson Kcoefficient described light curve shape:$\displaystyle \frac{\sum _{i}\left| \frac{m_{i} -\langle m\rangle }{\delta _{i}}\right| }{\sqrt{N\ \chi ^{2}}}$21VillarFitSeven fit parameters and goodness of fit (reduced $\chi ^{2}$) of the Villar function developed for supernovae classification:$f(t)=c+\frac{A}{1+\exp\frac{-(t-t_{0} )}{\tau _{rise}}} \times f_{fall}(t),$$f_{fall}(t) = 1-\frac{\nu (t-t_{0} )}{\gamma }, ~~~ t< t_{0} +\gamma,$$f_{fall}(t) = (1-\nu )\exp\frac{-(t-t_{0} -\gamma )}{\tau _{fall}}, ~~~ t \geq t_{0} + \gamma.$where $f(t) -$ flux observation, $A, \gamma , \tau _{rise} , \tau _{fall}  >0$, $\nu \in [0;1)$Here we introduce a new dimensionless parameter $\nu$ instead of the plateau slope $\beta$ from the original paper: $\nu \equiv -\beta \gamma /A$88WeightedMeanWeighted mean magnitude:$\displaystyle \overline{m} \equiv \frac{\sum _{i} m_{i} /\delta _{i}^{2}}{\sum _{i} 1/\delta _{i}^{2}}$11Meta-featuresMeta-features can accept other feature extractors and apply them to pre-processed data.PeriodogramThis feature transforms time-series data into the Lomb-Scargle periodogram, providing an estimation of the power spectrum. The peaks argument corresponds to the number of the most significant spectral density peaks to return. For each peak, its period and "signal-to-noise" ratio are returned.$$
\text{signal to noise of peak} \equiv \frac{P(\omega_\mathrm{peak}) - \langle P(\omega) \rangle}{\sigma_{P(\omega)}}
$$The optional features argument accepts a list of additional feature evaluators, which are applied to the power spectrum: frequency is passed as "time," power spectrum is passed as "magnitude," and no uncertainties are set.BinsBinning time series to bins with width $\mathrm{window}$ with respect to some $\mathrm{offset}$.
$j-th$ bin boundaries are $[j \cdot \mathrm{window} + \mathrm{offset}; (j + 1) \cdot \mathrm{window} + \mathrm{offset}]$.Binned time series is defined by
$$t_j^* = (j + \frac12) \cdot \mathrm{window} + \mathrm{offset},$$
$$m_j^* = \frac{\sum{m_i / \delta_i^2}}{\sum{\delta_i^{-2}}},$$
$$\delta_j^* = \frac{N_j}{\sum{\delta_i^{-2}}},$$
where $N_j$ is a number of sampling observations and all sums are over observations inside considering bin.Multi-band featuresAs of v0.8, experimental extractors (see below), support multi-band light-curve inputs.importnumpyasnpfromlight_curve.light_curve_pyimportLinearFitt=np.arange(20,dtype=float)m=np.arange(20,dtype=float)sigma=np.full_like(t,0.1)bands=np.array(["g"]*10+["r"]*10)feature=LinearFit(bands=["g","r"])values=feature(t,m,sigma,bands)print(values)Rainbow FitRainbow (Russeil+23) is a black-body parametric model for transient light curves.
It uses Bazin function as a model for bolometric flux evolution and a logistic function for the temperature evolutiion.
This example demonstrates the reconstruction of a syntetic light curve with this model.RainbowFitrequiresiminuitpackage and Python 3.8+.importnumpyasnpfromlight_curve.light_curve_pyimportRainbowFitdefbb_nu(wave_aa,T):"""Black-body spectral model"""nu=3e10/(wave_aa*1e-8)return2*6.626e-27*nu**3/3e10**2/np.expm1(6.626e-27*nu/(1.38e-16*T))# Effective wavelengths in Angstromband_wave_aa={"g":4770.0,"r":6231.0,"i":7625.0,"z":9134.0}# Parameter valuesreference_time=60000.0# time close to the peak time# Bolometric flux model parametersamplitude=1.0# bolometric flux semiamplitude, arbitrary (non-spectral) flux/luminosity unitsrise_time=5.0# exponential growth timescale, daysfall_time=30.0# exponential decay timescale, days# Temperature model parametersTmin=5e3# temperature on +infinite time, kelvinsdelta_T=10e3# (Tmin + delta_T) is temperature on -infinite time, kelvinsk_sig=4.0# temperature evolution timescale, daysrng=np.random.default_rng(0)t=np.sort(rng.uniform(reference_time-3*rise_time,reference_time+3*fall_time,1000))band=rng.choice(list(band_wave_aa),size=len(t))waves=np.array([band_wave_aa[b]forbinband])# Temperature evolution is a sigmoid functiontemp=Tmin+delta_T/(1.0+np.exp((t-reference_time)/k_sig))# Bolometric flux evolution is the Bazin functionlum=amplitude*np.exp(-(t-reference_time)/fall_time)/(1.0+np.exp(-(t-reference_time)/rise_time))# Spectral flux density for each given pair of time and passbandflux=np.pi*bb_nu(waves,temp)/(5.67e-5*temp**4)*lum# S/N = 5 for minimum flux, scale for Poisson noiseflux_err=np.sqrt(flux*np.min(flux)/5.0)flux+=rng.normal(0.0,flux_err)feature=RainbowFit.from_angstrom(band_wave_aa,with_baseline=False)values=feature(t,flux,sigma=flux_err,band=band)print(dict(zip(feature.names,values)))print(f"Goodness of fit:{values[-1]}")Note, that while we don't use precise physical constant values to generate the data,RainbowFituses CODATA 2018 values.Experimental extractorsFrom the technical point of view the package consists of two parts: a wrapper forlight-curve-featureRust crate(light_curve_extsub-package) and pure Python sub-packagelight_curve_py.
We use the Python implementation of feature extractors to test Rust implementation and to implement new experimental extractors.
Please note, that the Python implementation is much slower for most of the extractors and doesn't provide the same functionality as the Rust implementation.
However, the Python implementation provides some new feature extractors you can find useful.You can manually use extractors from both implementations:importnumpyasnpfromnumpy.testingimportassert_allclosefromlight_curve.light_curve_extimportLinearTrendasRustLinearTrendfromlight_curve.light_curve_pyimportLinearTrendasPythonLinearTrendrust_fe=RustLinearTrend()py_fe=PythonLinearTrend()n=100t=np.sort(np.random.normal(size=n))m=3.14*t-2.16+np.random.normal(size=n)assert_allclose(rust_fe(t,m),py_fe(t,m),err_msg="Python and Rust implementations must provide the same result")This should print a warning about experimental status of the Python classBenchmarksYou can run all benchmarks from the Python project folder withpython3 -mpytest --benchmark-enable tests/test_w_bench.py, or with slow benchmarks disabledpython3 -mpytest -m "not (nobs or multi)" --benchmark-enable tests/test_w_bench.py.Here we benchmark the Rust implementation (rust) versusfeetspackage and our own Python implementation (lc_py) for a light curve having n=1000 observations.The plot shows that the Rust implementation of the package outperforms other ones by a factor of 1.5—50.
This allows to extract a large set of "cheap" features well under one ms for n=1000.
The performance of parametric fits (BazinFitandVillarFit) andPeriodogramdepend on their parameters, but the typical timescale of feature extraction including these features is 20—50 ms for few hundred observations.Benchmark results of several features for both the pure-Python and Rust implementations of the "light-curve" package, as a function of the number of observations in a light curve. Both the x-axis and y-axis are on a logarithmic scale.Processing time per a single light curve for extraction of features subset presented in first benchmark versus the number of CPU cores used. The dataset consists of 10,000 light curves with 1,000 observations in each.See benchmarks' descriptions in more details in"Performant feature extraction for photometric time series".dm-dt mapClassDmDtprovides dm–dt mapper (based onMahabal et al. 2011,Soraisam et al. 2020). It is a Python wrapper forlight-curve-dmdtRust crate.importnumpyasnpfromlight_curveimportDmDtfromnumpy.testingimportassert_array_equaldmdt=DmDt.from_borders(min_lgdt=0,max_lgdt=np.log10(3),max_abs_dm=3,lgdt_size=2,dm_size=4,norm=[])t=np.array([0,1,2],dtype=np.float32)m=np.array([0,1,2],dtype=np.float32)desired=np.array([[0,0,2,0],[0,0,0,1],])actual=dmdt.points(t,m)assert_array_equal(actual,desired)CitationIf you found this project useful for your research please citeMalanchev et al., 2021@ARTICLE{2021MNRAS.502.5147M,author={{Malanchev}, K.~L. and {Pruzhinskaya}, M.~V. and {Korolev}, V.~S. and {Aleo}, P.~D. and {Kornilov}, M.~V. and {Ishida}, E.~E.~O. and {Krushinsky}, V.~V. and {Mondon}, F. and {Sreejith}, S. and {Volnova}, A.~A. and {Belinski}, A.~A. and {Dodin}, A.~V. and {Tatarnikov}, A.~M. and {Zheltoukhov}, S.~G. and {(The SNAD Team)}},title="{Anomaly detection in the Zwicky Transient Facility DR3}",journal={\mnras},keywords={methods: data analysis, astronomical data bases: miscellaneous, stars: variables: general, Astrophysics - Instrumentation and Methods for Astrophysics, Astrophysics - Solar and Stellar Astrophysics},year=2021,month=apr,volume={502},number={4},pages={5147-5175},doi={10.1093/mnras/stab316},archivePrefix={arXiv},eprint={2012.01419},primaryClass={astro-ph.IM},adsurl={https://ui.adsabs.harvard.edu/abs/2021MNRAS.502.5147M},adsnote={Provided by the SAO/NASA Astrophysics Data System}}