Package: ltsfit
The LtsFit PackageRobust Least Squares Regression with Uncertainties and Scatter in Any DimensionLtsFit is a Python package forvery robusthyperplane fitting in N dimensions,
with uncertainties in all coordinates and intrinsic scatter. It implements the
method described in Section 3.2 ofCappellari et al. (2013a)and uses the Least Trimmed Squares (LTS) technique to iteratively clip outliers(Rousseeuw & van Driessen 2006).ContentsThe LtsFit PackageAttributionInstallationDocumentationltsfitPurposeCalling SequenceInput ParametersOptional KeywordsOutput ParametersLicenseChangelogAttributionPlease also citeCappellari et al. (2013a)if you use this software for your research. This is the paper where the
implementation was described. The BibTeX entry for the paper is:@ARTICLE{Cappellari2013a,
    author = {{Cappellari}, M. and {Scott}, N. and {Alatalo}, K. and
        {Blitz}, L. and {Bois}, M. and {Bournaud}, F. and {Bureau}, M. and
        {Crocker}, A.~F. and {Davies}, R.~L. and {Davis}, T.~A. and {de Zeeuw},
        P.~T. and {Duc}, P.-A. and {Emsellem}, E. and {Khochfar}, S. and
        {Krajnovi{\'c}}, D. and {Kuntschner}, H. and {McDermid}, R.~M. and
        {Morganti}, R. and {Naab}, T. and {Oosterloo}, T. and {Sarzi}, M. and
        {Serra}, P. and {Weijmans}, A.-M. and {Young}, L.~M.},
    title = "{The ATLAS$^{3D}$ project - XV. Benchmark for early-type
        galaxies scaling relations from 260 dynamical models: mass-to-light
        ratio, dark matter, Fundamental Plane and Mass Plane}",
    journal = {MNRAS},
    eprint = {1208.3522},
    year = 2013,
    volume = 432,
    pages = {1709-1741},
    doi = {10.1093/mnras/stt562}
}Installationinstall with:pip install ltsfitWithout writing access to the globalsite-packagesdirectory, use:pip install --user ltsfitTo upgrade the package to the latest version use:pip install --upgrade ltsfitDocumentationSeeltsfit/examplesand the files docstrings.
They are copied bypipwithin the global foldersite-packages.ltsfitPurposeFit a linear function of the form:y = a + b1*x1 + b2*x2 +...+ bm*xm,to data with errors in all coordinates and intrinsic scatter, using a robust
method that clips outliers. The function can handle lines in 2-dim, planes in
3-dim, or hyperplanes in N-dim, wherex1,x2,...,xmare the independent
variables andyis the dependent variable. The method was introduced in
Sec. 3.2 ofCappellari et al. (2013a)and the treatment of outliers is is based on the FAST-LTS technique byRousseeuw & van Driessen (2006).
See alsoRousseeuw (1987).Calling Sequencefromltsfit.ltsfitimportltsfitp=ltsfit(x,y,sigx,sigy,clip=2.6,corr=True,epsy=True,frac=None,label='Fitted',label_clip='Clipped',legend=True,pivot=None,plot=True,text=True)print(f"Best fitting parameters:{p.coeff}")The output values are stored as attributes of thepobject.Input Parametersx: array_like with shape (n, m)Array ofnindependent variables formdimensions.EXAMPLE: To fit a line in 2-dim, one has a single vectorxof
lengthnwith the independent variable and a corresponding vector of
dependent variabley.EXAMPLE: To fit a plane in 3-dim, one has two vectors of lengthnof
independent variables(x1, x2). In this case,x =np.column_stack([x1,x2]).EXAMPLE: To fit a hyperplane in 4-dim, one has three vectors of
independent variables(x1, x2, x3). In this case,x =np.column_stack([x1,x2, x3]).y: array_like with shape (n,)Vector of measured values for each set ofxvariables.sigx: array_like with shape (n, m)Array of1sigmauncertainties for eachxcoordinate formdimensions. This has the same shape asx.sigy: array_like with shape (n,)Vector of1sigmauncertainties for eachyvalue.Optional Keywordsclip: floatClipping threshold insigmaunits. Values deviating more thanclip*sigmafrom the best fit are considered outliers and are
excluded from the fit. Default isclip=2.6, which would include
99% of the values for a Gaussian distribution.corr: boolifTrue, the correlation coefficients are printed on the plot.
Default isTrue.epsy: boolIfTrue, the intrinsic scatter is printed on the output plot.
Default isTrue.frac: floatFraction of values to include in the LTS stage.
Up to a fractionfracof the values can be outliers.
One must have0.5 <= frac <= 1. Default is0.5.NOTE: Setfrac=1to turn off outlier detection.pivot: array_like with shape (m,)If nonzero, thenltsfitfits the following line, plane or hyperplane:y = a + b0*(x0 - pivot[0]) + b1*(x1 - pivot[1]) + ...pivotare calledx_0,y_0in eq.(7) ofCappellari et al. (2013a).
Use of this keyword is strongly recommended, and suggested values arepivot = np.median(x, 0). This keyword has no effect on the best fit
but is important to reduce the covariance and uncertainty in the
intercepta.  However, the covariance is weakly dependent on the
precise value of thepivot. For this reason, it is generally better
to round thepivotvalues to nice numbers. Default is0.plot: boolIfTrue, a plot of the fit is produced. Default isTrue.text: boolIfTrue, the best fitting parameters are printed on the plot.
Default isTrue.Output ParametersThe output values are stored as attributes of theltsfitclass.p.coef: array_like with shape (m+1,)Best fitting parameters[a, b1,b2,...,bm].p.coef_err: array_like with shape (m+1,)1*sigmaformal uncertainties[a_err, b1_err,b2_err,...,bm_err].p.mask: array_like with shape (n,) and dtype boolBoolean vector indicating which elements ofzwere included in
the fit (True) and which were clipped as outliers (False).p.rms: floatRMS deviation between the data and the fitted relation.p.sig_int: floatIntrinsic scatter in theydirection around the line/plane/hyperplane.sig_intis calledepsilon_yin eq.(6) ofCappellari et al. (2013a).p.sig_int_err: float1*sigmaformal error onsig_int.p.xx: array_like with shape (n,)Values plotted along the x-axis. This is the linear combination of thexvariables that represents the plane/hyperplane edge-on:xx = a + b1*(x1 - pivot[0]) + b2*(x2 - pivot[1]) + ...For line fitting, these are just thexvalues.p.yy: array_like with shape (n,)The inputyvalues plotted along the y-axis.p.xerr: array_like with shape (n,)1*sigmauncertainties forp.xxin the x-axis of the plot.p.yerr: array_like with shape (n,)1*sigmauncertainties forp.yyin the y-axis of the plot.p.xline: array_like with shape (2,)xcoordinates of the best fitting relation as shown on the plot.p.yline: array_like with shape (2,)ycoordinates of the best fitting relation as shown on the plot.p.spearmanr: array_like with shape (2,)Spearmanrcoefficient and probabilitypbetween(p.xx, p.yy)without clipping outliers.p.pearsonr: array_like with shape (2,)Pearsonrcoefficient and probabilitypbetween(p.xx, p.yy)without clipping outliers.LicenseOther/Proprietary LicenseCopyright (c) 2012-2023 Michele CappellariThis software is provided as is with no warranty. You may use it for
non-commercial purposes and modify it for personal or internal use, as long
as you include this copyright and disclaimer in all copies. You may not
redistribute the code.ChangelogV6.0.1: MC, Oxford, 20 July 2023New functionltsfitto fit hyperplanes in N-dim. This procedure
generalizes and replaces bothlts_linefitandlts_planefit, which
are now deprecated wrappers forltsfit. This change was suggested and
motivated by Francesco D’Eugenio (cam.ac.uk), who shared his own 4-dimlts_hyperfitand his paper on a useful application.ltsfit: When fitting planes/hyperplanes, plot the independent variable
on the y-axis to be consistent with line fitting. Also plot a legend.Updated allltsfit_examples.Fixed inconsistency between the version number on PyPi and in the Changelog.V5.0.20: MC, Oxford, 3 October 2022Fixed program stop due to Matplotlib change.
Thanks to Hitesh Lala (Heidelberg) for the report.Extract documentation from docstrings and show it on PyPi.V5.0.19: MC, Oxford, 22 January 2021Fixed incorrect plot ranges due to a Matplotlib change.
Thanks to Davide Bevacqua (unibo.it) for the report.V5.0.18: MC, Oxford, 17 February 2020Properly print significant trailing zeros in results.V5.0.17: MC, Oxford, 22 January 2020Formatted documentation as docstring.Included p.rms output.Published on PyPi. Increased major version number by mistake.V2.0.16: MC, Oxford, 27 September 2018Fixed clock DeprecationWarning in Python 3.7.V2.0.15: MC, Oxford, 12 May 2018Dropped Python 2.7 support.V2.0.14: MC, Oxford, 13 April 2018Fixed FutureWarning in Numpy 1.14.V2.0.13: Michele Cappellari, Oxford, 26 July 2017Increased upper limit of intrinsic scatter accounting for
uncertainty of standard deviation with small samples.V2.0.12: MC, Oxford, 5 September 2016Fixed: store ab errors in p.ab_err as documented.
Thanks to Alison Crocker for the correction.V2.0.11: MC, Oxford, 4 July 2016Added capsize=0 in plt.errorbar to prevent error bar caps
from showing up in PDF.V2.0.10: MC, Oxford, 23 January 2016Check for non finite values in input.V2.0.9: MC, Oxford, 8 January 2016Use LimeGreen for outliers.V2.0.8: MC, Oxford, 9 December 2015Fixed potential program stop without outliers in Matplotlib 1.5.Increased maximum intrinsic scatter for brentq, to avoid possible
stops in extreme situations.V2.0.7: MC, Oxford, 1 October 2015Fixed potential program stop without outliers.V2.0.6: MC, Oxford, 5 September 2015Optionally pass a legend label.V2.0.5: MC, Oxford, 6 July 2015Fixed potential program stop without outliers.
Thanks to Masato Onodera for a clear report of the problem.Output boolean mask instead of good/bad indices.Removed lts_linefit_example from this file.Print verbose output during calculation.V2.0.4: MC, Baltimore, 9 June 2015Updated documentation.V2.0.3: MC, Oxford, 10 December 2014Uses np.std rather than biweight to estimate scatter upper limit.V2.0.2: MC, 6 November 2014Included _linefit function to avoid np.polyfit bug with weights.V2.0.1: MC, Oxford, 23 October 2014Fixed program stop with zero scatter.V2.0.0: MC, Portsmouth, 23 June 2014Converted from IDL into Python.V1.0.6: MC, Baltimore, 8 June 2014Check that all input vectors have the same size.V1.0.5: MC, Oxford, 19 September 2013Scale line spacing with character size in text output.V1.0.4: MC, Turku, 10 July 2013Fixed program stop affecting earlier versions of IDL.
Thanks to Xue-Guang Zhang for reporting the problem
and the solution.V1.0.3: MC, Oxford, 13 March 2013Added CLIP keyword.V1.0.2: MC, Oxford, 1 August 2011Added PIVOT keyword.V1.0.1: MC, Oxford, 28 July 2011Included _EXTRA and OVEPLOT, keywords.V1.0.0: Michele Cappellari, Oxford, 21 March 2011Created and tested.