Package: tfHuber
tfHuberPython implementation ofTuning-FreeHuberEstimation and RegressionDescriptionThis package implements the Huber mean estimator, Huber covariance matrix estimation and adaptive Huber regression estimators efficiently. For all these methods, the robustification parameterτis calibrated via a tuning-free principle.Specifically, for Huber regression, assume the observed data vectors (Y,X) follow a linear modelY = θ0+ X θ + ε, whereYis ann-dimensional response vector,Xis ann×ddesign matrix, andεis ann-vector of noise variables whose distributions can be asymmetric and/or heavy-tailed. The package computes the standard Huber'sM-estimator whend < nand the Huber-Lasso estimator whend > n. The vector of coefficientsθand the intercept termθ0are estimated successively via a two-step procedure. SeeWang et al., 2020for more details of the two-step tuning-free framework.Requirementnumpy
setuptools
wheelFunctionsThere are four functions in this package:mean(X, grad=True, tol=1e-5, max_iter=500): Huber mean estimation. Return a tuple of mean, $\tau$ and the number of iteration.X: A 1-d array.grad: Using gradient descent or weighed least square to optimize the mean, defaultTruetol: Tolerance of the error, default1e-5.max_iter: Maximum times of iteration, default500.cov(X, type="element", pairwise=False, tol=1e-5, max_iter=500): Huber covariance matrix estimation. Return a 2d covariance matrix.X: A 2-d array.type: If set to"element", apply adaptive huber M-estimation; or if set to"spectrum", apply spectrum-wise truncated estimation. Default"element"pairwise: Pairwise covariance or difference based covariance. Defaultfalse.tol: Tolerance of the error, default1e-5.max_iter: Maximum times of iteration, default500.one_step_reg(X, Y, grad=True, tol=1e-5, max_iter=500 two_step_reg(X, Y, grad=True, tol=1e-5, constTau=1.345, max_iter=500)One or two step adaptive Huber regression. Return a tuple of coefficients, $\tau$ and the number of iteration.X, Y: Arrays of data.grad: Using gradient descent or weighed least square to optimize the mean, defaultTrue.tol: Tolerance of the error, default1e-5.constTau: Default 1.345. Used only in two-step method.max_iter: Maximum times of iteration, default500.cvlasso(X, Y, lSeq=0, nlambda=30, constTau=2.5, phi0=0.001, gamma=1.5, tol=0.001, nfolds=3): K-fold cross validated Huber-lasso regression. Return a tuple of coefficients, $tau$, the number of iteration and minimun of $\lambda$.X, Y: Arrays of data.lSeq: A list of Lasso parameter $\lambda$. If not set, automatically find a range of $\lambda$ to be cross validated.nlambda: The number of $\lambda$ used for validation.constTau, phi0, gamma: Some parameters.tol: Tolerance of the error, default0.001.nfolds: Number of folds to be cross validated.ExamplesWe present an example of adaptive Huber methods. Here we generate data from a linear modelY = X θ + ε, whereεfollows a normal distribution, and estimate the intercept and coefficients by tuning-free Huber regression.importnumpyimporttfhuberX=np.random.uniform(-1.5,1.5,(10000,10))Y=intercept+np.dot(X,beta)+np.random.normal(0,1,10000)mu,tau,iteration=tf.mean(Y,grad=True,tol=1e-5,max_iter=500)cov=tf.cov(X,method=1,tol=1e-5,max_iter=500)theta,tau,iteration=tf.one_step_reg(X,Y,grad=True,tol=1e-5,max_iter=500)theta,tau,iteration=tf.two_step_reg(X,Y,grad=True,tol=1e-5,consTau=1.345,max_iter=500)theta,tau,iteration,lam=tf.cvlasso(X,Y)Simulation result can be viewed in thiscolab notebook.LicenseGPL (>= 3)Author(s)Yifan Daiyifandai@yeah.net, Qiang Sunqsun.ustc@gmail.comDescription and algorithms refer toXiaoou Pan's page.ReferencesGuennebaud, G. and Jacob B. and others. (2010). Eigen v3.WebsiteKe, Y., Minsker, S., Ren, Z., Sun, Q. and Zhou, W.-X. (2019). User-friendly covariance estimation for heavy-tailed distributions.Statis. Sci.34454-471,PaperPan, X., Sun, Q. and Zhou, W.-X. (2019). Nonconvex regularized robust regression with oracle properties in polynomial time. Preprint.PaperSanderson, C. and Curtin, R. (2016). Armadillo: A template-based C++ library for linear algebra.J. Open Source Softw.126.PaperSun, Q., Zhou, W.-X. and Fan, J. (2020). Adaptive Huber regression.J. Amer. Stat. Assoc.115254-265.PaperTibshirani, R. (1996). Regression shrinkage and selection via the lasso.J. R. Stat. Soc. Ser. B. Stat. Methodol.58267–288.PaperWang, L., Zheng, C., Zhou, W. and Zhou, W.-X. (2020). A new principle for tuning-free Huber regression.Stat. Sinicato appear.Paper