Package: bess
bess: A Python Package for Best Subset SelectionIntroductionOne of the main tasks of statistical modeling is to exploit the association between
a response variable and multiple predictors. Linear model (LM), as a simple parametric
regression model, is often used to capture linear dependence between response and
predictors. Generalized linear model (GLM) can be considered as
the extensions of linear model, depending on the types of responses. Parameter estimation in these models
can be computationally intensive when the number of predictors is large. Meanwhile,
Occam's razor is widely accepted as a heuristic rule for statistical modeling,
which balances goodness of fit and model complexity. This rule leads to a relative
small subset of important predictors.besspackage provides solutions for best subset selection problem for sparse LM,
and GLM models.We consider a primal-dual active set (PDAS) approach to exactly solve the best subset
selection problem for sparse LM and GLM models. The PDAS algorithm for linear
least squares problems was first introduced byIto and Kunisch (2013)and later discussed byJiao, Jin, and Lu (2015)andHuang, Jiao, Liu, and Lu (2017).
It utilizes an active set updating strategy and fits the sub-models through use of
complementary primal and dual variables. We generalize the PDAS algorithm for
general convex loss functions with the best subset constraint, and further
extend it to support both sequential and golden section search strategies
for optimal k determination.InstallPython Versionpython >= 3.5Modules needednumpyThe package has been publish in PyPI. You can easy install by:$pipinstallbessExample### PdasLm samplefrombess.linearimport*importnumpyasnpnp.random.seed(12345)# fix seed to get the same resultx=np.random.normal(0,1,100*150).reshape((100,150))beta=np.hstack((np.array([1,1,-1,-1,-1]),np.zeros(145)))noise=np.random.normal(0,1,100)y=np.matmul(x,beta)+noise### Sparsity knownmodel=PdasLm(path_type="seq",sequence=[5])model.fit(X=x,y=y)model.predict(x)### Sparsity unknown# path_type="seq", Default:sequence=[1,2,...,min(x.shape[0], x.shape[1])]model=PdasLm(path_type="seq",sequence=range(1,10))model.fit(X=x,y=y)model.predict(x)# path_type="pgs", Default:s_min=1, s_max=X.shape[1], K_max = int(math.log(p, 2/(math.sqrt(5) - 1)))model=PdasLm(path_type="pgs",s_max=20)model.fit(X=x,y=y)model.predict(x)### PdasLogistic samplenp.random.seed(12345)x=np.random.normal(0,1,100*150).reshape((100,150))beta=np.hstack((np.array([1,1,-1,-1,-1]),np.zeros(145)))xbeta=np.matmul(x,beta)p=np.exp(xbeta)/(1+np.exp(xbeta))y=np.random.binomial(1,p)### Sparsity knownmodel=PdasLogistic(path_type="seq",sequence=[5])model.fit(X=x,y=y)model.predict(x)### Sparsity unknown# path_type="seq", Default:sequence=[1,2,...,min(x.shape[0], x.shape[1])]model=PdasLogistic(path_type="seq",sequence=range(1,10))model.fit(X=x,y=y)model.predict(x)# path_type="pgs", Default:s_min=1, s_max=X.shape[1], K_max = int(math.log(p, 2/(math.sqrt(5) - 1)))model=PdasLogistic(path_type="pgs")model.fit(X=x,y=y)model.predict(x)### PdasPoisson samplenp.random.seed(12345)x=np.random.normal(0,1,100*150).reshape((100,150))beta=np.hstack((np.array([1,1,-1,-1,-1]),np.zeros(145)))lam=np.exp(np.matmul(x,beta))y=np.random.poisson(lam=lam)### Sparsity knownmodel=PdasPoisson(path_type="seq",sequence=[5])model.fit(X=x,y=y)model.predict(x)### Sparsity unknown# path_type="seq", Default:sequence=[1,2,...,min(x.shape[0], x.shape[1])]model=PdasPoisson(path_type="seq",sequence=range(1,10))model.fit(X=x,y=y)model.predict(x)# path_type="pgs", Default:s_min=1, s_max=X.shape[1], K_max = int(math.log(p, 2/(math.sqrt(5) - 1)))model=PdasPoisson(path_type="pgs")model.fit(X=x,y=y)model.predict(x)### PdasCox samplefrombess.gen_dataimportgen_datanp.random.seed(12345)data=gen_data(100,200,family="cox",k=5,rho=0,sigma=1,c=10)### Sparsity knownmodel=PdasCox(path_type="seq",sequence=[5])model.fit(data.x,data.y,is_normal=True)model.predict(data.x)### Sparsity unknown# path_type="seq", Default:sequence=[1,2,...,min(x.shape[0], x.shape[1])]model=PdasCox(path_type="seq",sequence=range(1,10))model.fit(data.x,data.y)model.predict(data.x)# path_type="pgs", Default:s_min=1, s_max=X.shape[1], K_max = int(math.log(p, 2/(math.sqrt(5) - 1)))model=PdasCox(path_type="pgs")model.fit(data.x,data.y)model.predict(data.x)ReferenceWen, C. , Zhang, A. , Quan, S. , & Wang, X. . (2017).Bess: an r package for best subset selection in linear, logistic and coxph modelsBug reportConnect to@Jiang-Kangkang, or send an email to Jiang Kangkang(jiangkk3@mail2.sysu.edu.cn)