Package: pyglmnet
A python implementation of elastic-net regularized generalized linear models[Documentation (stable version)][Documentation (development version)]Generalized linear
modelsare
well-established tools for regression and classification and are widely
applied across the sciences, economics, business, and finance. They are
uniquely identifiable due to their convex loss and easy to interpret due
to their point-wise non-linearities and well-defined noise models.In the era of exploratory data analyses with a large number of predictor
variables, it is important to regularize. Regularization prevents
overfitting by penalizing the negative log likelihood and can be used to
articulate prior knowledge about the parameters in a structured form.Despite the attractiveness of regularized GLMs, the available tools in
the Python data science eco-system are highly fragmented. More
specifically,statsmodelsprovides a wide range of link functions but no regularization.scikit-learnprovides elastic net regularization but only for linear models.lightningprovides elastic net and group lasso regularization, but only for
linear and logistic regression.Pyglmnetis a response to this fragmentation. It runs on Python 3.5+,
and here are some of the highlights.Pyglmnet provides a wide range of noise models (and paired canonical
link functions):'gaussian','binomial','probit','gamma', ‘poisson’, and'softplus'.It supports a wide range of regularizers: ridge, lasso, elastic net,group
lasso,
andTikhonov
regularization.Pyglmnet’s API is designed to be compatible with scikit-learn, so you
can deployPipelinetools such asGridSearchCV()andcross_val_score().We follow the same approach and notations as inFriedman, J.,
Hastie, T., & Tibshirani, R.
(2010)and the
accompanying widely popularR
package.We have implemented a cyclical coordinate descent optimizer with
Newton update, active sets, update caching, and warm restarts. This
optimization approach is identical to the one used in R package.A number of Python wrappers exist for the R glmnet package (e.g.hereandhere) but in contrast to
these, Pyglmnet is a pure python implementation. Therefore, it is
easy to modify and introduce additional noise models and regularizers
in the future.InstallationInstall the stable PyPI version withpip$pipinstallpyglmnetFor the bleeding edge development version:Clone the repository.$pipinstallhttps://api.github.com/repos/glm-tools/pyglmnet/zipball/masterGetting StartedHere is an example on how to use theGLMestimator.importnumpyasnpimportscipy.sparseasspsfrompyglmnetimportGLM,simulate_glmn_samples,n_features=1000,100distr='poisson'# sample a sparse modelbeta0=np.random.rand()beta=np.random.random(n_features)beta[beta<0.9]=0# simulate dataXtrain=np.random.normal(0.0,1.0,[n_samples,n_features])ytrain=simulate_glm('poisson',beta0,beta,Xtrain)Xtest=np.random.normal(0.0,1.0,[n_samples,n_features])ytest=simulate_glm('poisson',beta0,beta,Xtest)# create an instance of the GLM classglm=GLM(distr='poisson',score_metric='deviance')# fit the model on the training dataglm.fit(Xtrain,ytrain)# predict using fitted model on the test datayhat=glm.predict(Xtest)# score the model on test datadeviance=glm.score(Xtest,ytest)More pyglmnet examples and use
cases.TutorialHere is anextensive
tutorialon GLMs,
optimization and pseudo-code.Here areslidesfrom a
talk atPyData Chicago
2016,
correspondingtutorial
notebooksand avideo.How to contribute?We welcome pull requests. Please see ourdeveloper documentation
pagefor more
details.AcknowledgmentsKonrad Kordingfor funding and supportSara
Sollafor masterful GLM lecturesLicenseMIT License Copyright (c) 2016-2019 Pavan Ramkumar