Package: lrtree
Logistic regression treesTable of ContentsDocumentationInstallation instructionsTheorySome examplesOpen an issueReferencesContributeMotivationThe goal oflrtreeis to build decision trees with logistic regressions at their leaves, so that the resulting model mixes non parametric VS parametric and stepwise VS linear approaches to have the best predictive results, yet maintaining interpretability.This is the implementation of glmtree as described inFormalization and study of statistical problems in Credit Scoring, Ehrhardt A. (seemanuscriptorweb article)Getting startedThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes.PrerequisitesThis code is supported on Python 3.8, 3.9, 3.10.Installing the packageInstalling the development versionIfgitis installed on your machine, you can use:pipenvinstallgit+https://github.com/adimajo/lrtree.gitIfgitis not installed, you can also use:pipenvinstall--upgradehttps://github.com/adimajo/lrtree/archive/master.tar.gzInstalling through thepipcommandYou can install a stable version fromPyPiby using:pipinstalllrtreeTo run the provided scripts,lrtree-consistencyandlrtree-realdata, you need a
few additional dependencies:pipinstalllrtree[scripts]Installation guide for AnacondaThe installation with thepiporpipenvcommandshouldwork. If not, please raise an issue.For people behind proxy(ies)...A lot of people, including myself, work behind a proxy at work...A simple solution to get the package is to use the--proxyoption ofpip:pip--proxy=http://username:password@server:portinstalllrtreewhereusername,password,serverandportshould be replaced by your own values.If environment variableshttp_proxyand / orhttps_proxyand / or (unfortunately depending on applications...)HTTP_PROXYandHTTPS_PROXYare set, the proxy settings should be picked up bypip.Over the years, I've foundCNTLMto be a great tool in this regard.AuthorsAdrien EhrhardtVincent VandewallePhilippe HeinrichChristophe BiernackiDmitri GaynullinElise BayraktarLicenseThis project is licensed under the MIT License - see theLICENSEfile for details.AcknowledgmentsThis research has been financed byCrédit Agricole Consumer Financethrough a CIFRE PhD.This research was supported byInria Lille - Nord-EuropeandLille Universityas part of a PhD.ReferencesEhrhardt, A. (2019),Formalization and study of statistical problems in Credit Scoring: Reject inference, discretization and pairwise interactions, logistic regression trees(PhD thesis).ContributeYou can clone this project using:gitclonehttps://github.com/adimajo/lrtree.gitYou can install all dependencies, including development dependencies, using (note that
this command requirespipenvwhich can be installed by typingpip install pipenv):pipenvinstall-dYou can build the documentation by going into thedocsdirectory and typingmake html.You can run the tests by typingcoverage run -m pytest, which relies on packagescoverageandpytest.To run the tests in different environments (one for each version of Python), installpyenv(seethe instructions here),
install all versions you want to test (seetox.ini), e.g. withpyenv install 3.7.0and runpipenv run pyenv local 3.7.0 [...](and all other versions) followed bypipenv run tox.Python EnvironmentThe project usespipenv.An interesting resource.To download all the project dependencies in order to then port them to a machine that had limited access to the internet, you must use the commandpipenv lock -r > requirements.txtwhich will transform thePipfileinto arequirements.txt.InstallationTo install a virtual environment as well as all the necessary dependencies, you must use thepipenv installcommand for production use
or the commandpipenv install -dfor development use.TestsThe tests are based onpytestand are stored in thetestsfolder. They can all be launched with the commandpytestin at the root of the project.
The test coverage can be calculated thanks to thecoveragepackage, which is also responsible for launching the tests.
The command to use iscoverage run -m pytest. We can then obtain a graphic summary in the form of an HTML page
using thecoverage htmlcommand which creates or updates thehtmlcovfolder from which we can open theindex.htmlfile.UtilizationThe package providessklearn-likeinterface.Loadingsample datafor regression task:fromsklearn.datasetsimportload_bostonfromsklearn.model_selectionimporttrain_test_splitX,y=load_boston(return_X_y=True)X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.25)The trained model consists of a fittedsklearn.tree.DecisionTreeClassifierclass for segmentation of a data andsklearn.linear_model.LogisticRegressionregressions for each node a of a tree in a form of python list.The snippet to train the model and make a prediction:fromlrtreeimportLrtreemodel=Lrtree(criterion="bic",ratios=(0.7,),class_num=2,max_iter=100)# Fitting the modelmodel.fit(X_train,y_train)# Make a prediction on a fitted modelmodel.predict(X_test)If you installed the additional dependencies for scripts, you can also run directly from the command line:LOGURU_LEVEL="ERROR"DEBUG="True"lrtree-consistencyorLOGURU_LEVEL="ERROR"TQDM_DISABLE="1"lrtree-realdataBeware: if you don't setLOGURU_LEVELthen it is implicitly set on DEBUG
which will yield a lot of prints. Also, both scripts will take very long
to complete as they test the consistency of the method for various
hyperparameters and run cross-validation on 3 real datasets respectively.