Package: nodegam
NODE GAM: Differentiable Generalized Additive Model for Interpretable Deep Learning:NodeGAM is an interpretable deep learning GAM model proposed in our ICLR 2022 paper:NODE GAM: Differentiable Generalized Additive Model for Interpretable Deep Learning.
In short, it trains a GAM model by multi-layer differentiable trees to be accurate, interpretable, and
differentiable. See thisblog postfor an intro, and ourdocumentation website!InstallationpipinstallnodegamThe performance and the runtime of the NodeGAM packageWe compare NodeGAM with other GAMs (EBM, XGB-GAM), and XGB in 6 datasets.
All models use default parameters, so the performance of NodeGAM here is lower than what paper
reported. We find NodeGAM often performs better in larger datasets.3 classification datasets:Dataset/AUROCDomainNPNodeGAMEBMXGB-GAMXGBMIMIC-IIMedicine25K170.844 ± 0.0180.842 ± 0.0190.833 ± 0.020.845 ± 0.019AdultFinance33K140.916 ± 0.0020.927 ± 0.0030.925 ± 0.0020.927 ± 0.002CreditFinance285K300.989 ± 0.0080.984 ± 0.0070.985 ± 0.0080.984 ± 0.013 regression datasets:Dataset/RMSEDomainNPNodeGAMEBMXGB-GAMXGBWineNature5K120.705 ± 0.0120.69 ± 0.0110.713 ± 0.0060.682 ± 0.023BikeshareRetail17K1657.438 ± 3.89955.676 ± 0.327101.093 ± 0.94645.212 ± 1.254YearMusic515K909.013 ± 0.0049.204 ± 0.09.257 ± 0.09.049 ± 0.0We also find the run time of our model increases mildly with growing data size due to mini-batch
training, while our baselines increase training time much more.3 classification datasets:Dataset/SecondsDomainNPNodeGAMEBMXGB-GAMXGBMIMIC-IIMedicine25K17105.0 ± 14.06.0 ± 2.00.0 ± 1.01.0 ± 1.0AdultFinance33K14196.0 ± 56.015.0 ± 8.06.0 ± 0.01.0 ± 0.0CreditFinance285K30113.0 ± 36.037.0 ± 2.026.0 ± 7.016.0 ± 2.03 regression datasets:Dataset/SecondsDomainNPNodeGAMEBMXGB-GAMXGBWineNature5K12157.0 ± 86.04.0 ± 2.00.0 ± 0.00.0 ± 0.0BikeshareRetail17K16223.0 ± 23.015.0 ± 3.01.0 ± 1.02.0 ± 1.0YearMusic515K90318.0 ± 20.0501.0 ± 8.0376.0 ± 1.0537.0 ± 1.0Reproducing notebook ishere.See the Table 1 and 2 of ourpaperfor more comparisons.NodeGAM TrainingSklearn interfaceTo simply use it on your dataset, just run:fromnodegam.sklearnimportNodeGAMClassifier,NodeGAMRegressormodel=NodeGAMClassifier()model.fit(X,y)Understand the model:model.visualize()orfromnodegam.vis_utilsimportvis_GAM_effectsvis_GAM_effects({'nodegam':model.get_GAM_df(),})See thenotebooks/toy dataset with nodegam sklearn.ipynbhere.Notebook trainingIt is useful if you want to customize the NodeGAM training to your PyTorch pipeline.
You can find details of the training in this notebook:https://colab.research.google.com/drive/1C_gBoSc1AlQ7VvCXVWiU-7X3YjQZTiZI?usp=sharingAnd see more examples undernotebooks/Python fileYou can also train a NodeGAM using our main file.
To reproduce our results, e.g. NODE-GA2M trained in fold 0 (total 5 folds) of bikeshare, you can runhparams="resources/best_hparams/node_ga2m/0519_f0_best_bikeshare_GAM_ga2m_s83_nl4_nt125_td1_d6_od0.0_ld0.3_cs0.5_lr0.01_lo0_la0.0_pt0_pr0_mn0_ol0_ll1"pythonmain.py\--name0603_best_bikeshare_f0\--load_from_hparams${hparams}--fold0The models will be stored inlogs/0603_best_bikeshare_f0/. And the results including test/val error are stored inresults/bikeshare_GAM.csvWe provide the best hyperparmeters we found inbest_hparams/.Baseline GAMsWe also provide code to train other GAMs for comparisons such as:Spline: we use thepygampackage.EBM:Explainable Boosting Machine.XGB-GAM: Limit the XGB to have tree depth 1 that removes all interaction effects in the model.
It's proposed inour KDD paper.Sklearn interfaceTo train baselines on your dataset, just run:fromnodegam.gams.MySplineimportMySplineLogisticGAM,MySplineGAMfromnodegam.gams.MyEBMimportMyExplainableBoostingClassifier,MyExplainableBoostingRegressorfromnodegam.gams.MyXGBimportMyXGBOnehotClassifier,MyXGBOnehotRegressorfromnodegam.gams.MyBaggingimportMyBaggingClassifier,MyBaggingRegressorebm=MyExplainableBoostingClassifier()ebm.fit(X,y)spline=MySplineLogisticGAM()bagged_spline=MyBaggingClassifier(base_estimator=spline,n_estimators=3)bagged_spline.fit(X,y)xgb_gam=MyXGBOnehotClassifier()bagged_xgb=MyBaggingClassifier(base_estimator=xgb_gam,n_estimators=3)bagged_xgb.fit(X,y)Understand the models:fromnodegam.vis_utilsimportvis_GAM_effectsfig,ax=vis_GAM_effects(all_dfs={'EBM':ebm.get_GAM_df(),'Spline':bagged_spline.get_GAM_df(),'XGB-GAM':bagged_xgb.get_GAM_df(),},)See thenotebooks/toy dataset with nodegam sklearn.ipynbherefor an example.Python fileYou can train Spline, EBM, and XGB-GAM by the following commands.pythonbaselines.py--name0603_bikeshare_spline_f0--fold0--model_namespline--datasetbikeshare
pythonbaselines.py--name0603_bikeshare_ebm_f0--fold0--model_nameebm-o100-i100--datasetbikeshare
pythonbaselines.py--name0603_bikeshare_xgb-o5_f0--fold0--model_namexgb-o5--datasetbikeshareThe result is shown inresults/baselines_bikeshare.csvand the model is stored inlogs/{name}/.Visualization of the trained models stored underlogs/To visualize and compare multiple trained GAM models stored underlogs/, run this in a notebook:fromnodegam.vis_utilsimportvis_GAM_effectsfromnodegam.utilsimportaverage_GAMsdf_dict={'node_ga2m':average_GAMs(['0603_best_bikeshare_f0','0603_best_bikeshare_f1',],max_n_bins=256),'ebm':average_GAMs(['0603_bikeshare_ebm_f0','0603_bikeshare_ebm_f1',],max_n_bins=256),}fig,ax=vis_GAM_effects(df_dict)To avoid long computations, when visualizing we specify max_n_bins to do quantile binning of each
feature to have at most 256 bins (default). Theaverage_GAMstake average of multiple runs of GAMs to produce mean and stdev on the GAM
graphs.Seenotebooks/bikeshare visualization.ipynbherewhich we show bikeshare graphs for all GAMs
(NODE-GA2M, NODE-GAM, EBM and Spline) in our paper.CitationsIf you find the code useful, please cite:@inproceedings{chang2021node,
  title={NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learning},
  author={Chang, Chun-Hao and Caruana, Rich and Goldenberg, Anna},
  booktitle={International Conference on Learning Representations},
  year={2022}
}

@inproceedings{chang2021interpretable,
  title={How interpretable and trustworthy are gams?},
  author={Chang, Chun-Hao and Tan, Sarah and Lengerich, Ben and Goldenberg, Anna and Caruana, Rich},
  booktitle={Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining},
  pages={95--105},
  year={2021}
}ContributingAll content in this repository is licensed under the MIT license.