Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose the neural additive models which have an additive structure and which are more explainable than general neural networks. They apply this model for various tasks and applications and show their interpretability through visualizations. Its a good contribution and a reasonably well-written paper. 
main_review: The authors propose neural additive models, a linear combination of multiple small networks, which are trained jointly and can learn complex functions such as jagged shape functions. The claim is that because the models are modular, the predictions can be propagated back to the individual network functions and easily visualized. They show the explainability of the networks in applications like fraud detection and recidivism risk prediction.

The authors have explained the proposed structure reasonably well but its readability could be improved with a better presentation of the notations and the model structure. The experimental section is detailed and covers various single-task and multi-task learning applications. 
They have compared their solution with other explainable ML algorithms and shown comparable prediction accuracy. My only concern is that their analysis results on interpretability is very heuristic and subjective, rather than a formal validation. 

Overall, the paper is a good contribution and I recommend acceptance.

Update (after author response): The authors have addressed my main concerns. I recommend acceptance with the changes proposed in the author response.
limitations_and_societal_impact: This paper addresses a number of applications like healthcare and recidivism prediction which have a high societal impact. For example, they try to determine the features which contribute to mortality during ICU stays (using the MIMIC dataset) and the features that contribute to bias in the COMPAS dataset. Though the authors do some heuristic analysis, these results should be ideally formally validated. The authors should also address some of the limitations of their current work and potential future avenues of study.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper propose Neural Additive Model that provides more interpretability with DNNs. It learns a linear combination of neural networks that each attend to a single input feature. Through experiments, NAM is shown to be more accurate than other intelligible models such as logistic regression and explainable boosting machine (EBM), while also provides explanability with respect to the input features. A multitask learning architecture along with the single task architecture is proposed.
main_review: The proposed NAM is a DNN version of the generalized additive models (GAM). The proposed NAM allows one to exam the eact description of how NAM compute a prediction since one can sweep the input feature and see the prediction curve. The individual shape function provides the good intelligibility for the input feature. The proposed method contains certain amount of novelty in the NAM and the way to interprete the model, including the multi-task NAM architecture. However, I think the novelty of the proposed NAM is not that significant. Several experiments have been conducted to demonstrate the better accuracy compared with linear regression and the intelligible EBM. The paper also shows several examples on how to interpret the model and explain how the input features affect the performance. The paper is well-written and easy to understand.

While it is good to allow each network attend to only one feature, the limitation of the proposed NAM is also evident. The major advantage of deep learning models lie in its representation learning, which learns the non-linear combination and transformation of interleaving features, not individual features. The relationship and interaction of different features empowers the development in many areas where deep learning applies such as computer vision, natural language processing, recommendation, etc. I don't see how the proposed NAM can be more generalized to other areas, where the input feature is of high-dimension and requires more complex understanding.

Other concerns:
- The proposed exp-centered hidden units (ExU) uses the exponential function for the weight. Will this introduce instability in training? It roughly only allows the weight to be in a much smaller range, and could cause the gradient to go to nan easily.
limitations_and_societal_impact: The limitation of the proposed NAM is that it is a simple linear combination of multiple neural nets which each only attends to one feature. While it provides the intelligibility to the model when input feature is simple, its generalizability and expressiveness to more complex domains is lacking.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors introduce a new generalized additive model, called neural additive models, which use neural networks to learn a non-linear transformation of each input feature, independently. Along the way, they introduce a new activation function, called ExU, which enables a highly sensitive weights that can fit very large, seemingly noisy, changes of a given function. They demonstrate that NAM performs as well as other GAMs, such as EBMs, as well as black box models, such as DNNs and XGBoost. The major  benefit of NAM is that, similar to EBMs, they are intrinsically interpretable simply by visualizing the learned non-linear function for each input feature.  They compare the performance of these methods across several tasks. The paper is written very clearly with easy to follow examples. 
main_review: Overall, I found the proposed NAM very interesting and a worthwhile contribution to the toolbox of GAMs. The new activation, i.e. ExU, is also quite intriguing. In this regards, I think that the paper has legs to stand on. The work is a simple extension, using neural networks to learn a non-linear transformation of each input feature independently. It is thus learning a suitable non-linearity for each input feature such that its additive contribution is predictive of the task at hand. Instead of using a preset non-linear function, NAMs use an MLP to learn one. It is very clear how this approach is different from previous contributions. A major limitation to this study was that there was no clear example that highlights the benefits of NAM over previous methods, either in terms of performance or interpretability. Below, I highlight my main concerns:

- The argument that a ExU can learn jagged shape functions, whereas the same network with ReLU activations learns something smoother is clearly demonstrated for a given network. However, a deeper network with ReLU activations should be able to fit the noisier function, right? So, the statement is really more about the ability of ExU to express noisier functions using a smaller number of parameters compared to ReLU activations. 

- Many of the synthetic examples are a bit contrived. For instance, the example of learning a jagged shape function; It's actually unclear when learning this kind of jagged shape function would be required in practice. On the flip side, how well does ExU do when the underlying function is smooth? Does it over-exaggerate more easily due to its highly sensitive weights? Would a relu or smooth activation perform better in such a circumstance? If there is a big difference between these scenarios, perhaps some guidance would be beneficial. 

- Building on this question, how does NAM do when the underlying features are supposed to be additive (i.e. no nonlinearities). In this case, NAMs are overkill, but do their MLPs learn learn a linear function or do they learn complex non-linear functions? If that's the case, how "interpretable" is it? I supoose it is still interpretable in a strict sense of the model, but it is very different from being reliable. A comment on this could help an end user think more deeply about these considerations. 

- I do appreciate the attempt at comparing different methods across different datasets, but I think the examples don't really highlight NAMs ability to excel at tasks that previous methods struggle. The gains for each model is quite marginal. One question could be -- relative to previous methods, are NAMs robust to a wide setting of hyperparameters? Are there any benefits (beyond interpretability) with training time, hyperparameter search, inference time, scalability? If so, this should be highlighted as the performance alone is right in the pack (and EBMs are also interpretable). 

- Since interpretability means so many things these days, I think there should be a clear message what is meant by interpretability here. A comparison of global function vs local function via LIME was made, but it may be nice to contrast other definitions of interpretability, such as post hoc explanations with attribution methods. 

- In regards to interpretability, it is a true benefit that NAMs can just plot the learned function and one can get a sense for variations of the function. It's nice that variability is low where data is concentrated and then the uncertainty grows in data deprived regions of function space. it would also be nice to see  interpretations of other models for comparison. 

- How do EMB interpretations compare? Do you draw the same conclusions? This may be beyond the scope of the paper but for black box models that were in the comparison, perhaps attribution methods, like Shap, could be sufficient for seeing whether you can draw similar conclusions (or point out weaknesses in their explanations). 

- For multi-task learning, it's unclear whether connecting each MLP across tasks leads to lower interpretability  as there are now a linear combination of different non-linear functions for each feature -- not just one. The examples show the prediction of each task, but that doesn't necessarily highlight interpretability. The improved performance may be benefiting from the increased expressivity of the model and not necessarily from the fact that it is in a multi-task learning paradigm. A nice control experiment would be to keep multiple MLP modules for each feature (as if multi-tasking) but train on a single task. I suspect the model would now be able to capture the complex shapes that it was unable to with the original single-task NAM. If this experiment works, I think it would be beneficial to compare the interpretability of the quasi-multi-task vs single-task NAM. 
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper proposes Neural Additive Models (NAMs), a linear combination of neural networks that can be used as a highly accurate and interpretable model for tabular data. In order to account for the potentially detrimental bias of NNs to be smooth, the authors introduce an exp-centered hidden unit which enables the NN to better fit jagged shaped function, and compare this NAM to other tree-based and DNN based models. NAMs are able to achieve comparable accuracy to tree-based models with the added advantage of being easily adapted to the multitask setting, which can be helpful in increasing accuracy, discovering bias, constructing more helpful models for fields which need interpretability like healthcare. 
main_review: This paper is incredibly well written, easy to follow, and has some of the best visualizations I have seen. The motivation and core method are clear and the analysis was quite thorough and seems like it could be easily reproduced. Furthermore, the applications to things like covid-19 and COMPAS are well thought out and have impressive results. Overall a very fun read and seems particularly well suited to the NerulIPS audience. 

My only concerns are relatively small: 

(1) In section 2.1 the authors mention that they altered the weight initialization to better fit jagged curves; I would love to see a plot showing the difference in curves with this new initialization as well as the traditional weight initialization (similar to figure 2). I looked in A.3 but that section seems to be empty. 

(2) While the plots in section 2.1 are convincing that these ExU networks are better for jagged curves, it would be nice to add them as another benchmark in section 3 

Other small nits:

* Units of median income are missing in Fig 6
* line 327 I believe should be "the public has different..."
* a period needs to be added at the end of the conclusion
limitations_and_societal_impact: The authors addressed how this method may not be suitable for certain applications, as they may need a different type of interpretability. One other thing I would think would be an issue is the amount of resources needed/throughput, although I have not benchmarked this myself.
review_title: null

Meta-review: The reviewers were overall very enthusiastic about this paper, highlighting its novelty, and clarity in terms of writing and visualizations. There are a few changes that I expect you will make based on your responses to the reviewers.
Decision: Accept (Spotlight)