Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes to make Generalized Additive Models (GAMs) scalable, by tensor rank decompositions of polynomials. Specifically, the traditional GAMs are re-written into the tensor computation form. The weight matrices are then processed with rank decomposition, making the GAMs a series of inner products and are more computationally efficient. The authors also propose several tricks to help learning, including feature rescaling, sharing basis across classes, and extending GAMs with non-linear operations. Experiments under different data domains are conducted, including MLPs and CNNs. The experiments make use of the concept bottleneck backbone to further improve its performances. Human subject evaluations are also included since this paper is related with interpretability. This is a well-written paper, with in-depth understanding of GAMs and comprehensive experiments.
strengths_and_weaknesses: Strengths
1. The paper addresses an important problem in interpretable machine learning, i.e., how to build efficient and effective models besides interpretability. The paper chooses GAMs as prototypes, making a smart application of tensor decomposition to make GAMs more efficient. The paper is very well-written.
2. The paper considers a series of extensions for the proposed SPAM models. These extensions are consistent with the core contribution.
3. Comprehensive experiments are conducted. With both quantitative analysis and human studies.

Weaknesses
1. I would avoid adding "fully-interpretable models" in the abstract. Is SPAM really "fully-interpretable"? I think "inherently interpretable" could be better.
2. I think adding some more analysis about the learned SPAM models (i.e., which features or feature interactions are important) could help readers understanding what has been learned inside models for some of the tasks.
questions: Please see the Weakness part.
limitations: The paper proposes a series of extensions of SPAM to address some possible limitations.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes SPAM which extends Generalized Additive Models (GAM) with polynomials. SPAM learns a polynomial model that can capture any-order interactions among features. The authors leverages the proved symmetric property and assumed low-rank property of the weight matrices to simplify the optimization problem. For multiple-class classification tasks, the authors propose to use shared bases to further speed up training. The authors show under some assumption, the generalization error of SPAM scales exponentially w.r.t. the chosen low-rank order. The authors conduct experiments with several datasets and show that proposed SPAM achieve better results than various of baseline models. The authors also conduct human evaluation on how good SPAM is comparing with a black box model and the results show SPAM achieve much better results than using a black box model with post-hoc explanation.
strengths_and_weaknesses: Strengths:
- The idea of using polynomials to improve GAM is novel. The authors address the challenges of using polynomials well (mainly, scalability) by leveraging some nice properties of the problem.
- The authors provide thorough analysis for the proposed models. The charaterization of the generalization error, the geometric rescaling, data preprocessing, and shared bases idea for multi-class problems help the readers get a deeper understanding of the proposed model.
- The experimental section is thorough. Other than regular comparsion with baseline methods, the authors check the effects of order, sparsity assumtion for higher order interactions and verify the generalization error empirically. The human evaluation setting is novel and provide more evidence on why a fully-inpretably model might be better.

Weaknesses:
- For Table 1, I think the authors can include results for NA^2M since SPAM at lease uses order-2 interactions. 
questions: - Can you show some empirical resutls about the effects of using shared bases? I'm curious to see how much that trick would improve. Do you think it makes sense to use a shared bases for $u_1, u_{2i}, ..., u_{ki}$ in equation 3?
limitations: - I think it's worth noting the running time of the SPAM on different datasets to help understand the scalability of SPAM. I guess for extremely high-dimension datasets (like high-resolution images or videos or texts), SPAM might not be able to train efficiently. It would be great if the authors can provide some discussions on this part.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a fully-interpretable model which is in the line of generalized additive models (GAMs). In order to let the fully-interpretable model scalable, the authors reduce the conventional GAMs to a polynomial one. By doing tensor decomposition, the proposed model can be effortlessly scalable and model higher-order feature interactions without a combinatorial parameter explosion. Experimental results validate the effectiveness of the proposed method both in the perspectives of prediction and human evaluation. Theoretical results also guarantee their model’s performance. 
strengths_and_weaknesses: Strengths: 
1. The technique of this paper is sound. The authors provide a clear view and logic of the proposed idea with the basic form of the polynomial additive models and how it can be scalable by low-rank decompositions. The improving on geometric rescaling and shared bases for multi-class problems also make sense.
2. So is the clarity of this paper according to the above comments. 
3. The human subject evaluation is also convincing. 

Weaknesses: 
1. It would be better to provide the details in the human subject evaluation. For example, the authors could provide the explanation provided by the models in the bird prediction task. Then we may have a better understanding on how the model is interpreted. 
2. The significance of the proposed method seems fair. The proposed method is not post-hoc, which means the prediction and interpretation all count on the proposed method. Table 1 shows pretty good prediction results where the proposed method can mostly outperform all the baselines including DNN. But it seems a bit tricky since the DNN is just a simple MLP. Outperforming an MLP may not be a big advantage. In addition, what are the reasons to choose the four datasets presented in the paper? I am concerned that the results are cherry picking so that the proposed method cannot work well in prediction. If so, a slightly better interpretable ability may not be worth applying to real world tasks. Good prediction plus post hoc explanation may be sufficient.

questions: What if we continue to increase the length of explanation? Is it the 7 global optimal in Figure 2C?
limitations: Yes.
review_title: null

Meta-review: metareview: The paper notes that polynomial functions are inherently interpretable models, and takes algorithmic advantage of the connection between polynomials and tensors by learning the coefficients of the polynomials using a low-rank tensor factorization. The resulting algorithm is shown to outperform prior SOTA interpretable models and to match blackbox model performance on several data sets. 
confidence: Certain
Decision: Accept