Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: This paper studies the impact of the number of samples on the behavior of SHapley Additive exPlanations (SHAP), an approach to interpret predictive models. The paper studies the role of the number of samples used to estimate SHAP and provides guidance on how to estimate the number of samples needed to use SHAP in a meaningful manner
strengths_and_weaknesses: **Strengths**

- The paper studies the number of samples needed in a systematic manner. This is a good problem to solve especially as there is no consensus on what constitutes a correct setup
- The initial experiment with number of samples and per variable variance provides evidence that there is a problem related to the correct use of SHAP measure. A deeper study can lead to recommendations that can be very useful to the ML community, specifically for researchers/practitioners working on model interpretability
- Code provided to support reproducibility 

**Weaknesses**

- The reviewer is a novice with SHAP values. So I was not able to see a clear connection between the number of samples experiment and the BLEU/Jaccard scores shown in Tables 1 and 2. I hope the authors would consider making the connection between the main idea and these clear to help a broader audience appreciate their work
suggested_changes: - I observed that the values in Table 1 and Table 2 are identical. Is this correct or expected?
- The paper is off to a great start. Adding connections between results in main text (Table 1 & Table 2) and SHAP values seen in experiments would make the paper accessible to a broader audience.
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: The article considers the impact of background dataset size for computing Shapley values in the SHapley Additive exPlanations (SHAP) method. The authors provide preliminary experiments to understand this impact along with some pointers for selecting the background data size.
strengths_and_weaknesses: The article considers the impact of background data size while using SHapley Additive exPlanations (SHAP) for explaining the impact of input variables in Neural Networks.

Strengths:
- As expected, higher background dataset sizes tend to be stable by virtue of lower variability in their estimates.
- The authors also evaluate the impact of background dataset size on SHAP by comparing two predicted variable impact rankings using the BLEU index and Jaccard index that can compare same-length sequences. Generally, both measures provide similar assessments that increasing background size is good.
- Interestingly, fuzzy measures where an exact match of rankings is not important tend to perform better than exact rankings. This indicates some limitations of SHAP and provides a good way to measure SHAP or similar methods.

Weaknesses:
- The paper demonstrates some insights into SHAP. However, there are few unique actionable insights (besides the pilot study) for leveraging or improving SHAP. Furthermore, there does not seem to be a way to use SHAP for smaller datasets. Further details in the suggested changes.
- Minor concern: Are the values in tables 1 and 2 for the two indices supposed to be the same?
suggested_changes: - Extending the study to study simple methods that can leverage SHAP for smaller background datasets can be useful. Can fuzzily similar variable rankings be leveraged to get a better estimate of the true variable rankings?
- Experiments can be extended to larger and more complex networks. Furthermore, new experiments should focus on the small background dataset scenario.
- Comparing SHAP rankings with other explainability rankings (not based on SHAP) will be useful for seeing how well SHAP performs.
- Finally, new insights (without pilot studies) into the impact of background size will significantly improve this article.
review_title: null

Decision: Invite to revise