Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: High Potential (HP): a submission which meets the reviewing criteria and has potential to make an impact on the field
review_content: summary_of_contributions: This paper proposes a novel way of using topic modeling to compare the preferences of different pretrained models
strengths_and_weaknesses: This paper proposes TopEx, a method to condense feature attributions into a model-independent explanation using topic modeling. The approach is to identify important words in the context and checking the topics where these words belong to. By using a model-agnostic approach like this, the authors observe that different models assign different importance to various parts of the context. 

It's quite a simple and effective approach to get insight into how pretrained models work. There is no automation and in order to get the insight one has to look into examples case by case. Nevertheless it's an additional qualitative analysis of these models. 

suggested_changes: Question: it is not clearly explained how "word-level importance scores for each model" is computed. Is it the likelihood that the model assigns to each word at generation? How do the authors get word-level importance from subword-level importance?
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: An interesting paper for model comparsion
strengths_and_weaknesses: This paper introduces TopEx, an explanation method that facilitates fair comparison of language models by using model-agnostic topics. The paper demonstrates how TopEx can identify similarities and differences between DistilRoBERTa and GPT-2 across a range of NLP tasks.

Strengths:

1.It is an interesting approach to use model-agnostic topics for model comparison.

2.The paper includes several pivot experiments.

Weaknesses:

1.It is not easy to evaluate the impact of the model comparison on downstream tasks or model interpretability.

2.The technical novelty is limited, and the paper could benefit from the use of stronger topic models.
suggested_changes: 1.It is not easy to evaluate the impact of the model comparison on downstream tasks or model interpretability.

2.The technical novelty is limited, and the paper could benefit from the use of stronger topic models.
review_title: null

Meta-review: metareview: This is a nice, simple, and effective approach to provide explanation of model's decisions using topic modelling. The approach is to identify important words in the context and checking the topics where these words belong to. They use a model-agnostic approach like topic modelling and observe that different models assign different importance to various parts of the context. The paper is well-written and the hypothesis is supported by the experiments. 
Some clarifications is needed (see reviewers' comments, for instance clarifying how do the authors get word-level importance from subword-level importance?) but overall a valuable work.

confidence: 4: The area chair is confident but not absolutely certain
Decision: Invite to present