Model-diff: A Tool for Comparative Study of Language Models in the Input Space
Authors: Weitang Liu, Yuelei Li, Zihan Wang, Ying Wai Li, Jingbo Shang


Reviewer paper summaries: This paer proposes a new framework called Model-diff that avoids the bias that can be introduced by using datasets by sampling the entire input space for comparative analysis between language models, thus providing a more comprehensive assessment of the predictive similarities and differences of the models.

1. The paper addresses the research question of comparing the performance of two models by introducing a new framework - Model-diff - without using external models or benchmark datasets.
2. Model-diff is a sampling based comparative analysis framework which works by leveraging the output distributions and the corresponding representative inputs of the two models to understand how the models agree/disagree with the prediction in their own meaningful input space. This means that we do not have to rely on benchmark datasets or external generative models to generate data - which can bias comparison results and make it harder to obtain a comprehensive comparison.
3. Based on an in depth experimental section using both toy models and real world model (GPT-2 small, GPT-2 medium, Llama and Llama-2), the authors showcase the effectiveness of how Model-diff can be used to compare model iterations and differentiate between actual model improvements and noisy alterations.
Summary of strengths: The Model-diff framework does not require a specific dataset, and avoids the bias of dataset selection by sampling the entire input space to perform model comparisons.

Model-diff uses only the model's own output distribution for comparison, avoiding biases that may be introduced by external models.

1. Model-diff presents a method to compare two models in their respective input spaces without the need for benchmarking datasets or external judge models (both of which can introduce bias in the analysis).
2. Model-diff also offers insights into comparing model iteration (newer versions of open source models) and identify whether these models are sufficiently improved compared to their original counterparts, or merely noise-altered versions of the original models.
Summary of weaknesses: Analytical results of the Model-diff framework are highly dependent on the performance and accuracy of the sampler, and the quality of the sampler directly affects the accuracy and reliability of the analysis.

1. Although the theoretical premise of the paper is sound, it is a bit unclear to me how we can use this scalably for larger models and larger sequence lengths. The experiments presented in the paper are all done with <= 100 tokens, while using much larger sequence lengths is commonplace. As the authors also point out, more research into sampling methods to better estimate the output distribution would help with the adoption of this method. It would be great if the authors could provide guidance/scaling laws of how their analysis scales with an increase in sequence length, both in terms of compute and statistical significance of the results.
Comments, suggestions and typos: Question: Could you tell the difference between "LMdiff: A Visual Diff Tool to Compare Language Models"?

Why choose comparing the negative log likelihood (NLL) differences between the two language models over the entire input space?

How about multiple model comparsion (models >3)?

Would it be possible for the authors to release the associated code to reproduce their experiments? This would greatly help further this research area.
Overall assessment scores: 3

3