Paper: SKILL-MIX: a Flexible and Expandable Family of Evaluations for AI Models (ICLR-2024)

Reviewer comment (Theory):
Section 3.1 (and the appendix) describe how the authors pick the skills by hand based on textbooks on logical reasoning, rhetoric, and theory of mind. But what is the justification for using these types of skills? For example, there are likely many skills not related to any of those topics which still require some sort of intelligent capability to combine meaningfully. The authors should explain this better.

Author response:
We apologize for any confusion. The evaluation tests skills that are important in intelligence but is not meant to be a complete test of intelligence. Please see the response to all reviewers Q3.


In Appendix C.2, we discuss details of our skill selection process and how we remove skills from the original large set due to various reasons. Conceptually, the choice of topic should not affect the use of language skills. The capability to combine seemingly unrelated skills and topics is indeed what Skill-Mix aims to test.

Reviewer comment (Experiments):
One minor issue is that the models picked do not necessarily need to be instruction-finetuned, as regular, non-instruction-finetuned language models are still able to follow instructions in the prompt. In fact, given that some recent work has demonstrated that instruction-finetuning on smaller models reduces their ability to produce as diverse of a response distribution, it could artificially hurt the performance of the selected open-source models in evaluation due to the selected topics occurring with a Google n-gram probability of ~1e-6 and them likely being fine-tuned on a much smaller dataset that may not include the selected topics/skills.

Author response:
We agree that the instruction following can work reasonably for many tasks in the base models. But Skill-Mix is  more complicated. Despite multiple attempts we were unable to get any reasonable performance on non-instruction tuned models, especially small 7B models. We would welcome any suggestions (e.g., prompting strategies)!

Reviewer comment (Evaluation):
What is the variance among the automatically graded scores? It is stated that human variance is higher than the automatic ones, but neither seem to be presented anywhere.

Author response:
The standard deviation of GPT-4 grading is 0.050 (averaged across all points), which is much smaller than 0.261, the standard deviation of human grading as reported in Appendix B. The standard deviation of LLaMA-2 grading is 0.105, meaning it is noisier than GPT-4, but still much more stable than human grading.

Reviewer comment (Evaluation):
The analysis of Skill-Mix vs other benchmarks could be more comprehensive — it would be interesting to see correlation scores against other benchmarks to see how Skill-Mix differs overall with respect to individual LLM rankings.

Author response:
We agree with the reviewers that this would be ideal. However, none of the benchmarks have included all the models we tested, making it hard to compute the correlation. We find the ordering of the GPT family and the LLaMA family on Skill-Mix to be consistent with the ordering on other benchmarks. For ordering including newer models, we have pointed out all noticeable inversions in the paper.

Reviewer comment (Writing):
Intro: Why is a specific opinion from Hinton used as motivation for desiderata for the authors’ benchmark? While Hinton is an important figure in the field, a good justification should come from (1) an intuitive explanation of the problem, (2) consensus in the field for the problem, or (3) direct citation of evidence of the problem (or some combination of all 3). Citing a single person’s opinion is not generally good justification or intuitive for a reader presented with a new problem.

Author response:
We apologize for the confusion and have revised wording to following: “(d) capable of producing novel text generations corresponding to a mix of skills and a topic that it did not see in the training corpus.”  The ``stochastic parrots’’ question  (Bender et al., 2021)” has of course been debated for a few years. We cited the more recent (Hinton & Ng’23) conversation because Hinton emphasized that disagreements about this issue among experts is hindering a full discussion of AI safety. We would be happy to cite other sources if the reviewer has suggestions!

Reviewer comment (Presentation):
Table 1/2: Some sort of bolding on the best performing across each category for each k would be really helpful to parse all of these numbers.

Author response:
We thank the reviewer for their constructive comments, and have made appropriate changes in the revision (e.g., bolding on the best performing model; for this point in particular, we also include figures in the appendix to help visually assess how well models perform against each other for each metric).

Reviewer comment (Theory):
Section 6: Can the authors explain the derivation of the high probability statement for $O(T \log T + N \log N)$? Intuitively, it is quite surprising given “random prompts” to expose these low-probability skills and their topics.

Author response:
We apologize for the confusion. Here we are referring to a dataset that follows Skill-Mix, that is, the random prompts are sampled by uniformly picking $k$ skills out of $N$ total skills, and 1 topic out of $T$ total topics. Thus, each prompt reveals $k$ random skills and 1 random topic. By basic probability calculations, $O(T\log T+N\log N)$ random prompts should reveal all $N$ skills and $T$ topics with high probability even when $k=2$.

Reviewer comment (Evaluation):
I'd love to see some correlation of the presented results with human spot-checking and evaluation and if such correlation does not exist, discussion on what might be the reason

Author response:
We thank the reviewer for bringing up this point. In Appendix B, we mentioned that the difference between GPT-4 grading and the average human grading is ~0.25, meaning that the human and GPT-4 agree on most of the points. We will include details in the final version.

Reviewer comment (Experiments):
I'm curious to see how the experiments with fine-tuning while releasing 10% of skills pan out.

Author response:
We include some preliminary results on finetuning LLaMA2-7B on 50% skills and topics, and observe improvements across the various Skill-Mix metrics. We are excited to release a more detailed exploration of this in future work! Please also see our response to all reviewers, Q2.

Reviewer comment (Experiments):
Do you think that instead of have one strong evaluator, you can have a mixture of experts each capable to evaluate a single task exceptionally well?

Author response:
This is a good point! If one decides to fine-tune the grader, it is unclear to us whether it is better to fine-tune one model or use a mixture of fine-tuned models. One concern of using a mixture of experts is that some experts may be harsher than others. However, in terms of human grading, we believe it would be more efficient to split skills and assign them to specific graders. We leave this a future work to explore.

Reviewer comment (Writing):
Can you please provide reference for "While both models are creditable graders, they, like all current LLMs, were unreliable at simple arithmetic, which is important for calculating a total score."

Author response:
We provide some references below. We also include our experience in Appendix C.3.3.

[1] Exposing Attention Glitches with Flip-Flop Language Modeling
Bingbin Li, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

[2] Table 10, Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, Denny Zhou

Reviewer comment (Reproducibility):
In the current form, skill-mix defends against 'cramming' by proposing a held-out private test set of skills and topics. However, any evaluation design that relies on private evaluation data is fundamentally flawed - the field of LLM evaluation design needs to learn from cryptography algorithm design on this count. This is also problematic from an open science perspective - making it harder for peer review to evaluate the efficacy of the evaluation framework for instance.

Author response:
> no reason to believe that, once released, skill-mix won't also suffer the problem of 'cramming for the leaderboard' 

We release 10 skills and 10 topics (randomly sampled)  to avoid “cramming” for Skill-Mix. For more detailed explanations, please see our response to all reviewers, **Q2**.

Reviewer comment (Theory):
The theoretical justification for skill-mix is also unclear to me.

Author response:
> The theoretical justification for skill-mix is also unclear to me. For instance - does it make sense to combine any k of the N skills together, or are there some incompatible skill-tuples? The connections with pedagogy theory and the empirical observations from the Arora & Goyal 2023 paper are a good starting point, but it's not clear in what way combining skills relates to generalized intelligence. 

We apologize for any confusion and would like to emphasize that as of now, the goal of our evaluation is to test general-purpose text generation capability, rather than generalized intelligence. Please also see our response to all reviewers, **Q3**.

With respect to incompatible skill tuples: We  eliminated  skills that in our judgement inherently compose poorly with other skills (see Appendix C.2). It is possible that this intuitive weeding still left some pair of skills that don’t compose well. But this would equally affect the final performance number of all models, and not be expected to affect the ranking.

Reviewer comment (Experiments):
For instance - does it make sense to combine /any/ \(k\) of the N skills together, or are there some incompatible skill-tuples?

Author response:
> The theoretical justification for skill-mix is also unclear to me. For instance - does it make sense to combine any k of the N skills together, or are there some incompatible skill-tuples? The connections with pedagogy theory and the empirical observations from the Arora & Goyal 2023 paper are a good starting point, but it's not clear in what way combining skills relates to generalized intelligence. 

We apologize for any confusion and would like to emphasize that as of now, the goal of our evaluation is to test general-purpose text generation capability, rather than generalized intelligence. Please also see our response to all reviewers, **Q3**.

With respect to incompatible skill tuples: We  eliminated  skills that in our judgement inherently compose poorly with other skills (see Appendix C.2). It is possible that this intuitive weeding still left some pair of skills that don’t compose well. But this would equally affect the final performance number of all models, and not be expected to affect the ranking.

Reviewer comment (Theory):
The connections with pedagogy theory and the empirical observations from the Arora & Goyal 2023 paper are a good starting point, but it's not clear in what way combining skills relates to generalized intelligence. A clearer definition of 'skill' may help here.

Author response:
> The theoretical justification for skill-mix is also unclear to me. For instance - does it make sense to combine any k of the N skills together, or are there some incompatible skill-tuples? The connections with pedagogy theory and the empirical observations from the Arora & Goyal 2023 paper are a good starting point, but it's not clear in what way combining skills relates to generalized intelligence. 

We apologize for any confusion and would like to emphasize that as of now, the goal of our evaluation is to test general-purpose text generation capability, rather than generalized intelligence. Please also see our response to all reviewers, **Q3**.

With respect to incompatible skill tuples: We  eliminated  skills that in our judgement inherently compose poorly with other skills (see Appendix C.2). It is possible that this intuitive weeding still left some pair of skills that don’t compose well. But this would equally affect the final performance number of all models, and not be expected to affect the ranking.

Reviewer comment (Experiments):
Some 'late breaking' paper results are missing from the submitted version (e.g. footnote 3, Sec. 6) - the paper will be strengthened with the inclusion of these results in a future version.

Author response:
> Some 'late breaking' paper results are missing from the submitted version (e.g. footnote 3, Sec. 6) - the paper will be strengthened with the inclusion of these results in a future version.

We thank the reviewers for pointing this out, and have updated these results in our revised version of the paper. In particular,
- For k = 5 and k = 6, we show that GPT-4 is generating correct answers a good fraction of time, and a reasonable portion (say more than 1/3) of these correct answers contain combinations of skills and topics that have never jointly appeared in the training. 
    - We verify this claim by providing upperbounds on the average frequency of skills, the average frequency of topics, and the number of sentences in the training corpus (see Sec. 6 and Appendix D). 
- We include preliminary finetuning results for LLaMA2 7b in Appendix E.

Reviewer comment (Evaluation):
(End of Sec. 5) "Difference between model ranking on popular LLM leaderboards vs. SKILL-MIX provide evidence of "cramming for the leaderboard", further validating that SKILL-MIX is a good evaluation benchmark" - the conclusion here doesn't follow from the premise.

Author response:
>  (End of Sec. 5) "Difference between model ranking on popular LLM leaderboards vs. SKILL-MIX provide evidence of "cramming for the leaderboard", further validating that SKILL-MIX is a good evaluation benchmark" - the conclusion here doesn't follow from the premise.


We apologize for any confusion. We meant to convey that we believe Skill-Mix is a good evaluation benchmark because model rankings on Skill-Mix coincide with anecdotal human experience with these models (and that this is not the case with model rankings on e.g., Open LLM). Many derivatives of Llama2 models scored higher  than the Llama2 model on the Open LLM leaderboard but performed much worse on Skill-Mix, suggesting their general-purpose language skills were harmed along the way.

Reviewer comment (Evaluation):
It is great to see some human experiments done to compare against the automated grading scheme (however I note the lack of IRB or ethics approval in the paper related to human experiments). However, I note that variance in human grading responses (e.g. see end of Sec. 4) shouldn't automatically be interpreted as lower quality of responses - the variance can often be a signal, not noise. For instance, this could indicate combinations of skills and/or topics which are perhaps less compatible or sensible to query LLMs with.

Author response:
> lack of IRB or ethics approval

We thank the reviewer for bringing up this point. The human graders were carrying out research too and we understand IRB is not required in such cases. The final version will fully explain this status.