Auxiliary task demands mask the capabilities of smaller language models

Authors: jennifer  hu
michael c frank
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-22 18:53:30

Abstract: Developmental psychologists have argued about when cognitive capacities such as language understanding or theory of mind emerge. These debates often hinge on the concept of "task demands" -- the auxiliary challenges associated with performing a particular evaluation -- that may mask the child’s underlying ability. The same issues arise when measuring the capacities of language models (LMs): performance on a task is a function of the model's underlying knowledge, combined with the model’s ability to interpret and perform the task given its available resources. Here, we show that for analogical reasoning, reflective reasoning, word prediction, and grammaticality judgments, evaluation methods with greater task demands yield lower performance than evaluations with reduced demands. This "demand gap" is most pronounced for models with fewer parameters and less training data. Our results illustrate that LM performance should not be interpreted as a direct indication of intelligence (or lack thereof), but as a reflection of capacities seen through the lens of researchers' design choices.

Main paper: 
 introduction: 
 A shared goal in psychology and artificial intelligence is to ascribe cognitive capacities to black-box agents. For example, we might be interested in whether a young child has a "theory of mind", or whether a language model (LM) has the ability to distinguish grammatical and ungrammatical sentences. The trouble is, although we would like to infer an underlying psychological construct, we only have access to specific observable evaluations -a child's ability to answer a question about a character in a story, or a model's performance on a syntax benchmark. 1 Inferences from evaluations to constructs are ubiquitous in developmental psychology (Frank, 2023a) , and cognitive evaluations of artificial models are beginning to follow similar principles (e.g., Momennejad et al., 2023; Ivanova, 2023) .For both humans and machines, making inferences about failures is especially tricky, because failure on a task does not always indicate the absence of the underlying capacity. In developmental psychology, children often fail due to task demands: auxiliary challenges unrelated to the capacity of interest (e.g., Keen, 2003; Carruthers, 2013; Turan-Küçük & Kibbe, 2024) . Children of a given age may succeed in a simple version of a task, while failing at a more complex version that incorporates other demands. Sometimes the issue is as basic as children not understanding the question being asked, while at other times task demands can include issues like remembering multiple pieces of information or inhibiting a response. Both methods apparently measure the accuracy of target word prediction, but the high demand setting imposes additional auxiliary demands. B: Hypothesized pattern of results if task demands asymmetrically affect less capable agents (e.g., younger children or smaller models). C: Signature "demand gap" produced by hypothesized pattern in B.Here, we argue that task demands also play an important role in determining evaluation success or failure for LMs, especially when comparing models of different capacities. As a conceptual illustration, Figure 1A depicts hypothetical task demands faced by humans and machines for two different ways of evaluating word prediction ability. The high-taskdemand evaluation presents a metalinguistic prompt, which explicitly asks the agent for a prediction. A correct response for this stimulus requires interpreting the question, accessing internal predictions for that target sentence, and producing them after the prompt. In contrast, a lower-demand design for humans might be the measurement of reaction time upon hearing the target word ("sugar"); for LMs, a lower-demand task would be a readout of the probability of the target word. While both of these evaluations apparently measure the accuracy of the LM's predictions, models might fail in the high-demand situation due to difficulty with task demands unrelated to the accuracy of their actual target word prediction. Indeed, recent studies have demonstrated performance gaps between higher-and lowerdemand evaluations in a variety of linguistic domains (e.g., Hu & Levy, 2023; Hu et al., 2024; Kauf et al., 2024; West et al., 2024; Tsvilodub et al., 2024) .Our primary question is not whether these performance gaps exist, but whether task demands asymmetrically affect less capable language models, mirroring the findings in children. A null hypothesis might be that a low-capability model -for example, one with fewer parameters or less training data -would perform equally poorly on all cognitive evaluations, regardless of the method. Inspired by the task demand debate in developmental psychology, we instead predict that less capable agents should suffer more from task demands than more capable agents (Figure 1B ). If borne out, this prediction would produce a signature "demand gap", which would be larger in less capable LMs (Figure 1C ).We present experiments demonstrating this signature pattern -a larger demand gap for less capable models -across a variety of tasks and open-source LMs. We investigate two evaluation contrasts: production vs. forced choice, and metalinguistic judgment vs. probability measurement. For each of these contrasts, we measure the demand gap between the higher-and lower-demand evaluation methods on multiple cognitive domains, collectively covering analogical reasoning, reflective reasoning, word prediction, and grammaticality judgments. We find evidence that demand gaps get smaller as LMs increase in size (i.e., number of parameters) and training time. Our findings suggest that task demands can mask the abilities of less capable models, illustrating how inferences about LM abilities depend on our choice of evaluation metrics.The paper is structured as follows. In Section 2, we more fully describe the concept of task demands and how it relates to other issues in LLM evaluation. We describe the logic and methods behind our experiments in Section 3, and present the results in Section 4. Finally, in Section 5 we conclude and discuss broader implications for NLP and cognitive science. 
background and related work: 
 Psychology focuses on connecting observable measurements to latent (unobserved) psychological constructs. A valid measure is one that appropriately measures the construct (Cronbach & Meehl, 1955) . Task demands are one of many different factors that can undermine validity. In particular, the presence of auxiliary demands in a task means that the inference from observed performance to construct is no longer "pure." Reducing these demands can enable a more direct inference about the latent constructs of interest.While the psychology literature does not provide precise definitions of what counts as a task demand, informally a task demand can be thought of as a feature of the task that imposes challenges unrelated to the capacity that the task is designed to measure. For humans (and particularly in children), these challenges might be related to processes such as memory, attention, or inhibition, which are involved in performing the task but separate from the target capacity. It is unclear how these kinds of challenges manifest in LM computation, and our work aims to explore several such possibilities.An evaluation with fewer task demands is not necessarily easier or more lenient. Instead, a better gloss might be that it is a purer task, in the sense that it better measures the targeted construct. Consider an evaluation of arithmetic ability: the size of the numbers being manipulated and the range of operations would likely change the overall difficulty of the evaluation. In contrast, the use of word problems, pictorial representations, or framing in terms of real-world objects would all change the task demands of the evaluation, even if the difficulty of arithmetic components stayed constant. Some of these changes might serve to make the task easier -for example, if they engaged familiar content (Lampinen et al., 2024)while others might make the task harder (at least for some models).Our work here builds upon the field of LM evaluation -specifically, research on links between evaluation metrics and underlying constructs, or in other words "machine evaluation validity". For example, Schlangen (2019) points out that individual evaluation tasks are often only loosely related to the underlying construct that researchers want to engineer into their system. Other work has questioned the validity of inferences from forced choice metrics (Tsvilodub et al., 2024; Schaeffer et al., 2023) , and several studies have noted the misalignment between production/forced choice (Lyu et al., 2024; Röttger et al., 2024; West et al., 2024) and between prompting/measurement of underlying probabilities (Hu & Levy, 2023; Hu et al., 2024; Kauf et al., 2024) . A related line of work asks how to design "species-fair" comparisons between humans and models (Firestone, 2020; Lampinen, 2023) .These specific investigations of validity are part of the broader effort to design robust LM evaluations. The results of any evaluation depend in part on design choices, which necessarily reflect a researcher's broader goals or theoretical commitments. These design choices have been widely discussed (e.g., Linzen, 2020; Bowman & Dahl, 2021; Raji et al., 2021) , including factors such as size and scope (Srivastava et al., 2023) , dynamic updating (Kiela et al., 2021) , and adversarial approaches (Nie et al., 2020; Sakaguchi et al., 2021) . To our knowledge, most of this work has not specifically analyzed the interaction between evaluation method and model capability that is our focus (but see Schaeffer et al., 2023) . 
methods: 
 Our main research question is whether less capable language models are more sensitive to task demands. If this is the case, then we would expect to find the pattern in Figure 1C : the performance gap between low-and high-demand tasks should be smaller for more capable (2) *Rachelle had bought that chairs.Table 1 : Overview of task contrasts and cognitive domains tested in our experiments.models. In the remainder of this section, we discuss the evaluation contrasts, cognitive constructs, and models tested in our experiments (see Table 1 for summary).We focus on two evaluation contrasts that are relevant to most behavioral LM evaluations.When designing an evaluation, researchers have the choice to assess the LM in a generative (production) or discriminative (forced choice) setting. Independently, there is also the choice of formulating the task as a prompt or a direct measurement of string probabilities. Both of these decision points can be formulated as a contrast between methods with higher and lower levels of task demands, which we discuss in more detail below.Production vs. forced choice. The first contrast compares production of the correct answer (higher task demands) versus forced choice over a fixed set of answer options (lower task demands). Notably, children's ability to produce a certain linguistic form tends to lag behind their ability to comprehend that form (e.g., Hendriks & Koster, 2010) . Forced choice paradigms have revealed knowledge in children that would not otherwise be observed through production. For example, 6-9-month-old infants can direct their gaze to named objects, before being able to talk themselves (Bergelson & Swingley, 2012) .It is not obvious that this asymmetry would also apply for LMs, for which the task of generating linguistic output -which may impose auxiliary demands on children, including planning motor actions associated with speech production -is the primary rationale. But even without these demands, forced choice requires less precise estimates of a probability distribution than correct production. Conditioned on a context (e.g., the statement of a reasoning problem), the LM's ability to solve the task is encoded in its distribution over subsequent tokens. This distribution is the same, regardless of whether the LM is evaluated through forced choice or production. Production, which involves sampling an output from the distribution, might not demonstrate the model's abilities because of auxiliary factors (e.g., the decoding method or low absolute probabilities), whereas targeted comparisons of answer options might reveal distinctions that are present in the model's probability distribution but would not be sampled. In this sense, production and forced choice can target the same cognitive construct, while imposing different kinds of auxiliary demands.2 Metalinguistic judgment vs. probability measurement. The second contrast compares two ways of measuring linguistic knowledge: a metalinguistic judgment versus a direct measurement of how likely or well-formed a linguistic expression is. From the developmental perspective, children can identify well-formed sentences of their language, without necessarily being able to articulate the underlying grammatical rules that govern these sentences. As previously argued by Hu & Levy (2023) , the probabilities assigned to strings directly reflect a model's linguistic generalizations, whereas metalinguistic prompts impose additional demands such as correctly interpreting the question (see Figure 1A ).For the production versus forced choice contrast, we evaluated LMs in two cognitive domains: analogical reasoning and reflective reasoning. Reasoning abilities have been of great recent interest in LM evaluation (e.g., Kojima et al., 2022; Webb et al., 2023; Chang & Bergen, 2024) , and the test sets we used feature well-motivated answer option sets that enable forced choice evaluation.Analogical reasoning. We used the digit matrices task introduced by Webb et al. (2023) to measure LMs' ability to perform analogical reasoning. Each item presents a sequence of digits in matrix format, and the model is evaluated on its ability to predict the continuation of the sequence. The relationships between the sequence and continuation involve a variety of transformations and logical patterns; we refer readers to Webb et al. (2023) for details.We followed Webb et al.'s protocol for scoring. For the forced-choice evaluation, we compared the log probability assigned by the model to each of the possible answer choices (summed over tokens). The set of answer choices for each problem was defined by Webb et al. (2023) . For the production condition, we truncated models' responses where a closing bracket is generated, and then evaluated the response based on whether it contained the correct set of digits. We report results averaged over all problem types.Reflective reasoning. To evaluate LMs' reflective reasoning abilities, we used the cognitive reflection tests (CRTs) described by Hagendorff et al. (2023) . CRTs (Frederick, 2005) are problems that have a reflexive, intuitive solution, which must be overridden through reflective problem-solving to arrive at the correct answer. The CRTs used by Hagendorff et al. (2023) cover three conditions which probe intuitive biases observed in humans.To evaluate models in the forced-choice condition, we compared the mean log probability assigned to the correct and intuitive answers. We average probabilities over tokens because answer options can differ in length (e.g., "$10.00" vs. "$1.50"). The model succeeds on a given item if it assigns higher probability to the correct answer than the intuitive answer, conditioned on the problem statement. To score the freely generated responses, we manually inspected the outputs and labeled whether they corresponded to the intuitive answer, correct answer, or neither. Results are averaged over the three tested conditions.For the metalinguistic judgment versus probability contrast, we evaluated LMs on word prediction and grammaticality judgment because they are the clearest cases where metalinguistic prompting differs from established ways of using direct measurements.A natural locus for a gap between direct measurement and metalinguistic judgment is word prediction. Since the direct evaluation setting is analogous to models' autoregressive training objective, we can view these predictions as a "ground-truth" measure of models' underlying word prediction abilities, providing a useful reference for of the answer options in their study, which were designed to include adversarial "hard negatives". More broadly (and in keeping with our general theme), researcher goals (in this case, adversarial testing as opposed to capacity discovery) can modulate the difficulty of any evaluation method.evaluating metalinguistic abilities (Hu & Levy, 2023) . We evaluated models' word prediction ability using the LAMBADA dataset (Paperno et al., 2016) . The items consist of short multi-sentence passages, which are constructed such that correctly predicting the final word of the text depends on the entire context, and not just the final sentence. 3To evaluate models using direct probability measurements, we computed the log probability assigned by the model to the target final word conditioned on the prefix (summing over sub-word tokens). To evaluate models metalinguistically, we computed the log probability of the final word conditioned on the metalinguistic prompt shown in (1).( 2024). To evaluate models using direct measurements, we assessed whether the model assigned higher probability to the grammatical sentence than its ungrammatical counterpart. This method is designed to control for factors such as lexical items and length (see also Linzen et al., 2016; Marvin & Linzen, 2018) . To evaluate models metalinguistically, we prompted the model to compare the two sentences in the minimal pair:(2) Which of the following two sentences is more grammatically correct in English? Respond with 1 or 2 as your answer. Sentence 1: [sentence1] Sentence 2: [sentence2] Answer:For each minimal pair, we evaluated models on two versions of (2), swapping the order in which the sentences are presented. A model succeeds if it assigns higher conditional probability to the token "1" or "2" (whichever one corresponds to the index of the grammatical sentence). Accuracy is computed as the average success across these two orders.Note that we only tested a single prompt for each of the metalinguistic evaluation tasks.For results on a broader set of prompts, we refer readers to Hu & Levy (2023) . They tested models on 3 different metalinguistic prompts in both English and Chinese, and found similar qualitative findings across prompt types and languages.Recall that our main question is whether the demand gap shrinks over the course of model "development", or as its general capabilities increase (Figure 1C ). We tested two ways of operationalizing this concept for language models: varying the size (i.e., number of parameters) while keeping other architectural and training details as similar as possible, and varying the duration of training for a given model instantiation.This analysis is not meant to draw direct parallels between development in children and scaling in LMs: we don't claim, for example, that Llama-2 7B is a model of a young child and Llama-2 70B is a model of an adult. Instead, we use size and training as practical proxies for how LMs might gain resources that help with overcoming task demands. Table 2 : Models tested. For OLMo and Gemma, the first/second model size is associated with the first/second amount of training tokens. For Gemma and Mistral, the data cutoff date is not publicly known, but we assume it is before publication of the associated paper.Model size. We first use model size as a way of manipulating an LM's general capabilities. Models with more parameters are more expressive, suggesting that they might be able to handle auxiliary task demands that would be difficult for less expressive models (e.g., interpreting a metalinguistic question, or maintaining the result of a computation across intervening content in a prompt).We evaluated 13 open-source autoregressive language models using the Huggingface Transformers library. The models range in size from 1B to 70B parameters spanning across five families: deduplicated Pythia (Biderman et al., 2023) , OLMo (Groeneveld et al., 2024 ), Gemma (Gemma Team et al., 2024) , Llama-2 (Touvron et al., 2023) , and Mistral (Jiang et al., 2023) . Table 2 summarizes the sizes we tested for each model family, as well as other training details. We chose these models to cover a range of sizes, while also achieving reasonable performance (ruling out smaller models such as GPT-2) and ensuring reproducibility. Since our experiments require token-level logits, we could not evaluate closed-API models.Training time. Another way to manipulate the general capabilities of an LM is training time. While many studies examine emergence as a function of model size (e.g., Wei et al., 2022a) , a growing line of work is concerned with "developmental interpretability" (Hoogland et al., 2023) , or how abilities and structures emerge over the course of learning (e.g., Power et al., 2022; Simon et al., 2023) . For our experiments on training time, we evaluated OLMo 7B at 10 checkpoints over the course of training (including the final one).Across all experiments, we computed conditional and full-sequence probabilities using the minicons package (Misra, 2022) . All models were base models without any fine-tuning.Data contamination. Some of our tested datasets were publicly available before the data cutoff of all our tested models (LAMBADA and BLiMP), and all of our datasets were released before the publication of Gemma. This raises the concern of data contamination: i.e., that these datasets were seen during training. While this situation is not ideal, it is not a major concern because we are not interested in performance per se -rather, we are interested in the difference in performance across evaluation methods. We do not have strong expectations that data contamination would affect one method more than another. 
task demands vs. model size: 
 Figure 2 shows the results for our first tested contrast: production vs. forced choice. We first briefly discuss raw task performance. For the analogical reasoning domain, larger models generally achieve higher performance (compared to smaller models in the same family) under both evaluation methods (Figure 2A ). Interestingly, we do not find this pattern for the reflective reasoning domain (Figure 2C ), because the larger models have a strong preference for intuitive responses, whereas the small models slightly prefer correct > intuitive but have a much stronger preference for atypical answers (see Hagendorff et al. 2023 for details).Our main focus is the relationship between the demand gap (forced choiceproduction) and number of parameters (Figure 2B, D ). We measure the demand gap as the difference in log odds, computed from the accuracy scores. In both domains, the difference in log Published as a conference paper at COLM 2024 odds tends to decrease as the number of parameters increases, demonstrating the predicted signature pattern (Figure 1C ). This pattern holds for all model families except Pythia, whose log odds differences are relatively flat across model size.Figure 3 shows the analogous results for our second evaluation contrast: metalinguistic judgment vs. probability measurement. For word prediction as well as grammaticality judgment, the direct method generally yields higher performance than the metalinguistic method, in line with Hu & Levy (2023) (Figure 3A, C ). As before, we also find that the demand gap (directmetalinguistic) decreases as model size increases for both domains. This gap is measured as a log probability difference for the word prediction task (Figure 3B ), and log odds difference for grammaticality judgment (Figure 3D ). For the grammaticality judgment domain, we note that the log odds difference is fairly flat for the Pythia and OLMo families. The tested Pythia and OLMo models do not change in performance across sizes, and do not perform appreciably above chance (50%) under the metalinguistic method. It could be the case that due to their architectures or training, the "shrinking gap" effect might not be seen within the range of tested sizes, but could potentially emerge at larger sizes.To formally test our demand gap prediction (visualized in Figure 1C ) -i.e., that smaller models are differentially affected by task demands -we analyzed the interaction between model size and task demands. We used the lme4 package in R to fit the following mixed effects model for each of the domains with binary responses (analogical reasoning, reflective reasoning, and grammaticality judgment): correct ∼ size * evalMethod + (size * evalMethod | modelFamily)(1) We fit the following model for the word prediction domain:finalWordLogprob ∼ size * evalMethod + (size * evalMethod | modelFamily) (2) We found a significant interaction between model size and evaluation method for each domain except reflective reasoning (analogical reasoning: z = 2.631, p = 0.009; grammaticality judgment: z = 1.995, p = 0.0461; word prediction: t = 7.654; p = 0.005).Overall, our results provide evidence for the pattern illustrated in Figure 1C : the "demand gap" between high-demand evaluation methods and low-demand evaluation methods is smaller for more capable models, which we operationalized here using parameter count. Next, we turn to an alternate operationalization: training time of a single model.We evaluated intermediate checkpoints of OLMo 7B on analogical reasoning (production vs. forced choice) and word prediction (metalinguistic vs. direct).5 Figure 4 compares performance across the low-and high-demand evaluation methods across training time. In both domains, the low-demand method yields higher performance earlier during training. For analogical reasoning (Figure 4A ), the slope of improvement early during training is flatter for production than forced choice. For word prediction (Figure 4B ), the log probabilities achieved by the direct method quickly plateau at a high value, whereas they increase more consistently under the metalinguistic method.Again, we fit generalized linear models testing for an interaction between training steps and evaluation method. Here, there is no model family grouping factor. We fit the following models for analogical reasoning and word prediction, respectively: correct ∼ logTrainingStep * evalMethod (3) finalWordLogprob ∼ logTrainingStep * evalMethod (4) For both domains, we found a significant interaction between the number of training steps and evaluation method (analogical reasoning: z = 2.639, p = 0.008; word prediction: t = 24.41; p < 0.0001).In sum, evaluation methods with lower task demands can reveal abilities in a model earlier during training that might not otherwise be revealed by higher-demand methods. These findings complement the results from Section 4.1, which showed that low-demand settings can reveal abilities in smaller models that would not be revealed by higher-demand methods. 
discussion: 
 In studying the cognition of both humans and language models, one of the primary challenges is linking specific evaluations to more general constructs. Designing valid evaluations is thus critical for LM research. Here, we used a concept from developmental psychologyauxiliary task demands -to investigate the validity of a range of LM evaluations. We found that evaluations with higher demands led to lower performance, and especially so for LMs with fewer parameters or less training -a phenomenon we termed the "demand gap." Our findings also connect to emergence (Wei et al., 2022a) , and suggest that the scale at which particular abilities are observed in LMs is a function of both the evaluation that is being used and the auxiliary capability of the model to handle the task demands of that evaluation. These conclusions complement the prior findings of Schaeffer et al. (2023) .We investigated two evaluation contrasts in each of two cognitive domains, which establishes consistency across settings but are still only samples from the much broader literature on cognitive evaluation. More work will be needed to investigate the impacts of other popular cognitive evaluation techniques, such as in-context learning (Lampinen, 2023) or chain-of-thought prompting (Wei et al., 2022b) . Similarly, we made an effort to include a representative group of openly accessible LMs (Frank, 2023b) released in multiple sizes or with multiple training checkpoints. With the exception of the Pythia family, all models showed qualitatively similar results, suggesting that our findings are at least somewhat general. It is unknown, however, how our pattern might vary across a broader scale of models; we speculate that the order of magnitude less data used in training the Pythia models might have resulted in the somewhat different performance we observed.Our conception of what makes a particular task demand is informal. For example, the decompositions given in Figure 1A rely on intuitive concepts such as "question interpretation."In developmental psychology, we can reason about task demands using constructs such as memory, attention, or inhibition, supplementing with our knowledge about how these develop in childhood. In LMs, we judged task demands to be low when evaluations aligned with training objectives (as in measurements of token probability) or when they constrained the space of possibilities (as in forced choice tasks). A more granular notion of task demand in LMs might investigate the attentional and representational mechanisms triggered by different evaluations; we view this as a promising avenue for future work.Another interesting (and speculative) direction suggested by our work is using LMs to study task demands in humans. The relative ordering of performance in LMs under different evaluation methods could reflect more general properties about how probabilistic predictions or representations of knowledge manifest in different evaluation settings. These insights could help us generate predictions about what kinds of methods might place greater demands on children. Indeed, recent studies are consistently showing that probability measurements reveal knowledge that can be masked by metalinguistic methods, suggesting that direct predictions might be the most diagnostic of underlying ability.Finally, given the wide range of evaluation methods available to LM researchers, and the stark differences that can emerge across methods, a natural question is which method is the "right" one to use. When designing an evaluation, we encourage researchers to think about the validity of using a particular method to measure a particular construct of interest (Schlangen, 2019) . As our work demonstrates, an important step is to specify the auxiliary demands that the evaluation might impose on the model, and how these demands interact with one's broader evaluation goals (e.g., trying to be adversarial, versus trying to reveal knowledge). Conversely, researchers should interpret model performance not as a direct indication of some feature of intelligence (or lack thereof), but as a reflection of a model's true underlying capacities seen through the lens of researchers' design choices.Here, we use the term "evaluation" to encompass tasks, benchmarks, measures, and experiments, for both humans and machines.Forced choice evaluation does require researchers to design a set of answer options, which may introduce unintended artifacts (e.g., content or frequency effects). In line with this view,West et al. (2024) find that generative (production) performance is better than discriminative (forced choice) performance across a range of models and tasks. However, this finding could be due to the complexityWe use the test split pre-processed by OpenAI, as exposed by the EleutherAI/lambada_openai Huggingface dataset. We obtain the prefix and final target word by splitting on whitespace.We randomly sampleditems from each of the 13 categories, resulting in 650 items.We did not perform this analysis on the other two domains because the fully trained OLMo 7B model achieved poor performance under the high-demand evaluations (see Figure2Cand Figure3C). 
 
 Appendix:  


Decision: The paper argues that interpretation of LM performance should factor in task demands - factors extraneous to the ability being evaluated that might nonetheless influence model scores, such as the need to translate predictions across multiple different data formats. Specifically, the authors show that task demands have a particularly large influence in small LMs, potentially unfairly decreasing our impressions of how capable such systems are.
The reviewers noted many strengths:
- The paper shows how some ideas from psychology can be valuable for work about understanding LLMs. Reviewers noted that they found these ideas helpful and that others in the - NLP community would likely find them helpful too.
- The experiments are well-constructed and have a clear takeaway message (namely, that task demands have a particularly large influence in small models).
- The experiments are comprehensive
- The paper is well-written and enjoyable to read

Some important limitations were also noted:
- It is unclear what determines how high-demand a task framing is
- More exploration of prompting styles could have been helpful, given the sensitivity to prompts that LLMs display
- The competence/performance distinction has a complex history in connectionism that could have been helpful to discuss in more detail 
- The particular selection of task framings has some potential limitations - as noted by reviewer cyq6, some of the pairs that are studied might get at different competences from each other

All reviewers rated this paper above the acceptance threshold, most of them substantially so. I am recommending acceptance because this paper presents important ideas clearly and with convincing experimental support. The statistical test that the authors raised in a response would further strengthen the paper. To the extent that space permits, I encourage the authors to add to the paper the points they made in response to reviewer 11db about prompt engineering and in response to reviewer cyq6 about the competence/performance distinction and about the extent to which the different task framings target the same cognitive construct.

Reviews: review: paper_summary: paper_summary: This paper investigates how "task demands" in evaluating language models can mask their underlying cognitive capabilities. It argues that evaluations with greater auxiliary task demands beyond the targeted cognitive construct tend to yield lower performance, especially for weaker models. The authors adopt two evaluation contrasts to uncover the influence of task demands: 1) production vs forced choice, and 2) metalinguistic judgments vs direct probability measurements. These contrasts are applied to four test scenarios and five LLM families. The experimental results demonstrate that there is a "demand gap" where the higher-demand task results in lower performance compared to the lower-demand task, and this "demand gap" shrinks as model size/training increases.
strength_weakness: strength_weakness: Overall, this paper is good, but it appears to be under-explored or arbitrary on the following two points:

1. There is a lack of a clear definition and boundary of high-demand and low-demand tasks. What tasks are high-demand, and what are low-demand? Furthermore, what abilities are introduced by high-demand tasks that are not involved in low-demand tasks? Although the authors list some abilities like "interpreting questions", a complete and clear taxonomy or something similar is still missing. It is encouraged to supplement relevant psychological knowledge.

2. The selected evaluation methods (production, forced choice, metalinguistic judgment, probability measurement) and cognitive constructs (analogical reasoning, reflective reasoning, word prediction, grammaticality judgment) lack motivational explanations. It is unclear whether these selections are representative or can be generalized to other scenarios.
paper_summary: paper_summary: The authors study how task demands impact language model evaluation. Their approach is motivated from a developmental psychology perspective where it is well established that children might fail certain tasks due to higher task demands and not because they lack the capacity of interest. Applying this framework to language model evaluation, the authors ask whether task demands have a bigger effect on smaller models, thereby masking their “true capabilities”. 

The authors predict a demand gap, i.e., that less capable models (smaller models or those trained for less steps) suffer more from higher task demands, i.e., achieve lower performance, than more capable models. 

In the experimental section, the authors evaluate 5 different model families on 4 different datasets while controlling for task demand. The experimental findings are mostly consistent with the demand gap prediction. 

Finally, the authors discuss the implications of their work for LM evaluation, the study of emergence, and even developmental psychology.
strength_weakness: strength_weakness: I only have two minor concerns and they are not strong enough to serve as reasons for rejection. I will still mention them here. 

- The Pythia models clearly behave as an outlier for 3 out of 4 evaluations. The authors could do a better job at discussing and investigating what is going on here (they briefly speculate about the number of training steps but I’m not fully convinced by that).

- The meta linguistic evaluation considers only a single prompt (for each of the tasks where it is applied). It is however widely established that prompt engineering can have a dramatic influence on model performance. These results would be even more convincing if they would take different prompts into consideration.
questions: questions: **Evaluation methods**
- Are the answer options part of the input during forced choice eval? Or are you only extracting the logits from the answer tokens? This was not entirely clear to me. 
- It would be interesting to discuss the limitations of each evaluation method a bit more and also talk about whether they can be applied to any task. As an example, direct evaluation assumes that we already know the answer and therefore might not be applicable at test time.  

**Data contamination** 
- While I agree that performance per se is not so important here but rather the difference in performance, data contamination might impact the capabilities you are actually evaluating. If we assume that all models saw e.g., LAMBADA and bigger models are better on it, are we evaluating recall from memory or the actual task? I feel this could be discussed a bit more. 

**Error analysis for production and meta-linguistic (in the word prediction case)**
- It would be interesting to look at the errors the models make in this case and see whether there are interesting patterns here. 
Maybe this could help to understand in which sense the model is misinterpreting or misunderstanding the task?

**Figure 4**
- It could be interesting to look at the rank of the correct answer (for the word prediction task) as a function of the training steps. 

**Pythia results**
- The difference to other models might also be explained by the training data. I feel this aspect could in general be discussed a bit more.  
- It would be interesting to see the checkpoint evaluation (Figure 4) for pythia models as well.
paper_summary: paper_summary: This paper makes an interesting contribution to the study of LMs by looking at task demands as a potential moderator variable for task performance. 
The authors conduct a series of tests including varying the amount of training and find evidence that task demands do indeed pay a role.
strength_weakness: strength_weakness: I only have one concern (which I think can and should be addressed). The analysis is incomplete as it is essentially a descriptive analysis without any formal statistical testing. 
The authors could address this by looking at simple analyses such as an ANOVA or other models of the GLM family. 
This kind of Testing is not only desirable but essential: looking at individual comparisons without statistical justification to do so is incorrect.
questions: questions: -
paper_summary: paper_summary: This paper looks at whether a "demand gap" exists in tasks designed for LMs, whether this gap is larger for "simpler" models, it draws on a cognitive analogy.
strength_weakness: strength_weakness: I find it odd that the abstract assumes a competence-performance distinction in LMs. The distinction is widely but not universally accepted for humans in cognitive science, but it has been explicitly argued for decades that connectionist models (=> modern LMs) do not support such a distinction. Supporters of connectionist models of cognition saw/see this as a virtue. The consequence of this is that properties of the models relating to their architecture, eg number of parameters in this paper, and practical challenges emerging from those decisions, are fundamental to their "competence." They cannot be disentangled. While this doesn't evaluate the paper, it does bring the general narrative, as well as analogy with humans, into question.
Take a look at Schwarz (1992, Connection Science), and Allen & Seidenberg (2013, Emergentist Approaches to Language) for examples.

Buried under 3.3, there's a short paragraph disavowing direct parallels between child development and model scaling. This is of course true. Children doesn't grow more neurons as they develop, let alone by a factor of 70. However, since model size is one of the two measures of model power that this paper looks at, this severely undermines the analogy between childhood development and model power that the authors make throughout this paper. At the very least, this caveat should be brought up into the introduction, where I first noticed this problem, but a better solution would be to downplay the cognitive analogies being made here. That would also help solve the (lack of) competence-performance distinction issue raised above.

Regarding demonstrating the "signature pattern," this relies both on the theoretical assumptions about competence-performance and also practical assumptions surrounding the demand gap. Is forced choice, which is often effectively classification, really the "same" cognitive construct as the production equivalent? They're certainly very different tasks from a traditional ML perspective. If they aren't, then what is being measured isn't really the "demand gap" as stated. They're two different competencies. 

Related to whether or not they're the same task further assumes that the task setups are in fact evaluating the problem that the researchers think they are. This is particularly crucial in the forced choice setting, which the authors sort of get at in footnote 2, but they don't take it far enough. 

One example of this that I found particularly compelling was the following, which looked at VQA with multiple choice answers. It turns out that their model achieved most of its performance without even being exposed to the image, which means that it was simply not actually doing VQA as intended. Instead it was picking up on biases in the Q and Q construction. It took aggressive debiasing of the data set to create an evaluation where the authors were confident that the model was actually performing VQA
Wei-Lun Chao, Hexiang Hu, and Fei Sha. 2018. "Being negative but constructively: Lessons learnt from
creating better visual question answering datasets." NAACL

There was a more recent paper making the same point with regards to BLiMP. Because of the forced choice task setup and researcher-generated sentence pairs, models seem to be able to solve BLiMP without actually engaging in grammaticality. This is particularly relevant, since the authors rely on BLiMP for one of their evaluations.
Héctor Vázquez Martínez, Annika Lea Heuser, Charles Yang, and Jordan Kodner. 2023. "Evaluating Neural Language Models as Cognitive Models of Language Acquisition." GenBench

A better "demand gap" set up might, for example, compare solving math word problems with a lot of filler text and complex scenarios on one hand and more straightforward word problems on the other hand. These are more clearly the same ML task.

Given all of the above, the 2nd past paragraph of the discussion comes off as probably nonsense. It should be removed.

Typo at the bottom of page 5 "prompted asked"

Comments: title: Response to Authors
Thank you for the response
Response re. statistical model
comment: Thank you for your response. I appreciate the clarification on the second point. However, regarding the first point, I still recommend that the authors conduct a more extensive review of the psychology literature. Incorporating these two points into the revision will make the paper more convincing. Thank you, and I will maintain my score.
Thank you for your rebuttal, though I don't find it convincing

My comment about the competence/performance distinction is the to the applicability of this distinction to *models* just as you say. It is far from obvious that it even makes sense to talk about the distinction in reference to neural models in the first place, which is a fundamental assumption of the paper. This is why I brought up the connectionists. They argued that their neural *models* lacked the distinction and yet succeeded at their tasks, therefore humans lack the distinction. So there is relevant existing literature out there that disagrees with the fundamental assumption, yet it is not addressed.

Yes, I saw the disclaimer in Section 3.3. It doesn't obviate the problems with how the paper is written.
Thanks for this clear response, this makes this paper much sharper.
If you can explain this to the NLP audience (most will be [very] unfamiliar with these kinds of models) and maybe show the key test statistics and effect size of each effect (main effects and interactions, e.g., in a table) on the extra page, I'd be happy to increase my score.