Can Generative Multimodal Models Count to Ten?

Authors: sunayana  rane
alexander  ku
jason  baldridge
ian  tenney
thomas l griffiths
Venue: ICLR_cc_2024_Workshop_Re-Align
Published: 2024-02-02 00:45:49

Abstract: We adapt a developmental psychology paradigm to characterize the counting ability of the foundation model Parti. We show that three model scales of the Parti model (350m, 3B, and 20B parameters respectively) each have some counting ability, with a significant jump in performance between the 350m and 3B model scales. We also demonstrate that it is possible to interfere with these models' counting ability simply by incorporating unusual descriptive adjectives for the objects being counted into the text prompt. We analyze our results in the context of the knower-level theory of child number learning. Our results show that we can gain experimental intuition for how to probe model behavior by drawing from a rich literature of behavioral experiments on humans, and, perhaps most importantly, by adapting human developmental benchmarking paradigms to AI models, we can characterize and understand their behavior with respect to our own.

Main paper: 
 background and motivation: 
 With text-to-image multimodal models gaining widespread use, it is more important than ever to characterize and methodically study their behaviors. Recent research has focused on studying whether these models demonstrate compositionality, appropriately producing the right combination of abstract concepts (Thrush et al., 2022 ). Here we focus on an even simpler criterion for abstraction: understanding number. For example, we might wonder whether a model can reliably count to ten, and whether its internal understanding of number concepts matches what we would expect of a human. Many of the questions we are now asking of a model's "understanding" of concepts are the same questions we've asked previously about human children (Frank, 2023a; b) . Developmental cognitive psychologists have devised tests and measures that probe many aspects of a child's understanding of number. For example, researchers discovered that children often produce (speak) number words in order ("one, two, three") before they understand how to use them or what they mean (before they can produce 3 objects, or even correctly count the 3 objects placed in front of them) Sarnecka & Carey (2008) ; Frye et al. (1989); Fuson (2012) . Here we adapt these procedures to study multimodal models.Standard evaluations of multimodal models focusing on a broad set of capabilities also often include some measure of counting ability among a larger set of metrics (Cho et al., 2023a; b; Hu et al., 2023; Lee et al., 2023) . In contrast to these broader metrics designed to provide a standard measure of a wide range of model abilities, here we provide a deeper behavior-based analysis which systematically and specifically varies counting along controlled lines. 
task and data: 
 The task often used as the gold-standard for measuring a child's understanding of number concepts is known as the Give-N task (Frye et al., 1989; Fuson, 2012; Schaeffer et al., 1974; Wynn, 1990; 1992; Marchand et al., 2022) . The idea is simple: prompted with an instruction like "give five lemons," the child must physically count out and give five toy lemons to a puppet. Instead of other tasks that ask children to count sets of objects they are given (as in the How-Many task), the Give-N task is understood to provide a very rigorous standard for number understanding; many children who can verbally count up to 5 cannot successfully produce sets of 5 items, so their performance on the Give-N task indicates that they thoroughly understand a number (Wynn, 1990; 1992) .Generative vision and language models can now be probed using something similar to the Give-N task, prompted with text like "five lemons" and asked to generate an image from scratch. This is in contrast to classification or captioning models, in which you can only be asked to count the number of objects in an input image, corresponding to the more lax "How-Many" task used in developmental psychology (Wynn, 1992; Connor et al., 2024) . We gave the Parti model text prompts based on the Give-N task and engaged a pool of human raters through a crowdsourcing contractor to count objects in the generated images.To approximate the Give-N task in a way that makes sense with the modalities of our models, we start with child word learning data. The "object" words used to create the prompts of our version of the Give-N task were the 40 most easily learned food and animal words (20 of each) of all the words in the WordBank database of child vocabulary development (Frank et al., 2017) . These are words most children learn prior to 36 months, and are close analogues to the food and animal toys in the original Give-N experiments. We give each model prompts of these 40 objects, with counts from 1-15.Figure 1 : Images generated by each of the three scales of Parti models for the input text prompts "seven apples" and "six birds".In addition to the standard prompts, we also constructed a smaller set of prompts to probe how dependent counting performance is on the familiarity of the objects in the prompt -in other words, can we interfere with counting ability by modulating other things about the prompt? We refer to these as the common and uncommon prompts, fully listed in Table 1 . We use a subset of the objects used in the standard prompts: apples, oranges, bananas, cats, and dogs. For each of these object types, we modulate an adjective: we use one that is common for the object, such as "fluffy" for dogs, and one that is uncommon, such as "spiky" for dogs. "six red apples" "six spotted apples" "two black cats" "two green cats" "four yellow bananas" "four blue bananas" "three fluffy dogs" "three spiky dogs" "eight shiny oranges" "eight hairy oranges" 
models: 
 One way to characterize a developmental trajectory in models is to explore several different scales of one model type. The Pathways Autoregressive Text-to-Image (Parti) model (Yu et al., 2022) provides just such an opportunity, because it presents a common model architecture at multiple scales. For our experiments we use three different scales of the Parti model: 350M, 3B, and 20B parameters. The model architecture follows a Transformer-based (Vaswani et al., 2017) encoder-decoder framework, with the decoder receiving the major share of the increase in each increasingly large model size. The Parti models take text input such as "five lemons" and output generated images. 
results: 
 For each of the prompt categories (standard, common, and uncommon prompts) we provide a detailed breakdown of the correlation between the number of objects in the model-generated images (as recorded by human evaluators) and the true count from the text prompt the model was given (Table 2). In addition, figure 3 gives a detailed breakdown of each model's count-by-count performance on the standard prompts. In analyzing the results, we also draw from the knower-levels framework often used to understand Give-N task results in child psychology studies (Wynn, 1992; Sarnecka & Carey, 2008) . At the "one-knower" level, which most children reach by 2.5-3 years, they understand only the concept of 1. A few months later, a child becomes a "two-knower," when they reliably give 1 and 2, but not 3, 4, 5. Then slowly comes the "three-knower" and, some studies report, the "four-knower" level before the child learns the subsequent numbers not slowly, as before, but all at once and through induction (they have learned that adding one to a prior number results in the next number).Interestingly, our results suggest that this inductive step is missing from all three scales of the models' behavior. The 20B model seems to be inching along in this direction, getting fairly reliable results up to 4 and improved results on 5 and 6 compared to both the 350m and 3B models. However, this behavior obviously has not scaled past 5-6, and from 7 onward it is quite difficult for the models, in contrast to children who learn 5 onward quickly and inductively. Our approach illuminates this gap, and shows behavioral similarities between these models and children of approximately 3-4 years of age.Furthermore, the results for the common and uncommon adjective prompts demonstrate a significant gap in performance across all models, indicating that unusual adjective and noun combinations do indeed interfere with the model's counting ability. This highlights an area for further training, investigation, and improvement in these models. There are also areas of strength: notably, the 3B model demonstrates particularly strong performance for common adjective prompts, surpassing even the 20B model and painting a more nuanced picture of how model scale relates to performance.We hope this approach empowers model designers to address developmental gaps in knowledge and performance, and that the practice of using developmental psychology paradigms to probe model behavior continues to help us develop more reliable, responsible foundation models. 
 
 Appendix:  



Reviews: review: title: title: Can Generative Multimodal Models Count to Ten?
main_review: main_review: This paper frames the evaluation of counting ability in multimodal language-vision models from the lens of developmental psychology, where two different tasks are used to evaluate counting ability in infants: the "how many" task, analogous to image captioning (or image-to-text), and the "give N" task, analogous to image generation (text-to-image). There are interesting learning dynamics when human children ages 2.5-5 learn to count, and this motivates the authors to compare 3 different text-to-image models with varying size. Prior work considers counting ability in language-vision models as part of a broader evaluation suite, whereas this work contributes a more in-depth analysis of counting ability by systematically testing ability for counting numbers 1-15 with common entities in images and language corpora such as birds and apples. Their main conclusions from their results are:
1. "all three scales of the Parti model have some counting ability, with a steep increase in performance between the 350m model and the 3B model ... counting skills may be “unlocked” at the 3B model scale"
2. "this inductive step is missing from all three scales of the models’ behavior ... Our approach illuminates this gap, and shows behavioral similarities between these models and children of approximately 3-4 years of age."

I believe this work is well-motivated, as counting objects is often discussed as a domain for evaluating text-image models. I appreciated the analogies with developmental psychology and the "give N" vs. "how many" framing seems novel. The experimental stimuli are reasonable, and the common vs. uncommon adjective comparison is useful. The authors' empirical results in Figure 3 are interesting. I'm not familiar enough with the specific topic of evaluating counting in multimodal models to say for certain if this is the first work to run a correlation analysis between model-generated numbers of objects and what's specified in the prompt, but if so, this is a valuable contribution.

However, I think this paper has a few key weaknesses:
1. It's unclear to me whether some of the authors conclusions follow from their results, based on my knowledge of the relevant literature. For the behavioral similarities with children of 3-4 years of age, how do these results suggest that these models are acting like 5/6-knowers? In the heatmaps of Figure 3, shouldn't we see uniform probabilities across all model-generated numbers over some N-knower threshold?
2. There is no comparison between "give N" and "how many" tasks, i.e. comparing evaluating text-to-image counting vs. image-to-text counting. Since so much of the text in Sections 1 & 2 is about this distinction, and since there is prior work evaluating counting in text-to-image models ("give N" task; e.g. [1]), I'm not sure what to make of this point.
3. Needs at least a few sentences summarizing the most relevant prior work. There's a lot of text on dev psych background, but comparatively little about "What did previous counting evaluations of multimodal models find? What was missing?"
4. There isn't much discussion of the striking biases shown in Figure 3. What's up with the spike in accuracy around 9 for all 3 models? What about the models' bias towards producing "20" for any count above ~10? What do you make of the accuracy for numbers 4-6 being ~30-50% - is there a similar pattern in children where N-knowers have ~50% accuracy on the number N and ~100% for numbers <N?

Smaller suggestions/comments:
- Table 2: add hit/miss accuracy as a column, to match the y-axis in Fig 3 line plots
- Fig 3: It seems like the rightmost 5 columns of the heatmaps could be dropped - these are all 0 and with (1, 15) this would align better with the line plots.
- It seems like even a very capable text-to-image model should have major limitations counting large numbers of objects due to image resolution and limitations of vision as a modality.


[1] Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang (2023). Davidsonian scene graph: Improving reliability in fine- grained evaluation for text-image generation.
strength_weakness: strength_weakness: Novel results on a specific domain, counting objects, commonly discussed in multimodal AI evaluation. Novel framing of problem grounded in developmental psychology.
title: title: Interesting toy study
main_review: main_review: The paper investigates the counting ability of a generative multimodal large language model at different scales. 


**Clarity** 

I found the paper to be engaging and easy to read. However, I'm left wondering about the take-away: generative large language models do not learn counting like children? Is your point that as these models scale, they do not fit into the knower-level framework, and are thus unlike children in how they learn numbers? I think this could be made a bit more clear. 

**Correctness** 

Overall, I think the approach makes sense and the methods seem correct. However, there remain a few points I would like to have more information on:
- In 4 Results, I would like to know more details about the human evaluators (how many for instance).
- In Table 2, how do you explain that the 3B model correlates higher with the true input count compared to the 20B model? This seems counterintuitive to me.
- In Figure 3, how do you explain the spike at true count = 9?
- In Figure 3 second row, how come not all columns add up to 40 (for instance true count = 1)?
- In 5 Discussion: ''indicating that counting skills may be “unlocked” at the 3B model scale.'' seems like a stretch, since all models still perform poorly for most numbers above 4. Also [1] show that multimodal large language models with 8B parameters still struggle with counting and that even GPT-4V struggles for some data sets where there are more than 10 objects in a scene (so likely it isn't unlocked at the 3B model scale).

**Novelty** 

As far as I can tell this is a novel investigation. There are papers that investigate multimodal large language models' counting abilities (such as [1]) but not for generative multimodal large language models. 

**Interest to community** 

This paper is of interest to the research community in general but since it does not investigate representations I am not sure if it is of particular interest for visitors of the Re-Align workshop.

[1] Schulze Buschoff, Luca M., et al. "Have we built machines that think like people?." arXiv e-prints (2023): arXiv-2311.
strength_weakness: strength_weakness: Overall, while the scope of the paper is rather small, the paper is well-written and briefly outlines an interesting investigation. In any case, I feel like it is of some interest to the community as a whole and I would therefore recommend acceptance.
title: title: Nice short paper using developmental approach reveals fundamental gap between models and humans
main_review: main_review: Summary:
This paper investigates the alignment between text-to-image multimodal AI models and human children, focusing on the task of understanding numbers. Despite its seemingly straightforward nature, the authors demonstrate that large-scale multimodal models, such as Parti, struggle to generate images that accurately represent the quantity of objects specified in the text prompt. In a broader context, the paper suggests the importance of addressing developmental gaps in both knowledge and performance when developing foundational models. It advocates for the use of developmental psychology paradigms as a reliable approach to probe model behaviors and facilitate the development of more aligned models.

Overall, I find this paper to be of interest to the alignment research community. It is well-written, easy to follow, and the experiments are logically designed. I particularly like the authors' approach of prompting models to generate images as a means of testing their understanding of numbers, which offers advantages over methods such as counting objects in input images. Additionally, the experiment involving common and uncommon adjective prompts yields intriguing results, and I would welcome further insights from the authors into the underlying reasons for these findings.

I recommend the acceptance of this short paper at ReAlign.
strength_weakness: strength_weakness: N/A