Title: DevBench: A multimodal developmental benchmark for language learning
Benchmark: DevBench
Summary: We introduce DEVBENCH, a multimodal benchmark comprising seven language evaluation tasks spanning the domains of lexical, syntactic, and semantic ability, with behavioral data from both children and adults. '

'We evaluate a set of vision–language models on these tasks, comparing models and humans on their response patterns. '

'Across tasks, models exhibit variation in their closeness to human response patterns, and models that perform better on a task also more closely resemble human behavioral responses. DEVBENCH thus provides a benchmark for comparing models to human language development.' (abstract)
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: Compare models to human language development
Phenomenon defined by authors: Yes
Phenomenon definition: In order to characterise models’ language learning performance, we should evaluate multiple levels of linguistic representation, including the lexicon, syntax, and semantics – ideally how these correspond to children’s development at different ages.' (page 2)
Definition scope: Comprehensive
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: Evaluate how closely multimodal language models’ response patterns match those of children and adults on seven language understanding tasks. These tasks span lexical, syntactic, and semantic domains, capturing human-model response distribution similarity rather than raw accuracy.
Task item definition: A visual input (e.g., images or scenes) paired with a language cue (e.g., word, sentence, or prompt).

The model must choose or rate responses (e.g., matching images or associated words). Human responses to the same inputs are used for comparison. 
Task source: Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions), Expert-crafted task examples (e.g. hand-written examples)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset metadata available: No
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically)
Split reviewed: Test
Response format: Short free response (e.g. single word or number)
Metric definition: Exact Match (accuracy, F1, precision, recall)
Metric subscores: No
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: Yes
Authors discuss validity: No
Taxonomy: Language Modelling / 
New taxonomy: General Capability / Adaptability