Title: UOUO: Uncontextualized Uncommon Objects for Measuring Knowledge Horizons of Vision Language Models
Benchmark: UOUO
Summary: VLMs ability to handle rare objects, which fall into the long tail of data distributions, is less studied in the current literature. . To rigorously evaluate this aspect, the authors introduce the "Uncontextualized Uncommon Objects" (UOUO) benchmark which focuses on systematically testing VLMs on rare and specialized objects.
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: rare & uncommon visual grounding
Phenomenon defined by authors: No
Definition scope: Subset
Definition integrity: Single cohesive phenomenon
Definition integrity detail: No
Phenomenon contested: Contested
Task definition: They don't define the tasks. They briefly mention during the experimental setup that they will assess VLMs in object segmentation and object detection. 
Task item definition: The target object name, the image with 4 objects, the Wikipedia category/domain of the target object
Task source: Procedurally-generated task examples (e.g. Creating instances from a template), LLM-generated task examples (e.g. Filtered from responses to a prompt)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset metadata available: Yes
Dataset metadata detail: Domain/Category of the object based on Wikipedia
Sampling method: Random sample (creators defined a task space and sampled from it)
Split reviewed: Test
Response format: Short free response (e.g. single word or number)
Metric definition: Exact Match (accuracy, F1, precision, recall), Mean IoU (Intersection over Union)
Metric aggregation: Simple Mean
Metric statistics: simple mean
Metric subscores: Yes
Metric subscores detail: - Random subset: with randomly selected objects on each image
- Hard subset: with similar objects on each image
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: No, no link is provided
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Grounding / 
New taxonomy: General Capability / Grounding