Title: GAIA: A Benchmark for General AI Assistant
Benchmark: GAIA
Summary: Create a benchmark which is conceptually simple questions for humans. It contains 466 quesitons from realistic questions and annotated by humans. The questions require fundamental abilities: reasoning, code, tool use et al.
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: reasoning, multimodality, code, tool use
Phenomenon defined by authors: Yes
Phenomenon definition: they reply on annotators' metadata
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Widely-agreed
Task definition: The task is defined by using the prompt + image
Task source: Real task examples (e.g. GitHub issues)
Dataset size: 466
Dataset metadata available: Yes
Sampling method: Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Train
Response format: Multiple choice
Metric definition: Exact Match (accuracy, F1, precision, recall)
Metric subscores: No
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Industry
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No
Results realism: Yes
Human baseline: Yes
Authors discuss validity: Yes
Taxonomy: Agents / 
New taxonomy: General Capability / Core Agentic Capabilities