ARE: Scaling Up Agent Environments and Evaluations
Authors: name: Pierre Andrews; name: Amine Benhalloum; name: Gerard Moreno-Torres Bertran; name: Matteo Bettini; name: Amar Budhiraja; name: Ricardo Silveira Cabral; name: Virginie Do; name: Romain Froger; name: Emilien Garreau; name: Jean-Baptiste Gaya; name: Hugo Laurençon; name: Maxime Lecanu; name: Kunal Malkan; name: Dheeraj Mekala; name: Pierre Ménard; name: Grégoire Mialon; name: Ulyana Piterbarg; name: Mikhail Plekhanov; name: Mathieu Rita; name: Andrey Rusakov; name: Thomas Scialom; name: Vladislav Vorotilov; name: Mengjue Wang; name: Ian Yu
Published: 2025-09-21 16:59:45

Abstract: We introduce Meta Agents Research Environments (ARE), a research platform for
scalable creation of environments, integration of synthetic or real
applications, and execution of agentic orchestrations. ARE provides simple
abstractions to build complex and diverse environments, each with their own
rules, tools, content, and verifiers, helping to bridge the gap between model
development and real-world deployment. We also propose Gaia2, a benchmark built
in ARE and designed to measure general agent capabilities. Beyond search and
execution, Gaia2 requires agents to handle ambiguities and noise, adapt to
dynamic environments, collaborate with other agents, and operate under temporal
constraints. Unlike prior benchmarks, Gaia2 runs asynchronously, surfacing new
failure modes that are invisible in static settings. Our experiments show that
no system dominates across the intelligence spectrum: stronger reasoning often
comes at the cost of efficiency, and budget scaling curves plateau,
highlighting the need for new architectures and adaptive compute strategies.
Perhaps more importantly, ARE abstractions enable continuous extension of Gaia2
to other environments, empowering the community to rapidly create new
benchmarks tailored to their domains. In AI's second half, progress
increasingly depends on defining meaningful tasks and robust evaluations to
drive frontier capabilities forward.