Autodata: An agentic data scientist to create high quality synthetic data
Authors: name: Ilia Kulikov, name: Chenxi Whitehouse, name: Tianhao Wu, name: Yixin Nie, name: Swarnadeep Saha, name: Eryk Helenowski, name: Weizhe Yuan, name: Olga Golovneva, name: Jack Lanchantin, name: Yoram Bachrach, name: Jakob Foerster, name: Xian Li, name: Han Fang, name: Sainbayar Sukhbaatar, name: Jason Weston
Organization: AI at Meta

We introduce Autodata, a general method that enables AI agents to act as data scientists who build high quality training and evaluation data. We show how to train (meta-optimize) such a data scientist agent, so that it learns to create even stronger data. We describe the overall formulation, and a specific practical implementation, Agentic Self-Instruct. We conduct experiments on computer science research tasks, legal reasoning tasks and reasoning with mathematical objects, where we obtain improved results compared to classical synthetic dataset creation methods. Further, meta-optimizing the data scientist agent itself delivers an even larger performance uplift. Agentic data creation provides a way to convert increased inference compute into higher quality model training. Overall, we believe this direction has the potential to change the way we build AI data.