Rethinking Reflection in Pre-Training
Authors: name: Essential AI; name: Darsh J Shah; name: Peter Rushton; name: Somanshu Singla; name: Mohit Parmar; name: Kurt Smith; name: Yash Vanjani; name: Ashish Vaswani; name: Adarsh Chaluvaraju; name: Andrew Hojel; name: Andrew Ma; name: Anil Thomas; name: Anthony Polloreno; name: Ashish Tanwer; name: Burhan Drak Sibai; name: Divya S Mansingka; name: Divya Shivaprasad; name: Ishaan Shah; name: Karl Stratos; name: Khoi Nguyen; name: Michael Callahan; name: Michael Pust; name: Mrinal Iyer; name: Philip Monk; name: Platon Mazarakis; name: Ritvik Kapila; name: Saurabh Srivastava; name: Tim Romanski
Published: 2025-04-05 02:24:07

Abstract: A language model's ability to reflect on its own reasoning provides a key
advantage for solving complex problems. While most recent research has focused
on how this ability develops during reinforcement learning, we show that it
actually begins to emerge much earlier - during the model's pre-training. To
study this, we introduce deliberate errors into chains-of-thought and test
whether the model can still arrive at the correct answer by recognizing and
correcting these mistakes. By tracking performance across different stages of
pre-training, we observe that this self-correcting ability appears early and
improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4
trillion tokens displays self-correction on our six self-reflection tasks.