Phi-4 Technical Report
Authors: name: Marah Abdin, name: Jyoti Aneja, name: Harkirat Behl, name: Sébastien Bubeck, name: Ronen Eldan, name: Suriya Gunasekar, name: Michael Harrison, name: Russell J. Hewett, name: Mojan Javaheripi, name: Piero Kauffmann, name: James R. Lee, name: Yin Tat Lee, name: Yuanzhi Li, name: Weishung Liu, name: Caio C. T. Mendes, name: Anh Nguyen, name: Eric Price, name: Gustavo de Rosa, name: Olli Saarikivi, name: Adil Salim, name: Shital Shah, name: Xin Wang, name: Rachel Ward, name: Yue Wu, name: Dingli Yu, name: Cyril Zhang, name: Yi Zhang

We present phi-4, a 14-billion parameter language model developed with a
training recipe that is centrally focused on data quality. Unlike most language
models, where pre-training is based primarily on organic data sources such as
web content or code, phi-4 strategically incorporates synthetic data throughout
the training process. While previous models in the Phi family largely distill
the capabilities of a teacher model (specifically GPT-4), phi-4 substantially
surpasses its teacher model on STEM-focused QA capabilities, giving evidence
that our data-generation and post-training techniques go beyond distillation.
Despite minimal changes to the phi-3 architecture, phi-4 achieves strong
performance relative to its size -- especially on reasoning-focused benchmarks
-- due to improved data, training curriculum, and innovations in the
post-training scheme.