KORMo: Korean Open Reasoning Model for Everyone
Authors: name: Minjun Kim; name: Hyeonseok Lim; name: Hangyeol Yoo; name: Inho Won; name: Seungwoo Song; name: Minkyung Cho; name: Junhun Yuk; name: Changsu Choi; name: Dongjae Shin; name: Huige Lee; name: Hoyun Song; name: Alice Oh; name: Kyungtae Lim
Published: 2025-10-10 14:31:25

Abstract: This work presents the first large-scale investigation into constructing a
fully open bilingual large language model (LLM) for a non-English language,
specifically Korean, trained predominantly on synthetic data. We introduce
KORMo-10B, a 10.8B-parameter model trained from scratch on a Korean-English
corpus in which 68.74% of the Korean portion is synthetic. Through systematic
experimentation, we demonstrate that synthetic data, when carefully curated
with balanced linguistic coverage and diverse instruction styles, does not
cause instability or degradation during large-scale pretraining. Furthermore,
the model achieves performance comparable to that of contemporary open-weight
multilingual baselines across a wide range of reasoning, knowledge, and
instruction-following benchmarks. Our experiments reveal two key findings: (1)
synthetic data can reliably sustain long-horizon pretraining without model
collapse, and (2) bilingual instruction tuning enables near-native reasoning
and discourse coherence in Korean. By fully releasing all components including
data, code, training recipes, and logs, this work establishes a transparent
framework for developing synthetic data-driven fully open models (FOMs) in
low-resource settings and sets a reproducible precedent for future multilingual
LLM research.