Orca: The World is in Your Mind
Authors: name: Yihao Wang, name: Yuheng Ji, name: Mingyu Cao, name: Yanqing Shen, name: Runze Xiao, name: Huaihai Lyu, name: Senwei Xie, name: Euan Liu, name: Klara Tian, name: Tianfeng Long, name: Yichi Zhang, name: Zhengliang Cai, name: Ruike Chen, name: Jifan Zhao, name: Ruochuan Shi, name: Zihan Tang, name: Jing Lyu, name: Wenxing Tan, name: Ningbo Zhang, name: Yangtao Hu, name: Yuming Gao, name: Xiansheng Chen, name: Junkai Zhao, name: Congsheng Xu, name: Boan Zhu, name: Ziqi Wang, name: Yupu Feng, name: Qiongqiong Zhang, name: Yingli Zhao, name: Yulong Ao, name: Shaoxuan Xie, name: You Liu, name: Guocai Yao, name: Leiduo Zhang, name: Xiaodan Liu, name: Yunyan Zhang, name: Yance Jiao, name: Xinyan Yang, name: Jiaxing Wei, name: Xu Liu, name: Tengfei Pan, name: Shaokai Nie, name: Chunlei Men, name: Sen Cui, name: Xiaojie Jin, name: Hongyang Li, name: Jianlan Luo, name: Yao Mu, name: Yunchao Wei, name: Jun Yan, name: Hang Zhao, name: Xiaolong Zheng, name: Jiaming Li, name: Yonghua Lin, name: Tiejun Huang, name: Zhongyuan Wang, name: Pengwei Wang

We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multimodal readout interfaces. Rather than optimizing isolated next-token, next-frame, or next-action prediction, we are centered on Next-State-Prediction modeling, offering a unified state-transition modeling route toward understanding, predicting, and acting upon the world. Orca learns through two complementary paradigms: unconscious learning captures dense natural state transitions from continuous videos, and conscious learning models sparse meaningful state transitions by language-described events and VQA supervision. For pre-training, we construct a large-scale world-learning inventory data, including 125K hours of video data and 160M event annotations. After pre-training, Orca learns a unified world latent space. To examine whether the learned latent supports downstream, we evaluate it by three representative downstream readouts: text generation, image prediction, and embodied action generation. Orca's backbone is frozen, and only the lightweight modality-specific decoders are trainable. Experiments show the scalability of the proposed paradigm and verify that stronger world latent enables stronger downstream readouts. Orca outperforms similar-sized specialized baselines. These results show that Orca, as a general world foundation model, presents a promising approach to understanding, predicting, and acting upon the world. Finally, we discuss the current limitations, aiming to provide useful insights and inspiration for the community.