Technical Report of TeleChat2, TeleChat2.5 and T1
Authors: name: Zihan Wang, name: Xinzhang Liu, name: Yitong Yao, name: Chao Wang, name: Yu Zhao, name: Zhihao Yang, name: Wenmin Deng, name: Kaipeng Jia, name: Jiaxin Peng, name: Yuyao Huang, name: Sishi Xiong, name: Zhuo Jiang, name: Kaidong Yu, name: Xiaohui Hu, name: Fubei Yao, name: Ruiyu Fang, name: Zhuoru Jiang, name: Ruiting Song, name: Qiyi Xie, name: Rui Xue, name: Xuewei He, name: Yanlei Xue, name: Zhu Yuan, name: Zhaoxi Zhang, name: Zilu Huang, name: Shiquan Wang, name: Xin Wang, name: Hanming Wu, name: Mingyuan Wang, name: Xufeng Zhan, name: Yuhan Sun, name: Zhaohu Xing, name: Yuhao Jiang, name: Bingkai Yang, name: Shuangyong Song, name: Yongxiang Li, name: Zhongjiang He, name: Xuelong Li

We introduce the latest series of TeleChat models: TeleChat2,
TeleChat2.5, and T1, offering a significant upgrade over
their predecessor, TeleChat. Despite minimal changes to the model architecture,
the new series achieves substantial performance gains through enhanced training
strategies in both pre-training and post-training stages. The series begins
with TeleChat2, which undergoes pretraining on 10 trillion
high-quality and diverse tokens. This is followed by Supervised Fine-Tuning
(SFT) and Direct Preference Optimization (DPO) to further enhance its
capabilities. TeleChat2.5 and T1 expand the pipeline by
incorporating a continual pretraining phase with domain-specific datasets,
combined with reinforcement learning (RL) to improve performance in code
generation and mathematical reasoning tasks. The T1 variant is
designed for complex reasoning, supporting long Chain-of-Thought (CoT)
reasoning and demonstrating substantial improvements in mathematics and coding.
In contrast, TeleChat2.5 prioritizes speed, delivering rapid
inference. Both flagship models of T1 and TeleChat2.5 are
dense Transformer-based architectures with 115B parameters, showcasing
significant advancements in reasoning and general task performance compared to
the original TeleChat. Notably, T1-115B outperform proprietary models
such as OpenAI's o1-mini and GPT-4o. We publicly release TeleChat2,
TeleChat2.5 and T1, including post-trained versions with 35B
and 115B parameters, to empower developers and researchers with
state-of-the-art language models tailored for diverse applications.