TeleChat Technical Report
Authors: name: Zihan Wang, name: Xinzhang Liu, name: Shixuan Liu, name: Yitong Yao, name: Yuyao Huang, name: Zhongjiang He, name: Xuelong Li, name: Yongxiang Li, name: Zhonghao Che, name: Zhaoxi Zhang, name: Yan Wang, name: Xin Wang, name: Luwen Pu, name: Huihan Xu, name: Ruiyu Fang, name: Yu Zhao, name: Jie Zhang, name: Xiaomeng Huang, name: Zhilong Lu, name: Jiaxin Peng, name: Wenjun Zheng, name: Shiquan Wang, name: Bingkai Yang, name: Xuewei he, name: Zhuoru Jiang, name: Qiyi Xie, name: Yanhan Zhang, name: Zhongqiu Li, name: Lingling Shi, name: Weiwei Fu, name: Yin Zhang, name: Zilu Huang, name: Sishi Xiong, name: Yuxiang Zhang, name: Chao Wang, name: Shuangyong Song

In this technical report, we present TeleChat, a collection of large language
models (LLMs) with parameters of 3 billion, 7 billion and 12 billion. It
includes pretrained language models as well as fine-tuned chat models that is
aligned with human preferences. TeleChat is initially pretrained on an
extensive corpus containing a diverse collection of texts from both English and
Chinese languages, including trillions of tokens. Subsequently, the model
undergoes fine-tuning to align with human preferences, following a detailed
methodology that we describe. We evaluate the performance of TeleChat on
various tasks, including language understanding, mathematics, reasoning, code
generation, and knowledge-based question answering. Our findings indicate that
TeleChat achieves comparable performance to other open-source models of similar
size across a wide range of public benchmarks. To support future research and
applications utilizing LLMs, we release the fine-tuned model checkpoints of
TeleChat's 7B and 12B variant, along with code and a portion of our pretraining
data, to the public community.