BigTrans: Augmenting Large Language Models with Multilingual Translation
  Capability over 100 Languages
Authors: name: Wen Yang; name: Chong Li; name: Jiajun Zhang; name: Chengqing Zong
Published: 2023-05-29 14:07:52

Abstract: Large language models (LLMs) demonstrate promising translation performance
among various natural languages. However, many LLMs especially the open-sourced
ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of
natural languages, making the potential of LLMs on language translation less
explored. In this work, we present BigTrans which adapts LLaMA that covers only
20 languages and enhances it with multilingual translation capability on more
than 100 languages. BigTrans is built upon LLaMA-13B and it is optimized in
three steps. First, we continue training LLaMA with massive Chinese monolingual
data. Second, we continue training the model with a large-scale parallel
dataset that covers 102 natural languages. Third, we instruct-tune the
foundation model with multilingual translation instructions, leading to our
BigTrans model. The preliminary experiments on multilingual translation show
that BigTrans performs comparably with ChatGPT and Google Translate in many
languages and even outperforms ChatGPT in 8 language pairs. We release the
BigTrans model and hope it can advance the research progress.