###### Abstract
Firstly, we analyse the shortcomings of ROBT and find that it can be easily surpassed by a simple but effective pivot online back-translation (POBT) algorithm, where sentences of the pivot language (mostly English) are back-translated rather than the translated to the target language. Secondly, we propose selective online back-translation (SOBT) by dynamically selecting suitable training samples from synthetic sentence pairs and fine-tuning the MNMT model with contrastive loss to achieve continued improvements, especially for non-English directions. We thoroughly investigate several selection strategies involving cross-entropy and quality estimation. Thirdly, to minimize encoder representation divergence between different languages, we further propose cross-lingual \begin{table}
\begin{tabular}{l l} \hline Src & El doctor [PERSON] creep que algunos pacientes podrian haber contrádo el virus dentro del hospital. \\ \hline Ref & Einige Patienten haben sich möglicherweise im Krankenhaus mit dem Erreger angesteckt, meint Dr. [PERSON]. \\ \hline Pivot & Der Arzt [PERSON] glaubt, dass einige Patienten das Virus innerhalb des Krankenhauses behandelt haben. \\ \hline MNMT & Der Arzt [PERSON] believes that some patients might have had the virus in the hospital. \\ \hline Src & Los ingresos del fabricante superarán los 420.000 millones de yuanes, de acuerdo con el informe. \\ \hline Ref & Der Umsatz des Herstellers wird dem Bericht zufolge 420 Milliarden Yuan übersteigen. \\ \hline Pivot & Dem Bericht zufolge wird der Umsatz des Herstellers 420 Milliarden Yuan übersteigen. \\ \hline MNMT & Los ingresos del fabricante superarán los 420.000 millones de yuanes, de acuerdo con el informe. \\ \hline \end{tabular}
\end{table} 
Table 1: Comparison between zero-shot translation with a baseline MNMT model and a pivot-based translation (English as the pivot language) for Spanish \(\rightarrow\) German. The baseline MNMT model often translates into an incorrect language, such as translating into English rather than German or copying the source sentence. online substitution (CLOS) by generating hard or soft code-switched source sentences on-the-fly with implicit alignment information instead of explicit multilingual dictionaries or word alignment models. CLOS can be combined with SOBT to greatly improve the performance of many-to-many MNMT systems. To our best knowledge, we are the first to investigate and establish a complete online data augmentation algorithm for massively many-to-many MNMT. The main contributions of our work can be summarized as follows:  
* We propose the POBT and SOBT algorithms to generate and select suitable training samples to improve massively many-to-many MNMT, especially for non-English directions. * We thoroughly study different selection criteria, such as CE loss and QE score, concluding that the combination of CE loss and QE scores performs best. * We boost the SOBT algorithm with CLOS utilizing implicit alignment information instead of external resources to strengthen transfer learning between non-English directions. * We conduct experiments on two multilingual translation benchmarks with detailed analysis, showing that our algorithms can achieve significant improvements in both English-centric and non-English directions compared with previous works.