ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Authors: Wenxiang Jiao
Jen-tse Huang
Wenxuan Wang
Zhiwei He
Tian Liang
Xing Wang
Shuming Shi
Zhaopeng Tu
Venue: EMNLP 2023 Findings (2023)
Keywords: machine translation
instruction tuning
LoRA
human feedback
LLaMA
BLOOMZ
ChatGPT
GPT-4
TL;DR: The ParroT framework to enhance and regulate the Translation Abilities during Chat based on open-sourced LLMs (e.g., LLaMA-7b, Bloomz-7b1-mt), human written translation and evaluation data.

Abstract: Large language models (LLMs) like ChatGPT have exhibited remarkable abilities on a wide range of natural language processing (NLP) tasks, including various machine translation abilities accomplished during chat. However, these models are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. Therefore, we propose ParroT, a framework to enhance and regulate the translation abilities during chat based on open-source LLMs (e.g., LLaMA), human-written translation and feedback data. Specifically, ParroT reformulates translation data into the instruction-following style, and introduces a “Hint” field for incorporating extra requirements to regulate the translation process. Accordingly, we propose three instruction types for finetuning ParroT models, including translation instruction, contrastive instruction, and error-guided instruction. Experiments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs significantly while error-guided instruction can lead to further improvement, which demonstrates the importance of learning from low-quality translations annotated by humans. We also demonstrate the potential of automatic evaluation tools in providing quality information of translations, when constructing error-guided instructions for directions that lack human annotation data. Please refer to our Github project for more implementation details: https://github.com/wxjiao/ParroT.

ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Wenxiang Jiao Jen-tse Huang 1.2
Wenxuan Wang1.2 Zhiwei He 1. Tian Liang!
Xing Wang'
Shuming Shi' Zhaopeng Tu
Tencent Lab "The Chinese University of Hong Kong
Shanghai Jiao Tong University "Tsinghua Shenzhen International Graduate School
(joelwxjiao, brightxwang, s wpt.ttteecccnttccommmm com {jthuang, wxwang] cuhk. edu. hk owhe.cs@sjje edu. cn 1iangt21@maien 1 edu.
Abstract
Translate the following sentencees English.
Large language models (LL.Ms) like ChaiGPT have exhibited remarkable abilities wide range of natural language processing (NLP) tasks, including various machine translation abilities accomplished during chat. However, these models are only accessible through re- stricted APIs, which creates barriers to new research and advancements in the field. There- fore, we propose ParroT, a framework to en- hance and regulate the translation abilities dur- ing chat based on open-source LLMs (c.g.. LLaMA), human- written translation and feed- back data. Specifically, Parro'T reformulates translation data into the instruction-folloiing style, and introduces "Hint" field for incorpo- rating extra requirements to regulate the trans- lation process. Accordingly, we propose three instruction types for finetuning ParroT models, including translation instruction, contrastive in- struction, error-guided instruction. Experi- ments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs significantly while error-guided instruction can lead to further improvement, which demon- strates the importance of learning from low- quality translations annotated by humans. We also demonstrate the potential automatic evaluation tools in providing quality informa- tion of translations, when constructing error- guided instructions for directions that lack human annotation data. Please refer Github project for more implementation details: https://ght thub. com/wwjiaoo/ParoT.
7 A
Hint: preferredio errors/entity alignmentt...
Some parrols can respord to simple commands or questions with the appropriate words, establishing communication humans.
Figure I:
Framework of ParroT. Hints are (optional) extra requirements to regulate the translation process.
processing (NLP) tasks within a single architecture, including question answering (Omar et al., 2023), text summarization (Yang et al., 2023), grammati- cal error correction (Wu et al., 2023), and machine translation (Jiao et al., 2023). Consequently, they represent significant stride toward the realization of artificial general intelligence (AGI).
Machine translation, a quintessential NLP task, faces both challenges and opportunities presented by the emergence of LLMs. Traditional machinc translation encompasses several sub-tasks (Farhad et al., 2021), such as bilingual translation (Vaswani et al., 2017), multilingual translation (Johnson et al.. 2017: Jiao et al., 2022), terminology transla- tion (Wang et al., 2022; Hou et al., 2022), quality estimation (Rei al., 2020), and automatic post- editing (Pal et al., 2016), among others. These tasks are typically addressed by individual mod- els with limited cross-task interaction. However, current LLMs have the potential to revolutionize this inefficient approach and redefine the machine translation paradigm. On one hand, LLMs can leverage the benefits of various sub-tasks and seam- lessly transition between them using only natural language instructions. For instance, if user is dissatisfied with translation result, they can re- quest the LLM to refine the translation implicitly (i.e., through automatic post-editing) or explicitly,
1 Introduction
Large language models (LLMs), designed in the followii format, such ChatGPT and GPT-4 (OpenAI, 2023), have garnered consider- able interest due to their remarkable abilities in comprehending instructions and generating human- like responses. These versatile models can effi- ciently perform a wide range of natural language *Conresponding author.

---

by imposing constraints on specific entities (i.e.. terminology translation). On the other hand, LLMs are expected to enhance the explainability of ma- chine translation, ultimately leading to further im- provements in translation quality. For example, users may want LLMs to compare two transla- tions of sentence (i.e.. quality estimation) and provide an explanation for the discrepancies (i.e., error analysis), which can then be addressed in tar- geted manner by the LLM itself. However, superior LLMs like ChatGPT and GPT-4 are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. There- fore, developing comprehensive machine transla- tion abilities upon open-source LLMs has become a critical and challenging research problem.
importance of learning from low-quality trans- lations annotated by humans.
Parameter efficient finetuning with low-rank adaptation (LoRA, Hu et al., 2022) can pre- vent LLMs from overfitting, which achieves better performance on dominant languages but slows down the learning from other languages.
We demonstrate the potential of automatic evaluation tools (i.e., COMET) providing quality information of translations, when con- structing error- grror-guided instructions for direc- tions that lack human annotation data.
Instruction Pool
In this paper, we propose the ParroT frame- work to enhance and regulate the translation abil- ities of LLMs during chat by leveraging existing translation and feedback data. To be compatible with chat, our framework reformu- lates translation data into the instruction-follooinn style (Taori et al., 2023), and introduces "Hint" field for incorporating extra requirements to guide the translation process. Accordingly, we propose three distinct instruction types: (1) Translation Instruction, that asks LLMs to generate transla- tions based on source sentences. (2) Contrastive Instruction, that asks LLMs to generate the trans- lations of two different systems with the preferred one at first. (3) Error-Guided Instruction, that asks L.LMs to generate the translations with human- annotated errors the hint. The first instruction guarantees the basic translation ability of LLMs while the latter two regulate the LLMs align with human feedbacks (Ouyang al., 2022: Liu al., 2023). We adopt the open-source LLaMA (Tou- vron et al., 2023) and BLOOM (Scao et al., 2022) models, and conduct instruction tuning on previous WMT validation data and Multidimensional Qual- ity Metric (MQM) human evaluation data. The resulting РarroT models are evaluated on Flores subsets and WMT22 test sets.
In this section, we introduce the three distinct in- struction types: translation instruction, contrastive instruction, and error-guided instruction. The first instruction guarantees the basic translation ability of LLMs while the latter two regulate the LLMs to align with human-written translation and feedback.
2.1 Translation Instruction
As traditional translation systems, we rely on bilin- gual sentence pairs accomplish the basic trans- lation ability of LLMs. We follow Stanford Al- paca (Taori et al., 2023) to transform bilingual sen- tence pairs into the - format, named translation instruction, for finetuning.
Table I presents example of the translation instruction, which includes a preface fixed for all tasks, an "### Instruction:" to describe the transla- tion task (e.g., stating the language pair), an "#I#I Input:" with the source sentence, and "#### Re- sponse:" with the target sentence to be generated. To ensure the high quality of sentence pairs, we use human-written translations rather than public training data that could be noisy.
2.2 Contrastive Instruction
Besides the basic translation ability, we also want LLMs to understand the relative quality difference between translations. In this way, we may improve the quality of translations by asking LLMs to out- put the preferred ones. To realize this goal, we need multiple different translations for each source sentence, which can be acquired by the systems submitted to WMT competitions. Meanwhile, the human evaluation results of these systems also pro- vide scores to reflect the quality differences.
Our main findings are summarized below:
Translation instruction, as expected, can im- prove the translation performance of LLMs significantly, especially for directions from English to other languages.
Error Error-guided instruction can further improve the performance when asking ParroT to gen- erate translations with no error, indicating the

---

Table I: Instruction types for finetuning Parro'T' models.
Type
Instruction
Below is instruction that describes task, paired with input that provides further context. Write response that uppropriately completes the request.nnnn
Translation
#M# Instruction: in o'Trunslate the following sentences from Chinese to English, In'in
#HH Input: 2
A
*
& 1 S # i
3l t2 # 4-
Vnin
The inspection results showed that there grain, oil, meat, fruit, vegetable, milk, and eggs the market commodity prices basically remain stable, the udministration found no serious offensive and noncompliant conducts, and the market order remains stable on the whole.
### Response:
adequate supply of daily necessities, including
Contrastive
### Instruction: Translatee the following sentences from Chinese English.
### Input: in A #
*
# AR Tij
ts E # wtw..
#
Inin
##B Hint: We prefer to trunslate it toin'n
### Response: <p>The inspection results showed that there was adequate supply of daily necessities, including grain, oil, meat, fruit, vegerable, milk, and eggs in the market and commodity prices hasically remain stable, the administration found no serious offensive and noncompliant conducts, and the market order remains stable on the whole.</p> rather than <p>The results of the inspection indicate the sufficient supply of living necessities marketing including cereals and oils, meat, fruits, vegetables, eggs and milk, and the basicully stabilized commodity price. The inspection order is stable on an overall basis.</p>
found serious violation of laws and regulations. market
Error-Guided
### Instruction: In Traanssltee the following sentences from Chinese to FnngishhInnin
### Input: in 1 1h th
*
#
ih * t6 15
4-48 Inin
#0# Hint: A translation with major accuracy/mistranslatioo errors could beinin
The results the inspection indicate the sufficient supply of living necessities <v>on market- ing</v> including cereals and oils, meat, fruits. vegetables, eggs and milk, and the basically stabilized commudity price. The inspection hasn't found serious violation of laws and regulations. The market order stable overall basis.
#*# Response:
### Hint: translation with minor fluency/grammar errors could beinin
### Response: The results of the <v>inspection</v> indicate the sufficient supply of living necessities on marketing including cereals and oils, meat, fruits, vegerahles, eggs and milk, and the hasically stabilized commodity price. The inspection basn't 't found serious violation of laws and regulations. The market order is stable on an overall basis.
As shown in Table 1, we form the response by concatenating two translations (e.g., linked by "rather than"), which the first translation has a higher quality score. Meanwhile, we indicate that the first translation is preferred in the "### Hint:" field. Essentially, the second translation acts like a negative sample to this sentence pair, which ex- plains the name contrastive instruction.
may ask LLMs produce translations with no error so as improve the quality.
We propose error-guided instruction. As shown in Table 1, we use the translation with er- rors annotated by the "<v></v>" span to form the response. Similar to contrastive instruction, we adopt the "### Hint:" field to indicate the error types. This kind of fine fne-grained error annotation also comes from the human evaluation data.
2.3 Error-Guided Instruction
Experimental Setups
The potential problem of contrastive instruction is that, it only tells the LLMs that the two translations have quality differences but clarify which kind of translation errors lead such differences. How- ever, we want LL.Ms to learn the correspondence between the errors and the translations. With such a deeper understanding on the translation errors,
3.1 Training Data
Alpaca Data. This dataset is built by Stanford Al- paca (Taori et al., 2023)' project, which contains 52.0K instruction- foollwition-oolloiingg data of multi-tasks for
hhttps:/ thub. com/ http:/tatullabbsaee wtr.laaaaoaoaaaaaaasaaalo

---

tuning the LLaMA (Touvron et al., 2023)² mod- els. We call these data general instructions, which help the resulting ParroT models to maintain capa- bilities on general tasks.
including the one in Table 1, the seeds to ask GPT-4 (OpenAI, 2023) paraphrase them. In total, we have different instructions that are randomly combined with the training examples.
WMT Validation Data. We use human-written validation data from previous WMT compe- titions rather than public training data to avoid introducing noises into instruction tun- ing. In this version, we use the newstest2017- 2020 of Englissh (i.e., Eh+>En) and Englah+English (i.e., Eee tasks, which con- sist of 51.2K sentence pairs for all the four direc- tions. These sentence pairs are formed into the translation instructions.
3.2 Model Training
We conduct our experiments with HuggingFace Transformers' on open-source LLMs from both the LLaMA family (Touvron et al., 2023) and the BLOOM family (Scao et al., 2022). Specifically, choose LLaMA-7b and BLOOMMZ-7bl- with matched parameters, and also include LLaMA-13b and BLOOMZ-560m to study the effect of model sizes. We finetune them to the following variants:
Alpaca, as a reimplementation of the Stan- ford Alpaca model fine-tuned only on the Al- paca multi-task dataset.
MQM Human Evaluation Data. Our human feedback data comes from the Multidimensional Quality Metries (MQM) datasets (Freitag et al., 2021)², which annotate the different translation errors (e.g., major accuracylmisstaanslttio,, minor fluency/grammar) of top WMT systems. Due to its higher reliability than Direct Assessment, MQM was introduced to WMT competitions starting from WMT20 but only provided for few language pairs. In this version, we use the MQM data for the WMT20 En and En submissions. These data are formed into the contrastive instructions (i.e., 20K) based on the quality scores and the error- guided instructions (i.e., 26K) based on the error annotations, respectively.
ParroT-T, finetuned on the Alpaca multi-task dataset and only the translation instructions from WMT validation data.
ParroT, finetuned on the Alpaca multi-task dataset, and all the three types of instructions introduced above.
ParroT-LoRA, finetuned by low-rank adap- tation (LoRA) with default hyper-parameters from alpaca-lora', which results in only 4.2M tunable parameters based on LLaMA-7b.
The hyper-parameter for finetuning are basically consistent with Stanford Alpaca (Taori al., 2023), We finctune the Alpaca and ParroT-T models for 3 epochs on the corresponding data combination. For ParroT and ParroT-LoRA, we finetune them for 1.5 epochs maintain similar training steps as P-rroT-T. We conduct finetuning on Nvidia A100 GPUs and utilize DeepSpeed ZeRO stage 3 for model parallel.
Automatically Assessed Data. Although the Di- rect Assessment (DA) data of WMT systems pro- vide scores for language directions that lack MQM data (i.e., En, Zh), we find the DA score to be very unreliable as they could be quite differ- ent for two similar translations. Instead, we opt for automatic evaluation metrics like COMET to score the translations of WMT systems. We also heuristically determine rough error level for each translation based on the COMET score, namely, Major Error: [0, 85]; Minor Error: (85, 90]; No Er- ror: (90, 100]. This decision comes in part from the observation that top commercial systems achieve COMET scores of nearly 90 on the Flores sub- sets (Table 3). Finally, we obtain 24K contrastive instructions and 29K error-guided instructions.
3.3 Evaluation
Test Data. We evaluate the translation perfor- mance of LLMs two sources of test sets:
Flores Subset: This dataset is a subset of Flores benchmark, in which 50 sentences are sampled for German, English, Romanian and Chinese, respectively. for evaluating the translation per- formance of ChatGPT (Jiao et al., 2023)
Note: To obtain set of diverse instructions. we use the three instructions in Jiao al. (2023),
'https://gi htttsss//gttuub...ommgggignnnn/t http://wgggngggeeeeerrrrnrrrrrra hhttps://github com/tloen/alpacaa htttp:: thub. http://cmmmiiccofff///eeepeeeee
/https://githul wwt../cccmoomeeeeeoe wawpenn-hhmaaeeaaluttaaeeee

---

Table 2: Ablation study of key factors on Flores En= subset with Alpaca based on LLaMA-7b.
inference. Therefore, we study if such an operation makes any performance variation.
Prompt
Instruct.
Search
BLEU
COMET
Instruction Variation. Recent studies (Jiao al., 2023; Zhang et al., 2023) suggest that LLMs are sensitive to task instructions, which could vary the translation performance considerably. a brief study for this by comparing the TP1 and TP3 instructions Jiao et al. (2023). Tp1 is the one pre- sented in Table while TP3 is "Please provide the [TGT] translation for the following sentences." ", which was demonstrated better choice when tested on ChatGPT
sample
20.0
80.0
TP1
beam 4
22.1
79.1
no-input
TP3
sample
19.4
79.0
conduct
beam 4
21.5
79.0
sample
21.0
79.5
TPl
beam 4
23.3
80.5
input
sample
19.3
78.6
TP3
beam 20.6
80.0
WMT22 Test Sets: We also use the test sets from WMT22 competition (Koemi et al., 2022), which are constructed based on more recent con- tent from various domains, including news, so- cial, e-commerce, and conversational domains. The numbers of samples for De=>En, En=DDe, Zh-En and En-ZZh tasks are 1984, 2037, 1875 and 2037, respectively.
Search Algorithm. In machine translation, the beam search strategy (Sutskever et al., 2014; Fre- itag and Al-Onaizan, 2017; Vaswani et al., 2017) has been the standard search algorithm for in- ference. However, beam search requires high computation costs which becomes infeasible with the LLMs, since they can casily induce out-of- memory (OOM) issues. Therefore, more efficient search algorithms such as sampling may have to be the choice. Therefore, we compare the sampling strategy (Taori et al., 2023) and the beam search strategy with beam size of 4 for this factor.
For models based on BLOOM, we only evalu- ate them on WMT22 test sets since the Flores benchmark has been used in the development of BLOOMZ models.
Metrics. For automatic evaluation, we adopt BLEU (Papineni et al., 2002) implementated in SacreBLEU (Post, 2018)?, and COMET (Rei et al., which
Table presents the results of these ablation studies. We have the following observations: (1) The prompt-input performs slightly better than prompt-no-input though the gap is marginal. (2) The Tp1 instruction works better on Alpaca than TP3 which is different from that on ChatGPT. (3) Generally, beam search outperforms sampling significantly, especially in terms of BLEU score. Therefore, we use prompt-input + Tp1 + beam search as the default setting for inference.
2020 from are driven by n-gram similarity and cross- lingull pretrained models, respectively.
4 Results
4.1
Ablation Study
Before diving into more experiments, we investi- gate some factors that may affect the translation performance of LLMs. By default, we conduct the ablation studies on the Flores En subset with the Alpaca model based on LLaMA-7b.
4.2 Main Results
Table and Table 4 present the translation perfor- mance of LLaMA and BLOOM models on the test sets. For Flores subsets, we include the baseline results reported in Jiao et al. (2023).
Prompt Format. In the Alpaca multi-task dataset, about 60% examples contain empty "### Input:", which results in two different prompt for- mats during finetuning, i.e., prompt-input and prompt-no-input. During inference, they use -nopptt which combines the instruction and input to fill the "### Instruction:" field, intro- ducing the inconsistency between finetuning and
Instruction tuning exploits the potential of vanilla LLMs for machine translation. Table 3 shows that the vanilla LLaMA-17 7b without any fur- ther training performs badly the Flores subsets. By inspecting the outputs, we find that the vanilla LLaMA-7b model tends generate very long sen- tences (e.g., copy the instructions, continuing expansion), which makes the generated text not
hhttpp://ggtuuu..mmmooott/aaaaa "https: http://tthubbccmm//bbbbblll/////
hhttss://chat.ogenai..co

---

Table 3: Translation performance of LLaMA models on Flores subsets and WMT22 test sets.
De->En
En De
Zh->En
En->Zh
System
BLEU
COMET
BLEU
COMET
BLEU
COMET
BLEU
COMET
Flores
Subsets
Google
45.0
88.7
41.1
88.6
31.6
87.7
43.5
88.4
DeepL.
49.2
89.7
41.4
89.0
31.2
87.3
44.3
88.
ChatGPT
43.7
89.1
38.8
88.1
24.7
85.8
38.2
86.9
GPT-4
46.0
89.3
45.7
89.2
28.5
87.4
42.5
88.4
Base Model:
LLaMA-7b
Vanilla
3.4
60.1
49.0
1.8
53.7
0.1
47.6
Alpaca
36.6
86.8
23.3
80.5
15.1
81.2
9.8
58.6
Alpaca-LoRA
40.7
87.7
24.6
84.0
16.4
81.5
14.5
70.5
ParroT-T
41.3
87.7
28.5
83.3
19.5
83.1
24.7
79.9
ParroT
41.0
87.9
30.8
84.3
19.2
83.9
25.8
80.1
Infer w/ Prefer.
38.1
87.6
23.0
83.9
18.6
83.1
22.5
80.1
+ Infer w/ No Err.
42.2
88.7
32.1
84.9
83.7
27.4
81.8
T-LoAA
43.8
88.3
29.0
84,9
16.9
80.6
14.8
71.5
Infer w/ No Err.
42.0
88.0
29.8
85,4
17.4
81.3
19.8
76.7
WMT22
Test Sets
Google
33.3
84.8
38.4
87.1
28.6
80.9
49,9
87.4
DeepL.
32.8
84.7
36.2
87.9
24.2
79.3
44.5
86.4
GPT-4
33.4
84.9
34.5
87.4
24.8
82.3
41.3
87.0
Base Model:
LLaMA-7b
Vanilla
2.9
52.8
1.6
45.3
1.2
50.3
0.3
46.3
Alpaca
27.8
82.3
20.1
78.1
14.2
74.0
10.4
62.1
Alpaca-LoRA
28.9
22.1
81.3
16.1
75.6
16.3
70.6
ParroT-T
26.6
82.5
24.0
80.4
18.1
75.3
27.0
78.4
ParroT
27.3
82.4
24.6
81.2
18.9
75.2
28.1
79.3
+ Infer w/ No Err.
27.3
82.4
26.1
81.6
20.2
75.9
30.3
80.3
Parro ToRAA
28.8
82.8
24.0
81.4
18.2
74.7
19.9
73.7
+ Inffer w/ No Err.
29.8
83.0
24.8
81.6
19.2
75.0
20.7
74.5
Base Model:
LLaMA-13b
Alpaca
29.7
83.1
21.4
79.4
16.2
75.9
17.6
70.8
PairoTT
27.6
83.2
27.0
82.8
19.9
75.8
30.9
8I.I
+ Infer w/ No Err.
31.1
83.6
28.1
82.6
21.7
76.7
31.7
81.0
faithful to the source sentences and also not gram- matically correct. The reason could be the long context modeling during pretraining. Another rea- son is that we use the Alpaca inference format, which is basically zero-shot setting that exhibits no guidance for translation.
unseen in the pretraining of LLaMA mod- els (Touvron et al., 2023). The findings of these LLaMA-basd A-based models are also consistent with that on the WMT22 test sets.
Learning from low-quality translations anno- tated by humans is also important. While pre- senting the high-quality bilingual pairs to LLMs is important, as discussed above, we argue that low-quality translations annotated by humans also bring benefits. As shown in Table 3, without hint in inference, ParroT outperforms ParroT-T slightly on translation directions from English to other lan- guages (i.c., En En=Zh). However, when asking ParroT generate translations with ror, the performance can be significantly improved across translation directions and test sets. We spec- ulate that ParroT does learn the relationship be- tween errors and translations by error-guided in-
Tuning LLaMA-7b on the Alpaca multi-task dataset (i.e., Alpaca) can ameliorate the above is- sue, resulting in complete generations with proper lengths. We find that Alpaca performs much better on translation, which may benefit from the 0.5% translation instructions in the Alpaca multi -tas dataset. However, the best performance is mainly observed high-resource dircctions like De= En, due the dominant language of Alpaca dataset in English. Further introducing small amount of translation instructions (i.e., ParroT-T) in the four language directions can significantly improve the performance, especially for En+Zh, in which Chi-

---

Table 4; Translation performance of BLOOM models on WMT22 test sets.
De->En
En De
Zh->En
-n-
System
BLEU
COMET
BLEU
COMET
BLEU
COMET
BLEU
COMET
Base Model: BLOOMZ-560m
Alpaca
4.4
55.2
0.5
30.8
6.9
70.1
2.0
54.0
ParroT
16.4
68.9
13.3
57.7
16.0
74.8
25.4
79.0
+ Infer w/ No Err.
16.9
69.3
12.8
56.8
15.7
75.0
26.3
79.5
Base Model: BLOOMZ-70
Alpaca
17.6
73.0
3.1
44.5
13.0
76.4
23.9
81.8
ParroT
23.1
77.6
20.0
772.7
21.4
78.5
32.4
83.6
+ Infer w/ No Err.
24.9
78.0
20.5
73.6
22.7
79.0
34.5
83.5
Table Effects of error levels as hints during inference. Green degradation.
eters also hinder the learning of instructions from other translation directions. Obviously, the hyper- parameters of LoRA should also be properly ad- justed better learn from more instruction data.
Red
improvement;
En
Zh- En
Hint
BLEU
COMET
BLEU
COMET
LLMs families and sizes also matter. For both LLaMA and BLOOM families, larger models can achieve much better translation performance after instruction tuming. Our ParroT framework proves to be effective across all the models. Comparing the two LLMs families, the ParroT model based BLOOMZ-7bl-nt performs much better Zh->En and En->Zh directions than those based LLaMA-7b, which mainly results from the better modeling of Chinese during the pretraining process of BLOOM.
None
30.8
84.3
19.2
83.9
No Err.
32.1
84.9
21.5
83.7
Minor Err.
28.8
83.6
20.6
82.1
Major Err.
28.5
82.9
19.3
80.5
struction, such that it can avoid the translation er- much possible when the hint of error is provided.
bit unexpected is that when asking ParroT to generate preferred translations, the performance drops considerably. As stated in Section 2.3, con- trastive instruction only indicates that two trans- lations may have quality differences but not state why, which is difficult for LLMs to identify by themselves. Previous study by Min et al. (2022) also suggests that it easier for LLMs to learn the instruction formats rather than the input-response patterns, which may explain the phenomenon here.
Automatic evaluation tools can be effective in constructing error-guided instructions. In Sec- tion 3.1, we construct the automatically assessed data for De=EE and En that are not provided with the MQM data. As shown in Table 3 and Ta- ble 4, of error-guided instruction on these two translation directions. demonstrates the potential of auto- matic evaluation tools (i.e., COMET) in providing the quality information of translations, as an aug- mentation to translation directions that lack human annotation data.
observe considerable improvements
Parameter efficient finetuning may prevent LLMs from overfitting. We also try low-rank adaptation (LoRA, Hu al., 2022) to finetune par- tial parameters of LLMs for efficiency. Experimen- tal results in Table show that Alpaca-LoRA out- performs its full model counterpart noticeably. We speculate that LoRA can prevent LLMs from over- fitting the small Alpaca multi-tašs task dataset, leading to stronger generalization ability. However, ap- plying LoRA to ParroT exhibits distinct behaviors for high-resource and low-resource translation di- rections. Specifically, ParroT-LoRe outperforms the corresponding full model ParroT on De but performs much worse on the other directions. It seems that the small amount of tunable param-
4.3 Analysis
We conduct more analyses to understand the effects of our instruction types. By default, we the model variants based on LLaMA-7b, and the Flores subsets.
Effectiveness of Error-Guided Instruction. To understand how error- error-guided instruction works, we investigate the behavior of ParroT when asking it to generate translations with varied error levels as

---

Table 6: Examples from Flores Zh-En subset when translated with varied error levels. Red
translation errors.
COMET
Example
SRC
n/a
#1 f PL
# 33 1 A1 18
REF
n/a
For example visiting castles the Loire Valley, the Rhine valley or taking a cruise to interesting cites on the Danube or boating along the Erie Canal.
None
81.2
For example, visiting the castles in the
Rhine Valley and the Lahn Valley taking cruise along
along the river taking a trip along the Illi River Rhine Valley the Ruhr Valley , taking cruise the citics along the Rhine River or taking boat ride along the River
the Rhine River explore cities
No
81.4
For example, visiting the castles the
Minor Err.
78.1
For example, visiting the castles in the Rhine Valley and the Ruhr Valley taking cruise visit
the cities along the Rhine River taking boat to play in the cities along Elbe River
Major 65.3
For example, visiting the fortress the valley of River Volga and River Levan taking cruise to visit the cities along the River Volga or taking a boat to play in the cities along the River Volga
Table Effects of preference as hints during inference. Red improvement:
between the "preferred" and "unpreferred" (i.e., the second translation in the response) translations. Surprisingly, as shown in Table 7, the "unpreferred" translations obtain much higher BLEU score but the situation is different for the COMET score. It indicates that ParroT attempted to identify the qual- ity differences between the first and second transla- tions in the contrastive instructions through lexical choices, which low-level pattern reflect the translation quality. One potential reason is that the WMT systems are so competitive with each other that the quality differences between them are too subtle for the LLM to learn effectively. We will investigate more about contrastive instruction in future work.
Green degradation.
En
Zh->En En
Hint
BLEU
COMET
BLEU
COMET
None
30.8
84.3
19.2
83.9
Prefer.
23.0
83.9
18.6
83.1
Unprefer. 29.1
83.7
19.6
82.3
hints. As shown in Table 5, the translation quality is getting worse from no error to minor error to ma- jor error, especially in terms COMET score. The translations generated with no hint are usually com- parable with the minor error level. It demonstrates that ParroT can place erroneous translations into other locations of the probability space with the reg- ulation of human annotations. As result, ParroT is more likely generate high-quality translation with "no error".
Related Work
LLMs for MT. With the increasing capacity of LLMs, they have become good few-shot learn- ers (Brown et al., 2020; Lin et al., 2022) on various NLP tasks, including machine translation. A num- ber of recent studies focus on how prompt LLMs for machine translation, including prompt template comparison (Zhang et al., 2023), few-shot example selection (Agrawal et al., 2022; Vilar et al., 2022), domain adaptation (Moslem et al., 2023), and rare word translation (Ghazvininejad et al., 2023). How- ever, our ParroT framework aims to develop instant translation capability for chatbots without few-shot examples. This is consistent with the performance of ChatGPT and GPT-4 (OpenAI, 2023), which ex- hibit excellent translation ability (Jiao et al., 2023; Bang et al., 2023; He et al., 2023; Liang et al., 2023) during chat.
For qualitative analysis, we show an example from Flores Zh En subset in Table 6, in which we highlight all errors in each translation. Com- pared to no error level, minor and major error levels tend to produce more over-translations and mis- translations. It is also important to point out that no error level does not guarantee that completely correct translations will be generated, especially for named entities, which we attribute to the under- explored translation abilities of current LLMs.
Failure of Contrastive Instruction. We try to understand why contrastive instruction does not work. By examining the responses of ParroT when asking it to generate preferred translations, we observe significant differences in lexical choices

---

Instruction Tuning. To eliminate the reliance on few-shot examples, recent studies also try to finetune LLMs on a small amount instructions covering different NLP tasks, making the LLMs zero-shot learners (Mishra et al., 2022; Wei et al., 2022). With the emergence of various power- ful open-source LLMs such as BLOOM (Scao et al., 2022) and LLaMA (Touvron et al., 2023), there has been boom for creating instruction data and tuning customized chatbots, for example, Al- paca (Taori al., 2023), Vicuna, WizardLM (Xu
also translate previous chat records when users struggle comprehend responses in foreign lan- guages.
Contrastive Translations: In this study, we did not observe performance improvements related to contrastive instructions, possibly due to incor- rect instruction formatting. By exploring alter- native formats, such automatic post-editing (APE), we could potentially capitalize on the advantages of contrastive translations.
et al., 2023) and the like. However, most of these studies focus on developing chatbots that are ca- pable of general NLP tasks, while we pay more attention to machine translation. More importantly, apart from the instructions built from parallel trans- lation data, we also transform human feedback data into instructions and demonstrate its effectiveness in improving the translation performance.
The current analysis did not reveal consistent performance improvements when using LoRA as compared to full model training. It may be necessary to adjust the number of tunable parameters according the dataset size for better results.
LoRA Effectiveness:
6 Conclusion
References
We propose ParroT to enhance and regulate the translation abilities during chat based on open- source LLMs, human-written translation and feed- back data. We reformulate translation data into the inslluction-follooingg style, and introduce "Hint" field for incorporating extra requirements to regulate the translation process. Accordingly, we propose three instruction types for finetuning ParroT models, i.c., translation instruction, con- trastive instruction, and error-guided instruction. Experiments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs sig- nificantly while error- error-guided guided instruction can lead to further improvement, demonstrating the impor- tance of lcarning from low-quality translations an- notated by humans. While we only use three in- struction types in this paper, it is natural to ex- tend ParroT to other hints (e.g., entity alignments), which we leave for future exploration.
Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. 2022. context examples selection for machine translation. arXiv.
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie. Holy Lovenia. Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hal- lucination, and interactivity.
Tom Brown, Benjamin Mann, Nick Ryder, Mclanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models are few-shot learners. NewrIPS.
Akhbardeh Farhad, Arkhangorodsky Arkady, Biesialska Magdalena, Bojar Ondřej. Chatterjee Rajen, Chaud- hary Vishrav, Marta R Costa-jussa, España-Bonet Cristina, Fan Angela, Federmann Christian, et al. 2021. Findings of the 2021 conference on machine translation (WMT21). In WMT.
Markus Freitag and Yaser Al-Onaizan. 2017. Beam search strategies for neural machine translation.
Limitations
ACI.
This work performs a preliminary exploration on the instant translation capability for chatbots, which can be further improved in the following aspects:
Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. 2021. Experts, errors, and context: A large-scale study of human evaluation for machine translation. TACL.
Marjan Ghazvininejad, Hila Gonen, and Luke Zettle- moyer. 2023. Dictionary-basee phrase-level prompt- ing of large language models for machine translation. arXiv.
Instruction Variants: Presently, the instruc- tions only support the translation of incoming sentences. may be beneficial for chatbots to

---

Zhiwei He, Tian Liang Wenxiang Jiao, Zhuosheng Zhang. Yuju Yang, Rui Wang, Zhaopeng Tu, Shum- ing Shi, and Xing Wang. 2023. Exploring human- like translation strategy with large language models. arXiv.
Yasmin Moslem, Rejwanul Haque, and Andy Way. 2023. Adaptive machine translation with large language models. arXiv.
Reham Omar, Omij Mangukiya, Panos Kalnis, and Es- sam Mansour. 2023. ChatGPT versus traditional question answering for knowledge graphs: Current status and future directions towards knowledge graph chatbots. arXiv.
Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu, and Mrinmaya Sachan. 2022. Adapters for enhanced modeling of multilingual knowledge and text. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3902-3917.
OpenAL. 2023. GPT-4 technical report. arXiv.
Long Ouyang, Jeff Wu, Xu Jiang. Diogo Almeida, Car- roll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. 2022. Training language models to follow instruc- tions with human feedback. arXiv.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang, Lu Wang. Weizhu Chen, al. 2022. LoRA: Low-rank adaptation of large
language models. In ICLR.
Wenxiang Jiao, Zhaopeng Tu, Jiarui Li. Wenxuan Wang. Jen-tse Huang, and Shuming Shi. 2022. Tencent's multilingual machine translation system for WMT22 lcrge-scale african languages. WMT.
Santanu Pal, Sudip Kumar Naskar, Mihaela Vela, and Josef van Genabith. 2016. neural network based approach to automatic post-editing. In ACI..
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: method for automatic evaluation of machine translation. In ACL.
Wenxiang Jiao, Wenxuan Wang. Jen tse Huang, Xing Wang, and Zhaopeng Tu. 2023. Is ChatGPT a good translator? Yes with GPT-4 as the engine. In ArXiv.
Matt Post. 2018. A call for clarity in reporting BLEU scores. WMT.
Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, al. 2017. Google's multilingual neural machine translation system: Enabling zero-shot translation. TACL.
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: neural framework for MT evaluation. EMNLP.
Teven Le Scao, Angela Fan, Christopher Akiki, El- lie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. BLOOM: A 176b- parameter open-access multilingual language model. arXiv.
Tom Koemi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grund- kiewicz, Barry Haddow, et al. 2022. Findings of the 2022 conference on machine translation (WMT22). In WMT.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence sequence learning with neural networks. NeurIPS.
Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang. Yan Wang, Rui Wang. Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking large language models through multi-agent debate. arXiv.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction following 1lama model. https: gi thub. www...com/ttt..eeeeeeaeoe.ee...a wtwm commtaasu--lal lab/ stanford_alppsc
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Na- man Goyal, Shruti Bhosale, Jingfei Du, 2022. Few-shot learning with multilingual generative lan- guage models. In EMNLP.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foun- dation language models. arXiv.
Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. 2023, Chain hindsight aligns language models with feed- back. arXiv.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannanch Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In EMNLPP
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. 2017, Attention is all you need. NeurIPS.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generaliza- tion via natural language crowdsourcing instructions. In ACL.
David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. 2022. Prompt- ing PaLM for translation: Assessing strategies and performance. arXiv.

---

Shuo Wang, Peng Li. Zhixing Tan, Zhaopeng Tu. Maosong Sun, and Yang Liu. 2022. A template- based method for constrained neural machine transla- tion. In EMNLP.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2022. Finetuned language models are zero-shot learners. In ICLR.
Haoran Wu, Wenxuan Wang. Yuxuan Wan, Wenxiang Jiao, and Michael Lyu. 2023. ChatGPT or Gram- marly? Evaluating ChatGPT grammatical error correction benchmark. arXiv.
Can Xu, Qingfeng Sun, Kai Zheng. Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv.
Xianjun Yang. Yan Li, Xinlu Zhang, Haifeng Chen, and Wei Cheng. 2023. Exploring the limits of ChatGPT for query aspect-based text summarization. arXiv.
Biao Zhang, Barry Haddow, and Alexandra Birch. 2023. Prompting large language model for machine transla- tion: A case study. arXiv.
A Flores Full Sets
We adopted the subset of Flores order directly compare with the results of ChatGPT and com- mercial MT systems from the previous evaluation report (Jiao et al., 2023), as stated in Section 3.3. However, for more convineing evaluation, we also test the LLMs on the full sets of Flores (i.e.. with 1012 sentences), and list the results in Table 8. Obviously, the trend of performance across sys- tems still holds, and our "ParroT + Infer w/ No Err" setting performs the best.
B General Tasks
In our experiments, include the Alpaca multi- task dataset for finetuning ParroT preserve its ability on general tasks. We present examples in Table 9, from question answering and code gen- eration, respectively. The responses of both Alpaca and ParroT are correct, although the expressions may be different. For comprehensive comparison, it is better to perform quantitative evaluation and we leave it for future work.

---

Table 8: Translation performance of LLaMA models on Flores full sets.
De
En De
Zh En
En-Zh
System
BLEU
COMET
BLEU
COMET
BLEU
COMET
BLEU
COMET
Base Model:
LLaMA-7b
Vanilla
3.0
58.2
1.7
48.8
2.7
52.0
0.1
48.6
Alpaca
36.3
87.6
23.0
81.1
16.4
81.3
8.3
56.5
ParroT-T
37.4
87.9
26.9
83.7
19.7
82.8
26.1
79.2
ParroT
38.1
88.0
28.9
84.5
21.1
83.0
27.0
80.3
+ Infer w/ Prefer.
34.6
87.5
24.8
83.9
18.1
82.4
24.2
79.0
Infer w/ No Err.
39.4
88.4
30.7
84.6
21.3
83.4
29.1
81.2
Table 9: Examples of general tasks. The base model is L.a.MA-7b.
System
Question Answering
Tell me about alpacas.
Alpaca
Alpacas are a species of camelid native to South America. They domesticated bred their fiber, which is used in the production of knitted and woven items such as clothing, blankets, and rugs. Alpacas are typically kept as livestock, and can live up to 20 years in captivity. They are typically brown, white, or grey in color, and grow up inches tall the shoulder.
ParroT
Alpacas are a species of domesticated South American camelids, bred primarily for their fiber. They typically kept as pets, for their fleece, for their meat. Alpacas very social animals and live herds, with dominant male leader and number females and young. They are very intelligent and can be trained to respond to basic commands. Alpacas require a lot of space and good quality food in order to thrive.
System
Code Generation
Write Python program that prints the first 10 Fibonacci numbers.
Alpaca
numbers [0, I]
for i in range(2, 10):
numbers mmmbersspppend/mmmbrrsj/)) numbers(i-1])
print(numbers)
ParroT
def libonacci(n): if n = 0:
retur 0
clif n = 1:
return
else:
return fibonacci(n- fibonacidd-))fiiioaaceee..
prin(((bonacci(10)


Reviews:
[{"review_id": "TbYmYRiHw2", "reviewer": "Reviewer_k66w", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's efforts in reviewing our paper. \n\nWe would like to clarify our contributions here.\n1. We propose a novel framework to incorporate extra requirements into instructions to regulate the translation process. While we mainly introduce instruction types based on human written translations and annotations, the framework is also compatible with other data resources (e.g., entity alignment).\n\n2. We conduct extensive experiments to compare our instruction strategies (translation instruction, error-guded instruction, and contrastive instruction) with the prevelant strategies (i.e., Alpaca type). We demonstrate the effectiveness of our instructions across four translation directions, upon two LLMs (i.e., LLaMA-7b and BLOOMZ-7b1-mt), with full model finetuning and LoRA tuning.\n\n3. We also demonstrate the potential of automatic evaluation tools in providing the quality information of translations, as an augmentation to translation directions that lack human annotation data. In Section 3.1, we constructed the Automatically Assessed Data for De->En and En->Zh, which do not have the MQM data. In Table 3, we can also observe considerable improvements of error-guided instruction on De->En and En->Zh.\n\nNonetheless, there could be other aspects to further improve this paper. We would appreciate if the reviewer can suggest some specific directions, and are glad to refine the paper accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The topic of this paper is interesting and of broad interest to the community of NLP.", "reasons_to_reject": "Though the authors proposing a novel framework to incorporate human feedback for improving machine translation, this paper does not sufficiently demonstrate significant advantages over existing methods. The incremental contributions are not convincingly conveyed through thorough quantitative analysis and comparative experiments. More evidence is needed to highlight the novelty beyond incremental improvements to justify publication at top NLP venues.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors propose the ParroT, a framework to enhance and regulate the translation abilities during chat based on open-sourced LLMs~(e.g., LLaMA), human-written translation and feedback data.", "excitement": 1, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UJ19osJUPT", "reviewer": "Reviewer_jRLd", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's efforts in reviewing our paper. \n \n1. Concerning the number of languages, we have two considerations.\n    - While scaling languages is a promising direction for machine translation (MT), it is not our focus in this work. We aims to investigate different strategies of instruction-following on LLMs for MT, and conduct experiments on two representative language pairs (i.e., De-En and Zh-En) that have been widely used in WMT competitions and MT researches.\n \n    - Another reason is the scarcity of existing high-quality human annotation data for MT, as stated in Section 3.1. However, with the rapid development of LLMs, human annotators have been much more involved, making it feasible to obtain human evaluation data for other languages. Besides, we also demonstrate the potential of automatic evaluation tools (e.g., COMET) in providing the quality information of translations, as an augmentation to translation directions that lack human annotation data. In Section 3.1, we constructed the Automatically Assessed Data for De->En and En->Zh, which do not have the MQM data. In Table 3, we can also observe considerable improvements of error-guided instruction on De->En and En->Zh. In addition, recent studies on LLMs as evaluators (e.g., AutoMQM [1]) may also serve as potential alternatives to label the error patterns of translations for more languages.\n\n    Nonetheless, it will be interesting to see how our method performs when extended to more languages. We plan to investigate this in the near future. \n\n\n2. Thanks for the good advice. We adopted the subset of Flores in order to directly compare with the results of ChatGPT and commercial MT systems from a previous evaluation report, as stated in Section 3.2. However, for a more convincing evaluation, we also test the LLMs on the full set of Flores ( with 1012 sentences), and list the results as below (Base Model: LLaMA-7b). Obviously, the trend of performance across systems still holds, and our \"ParroT-Hint + Infer w/ No Err\" performs the best. \n\n    | System | De-En (BLEU) | De-En (COMET) |  En-De (BLEU) | En-De (COMET) |  Zh-En (BLEU) | Zh-En (COMET) |  En-Zh (BLEU) | En-Zh (COMET) | \n    | ------ | ------: | ------: | ------: | ------: | ------: | ------: | ------: | ------: |\n    | Vanilla |  3.0 | 58.2 | 1.7 | 48.8 | 2.7 | 52.0 | 0.1 | 48.6 |\n    | Alpaca |  36.3 | 87.6 | 23.0 | 81.1 | 16.4 | 81.3 | 8.3 | 56.5 |\n    | ParroT |  37.4 | 87.9 | 26.9 | 83.7 | 19.7 | 82.8 | 26.1 | 79.2 |\n    | ParroT-Hint |  38.1 | 88.0 | 28.9 | 84.5 | 21.1 | 83.0 | 27.0 | 80.3 |\n    |  + Infer w/ Prefer. |  34.6 | 87.5 | 24.8 | 83.9 | 18.1 | 82.4 | 24.2 | 79.0 |\n    |  + Infer w/ No Err. |  **39.4** | **88.4** | **30.7** | **84.6** | **21.3**  | **83.4** | **29.1** | **81.2** |\n\n\n[1] The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation. Fernandes et al. arXiv 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper does a detailed analysis of what is effective for instruction tuning of LLM for machine translation", "reasons_to_reject": "1. The paper conducts evaluation for few language pairs including De<->En and Zh<->En\n2. In the evaluation on FLORES they show results with a subset of only 50 sentences. Thus the results for FLORES may not be reliable.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes to instruction tune LLM for machine translation. They experiment with different ways of instruction tuning including instruction for direct translation, instruction for a preferred translation over a less preferred translation (contrastive instruction), and instruction for controlling the amount of error in the generated translation.\nThey evaluate with two LLMs: LLaMa 7B, and BLOOMZ-7B parameters. They also compare between full finetuning and LoRA based finetuning.\n\nContributions:\n1. The paper compares between different ways of instruction tuning for machine translation\n2. They find that direct instruction tuning and error-guided instruction tuning works well.\n3. They find that contrastive instruction is not effective.\n4. Also LoRA based finetuning is more effective for high resource language pair", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mNCvS5yCV8", "reviewer": "Reviewer_AfZp", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's efforts in reviewing our paper. \n\nConcerning the question of scaling languages, we would like to share our ideas with the reviewer.\n\n1. It is true that human annotation data is readily available only for few languages. However, with the rapid development of LLMs, human annotators have been much more involved, making it feasible to obtain human evaluation data for other languages.\nBesides, we also demonstrate the potential of automatic evaluation tools (e.g., COMET) in providing the quality information of translations, as an augmentation to translation directions that lack human annotation data. In Section 3.1, we constructed the Automatically Assessed Data for De->En and En->Zh, which do not have the MQM data. In Table 3, we can also observe considerable improvements of error-guided instruction on De->En and En->Zh. \nIn addition, recent studies on LLMs as evaluators (e.g., AutoMQM [1]) can also serve as potential alternatives to label the error patterns of translations for more languages. \n\n\n2. In this work, we focus on investigating different strategies of instruction-following on LLMs for MT, and conduct experiments on two representative language pairs (i.e., De-En and Zh-En) that have been widely used in WMT competitions and MT researches. Generally, our error-guided instructions can bring the most improvements over baselines across different translation directions. We believe our approach can generalize to more languages and plan to investigate it in the near future.\n\n[1] The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation. Fernandes et al. arXiv 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper is well written and easy to follow\n* The approach is interesting, the experiments indicate gains over vanilla baselines, and the paper comprises some interesting analyses and insights.\n* Different approaches for fine-tuning are included, i.e. regular FT and LoRA, showing interesting differences in performance w.r.t. language data sources.  ", "reasons_to_reject": "* The approach assumes the availability of human labeled data / data which comprises information on error patterns for translations, which appears to be readily available only for few datasets and languages. Scaling to further languages may thus be rather difficult.  \n* The choice of languages in the experiments is rather constrained (likely due to the previous point), and it is thus unclear how well it would work for further languages (given that differences are already being identified for the explored languages). \n\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper aims at improving machine translation performance of pre-trained LLMs (with the final goal of bringing MT capabilities of open-source LLMs closer to those of models which are only accessible via services, such as ChatGPT); this appears to be a relevant direction of work. Specifically, the paper proposes ParroT which aims at increasing LLM translation capabilities during chat by leveraging human-written translations as well as quality information about translations (specifically human annotations pointing to errors). The information is provided to the LLM via different types of instructions, which include “hints” fields. Specifically, the authors test with using contrastive instructions which comprise 2 translations of different quality, and error-guided instructions which explicitly hint the LLM on errors in example translations. The approach is being evaluated with one LLM mostly trained on English data (i.e. LLaMA) and one multlingual model (i.e. BLOOMZ-7b), indicating that the proposed approach with error-guiding yields improvements over the vanilla approach as well as over using instruction tuning with alpaca dataset. The authors compare full fine-tuning vs fine-tuning with LoRA, indicating interesting difference in performance across language pairs, and some further analyses and interesting insights are provided. However, results are constrained to only very few languages, as data comprising annotations on translation quality/errors appears to be limited, and thus the approach may not be easily applicable to many languages (except if corresponding data are being collected which may be expensive), and it is unclear how well it would work for further languages (given that differences are already being identified for the explored languages). ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this work, the authors propose PARROT, a instruction-guided technique to improve the translation abilities of open-sourced LLMs. Translation data is rewritten in instruction-style in three different ways: 1) Using standard translation instructions 2) Using contrastive instructions and 3) Using error-guided instructions. The authors find translation instructions to be very helpful and error-guided instructions to further improve performance. However, contrastive instructions do not yield performance improvements.They also find LoRA-based finetuning to be more effective for high resource language pairs.

R2 and R3 have both rated this work highly on soundness (4/4). R1 listed a number of limitations including the need for more baseline comparisons (esp with larger models), the difference between error-guided and chain-of-thought based methods and the main motivation behind the work. While R1's comments came quite late during the rebuttal window, the authors have offered a fairly thorough response with additional experiments.
Decision: Accept-Findings