Prompting LLM s: Length Control for Isometric Machine Translation (2025)

Abstract: In this study, we explore the effectiveness of isometric machine translation across multiple language pairs (En o De, En o Fr, and En o Es) under the conditions of the IWSLT Isometric Shared Task 2022. Using eight open-source large language models (LLMs) of varying sizes, we investigate how different prompting strategies, varying numbers of few-shot examples, and demonstration selection influence translation quality and length control. We discover that the phrasing of instructions, when aligned with the properties of the provided demonstrations, plays a crucial role in controlling the output length. Our experiments show that LLMs tend to produce shorter translations only when presented with extreme examples, while isometric demonstrations often lead to the models disregarding length constraints. While few-shot prompting generally enhances translation quality, further improvements are marginal across 5, 10, and 20-shot settings. Finally, considering multiple outputs allows to notably improve overall tradeoff between the length and quality, yielding state-of-the-art performance for some language pairs.

## introduction accurate and concise translations are increasingly needed in media applications such as subtitling @xcite and dubbing @xcite @xcite @xcite , where length constraints are critical. dubbing, in particular, requires translations to stay @xmath0, a constraint known as isometric machine translation. the 2022 isometric mt shared task @xcite found that most participating systems used lead tokens for length control, with some incorporating reranking or adjusted positional embeddings. recent work also explored rein- with the rise of large language models (llms) @xcite , there has been a shift toward prompting @xcite @xcite and fine-tuning @xcite for mt. prompting strategies notably affect performance, especially in few-shot settings @xcite . studies found that randomly selected examples often improve results @xcite yvon, 2023), though performance gains plateau beyond five examples @xcite . while models like bloom tend to overgenerate in zero-shot settings (bawden and yvon, 2023), fine-tuning methods such as qlora @xcite have shown superior performance over few-shot learning. real-time adaptive mt has also demonstrated strong results, with models like chatgpt rivaling traditional mt systems @xcite . the use of llms for mt has led to the exploration of various prompt templates, with simple structures like '[src]: [input] \n [tgt]:' proving effective @xcite @xcite ). the impact of example selection has also been examined, confirming that beyond five-shot settings, improvements become marginal @xcite @xcite . given these insights, we explore the application of llms to isometric mt, focusing on length control strategies. we analyze four prompting approaches: (1) uncontrolled translation, (2) isometric translation (±10% length variation), (3) samelength translation, and (4) shorter translation, each paired with corresponding demonstration sets. experiments are conducted on eight open-weight models (llama 3, gemma 2, qwen 2 of two sizes each, and mistral and mixtral) across 0, 5, 10, and 20shot settings for @xmath2, @xmath3, and @xmath4, following the 2022 isometric shared task setup @xcite . our results show that few-shot demonstrations affect translation outputs, but precise length control requires well-aligned instructions reflecting example properties, as summarized in figure 1 . additionally, we show that generating multiple outputs with different example sets substantially improves length control, matching competitive isometric mt systems and offering high potential for synthetic data creation in training encoder-decoder models. we publicly release all collected data for potential future analyses. foot_0 ## experimental setup development first, we conduct experiments with multiple settings (varying prompt type, the type of pools of demonstrations, and shot count in few-shot learning) to identify the best-performing configuration for length control. we refer to this as the development setup and use the following data: ## prompts in our experiments, we use english as the language of the prompts @xcite and explicitly specify the source and target languages within the prompt @xcite . our focus is on length control when testing various prompt formulations. while large language models (llms) show strong performance in machine translation, they sometimes lag behind supervised neural models @xcite @xcite . to our knowledge, length control has not been extensively explored for llms in machine translation. prompt construction we construct prompts by concatenating template parts and replacing placeholders with the appropriate values. the random (uncontrolled) template instructs the model to generate a translation of the source sentence without any length restrictions. in the isometric template, the model is instructed to generate a translation @xmath5. the same template instructs the model to produce a translation that exactly matches the source text length, while the short / tiny template directs the model to generate a shorter translation, as the length ratios between studied language pairs often exceed 1, and a standard translation (typically longer) is not desired. a detailed overview of the prompt templates is in we evaluate models in zero-shot and few-shot settings. they often overgenerate, adding explanations or extra translations, as noted by @xcite . while these authors used regular expressions to extract translations, we prevent this by explicitly instructing models to output only the translation, which proves effective. further analysis is in appendix a. ## analysis in all experiments, the prompts remain identical across all models within a given setting. to reduce the bias of sampling from demonstration sets, we performed 10 runs for every setting. ## final evaluation since llama and gemma models achieve the best performance on all language pairs on average, we select and evaluate them in final evaluation using the isometric shared task blind set. we generate outputs using 10 distinct sets of 10 examples (10-shot), each drawn from the tiny pool as it yields the best length control. we keep only outputs in @xmath6, i.e. cometkiwi foot_7 score @xcite . we then compare these results with the submissions from the iwslt 2022 isometric shared task, specifically those from the apptek (wilken and matusov, 2022), hw-tsc @xcite , amazon prime video (apv), and nuv teams @xcite , in addition to the two (strong and weak) baselines provided by the organizers. for a brief overview of each system, please refer to @xcite . additionally, we compare our results to an oraclebleu setting, where the best translation is selected according to the sentence bleu score across all configurations after filtering out translations that fall @xmath7. the results are summarized in our results show that for the @xmath8, the gemma models achieve output quality comparable to the strong baseline. while the translation quality metrics surpass that of the strong baseline, the length control is slightly less precise. for @xmath9, however, the strong baseline continues to outperform our models in terms of quality as well as lr. although generating 10 different outputs for each source sentence may not be feasible in practice, this approach could be beneficial for producing synthetic data for training isometric machine translation models. ## conclusion in this paper, we explored the use of llms for isometric machine translation, focusing on strategies to control the translation length. our key findings are as follows: first, effective length control in fewshot prompting requires the simultaneous use of appropriate demonstrations and matching instructions. second, generating multiple outputs achieves the best trade-off between length control and translation quality, indicating the high capability of llms to generate desired outputs. it might be also useful for creating synthetic training data. although prompting 10 times may seem inefficient, it would not be necessary for every sample in practice. since half of the samples are already length-complianteven with the uncontrolled random prompt -compliance for the rest can be achieved iteratively by generating translations until the length constraint is met. future work might benefit from fine-tuning llms or from a more in-depth analysis of the internal representation of length in llms to avoid many samples to generate.