Learning-From-Mistakes Prompting for Indigenous Language Translation (2024)

Abstract: Using large language models, this paper presents techniques to improve extremely low-resourced indigenous language translations. Our approaches are grounded in the use of (1) the presence of a datastore consisting of a limited number of parallel translation examples, (2) the inherent capabilities of LLMs like GPT-3.5, and (3) a word-level translation dictionary. We harness the potential of LLMs and in-context learning techniques in such a setting for using LLM as universal translators for extremely low-resourced languages. Our methodology hinges on utilizing LLMs as language compilers for selected language pairs, hypothesizing that they could internalize syntactic structures to facilitate accurate translation. We introduce three techniques: KNN-Prompting with Retrieved Prompting Context, Chain-of-Thought Prompting, and Learning-from-Mistakes Prompting, with the last method addressing past errors. The evaluation results suggest that, even with limited corpora, LLMs, when paired with proper prompting, can effectively translate extremely low-resource languages.

## introduction in recent years, llms have showcased astonishing capabilities in the realm of natural language processing, particularly in tasks like language translation @xcite ), text generation @xcite , and contextual understanding @xcite . the robust functionality of these models has led us to reconsider their potential role in indigenous language translation. in our pursuit to facilitate translations from chinese to taiwanese indigenous languages, we leverage the power of llms, buttressed by three foundational pillars: the presence of a datastore consisting of a limited number of parallel translation examples, the inherent capabilities of llms like gpt-3.5, and the integration of a word-level translation dictionary. in this paper, we delineate three translation methodologies that build upon each other in a cumulative fashion. each method represents a layer in our stratified approach, starting from leveraging contextual similarity in knn-prompting with retrieved prompting context (rpc) to harnessing the didactic potential of chain of thought @xcite prompting, and culminating in the learning-from-mistakes (lfm) prompting technique that incorporates feedback mechanisms for continuous improvement. figure 1 provides an overview of our methodologies, illustrating a step-by-step translation enhancement process designed for the taiwanese indigenous language context. this paper is structured as follows: in section 2, we review the literature and discuss the position of this study. in section 3, we explore the cot prompting methodology, followed by an indepth analysis of the lfm prompting approach. in section 4, we report the evaluation results. through empirical evaluation and expert reviews, we demonstrate the effectiveness of the proposed methodologies. ## related work llms have exhibited excellent performance in language translation tasks, particularly evident in well-represented source languages like english and chinese. despite significant strides in translation performance for these languages, there remains a notable gap in the exploration of llms for low-resourced languages or those that have not been pre-trained. this aspect represents an underexplored area within the domain of research. ## methodology problem setting and assumptions the primary objective of this research is to enable the translation of chinese sentences into taiwanese indigenous languages through the utilization of llms. in pursuit of this goal, we make the following assumptions for the methods proposed in this study: figure 1 outlines our study's methods for enhancing translation in a cumulative manner. the knn-prompting with rpc method forms the base, merging contextually similar sentences and word translations to inform the llm's understanding of grammar and context. the cot prompting adds cot demonstrations, showing rpc integration for effective translation. the lfm prompting expands upon these with a feedback loop, leveraging previous translation errors to refine outcomes. this progressive strategy not only enhances llm's translation proficiency but also promotes continual learning and accuracy improvement. ## conclusion this study delves into the capabilities of llms in translating indigenous languages. despite a limited datastore of parallel translations, our introduced methodologies: knn-prompting with rpc, cot prompting, and lfm prompting demonstrate effectiveness in harnessing llms for this task. emphasizing our technical contribution, empirical results highlight the superior performance of the cot prompting and lfm strategy over the compared baseline, signifying its adeptness at capturing intricate linguistic nuances and offering an advanced approach to preserving linguistic diversity.