Exploring the Advantages and Challenges of a Concept-Guided Approach in Large Language Model Aided Machine Translation: Integrating Generative AI And Human-like Cognition (2024)

Abstract: Humans outperform large language models (LLMs) on sophisticated tasks because human cognition involves a range of cognitive functions and their dynamic interactions. This study explores how integrating human cognition through concept-guided instruction and few-shot teaching in the prompt can guide LLMs to improve translation outcomes. We first demonstrate that for simple and widely used concepts, concept-guided prompting approaches offer significant benefits. We then test prompt engineering with Chinese-to-English translation examples, using hypothetical spaces—generated by GPT4—to estimate the complexity of various concepts and Likert scores—generated by human experts—to evaluate the translation performance. Our findings show that LLM translation performance declines as concept complexity increases. We also identify additional challenges: LLMs struggle with continuity in explaining and practicing sophisticated concepts due to the lack of human-like cognitive functions, such as cognitive dissonance. Additionally, LLMs lack a graceful speed-accuracy tradeoff because they do not possess the dynamic information processing, response strategies, and performance assessment that humans do. However, LLMs can mitigate some of these challenges by using Chain-of-Thought (CoT) reasoning, which is especially effective for problems requiring consistent, well-structured reasoning steps. Despite this, LLMs can only represent the effects of complex human cognitive functions through (often) fragmented linguistic descriptions, whereas humans excel at understanding critical and broader contexts and the interconnections between cognitive aspects.

## introduction humans possess superior capabilities compared to large language models (llms) because human cognition emcompasses a range of mental functions: language processing, perception, attention, memory, reasoning, problem-solving, planning, decision-making, and executing actions. according to triadic cognition theory @xcite , dual process theory @xcite , embodied cognition (shapiro 2014), and cognitive psychology (neisser 2014), human cognition arises from the dynamic interaction of these functions, yielding a more nuanced, multifaceted understanding of real-world contexts than a simplistic and static computational model. recent advancements in large language models (llms) have shown that while these models still do not match the full breadth of human cognition, they are making significant strides in specific areas such as machine translation @xcite . unlike traditional machine translation (mt) tools such as google translate and deepl, which rely solely on the source language text, llms possess the ability to follow instructions and can even learn from bilingual training examples provided within the prompt. this capability allows llms to provide context-sensitive translations with improved performance, making them as a more adaptable and user-friendly option to traditional methods @xcite @xcite . concepts serve as a common entity used across multiple cognitive functions, both in human minds and large language models. john mccarthy's insight, "to understand natural language is to understand the concepts in the language, not just the words," @xcite ) captures the fundamental cognitive principle that language is a tool for conveying and manipulating concepts, and true understanding requires engaging with those underlying mental constructs rather than simply processing linguistic symbols. one important approach to guide the llmbased translation outcome is to provide concept definitions and examples through prompt engineering. in section 2, by experimenting with many examples associated with common sense concepts (e.g., systems of measurement such as the metric system and the imperial system) and crosscultural conceptualization, we show that, given proper prompts in the form of pre-editing analysis, concept-guided translation prompt instruction, and post-editing revision, llms can successfully reflect the relevant concepts in the translation outcome. on the other hand, for more sophisticated translation concepts, we want to test whether performance of the concept-guided approach degrades as the concepts become more complex. in section 3, we test prompt-engineering-based implementations of multiple translation-related concepts using chinese-to-english translation examples. by using hypothetical spaces to represent the complexity levels of various concepts, we verify that llms' translation performance degrades as concept complexity level increases. also in section 3, we identify and analyze several challenges faced by the conceptguided approach: llms lack continuity to explain and practice sophisticated concepts; and compared with humans, llms lack graceful speed-accuracy tradeoff. these challenges stem from the fundamental differences between llms and human cognition. as artificial systems designed solely for generating text, llms' internal processes do not mimic or engage the complex cognitive functions and dynamic interactions typical of the human brain. consequently, these limitations in simulating human-like cognition contribute to the observed deficiencies. nevertheless, llms have been shown to exhibit human-like cognitive functions such as reasoning patterns @xcite . section 4 demonstrates that llms can use chain-of-thought (cot) reasoning @xcite ) to overcome some challenges and enhance the performance. this is particularly effective for subproblems that benefit from well-formed, consistent reasoning steps, ultimately leading to the correct conclusion. however, while humans use multiple cognitive functions to solve complex problems, llms can only project this complexity, often in fragmented form, as linguistic descriptions. again, these broader perspectives and interconnections are critical strengths of the complex cognitive functions and dynamic interactions typical of the human brain. section 5 concludes our findings. ## advantages of concept-guided preediting analysis, translation instruction, and post-editing revision the role of concepts across multiple cognitive functions is multifaceted and central to human cognition: facilitating reasoning and generalization (william 2010), guiding decision-making by providing a framework for understanding and categorizing new information @xcite , integrating multiple features and dimensions of information @xcite to support flexible cognition @xcite , bridging episodic and semantic memory @xcite , and creating functionally unitary representations that can be accessed and used across different cognitive tasks @xcite . in this section, we showcase several examples where concept-guided prompting has significant advantage by using llm-based pre-analyses, translation instructions and post-editing revisions. ## challenges faced by the concept-guided approach in this section, we test chinese-to-english translation concepts with varying levels of difficulty. we found that while llms perform well with simple and straightforward concepts, they encounter significant challenges when handling more sophisticated concepts (qian and kong 2024a, qian and kong 2024b). ## chain-of-thought (cot) reasoning provides a structured way for llms to support sophisticated concepts in this section, we explore methods to address the limitations identified previously. we focus on chain-of-thought (cot) reasoning prompting as a solution and evaluate the pros and cons of this approach. ## conclusion humans outperform large language models (llms) because human cognition involves a range of cognitive functions and their dynamic interaction. this study explores how integrating human cognition through concept-guided instruction and few-shot teaching can guide llms to improve translation outcomes. we first demonstrate that for simple and widely used concepts, concept-guided approaches offer significant benefits. we then tested prompt engineering with chinese-to-english translation examples, using gpt-4 to estimate the complexity of various concepts and human experts to evaluate translation outcomes' quality. our preliminary findings show that llm translation performance declines as concept complexity increases. we also identify several challenges in the concept-guided approach: llms struggle with continuity in explaining and practicing sophisticated concepts due to the lack of human-like cognitive functions, such as cognitive dissonance. additionally, llms lack a graceful speed-accuracy tradeoff because they do not possess the cognitive functions such as dynamic information processing, response strategies, and performance assessment that humans do. however, llms can mitigate some of these challenges by using chain-of-thought (cot) reasoning, which is especially effective for problems requiring consistent, well-structured reasoning steps. despite this, llms can only model cognitive complexity as fragmented linguistic descriptions, unlike humans, who excel at understanding broader contexts and interconnections between cognitive aspects. this limitation makes it unrealistic to expect llms to effectively define solution steps automatically, a task that humans can accomplish by naturally considering critical context. the findings from this preliminary study on the english-to-chinese language pair indicate that further research is necessary to validate the key results. deverbalization is a concept in translation studies, par-ticularly relevant in the context of translating between languages with significantly different structures, such as chinese to english. the core idea of deverbalizatio is to move away from a direct, word-for-word translation, which often leads to awkward or inaccurate results due to the differences in grammar, syntax, and cultural contexts. instead, the translator interprets the meaning of the source text at a deeper, more abstract level, and then re-expresses this meaning in the target language. three examples are listed below to illustrate the chinese-to-english translation reflecting the concept above： chinese: 过去带动高增长的主要是高投资，而 高投资主要是三大需求来源，出口、基础建 设、房地产。 english: the key driver of rapid growth has been investment, which is needed in three areas/goes to three areas, export, infrastructure, and real estate. chinese: 本土化人才了解中国市场上的需求和 消费者的心理。 english: local talents understand the market needs in china and know what customers want. chinese: 中国现在很多地区复工率很高。 english: many areas in china have reopened their economies.