Using C hat GPT as a CAT tool in Easy Language translation (2023)

Abstract: This study sets out to investigate the feasibility of using ChatGPT to translate citizen-oriented administrative texts into German Easy Language, a simplified, rule-based language variety that is adapted to the needs of people with reading impairments. We use ChatGPT to translate selected texts from websites of German public authorities using two strategies, i.e. linguistic and holistic. We analyse the quality of the generated texts based on different criteria, such as correctness, readability, and syntactic complexity. The results indicated that the generated texts are easier than the standard texts, but that they still do not fully meet the established Easy Language standards. Additionally, the content is not always rendered correctly.

## introduction generative pre-trained transformer (gpt) models show remarkable advances not only in natural language generation @xcite , but also in automated translation @xcite . however, their performance in specific machine translation tasks has not been yet extensively explored. while the online chatgpt is also known for its ability to translate texts from one language to another (so-called interlingual translation), so far very little is known about its ability to translate texts from standard language into a complexity-reduced language variety of the same language (so-called intralingual translation). in this study, we investigate the feasibility of using chatgpt to translate citizenoriented administrative texts from public agencies into easy language for people with reading impairments. the aim of our study is twofold: first, to answer the question of whether and to what extent large language models like chatgpt are able to generate translations from standard german into easy language and second, to determine whether a holistic or a text-based approach leads to a more comprehensible output. in our study, we aim to test whether chatgpt is fit to be used as a tool for translating texts into german easy language. we selected a number of texts and ordered the tool to perform several translation tasks with two different strategies: linguistic level dependent and holistic. in this paper, we present the results of the qualitative and quantitative data from the analysis and on their basis, we draw the first conclusions on the usability of chatgpt as an easy language cat 1 tool. the remainder of this paper is structured as follows. in section 2, we outline the relevance of the demand for the current situation with easy language in germany. in section 3, we summarize the existing related work. in section 4, we describe the data used in this study and outline our research design. in section 5, we present our main results and lastly, in section 6, we discuss our main findings and suggest promising directions for future research. ## easy language in germany in germany, the federal act on equal opportunities of persons with disabilities @xcite states that public authorities must on request explain official notices, general rulings, public-law contracts and forms in easy language to individuals with intellectual or psychological disabilities. however, since many german public authorities still do not provide sufficient information in easy language, these rights to information in easy language are often disregarded. one of the main reasons for the lack of information in easy language is that translating texts into easy language is very timeconsuming and costly and requires professional training (cf. @xcite . consequently, the problem is twofold: on the 1 computer-aided translation 2 one hand there is an increasing demand for texts in easy language and, due to the legal situation, a significantly rising text volume to be translated @xcite , and on the other hand, people who are responsible for translating texts into easy language often lack sufficient translation experience and expertise @xcite . at the moment, easy language translations are not always carried out by academically trained translators but also often by persons without academic translation training, such as employees of public authorities or organizations or social education workers, who have access to the easy language target groups but who do not have "the necessary text expertise and adequate formal training in intralingual translation" @xcite . this lack of professional academic training leads to a heterogeneous and often very poor text quality. this is problematic for several reasons: firstly, texts in easy language are only functional when they are of high quality. this is especially true for target groups with special communication needs who may not be able to understand poorly written texts (for an overview of the easy language target groups see @xcite . in addition, poorly written texts can also negatively affect the public image of easy language and may even stigmatize its users (cf. maaß 2020). furthermore, texts of poor quality are detrimental to the development of machine translation systems for easy language, because the successful training of such systems requires a large corpus of rule-consistent high quality easy language translations (cf. hansen-schirra et al. 2020). thus, texts of poor quality hinder the compilation of such a corpus and therefore slow down the advancement of automatic text simplification systems that can be used for intralingual translation. however, due to the significantly rising text volume and the lack of professional translators, the need for technological assistance is obvious. as texts in easy language are based on defined rules on the word, sentence and text level, easy language is often treated as a controlled language (cf. @xcite . this in turn means that, seen from a theoretical perspective, they offer a high automation potential. yet, due to the above-mentioned reasons, texts in easy language are mostly translated manually. as large language models (llms) like chatgpt are trained on large amounts of data from the inter-net and use this data to generate new content, it is conceivable that llms like chatgpt are able to reduce the complexity of a text by applying strategies of lexical and syntactic simplification. seen from a theoretical perspective, it therefore seems plausible that llms have the potential to convert a source text into a text version that is easier to read and understand. however, at this time we are not aware of any studies that evaluate the feasibility of such systems for intralingual translation. ## research design 4.1 data collection to test the chatbot chatgpt foot_1 for intralingual translation into german easy language, we used twenty texts from three different websites of german public authorities. each text contained between 179 and 672 words. the texts contained information about different citizen-oriented topics, such as how to report lost and found items, how to take parental leave, or how to obtain a criminal record certificate. in our study, we tested two different approaches: as human translators usually follow a holistic approach when translating a text, our first approach corresponds to a natural translation strategy. however, as german easy language is a controlled language that is characterized by specific rules on text, sentence, and word level, it is also conceivable that simplifying the linguistic levels separately improves the machine generated output. in our second approach, the so-called linguistic level dependent approach, we therefore adapted our prompts to the text, sentence and word level respectively. starting with the holistic approach, we first asked the tool to translate the following text into german easy language. however, when looking at the generated output, it quickly became clear that the texts did not follow the common german easy language rules and were still too complex. for example, the independent clause-only principle was violated and the texts still contained complex nominal phrases. therefore, in a second step we requested chatgpt to make the text easier. this request was formulated twice. afterwards, we tested the second approach. in this approach, we tried to simplify the source texts step by step, according to the strategies that are applied in easy language translation. we differentiated between simplifying strategies on text level, sentence level and word level. starting from the text level, we first asked chatgpt to reformulate the text but to leave out unimportant information. in a second step, we requested the tool to reformulate the text without compound sentences and with simple syntactic structures. in a third step, we requested chatgpt to add explanations of difficult words in the text. in our analysis, we only considered the final outputs of the two approaches, i.e., the version the tool generated after each of the respective last query. ## summary and discussion the present paper focused on the feasibility of using chatgpt for intralingual translation, i.e. translation of administrative texts into german easy language. our results show that in terms of readability, the generated texts are easier than the source texts, however, most of the texts still do not meet the easy language standards. in other words, the texts are easier, but not easy enough. furthermore, the content of the texts was not always correct. however, in terms of correctness, it should be noted that classifying a text as "incorrect" does not mean that the entire content was incorrect. in most texts that were labelled as incorrect, most of the content was transferred correctly and only one small piece of information was incorrect, or some crucial information was missing, which in turn led to the fact that the message differed from the source text. all in all, our results allow us to conclude that so far, chatgpt might be used as a template for professional translators rather than a standalone easy language translation tool. the conclusion that in easy language translation, human translators are still indispensable is also due to the fact that only parts of the translation can be performed by adhering to simplification rules. even when all rules are applied, there are still some tasks that require the translator's specialized knowledge, creativity and understanding and awareness of the target group. therefore, in addition to the text perspective, a functional easy language translation also has to focus on the reader and has to be adapted to the reader's prior knowledge. this for example means that the translator, on the one hand, has to select and prioritize the information for its users and, on the other hand, has to add paraphrases, examples and explanations. as information is processed and retained more easily if presented in a multimodal and multicodal way, the translator also has to include images to reflect, clarify, or exemplify the subject-related information and to highlight core concepts and associations. this shows that even though in easy language translation there clearly is a potential for automation, the translation task consists of much more than applying text-based rules. thus, if translators use chatgpt to translate texts into german easy language, they need to have professional post-editing competences for intralingual translation, such as error detection, research, and correction skills. however, the more we engaged with the topic, the more we learnt how to get more precise and tailored outputs, i.e., we learnt that other -more appropriate -prompts can improve the comprehensibility of the generated texts. one way to improve the quality of the answers is to assign chatgpt a role. for example, when telling the tool that it is a translator for easy language before asking it to translate a text into easy language, it seems that the output is less complex than without the previous role assignment. chatgpt seems also to deliver more appropriate outcomes if a context is set before asking for a translation. for instance, it may be helpful to ask chatgpt about german easy language rules and then ask for a translation into german easy language. a set contextual framework may deliver more appropriate results. still, the extent to which the versions differ from each other still has to be investigated in a larger-scale study. considering that there are no prompting instructions when opening chatgpt, we expect that the av-erage user is not aware of these techniques i.e., they do not know that assigning a role or setting a context improves the quality of the output. this highlights the paramount importance of professional competences when using these kind of tools in intralingual easy language translation. in our future work, we will extend the evaluation techniques applied, as we have focused on those commonly used in german easy language research so far. we will also include further automated evaluation and quality estimation methods derived from automatic text simplification. moreover, we would like to more closely look into different cases of partial correctness mentioned above, where only piece of information was incorrect or missing.