K pop MT : Translation Dataset with Terminology for Kpop Fandom (2024)

Abstract: While machines learn from existing corpora, humans have the unique capability to establish and accept new language systems. This makes human form unique language systems within social groups. Aligning with this, we focus on a gap remaining in addressing translation challenges within social groups, where in-group members utilize unique terminologies. We propose KpopMT dataset, which aims to fill this gap by enabling precise terminology translation, choosing Kpop fandom as an initiative for social groups given its global popularity. Expert translators provide 1k English translations for Korean posts and comments, each annotated with specific terminology within social groups’ language systems. We evaluate existing translation systems including GPT models on KpopMT to identify their failure cases. Results show overall low scores, underscoring the challenges of reflecting group-specific terminologies and styles in translation. We make KpopMT publicly available.

## introduction one of the most profound distinctions between humans and machines lies in their ability to form a new language system. while machines learn from and rely on existing corpora, humans have the unique capability to establish new social conventions, such as agreeing to call an apple "apple". this inventive ability is deeply intertwined with the social nature of language, wherein words gain influence and become integrated into the vernacular of social groups -termed as social dialect -through social interactions and contexts. in light of this, our research places a particular emphasis on specific phenomena: social groups often develop their unique linguistic systems, replete with specific terminologies and jargon, as in in order to promote research in this direction, we argue for the necessity of a benchmark dataset that encompasses the language systems of social groups. this dataset is also crucial for demonstrating the capabilities of the current translation systems specifically tailored to handle the nuances of social group languages. in our study, we reference the field of terminology-based mt @xcite , which focuses on accurately translating specialized terminology in machinegenerated output. notably, the medical domain has seen significant advancements in this area @xcite , highlighted by the release of datasets that aid the nlp community, particularly those interested in terminology translation and constrained neural mt tasks @xcite . we propose a terminology-tagged mt dataset tailored for social groups, named kpopmt. through human survey, we demonstrate participants in a social group show a strong preference for reference translations in kpopmt compared to translations without terminology. we choose the kpop fandom as the social group for this initiative, given its global popularity and the widespread sharing of content on social media platforms among fans from diverse countries @xcite . kpopmt is collected from korean posts and comments found on fan-related websites, including x (twitter). expert translators fluent in the terminologies used within this social group provide english translations, resulting in 1k sentence pairs, each annotated with specific terminologies. we evaluate existing translation systems, including state-of-the-art models such as gpts, on kpopmt to identify areas of improvement and establish baseline performance for future research. in to ensure translation reliability, we get confirmation from five native english kpop fans who know both korean fandom terms and english fandom terms. ## experiments we evaluate existing machine translation systems on kpopmt to assess its difficulty. ## conclusions our approach acknowledges the dynamic and evolving nature of language, especially in digitally mediated communities, which are often underrepresented in traditional linguistic resources. we propose a benchmark dataset for social groups' language systems, named kpopmt. 1k parallel dataset contains not only korean-english parallel sentences but also terminology information of kpop fandom group. we also make termbase and monolingual data publicly available. we evaluate existing translation approaches on kpopmt to identify their failure cases. our future plan includes expanding kpopmt to encompass other social groups, such as sports and global movie communities.