Visualizing C ross‐ L ingual Discourse Relations in Multilingual TED Corpora (2021)

Abstract: This paper presents an interactive data dashboard that provides users with an overview of the preservation of discourse relations among 28 language pairs. We display a graph network depicting the cross-lingual discourse relations between a pair of languages for multilingual TED talks and provide a search function to look for sentences with specific keywords or relation types, facilitating ease of analysis on the cross-lingual discourse relations.

## introduction discourse relations in texts describe how two discourse segments-which can be phrases, sentences, or even paragraphs-are connected logically to each other. for example, given the following two sentences, "i have cooked some pasta." and "i am hungry.", we may infer that the latter is a cause for the former, in which case, the discourse relation between them can be classified as an implicit causation. @xcite now, when we translate these sentences into another language, the grammar and pragmatics of that language may prefer a certain discourse relation over the others that may not be the same as that of the original language. for example, in the case of korean, which is an agglutinative language, the heavy usage of korean postpositions (particles) dictates that the causal relation in the above example is revealed explicitly (regardless of the usage of explicit discourse connectives). such a cross-lingual discourse analysis can provide some insights for improving the quality of (machine) translation (meyer and webber, 2013; meyer and poláková, 2013; guzmán et al., 2014; iruskieta et al., 2015; chen et al., 2020). this paper presents an interactive system that visualizes the cross-lingual discourse relations using two human-annotated multilingual discourse datasets @xcite it is implicit because these sentences are not connected with an explicit discourse marker such as "because". (zeyrek et al., 2019; long et al., 2020) derived from ted talks following the penn discourse treebank (pdtb) (prasad et al., 2008) framework. our interactive data dashboard provides users with an overview of relation preservation among all language pairs. we also display a graph network depicting the cross-lingual discourse relations between a pair of languages for a given ted talk, and provide a search function to find sentences with specific query or relation types. we believe that, with this visualization tool, users can easily browse through the multilingual talks and discover which parts of the talks share similar discourse relations, and where they diverge. ## preprocessing datasets we abbreviate the two discourse-annotated datasets, zeyrek et al. (2019) and long et al. (2020), as ted-mult and ted-zh, respectively. ted-mult involves seven languages (english, german, lithuanian, polish, portuguese, russian, and turkish); and ted-zh, two languages (english and chinese). thus, our analysis is conducted across the 28 possible language pairs. in order to facilitate cross-lingual analysis, we first performed sentence level alignment using a sentence segmentation tool foot_0 based on conditional random fields and a set of heuristic rules. we note that the reason for using this particular tool is that it guarantees the reconstruction of original texts when segmented sentences are joined together; and that it supports custom regex-based rules to add to allowlist or denylist if need be. once the ted talk scripts for all eight languages were segmented into sentence level, we cross-lingually matched the english sentences with sentences from other languages by the following procedure: when aligning two non-english documents (lang.x:lang.y), we used the alignments with the english (lang.x:english and lang.y:english) as a pivot to cross-lingually match them. to ensure the quality of the resulting alignments, we checked and revised each of them manually. we note that the manual correction was seldom required. level-1 level-2 level-3 temporal synchronous -asynchronous precedence succession contingency cause reason result neg. result cause+belief reason+belief result+belief cause+speech act reason+speech act result+speech act condition arg1-as-cond arg2-as-cond condition+speech act -negative-condition arg1-as-neg. cond arg2-as-neg. cond negative-condition+ speech act -purpose arg1-as-goal arg2-as-goal comparison concession arg1-as-denier arg2-as-denier concession+speech act arg2-as-denier+speech act contrast -similarity -expansion conjunction -disjunction -equivalence -exception arg1-as-excpt arg2-as-excpt instantiation arg1-as-instance arg2-as-instance level-of-detail arg1-as-detail arg2-as-detail manner arg1-as-manner arg2-as-manner substitution arg1-as-subst arg2-as-subst ted-mult and four talks from ted-zh, which are aligned with other languages. overall, there are about 400 sentences per language, and ## system description our system is implemented using streamlit 3 which is an open-source web application framework in python. the aligned dataset with codes 4 and the demo system 5 are available online. the system is organized into three sections, "overall patterns", "pairwise talks", and "search", where each section can be accessed via a navigation panel on the left-hand side. ## conclusion we presented an interactive system that visualizes the cross-lingual discourse relations among multilingual ted talks, along with other related statistics. as future work, we plan to integrate an online multilingual discourse parser and a multilingual machine translation model into the system so that users can translate and analyze any given document and its translation on the fly.