Tracing Source Language Interference in Translation with Graph-Isomorphism Measures (2021)

Abstract: Previous research has used linguistic features to show that translations exhibit traces of source language interference and that phylogenetic trees between languages can be reconstructed from the results of translations into the same language. Recent research has shown that instances of translationese (source language interference) can even be detected in embedding spaces, comparing embeddings spaces of original language data with embedding spaces resulting from translations into the same language, using a simple Eigenvector-based divergence from isomorphism measure. To date, it remains an open question whether alternative graph-isomorphism measures can produce better results. In this paper, we (i) explore Gromov-Hausdorff distance, (ii) present a novel spectral version of the Eigenvector-based method, and (iii) evaluate all approaches against a broad linguistic typological database (URIEL). We show that language distances resulting from our spectral isomorphism approaches can reproduce genetic trees on a par with previous work without requiring any explicit linguistic information and that the results can be extended to non-Indo-European languages. Finally, we show that the methods are robust under a variety of modeling conditions.

## introduction the study of cross-linguistic variation has been a key focus of linguistics for genetic or typological classification of languages. historical comparative linguistic methods determine genetic relationships between languages using concept lists of words with a common origin in multiple languages that share similar meaning and pronunciation @xcite . linguistic typology studies how distinct languages are, and what generalizations can be made regarding cross-linguistic variation on different levels of linguistic analysis and representations @xcite . @xcite , for example, studies language variance in terms of their functional processes, whereas @xcite measures language distance using structural features. more recent research indicates that semantic similarity between languages can serve as a quantitative means to determine cross-linguistic variation across languages. seminal work of @xcite provides evidence that semantic alignment between languages can be explained by geographical factors. likewise, @xcite find that differences correlate with cultural distances among societies speaking the languages. conversely, it has also been shown that language differences are so profound that the structure of a language is approximately preserved even when translated into another language. this is often referred to as source language interference @xcite . @xcite show that source languages of translations into the same target language can be clustered solely based on interference phenomena in the translations in the target language using simple linguistic features and that these clusters correspond with genetic distance. in a similar vein, @xcite find that comparable results can be established by clustering neural language model (nlm) based vectors using raw words, part-of-speech (pos) tags, phrasestructure or dependency-based input sequence representations of the data, showing that the distances between these learned language representations are more reflective of syntactic (structural) similarity rather than genetic relationship. @xcite show that source language interference is even evident in simple word, pos, synset and semantic tag based embedding spaces computed from originally authored and data translated into the same target language. they use a graph-based eigenvector (ev) divergence from iso-morphism distance measure @xcite , originally used for bilingual dictionary induction, to capture divergence from isomorphism between monolingual original and translation embedding spaces. with this, they quantify distances between the source languages of the translations and predict phylogenetic trees, and analyse the correlation between isomorphism measure based distances and genetic relations in language families. however, to date, (i) alternative graph-based distance metrics have not yet been explored for embedding-based approaches to detect translationese; (ii) it is not clear how word-embedding based approaches fare under different data settings including a) varying the number of most frequent words considered in the graphs, b) different corpus sizes and c) different word embedding architectures; (iii) it is not clear how the previous approaches (using either linguistic feature vectors, nlm based feature vectors, or divergence from isomorphism graph-based distance between embedding spaces) compare on the same data against the commonly used gold standard phylogenetic tree of @xcite (sp08); (iv) it is not clear how function words would affect graph-based distances; (v) evaluation of the graph-and embeddingspace approach against the broader uriel typological data base @xcite has not been carried out; and (vi) it is not clear if the scope of this research can be expanded to include non-indo-european languages. in this paper, we show (i) that gromov-hausdorff (gh) distance can be used as a distance metric to quantify divergence from isomorphism between simple embedding spaces in monolingual settings and develop a novel spectral graphbased (sgm) distance measure, extending the original ev-based approach; (ii) that graph-and embedding-based distances are fairly robust under different data settings and that they are not sensitive to skip-gram or cbow-based embeddings; (iii) divergence from isomorphism graph-based measures using embeddings can reproduce genetic trees on a par with linguistic feature vector and nlm based approaches @xcite ; (iv) that function words and concept lists are still relevant within this general approach; (v) that graph-and embedding space-based distance metrics correlate not only with genetic features but also with geographical and syntactic ones @xcite ; and (vi) that this research can be extended to translations from non-indo-european languages. the rest of the paper is organised as follows. we review related work in section 2. section 3 introduces the concept of graph isomorphism and our sgm measure, together with ev and gh. we describe the experimental setting in section 4. in section 5, we report results on the isomorphism metrics, infer language family relationships and correlate them with linguistic benchmarks. we describe robustness experiments in section 6 and compare to previous work in section 7. finally, we extend our analysis to non-indo-european source languages in section 8 and summarize and draw conclusions in section 9. ## related work representational distance between two languages refers to how different one language or language variety is from another. several analyses @xcite have attempted to disentangle the typological factors that influence language representational distance. @xcite clustered languages based on linguistically inspired features of their translations into the same target language and show that syntactic footprints of the source language in the translations can be used to estimate phylogenetic similarities between their source languages. they use agglomerative clustering with variance minimization @xcite as linkage procedure and compare their generated trees (p ) to the pruned gold-tree (g) (sp08) of @xcite . their comparison metric is the sum of squared deviations between each language pair's gold-tree distance d g and corresponding distance in their computed tree d p : it is worth noting that the use of sp08 as a gold standard has also been questioned in the literature. @xcite observes that the sp08 approximation is only suited for a small subset of languages and that it fails to explain finer-grained inconsistencies in the indo-european language family. @xcite expand the work of @xcite in their nlm-and sequencebased approach and argue that representational distance between languages can be better explained by structural relatedness than by language genetics. chowdhury et al. (2020) use departures from isomorphism based on the ev measure @xcite on simple embeddings to infer genealogical distances. they compare different embedding spaces (word, pos, synset or semantic tags) constructed from translations into a single target language and the target language in terms of how similar their corresponding nearest neighborhood graphs are by analyzing their eigenvalues. the similarity between languages can also be measured using the (dis)similarities between their discrete linguistic properties. such properties are typically handcrafted and collected in typological databases such as uriel @xcite which lists a large inventory of properties for 8000 languages of various typological characteristics, such as overlap in syntactic features, or proximity along phoneme features @xcite . uriel is a compilation of a variety of linguistic resources including the world atlas of language structure, wals @xcite , phoible @xcite , ethnologue @xcite , and glottolog @xcite . based on linguistic feature vectors, uriel provides precomputed distance statistics between any language pairs stored in the database in terms of various metrics including genetic, geographical, syntactic, phonological, and phonetic inventory distances. in this work, we follow the approach of @xcite and evaluate our geometrical measures against the phylogenetic benchmark sp08. we compute the branching length directly from sp08, assuming it reflects the actual proportions. additionally, we follow @xcite to compare our generated trees against the average of three precomputed measures of language distance, namely genetic, geographic, and syntactic distances based on the uriel database. ## graph isomorphism we define the distance between languages based on word usage and the notion of isomorphism. an isomorphism f between two metric spaces (x , d x ) and (y, d y ), where x and y are two sets of words in two languages and d x and d y are the metric distances, is a function f : @xmath0.e.: for all pairs of points x 1 and x 2 in x such that, for a vocabulary @xmath1, v 1 , ...., v n in language ℓ, we define its graph as g(v, e, w), where v denotes the set of vertices corresponding to the vo-cabulary words; @xmath2, e 1 , ..., e m is a set of edges; and every pair {v i ,v j } has a non-negative edge weight w ij associated with it. our approach starts with mapping words v ℓ i in language ℓ onto points v i ℓ using distributional semantics methods. each language is then represented with its own graph g ℓ . after mapping words onto points v i ℓ as vectors, the distance between words is defined as the distance between their vectors. we quantify the similarity between languages ℓ 1 and ℓ 2 through a distance function between their graphs in what follows we make the concept mentioned above more concrete. ## experimental setting in this section, we provide information on the data, and the vector spaces used for computing devia-tion of isomorphism. since we quantify language similarity based on the degree of isomorphism in monolingual spaces, we independently train monolingual word embeddings for the target language and translations into that target language. ## results and evaluation we analyze the behaviour of the different distance measures: the gromov-hausdorff distance (gh) and the two eigenvector similarity-based ones (sgm and ev). we apply them to (i) infer language families, (ii) reconstruct phylogenetic trees, and finally (iii) perform correlation analysis against two benchmarks (sp08 and uriel). ## robustness analysis after showing that exploiting departures from isomorphism between spaces can be used to predict relations between languages, we analyze the impact of various modeling assumptions and different training conditions that might have an effect in skewing the results. ## analysis for non-indo-european source languages previous research @xcite @xcite focused on investigating translationese and source language interference for european language families. here we extend this work, for the first time, to the best of our knowledge, to translations from non-indo-european languages into english. we explore the language distance measures of section 5.1 on the un corpus @xcite which consists of translations covering typologically different languages such as arabic (ar) and chinese (zh), as well as indo-european languages (i.e., russian (ru), spanish (es) and french (f r)). the embedding spaces are created in the same manner as for europarl dataset. we show results with 1500 words foot_1 in figure 4 and observe the following trends: compared with translations from ar and zh, the difference between original english and translations from f r and es tend to be smaller, the distance to zh is the largest, and that within the european language family distances is mostly f @xmath3. this is in line with our previous results in figure 1 on the same-domain monolingual europarl data under different data settings. however, despite these general trends, gh and ev measured distance scores are similar for es, ar and ru, while ev has f r more distant to en than es, ru and ar. this is something that would need to be further explored in future work. of all measures, sgm, which captures more context from the interaction between data-points and their neighborhoods, accords best with linguistic expectations about language (dis)similarities. ## conclusion in this paper we contribute to the ongoing line of research in computational typology, exploring the potential of translations into a single target language that retains the traces of the source languages to reflect the distances between them. specifically, we propose an alternative graph-based distance measure to explore (dis)similarities between languages. our results show that simple graph-and embedding-based distance based methods perform on a par with the best results achieved by previous approaches based on linguistic features in detecting source language interference in translations. we compare gromov-hausdorff and our novel spectral graph based approach with the original eigenvector-based divergence from isomorphism measure (ev) against uriel and sp08, show that our alternative graph isomorphism measures outperform ev and, for the first time, expand translationese research to non-indo-european languages. we perform robustness tests to verify that our methods are stable under a variety of modeling conditions. in future work, we aim to leverage our estimated similarities to better explain transfer behavior (local information spreading from one language to the other) in downstream applications such as machine translation.