Edit distance is commonly used to relate cognates across languages. This technique is particularly relevant for the processing of lowresource languages because the sparse data from such a language can be significantly bolstered by connecting words in the lowresource language with cognates in a related, higher-resource language. We present three methods for weighting edit distance algorithms based on linguistic information. These methods base their penalties on (i) phonological features, (ii) distributional character embeddings, or (iii) differences between cognate words. We also introduce a novel method for evaluating edit distance through the task of low-resource word alignment by using editdistance neighbors in a high-resource pivot language to inform alignments from the lowresource language. At this task, the cognatebased scheme outperforms our other methods and the Levenshtein edit distance baseline, showing that NLP applications can benefit from information about cross-linguistic phonological patterns.
REF use contextbased character embeddings for cognate discovery and propose a method to discover cognates in a low-resourced language via a better-resourced pivot language.
Phonologically Informed Edit Distance Algorithms for Word Alignment with Low-Resource Languages
Computer Science