Exploring Word Alignment towards an Efficient Sentence Aligner for F ilipino and C ebuano Languages (2022)

Abstract: Building a robust machine translation (MT) system requires a large amount of parallel corpus which is an expensive resource for low-resourced languages. The two major languages being spoken in the Philippines which are Filipino and Cebuano have an abundance in monolingual data that this study took advantage of attempting to find the best way to automatically generate parallel corpus out from monolingual corpora through the use of bitext alignment. Byte-pair encoding was applied in an attempt to optimize the alignment of the source and target texts. Results have shown that alignment was best achieved without segmenting the tokens. Itermax alignment score is best for short-length sentences and match or argmax alignment score are best for long-length sentences.

## introduction word alignment is the task of discovering the corresponding words or terms in a bilingual sentence pair @xcite . word-aligned corpora are a great source of translation-related knowledge. the estimation of translation model parameters usually relies heavily on word-aligned corpora @xcite . therefore, the alignment of words is a crucial stage in the process of building a machine translation system @xcite . sentence alignment is the task of aligning sentences in a document pair @xcite or in a parallel corpus. in most cases, these sentence pairs share the same meaning or are contextually translated. abundance of these parallel corpora is very evident for highly resourced languages while the collection and even building of such parallel corpus for low-resourced language is very difficult and is a very tedious task @xcite . the problem of aligning words in massively parallel texts containing hundreds or thousands of languages remains mostly unexplored @xcite and that includes the filipino and cebuano languages . the philippines has a scarcity of language resources, particularly parallel corpora. several studies have been conducted in an attempt to build a parallel corpus involving philippine languages and most of them are paired with the english language @xcite @xcite . the study of @xcite had focused on the collection and building of both monolingual and parallel corpus for filipino and cebuano to build an @xcite for the said languages. the abundance of the collected monolingual corpora for the said language pair presents an opportunity for it to be transformed into a parallel corpus using an aligner. cebuano and filipino are the two most spoken languages in the philippines where the structure of these languages are morphologically-complex. filipino language is flexible when it comes to word order. in fact, some filipino sentence can be rearranged up to 6 different ways since sentence structures like svo, vso, vos are accepted. while the cebuano language, it is said to be predicated which explains why it follows the vso format @xcite . these languages can contribute and get benefits from our existing technology in different aspects, especially for machine translation. translation studies and contrastive linguistics rely heavily on parallel corpora which are crucial for developing high-quality machine translation systems @xcite . the transformation of the available monolingual corpus would be an addition to the existing filipino-cebuano parallel corpus. to date, the checking of the translation and the generation of parallel corpus is done manually which is a very laborious and tedious task especially when you have hundreds of thousands of sentences. to the best of our knowledge, there is still no word and sentence aligner effective for cebuano and filipino. in this paper, the researchers aim to conduct a preliminary investigation on the use of a word aligner for cebuano and filipino languages towards the development of an efficient sentence aligner and evaluate its performance in accordance to some ground truth. ## related works there have been different word alignment approaches that are widely used especially for machine translation. this section discusses the related studies of word and sentence alignment for machine translation. the study of @xcite describes a method for improving statistical machine translation (smt) performance in multiple bridge languages by leveraging multilingual, parallel, sentence-aligned corpora. their solution includes a simple way for creating a word alignment system using a bridge language and a mechanism for integrating word alignment systems from various bridge languages. the researchers provide studies that show how this framework can be used to improve translation performance on an arabic-to-english problem by using multilingual, parallel material in spanish, french, russian, and chinese. the paper goes over the many ways and challenges that come up when it comes to word alignment. considering hindi is based on subject object verb "sov" and english is based on subject verb object "svo," this study focuses on the major problem that occurs in word alignment. the report gives a survey on word alignment in the application of machine translation for foreign and indian languages @xcite . @xcite described a strategy for enhancing word alignments by comparing words. this strategy is based on encouraging semantically comparable words to align in the same way. to estimate similarity, they employ word vectors trained on monolingual data. additionally, by increasing the alignments of infrequent tokens, the researchers increase word alignments and machine translation in low-resource settings. to improve the quality of chinese-vietnamese word alignment, @xcite incorporate linguistic relationship factors into the word alignment model. these are sino-vietnamese and content word linguistic relationships. the results of the experiments demonstrated that their strategy en-hanced word alignment as well as machine translation quality. @xcite presents a new statistical machine translation strategy that uses monolingual english semantic parsing to bias inversion transduction grammar (itg) induction and is specifically oriented to learning translation from low resource languages. the study shows that, in contrast to traditional statistical machine translation (smt) training methods, which rely heavily on phrase memorization, the approach proposed focuses on learning bilingual correlations that aid in translating low-resource languages, with the output language semantic structure being used to further narrow itg constraints. @xcite presented a novel approach for constructing and merging complementary word alignments for low-resource languages in order to increase word alignment quality and translation performance. in the study, they construct numerous sets of diverse alignments based on different incentives, such as linguistic knowledge, morphology, and heuristics, rather than focused on improving a single set of word alignments. by integrating the alignments acquired from syntactic reordering, stemming, and partial words, they demonstrate their strategy on an english-to-pashto translation task. with much higher f scores and higher translation performance, the combined alignment surpasses the baseline alignment. the researchers demonstrate that attention weights do accurately capture word alignments and propose two new word alignment induction methods, shift-att and shift-aet. the fundamental idea is to induce alignments when the tobe-aligned target token is the decoder input, rather than the decoder output, as in prior work @xcite . in the study of @xcite , they propose a word-level contrastive objective for many-to-many nmt that takes advantage of word alignments. for various language combinations, empirical studies demonstrate that this results in 0.8 bleu gains. analyses show that the encoder's sentence retrieval efficiency in many-to-many nmt is substantially correlated with translation quality, which explains why the suggested method has an impact on translation. a study where the researchers used hmm-based models that were designed for bitext word and phrase alignment. the models are written in such a way that parameter estimation and alignment can be done quickly. even with massive training bitexts, it has been founnd that chinese english word alignment performance is comparable to ibm model-4 @xcite . ## results and discussions in this section, we discussed the results for evaluating the aligned texts of the sentences with bpe and without bpe using the 4 evaluation measures namely precision, recall, f1, and aer. to determine the best alignment score, ## conclusion sentence aligned parallel corpora are crucial in machine translation and choosing the most efficient aligner in different languages will be of great help in doing nlp tasks. in this study, we have observed that when aligning words, results are favorable when tokens are not segmented with bpe. also, in the alignment from similarity matrices match or argmax are preferred for long-length sentences and itermax for short-length sentences. for future studies, it is recommended to increase the number of sentence pairs in the experimentation of the simalign to maximize the performance of algorithm. it is also recommended to explore a different embedding model that is specific to this kind of language to evaluate how embedding models affect the results of the alignment.