The Effect of Data Encoding on Relation Triplet Identification (2023)

Abstract: This paper presents a novel method for creating relation extraction data for low-resource languages. Relation extraction (RE) is a task in natural language processing that involves identifying and extracting meaningful relationships between entities in text. Despite the increasing need to extract relationships from unstructured text, the limited availability of annotated data in low-resource languages presents a significant challenge to the development of high-quality relation extraction models. Our method leverages existing methods for high-resource languages to create training data for low-resource languages. The proposed method is simple, efficient and has the potential to significantly improve the performance of relation extraction models for low-resource languages, making it a promising avenue for future research.

## introduction relation extraction (re) is a task in the field of natural language processing, aimed at identifying and extracting semantically meaningful relationships between entities present in text. a relation is generally extracted as an ordered triple (e 1 , r, e 2 ) where e 1 and e 2 refer to the entity identifiers and r refers to the relation type. this task holds great significance in several practical applications, including information retrieval, knowledge management, and question-answering systems, among others (for a recent review, see @xcite ). the increasing availability of unstructured text data on the web has only served to underline the importance of relation extraction, as there is a pressing need to convert this data into structured information that can be eas-ily accessed and analyzed. this challenge is especially pertinent for low-resource languages like icelandic, where the limited availability of annotated data presents a significant impediment to the development of high-quality relation extraction models. relation extraction methods for english have evolved over the years, with early methods relying on hand-crafted rules, patterns, and statistical analyses @xcite @xcite . with the advent of deep learning and the availability of large annotated corpora, more sophisticated methods have emerged @xcite @xcite . deep learning models have shown promising results in extracting relationships between entities, outperforming traditional methods. current state-of-the-art model, rebel (cabot and navigli, 2021), performs joint relation extraction foot_0 . the progress in relation extraction for english has demonstrated the potential for using advanced techniques to extract meaningful relationships from large amounts of text data. the challenge of developing effective re methods for languages like icelandic lies in the scarcity of annotated data. the performance of machine learning models heavily relies on the availability of large amounts of annotated training data. the limited availability of annotated text in lowresource languages creates a major challenge for training high-quality relation extraction models. to overcome this challenge, one way is to study methods to efficiently create training data based on existing methods for english. our main question is whether models from high-resource languages can be used to efficiently create training and testing data for low-resource languages. in this paper, we present a novel method for efficiently creating relation extraction data for low-resource languages like icelandic. our method is based on replacing entities in the text with unique identifiers, translating the text to a high-resource language using machine translation, and then replacing the entities back in the translated text. finally, we perform relation extraction on the translated text to obtain the relationships between entities. our method is simple and only requires the location of entities in the text and a machine translation model. this approach leverages the availability of specialized models in high-resource languages to create training data for low-resource languages, thereby addressing the challenge posed by the scarcity of annotated data. our method has the potential to significantly improve the performance of relation extraction models for low-resource languages, making it a promising avenue for future research. ## purpose in a previous paper, we proposed a way of bootstrapping re training data for low-resource languages (lrl) using a combination of machine translation and open re methods @xcite . by automatically translating the lrl data into english, we were able to feed it directly into the high-resource language sota model before translating the relation triplets back to the lrl where it can serve as training data for a new lrl model. while this method showed potential, the resulting data was jumbled by errors in translation. some examples of this include people's names being perturbed by the multilingual translation model (resulting in alfreð being changed into alfredo, sveinn into sweene etc.), entities getting directly translated and thereby loosing their meaning (such as when the icelandic name erlendur gets directly translated as foreign) and unfortunate translation mishaps (such as when dauðarósir, a novel by the icelandic author arnaldur indriðason, gets translated as deathly hallows, a real novel by a different author). in this paper, we hypothesize that these translation errors can be ameliorated by encoding the entities within the data before it gets translated, and then decoding them before they get sent into the high-resource re model. whereas the proposed method remains the same, this extra step in preprocessing the input data should result in more accurate predictions made by the re model, which in turn makes for better training data. ## previous work machine-translation has previously been used to create cross-lingual named entity recognition (ner) datasets, which led to improvement in ner for several languages @xcite . in these earlier works, the text was translated directly, without any modifications, and the entities in the resulting text were matched heuristically to the entities in the untranslated text using word alignment methods. this works well for entities that translate correctly or change little in the translation process but can be limited by the translation system, specifically if the system translates entities incorrectly. for re in low-resource languages, there has been limited focus on building training and testing data efficiently. however, crosslingual transfer methods have been applied to improve re models, such as using multilingual bert @xcite . universal dependencies and sequence-tosequence approaches have also been employed for re in low-resource languages @xcite . finally, recent sequence-to-sequence approaches for english have focused on extracting both relations and relation types @xcite . ## methodology a general overview of the annotation approach is demonstrated in figure 1 . below, we outline the methods used in the process. ## results our results indicate that using the encoding method proposed can increase the number of correctly identified relations between two entities by up to 9.7% ( it's worth noting that the text categories from our corpora that contain news have the highest precision scores and improvements using our method. this is not surprising since they were translated using wmt21's newsdomain parameter which should make them better translated. all categories score higher in correctly identified relation triplets when using the encoding method except one, the adjudications category. this is likely because there are few entities reported in that category and most of them are anonymized, and not counted as a relation because an anonymous person a being related to an anonymous person b is not informative for a knowledge base. when looking at the overall performance of our model in ## discussion in this work, our focus was on evaluating an encoding method that can lead to improved automated relation extraction in text such as icelandic. to eliminate any errors due to the recognition of entities, we based this evaluation on goldannotated entities. however, for labelling relations, our method can be combined with existing ner models such as from @xcite that achieves an accuracy of 98.8% for icelandic. we further believe that it would be interesting to study our approach for relation extraction methods that only report possible relations between entities instead of all possible relations in the text, i.e., where the entities can be provided as input to the relation extraction model. for further study of efficiency, it could be interesting to compare this method to heuristics that match entities in translated text. one limitation of our study is the strict evaluation approach. we deemed the output of rebel to be incorrect if the entities were not in their lemmatized form, shortened or otherwise modified. for example, when talking about someone using their first name only, rebel does not have sufficient context to disambiguate the entity for insertion into a knowledge-base. this could be addressed by first disambiguating the entities before the text is processed in this manner. we did not evaluate how much the performance could increase, but we believe that a good disambiguation model could have a significant effect on the result. our approach addresses the low-performance of modern machine translation systems in translating entities correctly. therefore, we would expect that improvements in machine translation would make our approach obsolete. however, that would require better translations of named entities. unfortunately, for icelandic, we do not have a corpus of entities translated to other languages such as english. transliteration of named entities is the process of translating entities across languages and has been performed for english and several other languages @xcite . transliterating named entities could be an approach to improve machine translation for icelandic and would possibly make it more reliable to translate without any modifications to the source text and use word alignment to match entities between the source text and the translated text. for creating data on relation extraction, we use machine-translation as an aid. however, to build a good relation extraction system for icelandic, it might not be necessary to fine-tune the system on icelandic. as an example, multilingual qa systems have shown good performance on icelandic although they were not fine-tuned in qa for the language @xcite . we expect to see similar results for icelandic and the data from this work can serve as a test set to measure the performance. ## conclusion in conclusion, the proposed encoding method shows great potential for lrl, improving the percentage of correctly identified relations between entities by up to 9.7% for various categories of text. the method is simple and does not require any additional cost, making it ideal for languages where data is scarce and budget is limited. we note that this method can only be as good as the quality of the machine translation models as well as the re methods for higher resourced languages. however, the encoding avoids several issues introduced by the bootstrapping method, making it more efficient with minimal effort.