Multilingual Fact Linking

a Knowledge Graph ( KG ) . Although facts themselves are language-agnostic , the fact labels ( i.e. , language-specific representation of the fact ) in the KG are often present only in a few languages . This makes it challenging to link KG facts to sentences in languages other than the limited set of languages . To address this problem , we introduce the task of Multilingual Fact Linking ( MFL ) where the goal is to link fact expressed in a sentence to corresponding fact in the KG , even when the fact label in the KG is not available in the language of the sentence . To facilitate research in this area , we present a new evaluation dataset , INDICLINK . This dataset contains 11,293 linked WikiData facts and 6,429 sentences spanning English and six Indian languages . We propose a Retrieval+Generation model , REFCOG , that can scale to millions of KG facts by combining Dual Encoder based retrieval with a Seq2Seq based generation model which is constrained to output only valid KG facts . REFCOG outperforms standard Retrieval+Re-ranking models by 10.7 pts in Precision @ 1 . In spite of this gain , the model achieves an overall score of 52.1 , showing ample scope for improvement in the task . REFCOG code and INDICLINK data are available at github.com/SaiKeshav/mfl 1 . Introduction . Knowledge Graphs ( KG ) are a large and useful source of information about the world and are composed of curated facts regarding entities . Each fact asserts a relation that exists between two entities , which can be expressed as ( subject ; relation ; object ) . KGs have proven to be helpful for NLP tasks like Question Answering [ Saxena et al. , 2020 ] and are used in a variety of industry applications [ Dong , 2017 , Fensel et al. , 2020 ] . KG facts represent knowledge explicitly in a structured format that allows for greater interpretability . Linking KG facts with text has several benefits . Fact linking can potentially be used by systems to verify claims and detect misinformation . Moreover , KG facts allow for rich search experiences by supporting knowledge panels which require triggering the correct panels by connecting search queries to KG facts . Recent works [ Verga et al. , 2021 ] have shown that augmenting pretrained transformers with KG facts allow for modular management of knowledge that can be tuned according to the end application , such as adding new facts or removing stale ones . Improved fact-linked text can benefit such models even further . * . Work done at Google . Knowledge graphs such as Wikidata1 often contain entity and property labels in multiple languages ( although with severe skew [ Kaffee et al. , 2017 ] ) . For example , Argovia and Argau are the English and Spanish entity labels for entity Q11972 in Wikidata . Similarly , country and paı́s are the English and Spanish property labels for property P17 . Combining these , we define the notion of fact labels which are language-specific representations of an otherwise language-agnostic fact . For example , ( Aargau ; country ; Switzerland ) and ( Argovia ; paı́s ; Suiza ) are the English and Spanish fact labels for the fact ( Q11972 ; P17 ; Q39 ) in Wikidata . While linking KG facts to text has received some attention in literature [ Elsahar et al. , 2018 ] , most of that work has been restricted to English fact labels and text . There is a growing need to connect fact and their mentions in text , especially when languages of the fact label and text are different . For example , we would like to link a fact with an English fact label with a text in Hindi or Telugu . We refer to this problem as Multilingual Fact Linking ( MFL ) . We define the task as predicting all the KG facts that are implied by a sentence even when the the sentence and KG fact labels are present in different languages . To the best of our knowledge there has been no prior work on this important problem . languages tagged with the corresponding Wikidata facts expressed in them . The English sentences and facts are taken from the relation extraction dataset , WebRED [ Ormándi et al. , 2021 ] . The test examples are manually translated into different languages and automatic translation of sentences are used for training . We use KG facts from Wikidata and explore different strategies to use their fact labels in English and other languages , wherever available . Since multilingual fact linking involves selecting a subset of facts from the entire set of KG facts , the label space may be in the scale of millions . To handle this , we make use of Dual Encoder-Cross Encoder paradigm , where a dual encoder [ Reimers and Gurevych , 2019b ] is used to quickly retrieve 1. https : //www.wikidata.org/ the potential top–k facts . These top–k facts are typically re-ranked using classification based cross encoder architectures . However , due to the pipelined nature of the system , performance of the reranking model is limited by that of the retrieval model . We propose a novel model , Retrieval based Fact-Constrained Generation ( ReFCoG ) , which replaces re-ranking cross encoders with a Seq2Seq based cross encoder that is constrained to generate facts from the KG , even those which are absent in the retrieved set of facts . We show that such generative models outperform classification based re-ranking , achieving 10.7 pts improvement in precision . In summary , the main contributions of this work are as follows : 1 . We introduce the task of Multilingual Fact Linking to link KG facts with their mentions in text , especially when there is a mismatch between the languages of the fact label and text . 2 . We present INDICLINK , an evaluation dataset for MFL in English and six Indian languages , a widely used but less studied set of languages in the NLP community . To the best of our knowledge , this is the first dataset of its kind for these languages . 3 . We propose REFCOG , a novel Retrieval+Generation model for the task of MFL . The proposed method significantly outperforms standard Retrieval+Re-Ranking models . 2 . Related Work . External sources of knowledge are helpful for NLP tasks such as question answering and fact verification . KILT [ Petroni et al. , 2020 ] uses a retrieval+seq2seq model with Wikipedia documents as the knowledge source to solve many knowledge-intensive tasks . FAE [ Verga et al. , 2021 ] uses WikiData KG to inject knowledge into pretrained language models in a modular fashion . Colon-Hernandez et al . [ 2021 ] surveys various methods of combining structured knowledge such as KGs and pretrained language models . However , they don ’ t handle multilingual examples in both the text or KG . Linking text to KGs has been traditionally explored in various settings , such as entity linking , relation classification and extraction , fact extraction and fact linking . We list various KG-related tasks and their corresponding inputs and outputs in Table 1 . Entity Linking : Entity linking involves finding the KG entity referred by a mention marked in the input sentence . Multilingual pretrained models have advanced Entity Linking across languages . MEL [ Botha et al. , 2020 ] uses a Dual Encoder , Cross Encoder pipeline trained on hard negatives to achieve strong performance on 104 languages . mGENRE [ Cao et al. , 2021 ] is an entity linking model that autoregressively generates the linked entity name using a Seq2Seq model . Moreover , it augments the decoder with a prefix trie in order to generate only valid entity names . We find that combining both of these models by using a Dual Encoder , Constrained Generation pipeline leads to strong multilingual fact linking performance . Relation Classification : The task involves identifying the relation between a pair of entity mentions . It is also referred to as relation extraction . The recently released WebRED [ Ormándi et al. , 2021 ] provides a dataset of English sentences annotated with the corresponding Wikidata relation/predicate and the linked entities . We create INDICLINK using these examples . Other multilingual relation classification datasets such as RELX [ Koksal and Ozgur , 2020 ] are unusable for fact alignment as they don ’ t provide the linked entities . Fact Extraction : The task involves joint entity and relation extraction [ Zhong and Chen , 2021 , Sui et al. , 2020 ] that focus on discovering new facts that are not present in the KG . Whereas , fact linking deals with connecting existing KG facts with text . Therefore , fact linking models make use of KG facts ( which maybe in the scale of millions ) whereas fact extraction systems do not . Fact Linking : Existing fact linking/alignment systems such as T-REx [ Elsahar et al. , 2018 ] align English DBPedia abstracts with Wikidata triples and provide a corpus of 11 million high quality alignments . They use adhoc pipelines of entity linking , coreference resolution and string matching based predicate linkers . In our experiments , we find that our end-to-end linker , REFCOG outperforms such pipeline systems . In another line of work , multilingual fact retrieval [ Jiang et al. , 2020 ] is used to judge the factual knowledge captured within LM parameters by predicting masked entities in facts . However , we are only concerned with retrieving the facts present in input text . 3 . Multilingual Fact Linking : Problem Overview . A knowledge graph ( KG ) contains a list of entities E , relations R and facts F , where the ith fact links two entities ( ex , ey ∈ E ) with a relation ( ry ∈ R ) and is defined as , Fi = ( ex ; ry ; ez ) . Multilingual KGs like Wikidata also contain textual labels of entities and relations in multiple languages . Given the set L of languages in the KG , for each language l , we construct the fact label by concatenating the labels of the entities and relation , if available . The label of Fi in language l ∈ L , F li , exists if the entity and relation labels are available in language l , i.e. , if e l x , r l y and e l z exist in KG , then , F li = ( e l x ; r l y ; e l z ) . 2 For example , in Table 1 , ( Q12146 ; P17 ; Q39 ) and ( Landschaft ; country ; Switzerland ) represent the fact and it ’ s English label . Multilingual Fact Linking ( MFL ) aims to discover the set of linked KG facts , LinkedFacts ( Tm ) ⊆ F that are explicitly expressed in text T of language m. The output set can be formally defined as ( ⇒ is used to represent the fact expressed in text ) : LinkedFacts ( Tm ) = { Fi : Fi = ( ex ; ry ; ez ) ∧ Fi ∈ F ∧ Tm ⇒ Fi } . ( 1 ) In order to predict this set , the fact labels available in different languages are used . In Section 4 , we describe the INDICLINK dataset curated for the MFL task and in Section 5 , we discuss the baselines and proposed models for the task . 2 . For right to left languages like Urdu , we use ( elz ; rly ; elx ) as the fact description .

Summary: This paper studies the problem of fact linking -- linking facts from a knowledge graph (KG) to a given sentence where the fact is mentioned. More specifically, recognizing the sparsity and skewness of KG entity and relation labels in terms of language coverage, this paper proposes to study the multilingual setting, where the input sentence is in an underrepresented language and one needs to link KG facts, often only expressed in more high-resource languages, to the input sentence. A new test set for 6 Indian languages is created by manually translating sentences from WebRED with professional translators. The paper also proposes a model for this task based on the Dual Encoder - Cross Encoder architecture, but unlike conventional Cross Encoder, it follows mGENRE and proposes a constrained autoregressive decoder to generate valid facts. It is shown that the proposed model outperforms re-ranking based Cross Encoders by a large margin on the new test set.