Align-then-Enhance: Multilingual Entailment Graph Enhancement with Soft Predicate Alignment
Year: 2023

Abstract:
Entailment graphs (EGs) with predicates as nodes and entailment relations as edges are typically incomplete, while EGs in different languages are often complementary to each other. In this paper, we propose a new task, multilingual entailment graph enhancement, which aims to utilize the entailment information from one EG to enhance another EG in a different language. The ultimate goal is to obtain an enhanced EG containing richer and more accurate entailment information. We present an alignthen-enhance framework (ATE) to achieve accurate multilingual entailment graph enhancement, which first exploits a cross-graph guided interaction mechanism to automatically discover potential equivalent predicates between different EGs and then constructs more accurate enhanced entailment graphs based on soft predicate alignments. Extensive experiments show that ATE achieves better and more robust predicate alignment results between different EGs, and the enhanced entailment graphs generated by ATE outperform the original graphs for entailment detection1.

heading: 1 Introduction
text: Predicate entailment detection is the task to determine if the meaning of one predicate is entailed (can be inferred) from the other predicate, which benefits many core natural language processing tasks such as question answering and semantic parsing. Usually, a question like “Did Lionel Messi appear at the FIFA World Cup Stadium Gelsenkirchen in 2006” might be answered by a sentence that does not directly correspond
∗Corresponding author. 1Code and data available at https://github.com/
StephanieWyt/Align-then-Enhance.
to the question, but is an expression like “Lionel Messi made his debut at the FIFA World Cup Stadium Gelsenkirchen in 2006”, since the predicate “make one’s debut at” entails predicate “appear at”. To bridge such semantic gap between queries and answers, recent approaches (Berant et al., 2011, 2015; Hosseini et al., 2018, 2019; Hosseini, 2021) have looked into learning typed Entailment Graphs (EGs) with predicates as nodes and entailment relations as edges. These methods first extract predicate-argument (entity) relation triples from large text corpora, and construct typed entailment graphs based on the Distributional Inclusion Hypothesis (Dagan et al., 1999; Geffet and Dagan, 2005; Herbelot and Ganesalingam, 2013; Kartsaklis and Sadrzadeh, 2016). Predicates are then grouped into typed entailment subgraphs based on the types of entity arguments they take. Such an EG is in an effective and machine-readable form to organize the context-independent entailment relations between predicates and can facilitate reasoning without extra context or resources, which can be regarded as a special kind of Knowledge Graph (KG) for natural language understanding. Figure 1 shows excerpts from two typed EGs in different languages with arguments of types <person, location>.
However, EGs frequently suffer from incompleteness, and EGs in different languages are often complementary to each other. This makes a compelling case for developing a method that can integrate entailment information from EGs in different languages and construct an enhanced EG containing richer and more accurate entailment information. For instance, to answer the aforementioned question “Did Lionel Messi appear at the FIFA World Cup Stadium Gelsenkirchen in 2006”, given
881
the sentence “Lionel Messi made his debut at the FIFA World Cup Stadium Gelsenkirchen in 2006”, we queried the popular English EG published by Hosseini et al. (2018) and found no entailment edge where “make one’s debut at” entails “appear at”. However, as shown in Figure 1 (b), in the Chinese EG constructed by Li et al. (2022), we find an entailment edge where the predicate “在·X·首次亮 相” entails “在·X·出现” . Significantly, “在·X·首次亮 相” and “make one’s debut at” as well as “在·X·出 现” and “appear at” are two pairs of equivalent predicates. If such equivalent predicates between Chinese and English EGs could be aligned, we can use the entailment information in one EG to enhance another. For example, according to the Chinese entailment edge where “在·X·首次亮相” entails “在·X·出现”, we can add the equivalent entailment edge where “make one’s debut at” entails “appear at” to the English EG. The enhanced EG fuses the entailment information from different EGs, further boosting the entailment detection performance of the original graph.
Recently, a few efforts have been made to improve the quality of an EG by integrating entailment information from another EG. Weber and Steedman (2019) have tried to align the English and German EGs by learning the predicate representations towards alignment through a link prediction model and showed that the stronger English EG can aid in German entailment detection. Whereas Weber and Steedman (2019) only proves that an EG in a higher resource language can improve the quality of an EG in a lower resource language, Li et al. (2022) further demonstrates that the cross-
lingual complementarity between different EGs can be used in both directions by ensembling the predictions from the two graphs. However, they did not really realize the alignment of EGs. In order to achieve an ensemble, their model needs the parallel Chinese translations of the English questions to query the Chinese and English EGs separately.
In this paper, we propose a new task, Multilingual Entailment Graph Enhancement (MEGE), which aims to automatically align EGs in different languages and utilize the entailment information from one EG to enhance the other. We emphasize that the enhancement should be effective in both directions, which is demonstrated and discussed in Section 6.2.
For the multilingual entailment graph enhancement task, we present an align-then-enhance framework, ATE, which first automatically discovers equivalent predicates between EGs in different languages and then constructs more accurate enhanced EGs based on soft predicate alignments. In order to achieve accurate predicate alignment, an effective approach is to learn better predicate representations toward alignment. However, since EGs usually suffer from severe sparsity issues, it is difficult to directly embed EG structures to learn effective predicate representations. For example, 79% of nodes in the popular English EG released by Hosseini et al. (2018) have degrees no more than 2. To tackle this issue, we introduce rich context information of predicates extracted from the large opendomain encyclopedia KG Wikidata (Vrandecic and Krötzsch, 2014) into original EG to assist in learning better predicate representations, constructing a predicate-centric graph. Simultaneously, we also build the entity-centric graph, aiming to improve the predicate representations by utilizing the information of entities closely associated to the predicates. And we propose a cross-graph guided interaction (CGI) mechanism to encourage sufficient interaction between the predicate-centric graph and the entity-centric graph and learn better predicate representations for alignment.
To our best knowledge, there is no standard dataset to directly evaluate the predicate alignment performance for entailment graphs. We thus build a new alignment evaluation dataset EGAlign. Experiments on EGAlign indicate that our model achieves the state-of-the-art performance on predicate alignment between different EGs. The key technical contributions of this paper are as follows:
• We propose a new task, multilingual entailment graph enhancement (MEGE), to improve the quality of an EG with another EG in a different language and construct an enhanced EG which can better support entailment detection. We present an align-then-enhance framework for this task.
• We design a cross-graph guided interaction mechanism to overcome the sparsity of EGs, which encourages the information interaction between the enriched predicate-centric graph and entity-centric graph and learns better predicate representations towards alignment.
• We build a new EG predicate alignment evaluation dataset, and our model achieves the state-of-the-art performance on it. We further evaluate the enhanced EGs on two benchmark datasets for entailment detection, and demonstrate that the enhanced entailment graphs are superior to the original graphs.

heading: 2 Related Work
text: Most previous works on entailment graphs only focused on constructing an entailment graph in a single language, and usually ignore the complementarity between different EGs. Similar to the idea of entailment graph enhancement, (Lewis and Steedman, 2013b) do not construct an enhanced entailment graph but learned clusters of semantically equivalent English and French predicates based on their named-entity arguments. They create predicate representations and align the predicates by the cosine similarity between representations. (Lewis and Steedman, 2013a) solve the problem of paraphrasing in a multilingual context by creating aligned paraphrase clusters. They take the Wikipedia articles describing the same topic as parallel texts, and use the Wikipedia inter-language links between named entities to align predicates in different languages. The study on creating paraphrase clusters lays a foundation for the construction of entailment graphs. Berant et al. (2015) first propose typed predicate entailment graphs but can not be scalable to a large amount of data. To tackle this problem, Hosseini et al. (2018) present a scalable method that learns globally consistent similarity scores for entailment graph construction.
Based on the typed entailment graphs, Weber and Steedman (2019) are the first to try to align an English entailment graph with a German entail-
ment graph. However, as discussed in Section 1, the serious sparsity issues of EGs hinder their alignment performance and they only perform alignment in one direction. Most recently, Li et al. (2022) demonstrate the cross-lingual complementarity between an English EG and a Chinese EG in both directions. However, instead of really aligning the two EGs, they ensemble the predictions from the two graphs, which needs the parallel translations between English and Chinese questions for querying the English and Chinese EGs separately.

heading: 3 Problem Formulation
text: Let P be the set of all typed predicates and T be the set of types, V(t1, t2) denotes the set of typed predicates p with unordered argument types t1 and t2, where p ∈ P and t1, t2 ∈ T . The argument types of each predicate are determined by the types of entities that instantiate the argument slots. Formally, we represent a typed EG as G(t1, t2) =< V(t1, t2), E(t1, t2) >, where V(t1, t2) is the set of typed predicate nodes and E(t1, t2) denotes the set of weighted edges. We represent the edges as the sparse score matrix W (t1, t2) ∈ [0, 1]|V(t1,t2)|×|V(t1,t2)|, containing the entailment scores between predicates of types t1 and t2.
Without loss of generality, we consider the entailment graph enhancement task between two typed EGs, G1(t1, t2) =< V1(t1, t2), E1(t1, t2) > and G2(t1, t2) =< V2(t1, t2), E2(t1, t2) >. The goal of this task is to utilize the entailment information (i.e., the entailment scores between typed predicates) from one EG to enhance the other. In this paper, we achieve this in two steps: 1) Given a set of pre-aligned typed predicate pairs L = {(pi1 , pi2)|pi1 ∈ V1(t1, t2), pi2 ∈ V2(t1, t2)} between G1(t1, t2) and G2(t1, t2), our approach first finds more aligned typed predicates as much as possible based on the existing ones; 2) With these predicate alignments in place, we utilize G2(t1, t2) to enhance G1(t1, t2), and construct an enhanced entailment graph Ĝ2→1(t1, t2) =< V̂(t1, t2), Ê(t1, t2) > with updated entailment score matrix Ŵ (t1, t2). Similarly, we can also obtain the enhanced entailment graph Ĝ1→2(t1, t2) generated by using G1(t1, t2) to enhance G2(t1, t2).

heading: 4 Our Approach: ATE
text: To obtain an enhanced EG, we propose a model, ATE. As depicted in Figure 2, ATE takes two typed EGs as input, and follows a two-stage pipeline: predicate alignment and entailment graph enhancement, and finally output an enhanced EG with richer and more accurate entailment information.

heading: 4.1 Predicate Alignment
text: In this stage, we aim to discover as many aligned predicate pairs as possible between G1(t1, t2) and G2(t1, t2). As discussed in Section 1, EGs often suffer from sparsity issues, which makes it difficult to learn good node (predicate) representations. To tackle this problem, we use context information of predicates extracted from Wikidata to enrich original EGs and construct denser predicatecentric graphs. We also introduce entity-centric graphs with entities as nodes and predicates as edges, which provide rich information for entities closely associated with the predicates. We propose a cross-graph guided interaction mechanism to encourage sufficient interaction between predicatecentric and entity-centric graphs, learning better predicate representations for alignment.

heading: 4.1.1 Entity/Predicate-centric Graph Construction
text: Let E1 and E2 be the entity instances of the argument slots of typed predicates in G1(t1, t2) and G2(t1, t2). We take entities as nodes and predicates as edges to construct the entity-centric graphs Ge1 = (E e 1, P e 1 , T e 1 ) and G e 2 = (E e 2, P e 2 , T e 2 ) for G1(t1, t2) and G2(t1, t2), respectively, where Ee1 and Ee2 are the entity sets and P e 1 and P e 2 are the predicate sets and T e1 ⊂ Ee1 × P e1 × Ee1 and T e2 ⊂ Ee2 × P e2 × Ee2 are the binary relation triples. Besides the structural information contained in the predicate-entity relation triples, we also introduce the neighborhood information of en-
tities in Wikidata to further enrich the entity-centric graphs. Therefore, the node sets Ee1 = E1 ∪ N1 and Ee2 = E2 ∪ N2, where N1 and N2 are the one-hop neighbors extracted from Wikidata2; the predicate sets P e1 = V1(t1, t2) ∪ Pwiki1 and P e2 = V2(t1, t2) ∪ Pwiki2 , where Pwiki1 and Pwiki2 are the Wikidata predicates associated with N1 and N2, respectively.
We also add the newly introduced Wikidata predicates Pwiki1 and P wiki 2 into G1(t1, t2) and G2(t1, t2) respectively to enhance the connectivity of two EGs and introduce richer contextual information of typed predicates of G1(t1, t2) and G2(t1, t2), obtaining the predicate-centric graphs Gp1 = (Vp1 , Ep1 ) and Gp2 = (Vp2 , Ep2 ), where Vp1 = P e1 = V1(t1, t2) ∪ Pwiki1 and Vp2 = P e2 = V2(t1, t2)∪Pwiki2 are the node sets, and Ep1 and Ep2 are the edge sets. Besides the existing entailment edges, if two predicates share the same head or tail entities in the entity-centric graphs, we will create an edge connecting the two predicate nodes vpi and vpj , and weight the edge with weight sij according to how likely the two predicates share similar heads or tails in entity-centric graphs:
sij = |Hi ∩Hj | |Hi ∪Hj | + |Ti ∩ Tj | |Ti ∪ Tj |
(1)
where Hi and Ti are the sets of head and tail entities for predicates pi in entity-centric graphs. Considering the original entailment scores on the entailment edges together, the final weight ϵpij between predicate nodes vpi and v p j is computed as:
ϵpij =
{ sij , v p i or v p j ∈ Pwiki1 ∪ Pwiki2
wij + sij , v p i , v p j ∈ V1(t1, t2) ∪ V2(t1, t2)
(2)
where wij is the entailment score between predicate pi and pj in original EGs. Note that we also modify the entailment scores of gold edges, which allows us to additionally incorporate richer features of typed predicates from the introduced world knowledge.
In order to facilitate the implementation of our model, we put Ge1 and G e 2 together as the final entity-centric graph Ge = (Ee, P e, T e), where Ee = Ee1 ∪ Ee2 and P e = P e1 ∪ P e2 and T e = T e1 ∪ T e2 . Similarly, we put Gp1 and Gp2 together as the final predicate-centric graph Gp = (Vp, Ep), where Vp = Vp1 ∪ Vp2 and Ep = Ep1 ∪ Ep2 .
2We simply link entities in E1 and E2 to Wikidata entities through the exact match of entity names. However, not all entities can be linked to Wikidata, so we introduce one-hop neighbors of entities that can be linked to Wikidata.

heading: 4.1.2 Cross-graph Guided Interaction
text: With the entity-centric graph Ge and the predicatecentric graph Gp in place, we propose a cross-graph guided interaction (CGI) mechanism to encourage interactions between the two graphs to obtain better predicate representations for alignment, which utilizes a graph attention mechanism (GAT) guided by counterpart graph to learn the node representations of Ge and Gp iteratively. Each cross-graph guided interaction consists of two layers, the predicate attention layer and the entity attention layer. By stacking multiple interactions, we can achieve more mutual improvements on both graphs. We further apply two Graph Convolutional Networks (GCNs) with highway gates on Ge and Gp respectively to model their structural information. The final predicate representations will be used to determine whether two predicates should be aligned. Predicate Attention Layer. Let Xp ∈ R|Vp|×d denote the input node representation matrix of Gp. Different from the vanilla GAT (Veličković et al., 2018), each node integrates its neighbor node information under the guidance of Ge. Specifically, the predicate attention score regarding each neighbor node is calculated with the entity node features X̂ e (computed by Eq. 9) produced by the entity attention layer from the previous interaction module:
x̃pi = ReLU( ∑
j∈Npi
αpijx p j ), (3)
αpij = exp(η(ϵpija p[zi∥zj ]))∑ k∈Npi exp(η(ϵ p ika p[zi∥zk])) , (4)
where x̃pi is the output representation of predicate node vpi ; x p j is the representation of v p j produced by the previous predicate attention layer; Npi indicates the set of neighbor indices of vpi ; a
p[·] is a fully connected layer; η is the Leaky ReLU; zi is the approximate predicate representation for predicate pi , which is computed as:
zi = [
∑ k∈Hi x̂ e k |Hi| ∥ ∑ l∈Ti x̂ e l |Ti| ], (5)
where x̂ek and x̂ e l are the output representations of the k-th head entity and l-th tail entity of predicate pi from previous entity attention layer. Note that we use Glove (Pennington et al., 2014) word embeddings of predicate names to initialize these predicate nodes, which are useful features for predicate alignment. To retain this useful information,
we integrate the initial features {xp_initi } with the output of predicate attention layer as follows:
x̂pi = φ p l ∗ x̃ p i + x p_init i , (6)
where x̂pi denotes the final output predicate representation of the interaction module for predicate node vpi ; φ p l is a weighting hyper-parameter for the l-th attention layer. We show the effectiveness of this skip connection design in Section 6.1. Entity Attention Layer. Similar to the predicate attention layer, we apply GAT on the entitycentric graph guided by the predicate-centric graph. Specifically, let Xe ∈ R|Ee|×d be the input node representation matrix of Ge. The representation x̃eq of entity eq in Ge can be computed as:
x̃eq = ReLU( ∑
t∈Neq αeqtx e t ), (7)
αeqt = exp(η(ae[x̂pqt]))∑
k∈Neq exp(η(a e[x̂pqk]))
, (8)
where x̂pqt is the representation for the predicate between entity eq and et obtained from Gp. We also initialize the entity node representations with entity names, and the final output representation x̂eq of the interaction module for entity eq are the weighted sum of the initial entity representations and the output of entity attention layer:
x̂eq = φ e c ∗ x̃eq + xe_initq , (9)
where φec is a weighting hyper-parameter for the c-th entity attention layer. Graph Structure Embedding. After multiple rounds of interaction between Gp and Ge, we can obtain enhanced predicate and entity representations. Following previous practice (Rahimi et al., 2018; Wu et al., 2019a), we respectively feed the two graphs into two different two-layer GCNs (Kipf and Welling, 2017) with highway gates (Srivastava et al., 2015) to incorporate evidence from their neighboring structures. Training. With the final predicate representations X̄p output by Highway-GCNs, predicate alignment can be performed by simply measuring the distance between two predicates:
d(p1, p2) = 1− cos(x̄p1, x̄p2), (10)
For training, we expect the distance between aligned predicate pairs to be as close as possible,
and the distance between negative predicate pairs to be as far as possible. We use the following marginbased scoring function as the training objective for predicate alignment.
Lp = ∑
(p,q)∈Lp
∑
(p′,q′)∈L′p max{0, d(p, q)− d(p′, q′) + γp},
(11)
where γp > 0 is a margin hyper-parameter; Lp indicates the pre-aligned predicate pairs for training; L′p is the set of negative instances generated through nearest neighbor sampling (Kotnis and Nastase, 2017).
Similarly, with the final entity representations X̄e, we can also calculate the training loss for entity alignment like Eq. 11, and learn the alignmentoriented entity representations. Predicate alignment and entity alignment can enhance each other in our model, and ultimately achieve more accurate alignment results.

heading: 4.2 Entailment Graph Enhancement
text: After obtaining the final alignment-oriented representation of each predicate in G1(t1, t2) and G2(t1, t2), we perform soft predicate alignment between two EGs by computing an alignment score π(p1, p2) for each predicate pair (p1, p2) where p1 ∈ V1(t1, t2) and p2 ∈ V2(t1, t2). Specifically, we calculate the cosine similarity of the representations of p1 and p2. Next, we will perform EG enhancement according to these alignment scores.
As discussed in Section 1 and 3, EG enhancement can be performed in two directions. Here, we take the enhancement process of Ĝ2→1(t1, t2) as an example. Given (p1i , p 1 j ) as a predicate pair in G1(t1, t2) and wij as the original entailment score between them, we aim to find the predicate pair (p2x, p 2 y) in G2(t1, t2), which is aligned with (p1i , p 1 j ), and enhance wij based on the entailment score wxy between p2x and p 2 y. Specifically, for (p1i , p 1 j ), we collect the top k similar predicates of p1i and p 1 j in G2(t1, t2) as TopK2(p1i ) and TopK2(p1j ), respectively. Then, we can get a set of candidate aligned predicate pairs from G2(t1, t2), namely C2(p1i , p1j ) = {(p2x, p2y)|p2x ∈ TopK2(p1i ), p 2 y ∈ TopK2(p1j )}.
We combine the entailment scores of all candidate predicate pairs in C2(p1i , p 1 j ), according to their alignment probability, which is computed as:
w̄2ij =
∑ (x,y)∈C2(i,j) AV G(π(i, x), π(j, y)) ∗ wxy
|C2(i, j)| . (12)
Finally, the new enhanced entailment score ŵ2→1ij between p 1 i and p 1 j is updated as:
ŵ2→1ij = AV G(ρ 2→1 ∗ w̄ij , wij). (13)
where ρ2→1 is a hyperparameter tuned on the development set, and AVG(·) denotes average pooling.

heading: 5 Experimental Setup

heading: 5.1 Predicate Alignment Evaluation
text: Datasets. Since there is no publicly available benchmark dataset to directly evaluate the predicate alignment performance for EGs, we construct a new alignment evaluation dataset, EGAlign, based on the popular English entailment graph (EGen) released by Hosseini et al. (2018) and the Chinese entailment graph (EGzh) constructed by Li et al. (2022). The labeling of aligned predicate pairs between two EGs is relatively labor-intensive, so we only manually aligned a set of equivalent predicates with argument types person and location. Thus, we extract the typed entailment subgraphs EGp,len and EG p,l zh with arguments of types <person,location> from EGen and EGzh respectively. We annotated 5784 Chinese-English predicate pairs with three annotators per pair and reached an average inter-annotator agreement of 0.83 considering the same annotation of a pair as an agreement. In addition, we link the argument entities from EGp,len and EGp,lzh respectively to English and Chinese versions of Wikidata, and obtain a set of aligned entity pairs through the inter-language links from entities of English version of Wikidata to those in Chinese. Table 1 shows the statistics of EGAlign, and we provide the annotation details for EGAlign in Appendix A. Following previous works (Wu et al., 2019b; Zhu et al., 2021), we use 30% of the prealigned predicate pairs and entity pairs as training data and 70% for testing. Metrics. We use Mean Average Precision at K (MAP@K) as the evaluation metric for predicate alignment, and report the results of both directions of alignment. As discussed in Section 1,
Models EN→ ZH ZH → ENMAP@1 MAP@10 MAP@50 MAP@1 MAP@10 MAP@50
currently, there is no public and complete implementation of predicate alignment for EGs, we thus compare our CGI with 4 state-of-the-art models , BootEA (Sun et al., 2018), RDGCN (Wu et al., 2019a), HGCN-JE/JR (Wu et al., 2019b) and RNM (Zhu et al., 2021), for the knowledge graph relation alignment task, which is similar in spirit to our task. We also implement two baselines Glovesim and BERT-sim, which directly take the Glove word embeddings (Pennington et al., 2014) and pre-trained BERT (Devlin et al., 2019) representations at [CLS] tokens of predicate names as the representations of predicates and perform predicate alignment by calculating the distance between them.

heading: 5.2 Entailment Detection
text: Datasets. To evaluate the quality of enhanced EGs and explore whether it can better support entailment detection than original graphs, we use the popular entailment detection datasets Levy/Holt (Levy and Dagan, 2016; Holt, 2019) and Berant (Berant et al., 2011). Each example in these two datasets contains a premise and a hypothesis (a pair of relation triples with the same arguments), and the entailment detection task is to judge whether the premise entails the hypothesis.
After doing EG enhancement according to Section 4.2, we obtain the enhanced EG Enhanp,lzh→en generated by using EGp,lzh to enhance EG p,l en. By enhancing in the opposite direction, we can get the enhanced Enhanp,len→zh as well. For fair comparison, we extract subsets with the types of arguments person and location, the portions of which are 6.3% (6107 examples) and 7.1% (2756 ex-
amples) in Levy/Holt and Berant datasets respectively. We also translated these subsets into Chinese to evaluate the performance of Enhanp,len→zh and EGp,lzh. Following Hosseini et al. (2018), we split Levy/Holt dataset into development (30%) and test (70%) sets. And we evaluate our model on the test set of Levy/Holt and the whole Berant dataset. We also compare with the BERT-based baselines for etailment detection. We provide more details of the evaluation process and the construction of the BERT-based baselines in Appendix B.
Metrics. Following Hosseini et al. (2018), we evaluate our methods on LevyHolt and Berant with the area under curves (AUC) of Precision-Recall Curves (PRC). Hosseini et al. (2018) mentioned that AUC for precisions in the range [0; 0.5) should not be taken into account, since model performs as random guess and is not applicable to down-stream applications. We thus report the AUC of PRC with precision range in [0.5, 1]. More details of our configuration please refer to Appendix C.

heading: 6 Experimental Results

heading: 6.1 Predicate Alignment
text: From Table 2, CGI substantially outperforms all baselines across all metrics and alignment directions. The four KG relation alignment models all deliver inferior performance on EGAlign. This might be because these models approximate predicate representations via entity representations which are inferior to ours in achieving predicate alignment. We observe that Glove-sim and BERTsim outperform other baselines, showing the importance of the semantics of predicate names.
Ablation Study. Without introducing the Wikidata entities and predicates into our entity/predicate-centric graphs (CGI w/o Wikidata), there is a significant performance drop of 11.98 % on average regarding MAP@1. This shows the benefits of the additional Wikidata information for learning enhanced predicate representations. Removing the GCNs from our model leads to an average drop of 3.77% on MAP@1, showing the importance of the GCNs in capturing the structural information of the predicate/entity-centric graphs. When removing the cross-graph interaction, we see an average drop of 2.65% on MAP@1. This confirms the effectiveness of our cross-graph guided interaction mechanism. To explore the impact of the skip connection design that retains the useful predicate and entity name semantics contained in the initial node embeddings, we implement three model variants CGI w/o relSkip, CGI w/o entSkip and CGI w/o bothSkip, which respectively deletes the skip connections from predicate attention layer, entity attention layer and both attention layers. We observe that the three variants all deliver much worse results than CGI, which demonstrates the effectiveness of keeping the initial semantics of predicate names using skip connections.

heading: 6.2 Entailment Detection
text: From Table 3, we observe that, on both English and Chinese datasets, EGp,len and EG p,l zh, as well as Enhanp,lzh→en and Enhan p,l en→zh, all greatly outperform BERT. This shows that entailment graphs seem to be more beneficial for entailment detection than BERT. Compared with EGp,len and EG p,l zh, the enhanced graphs Enhanp,lzh→en and Enhan p,l en→zh show obvious advantages, with average improvements of 3.7% and 6.7% on Levy/Holt and Berant subsets. Besides, as shown in Figure 3, in
the moderate precision range, Enhanp,lzh→en and Enhanp,len→zh can achieve higher recall than original graphs EGp,len and EG p,l zh across all datasets, and both significantly outperform BERT. These results demonstrate the significance of performing entailment graph enhancement as well as the effectiveness of our enhancement model.
In addition, we note that EGp,len outperforms EGp,lzh by 3.1% and 1.6% on two datasets respectively, indicating that EGp,len is more complete in entailment information. Hence after enhancement, comparing to the original EGs, Enhanp,len→zh achieves the improvements of 6.1% and 7.3% on Levy/Holt and Berant, which are both greater than Enhanp,lzh→en’s improvement of 1.8% and 5.6%. The results further demonstrate that the enhancement between two EGs are effective in both directions, and using stronger graphs to enhance the relatively poor ones will bring greater improvements.
Error Analysis. We conduct a case study to further analyze the circumstances where the predictions of the original EG are correct while the enhanced EGs have made wrong predictions. We randomly sampled and analyzed 100 examples that were pre-
dicted accurately by EGzh but wrongly predicted by Enhanp,lzh→en from Levy/Holt. As shown in Table 4, the error type Inaccurate Entailment Scores in EGzh indicates that the aligned predicate pairs predicted by ATE have inaccurate entailment scores in EGzh, which negatively affect the original entailment scores in EGen and lead to inaccurate updated entailment scores for Enhanp,lzh→en. This error type accounts for more than half of the total (52%), which shows that the quality of EGs has a significant impact on the achievement of accurate EG enhancement. With EGs of higher quality, our method could generate better enhanced EG. The remaining errors are basically due to inaccurate predicate alignment between two EGs. Specifically, in 4% of the cases, the English predicates are incorrectly aligned to their synonyms in Chinese, in 5% to their antonyms, in 8% to their hyponyms (namely the Chinese predicates entail the English predicates), and in 31% to the unrelated predicates. These results inspire us to improve the quality of predicate alignment by further distinguishing synonyms, antonyms, and hyponyms in the future.

heading: 7 Conclusion
text: We present a new task, multilingual entailment graph enhancement, aiming to enhance the quality of one EG with another graph in a different language. We design an align-then-enhance method for this task, which utilizes a cross-graph guided interaction mechanism to tackle the sparsity issues of EGs and achieves EG enhancement based on soft predicate alignment between different EGs. Experiments show that our align-then-enhance framework can effectively mine equivalent predicates in other EGs through sufficient cross-graph interaction and
better achieve accurate enhancement. We build a new dataset EGAlign to evaluate the predicate alignment performance of our model, and the results show that our model achieves the best performance. Furthermore, we show that the enhanced EGs outperform the original graph as well as BERT on entailment detection.
Limitations
There are two main limitations of our work: (1) Our approach requires a set of previously aligned predicate pairs as training data to achieve predicate alignment between different KGs, which limits the generalization ability of our method. In our experiments, since we manually aligned a set of equivalent predicates with arguments of types person and location between the English and Chinese EGs, we can only perform predicate alignment and entailment graph enhancement between the <person,location> subgraphs of two EGs. We will explore the semi-supervised or unsupervised predicate alignment method between different EGs in our future work. (2) Our current enhancement strategy introduced in Section 4.2 is straightforward. It might not be robust enough when dealing with entailment graphs of poor quality. We will explore more adaptive EG enhancement methods in the future.
Ethics Statement
In this paper, we construct a new EG alignment evaluation dataset based on two publicly available EGs, and manually annotated a set of equivalent predicates with argument types person and location. Annotators are offered a competitive pay of ¥60 per hour, which is more than double the local minimum wage. This remuneration applies to both the annotation stage and the discussion stage, ensuring that annotators are compensated for their time and effort. Annotators are required to familiarize themselves with the ACM Code of Ethics and Professional Conduct and promptly report any instances that violate the code. Inappropriate cases that breach the code are promptly eliminated from the selected documents. The resulting annotations, based on the consensus of three annotators, provide a respectable approximation of the gold labels. Note that they may not represent the absolute ground truth due to natural error rates. Users who wish to utilize the dataset should be mindful of its limitations. We are not responsible for problems en-
countered in subsequent model training processes utilizing our data.

heading: Acknowledgements
text: This work is supported by the Talent Fund of Beijing Jiaotong University (No. 2023XKRC032), NSFC (62161160339), ERC Advanced Fellowship GA 742137 SEMANTAX, a Mozilla PhD scholarship at Informatics Graduate School and the University of Edinburgh Huawei Laboratory. We would like to thank the anonymous reviewers for their helpful comments and suggestions. For any correspondence, please contact Yansong Feng.

heading: A Annotation Details for EGAlign Dataset
text: In this section, we will introduce the specific alignment rules when constructing the EGAlign
datasets. For entity alignment, we link the argument entities from EGp,len and EG p,l zh respectively to English and Chinese versions of Wikidata, and obtain a set of aligned entity pairs through the inter-language links from entities of English version of Wikidata to those in Chinese. For predicates alignment, human annotators are asked to grade scores (1-3) to the sampled relation pairs according to criterion as follow: Score 1: Seldom Align. Two predicates can not replace each other in any context. Score 2: Sometimes Align. Two predicates have similar usage and interpretation in some contexts (e.g., polysemy). Score 3: Always Align. Two predicates have the same usage and semantics in any context. The predicate pairs with average score greater than 1.5 are selected as predicate alignment seeds. We recruited the annotators from our school, and they are college students who are proficient in Chinese and English. Before starting annotation, annotators were informed what we will use this dataset for and the data collection protocol was approved by an ethics review board. Besides, they were paid with ¥60 per hour, which is a reasonable payment in our country.

heading: B More Details of Entailment Detection Evaluation
text: When evaluating on Levy/Holt and Berant datasets, for each pair of premise and hypothesis, we search the EGs for entailment edges from the predicate of premise to the predicate of hypothesis, and return the entailment scores associated with these edges.
For entailment detection, we compare our enhanced EGs with a strong baseline BERT. On Chinese subsets, for each premise-hypothesis pair, we compute the cosine similarity between their pretrained BERT representations of [CLS] tokens, denoted by BERTzh. For the English subsets, we average the BERT hidden states of the predicate’s start and end tokens as the final representations for premise or hypothesis, and calculate the cosine similarity of the representations, denoted by BERTen.
C Implementation Details
The implementation details of our ATE are summarized in Table 5. Our model were trained on Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz, and the training converged to be stable in 100 epochs. The training time of ATE for 100 epochs are about 43 minutes.
D Impact of Available Training Data for Predicate Alignment
To explore the impact of the size of training data on our model, we compare our CGI with Glovesim and the strongest baseline RNM by varying the proportion of seed predicate and entity alignments from 20% to 60% with a step of 10%. Figure 4 illustrates the MAP@1 for predicate alignment from English to Chinese of the three models on EGAlign dataset. As the amount of seed alignments increases, the performances of all three models gradually improve. Our CGI consistently obtains superior results compared to Glove-sim and RNM. These results show the promising performance of our model. Furthermore, according to the current trend of CGI, we believe that our model will achieve much better performance with more training data.

heading: E Entity Alignment
text: Our model can also achieve accurate entity alignment simultaneously. For entity alignment, we compare with BootEA, RDGCN, HGCN-JE/JR, Glove-sim, BERT-sim as well as the state-of-theart entity alignment model NMN (Wu et al., 2020) which presents a graph sampling method for identifying the most informative neighbors towards entity alignment and utilize a cross-graph attention-based matching mechanism to compare the neighborhood subgraphs of two entities for entity alignment.
Table 6 shows the entity alignment performance
on EGAlign dataset. We can observe that our CGI outperforms all the compared baselines across all the metrics, which shows that CGI can also learn better entity representations towards alignment and achieve accurate entity alignment.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitation section
A2. Did you discuss any potential risks of your work? Not applicable. Our work do not have potential risks.
3 A3. Do the abstract and introduction summarize the paper’s main claims? abstract and section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? section 6
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix B
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? section 5
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? section 6
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. No, we do not use them.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Appendix A
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Appendix A
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Appendix A
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Appendix A
3 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Appendix A
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Appendix A

heading: Future Work
text: We will explore the semi-supervised or unsupervised predicate alignment method between different EGs in our future work. (2) Our current enhancement strategy introduced in Section 4. 2 is straightforward. It might not be robust enough when dealing with entailment graphs of poor quality. We will explore more adaptive EG enhancement methods in the future