SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Split	Query Types	FB15K	FB15K-237	NELL995
Train	1p, 2p, 3p, 2i, 3i	273,710	149,689	107,982
2in, 3in, inp, pin, pni	27,371	14,968	10,798
Valid	1p	59,078	20,094	16,910
Others	8,000	5,000	4,000
Test	1p	66,990	22,804	17,021
Others	8,000	5,000	4,000
Table 1: Statistics on the different types of query structures in FB15K, FB15K-237, and NELL995.
Figure 3: Query structures considered in our experiments, as proposed by Ren and Leskovec [2020] – the naming of each query structure corresponds to projection  $(\mathbf{p})$ , intersection  $(\mathbf{i})$ , union  $(\mathbf{u})$  and negation  $(\mathbf{n})$ , reflecting how they were generated in the BetaE paper [Ren and Leskovec, 2020]. An example of a pin query is  $?T : \exists V.p(a,V)$ ,  $q(V,T) \neg r(b,T)$ , where  $a$  and  $b$  are anchor nodes,  $V$  is a variable node, and  $T$  is the query target node.
Equation (4) identifies the variable-to-entity substitution  $S$  that 1) maximises the query score score  $(\mathcal{Q},S)$ , defined in Equation (2), and 2) associates the answer variable  $A$  with  $a^\prime \in \mathcal{E}$ , i.e.  $A\gets a^{\prime}\in S$ . For computing  $S$  with the additional constraint that  $A\leftarrow a^{\prime}\in S$ , we use the complex query answering procedure outlined in Section 3. We optimise the additional parameters  $\mathbf{W}$  introduced in Section 4, by gradient descent on the likelihood of the true answers on a dataset  $\mathcal{D} = \{(\mathcal{Q}_i,a_i)\}_{i = 1}^{|\mathcal{D}|}$  of query-answer pairs by using a 1-vs-all cross-entropy loss, introduced by Lacroix et al. [2018], which was also used to train the neural link prediction model:
$$
\mathcal {L} (\mathcal {D}) = \sum_ {\left(\mathcal {Q} _ {i}, a _ {i}\right) \in \mathcal {D}} - \operatorname {s c o r e} \left(\mathcal {Q} _ {i}, A \leftarrow a _ {i}\right) + \log \left[ \sum_ {a ^ {\prime} \in \mathcal {E}} \exp \left(\operatorname {s c o r e} \left(\mathcal {Q} _ {i}, A \leftarrow a ^ {\prime}\right)\right) \right]. \tag {5}
$$
In addition to the 1- vs. all [Ruffinelli et al., 2020] loss in Equation (5), we also experiment with the binary cross-entropy loss, using the negative sampling procedure from Ren and Leskovec [2020].
Datasets To evaluate the complex query answering capabilities of our method, we use a benchmark comprising of 3 KGs: FB15K [Bordes et al., 2013], FB15K-237 [Toutanova and Chen, 2015] and NELL995 [Xiong et al., 2017]. For a fair comparison with previous work, we use the datasets of FOL queries proposed by Ren and Leskovec [2020], which includes nine structures of EPFO queries and 5 query types with atomic negations, seen in Figure 3. The datasets provided by Ren and Leskovec [2020] introduce queries with hard answers, which are the answers that cannot be obtained by direct graph traversal; in addition, this dataset does not include queries with more than 100 answers, increasing the difficulty of the complex query answering task. The statistics for each dataset can be seen in Table 1. Note that during training, we only use  $2i$ ,  $3i$ ,  $2in$ , and  $3in$  queries, corresponding to  $\leq 30\%$  of the training dataset, for the adaptation of the neural link predictor. To assess the model's ability to generalise, we evaluate it on all query types.
Model	\(avg_p\)	\(avg_n\)	1p	2p	3p	2i	3i	pi	ip	2u	up	2in	3in	inp	pin	pni
FB15K
GQE	28.0	-	54.6	15.3	10.8	39.7	51.4	27.6	19.1	22.1	11.6	-	-	-	-	-
Q2B	38.0	-	68.0	21.0	14.2	55.1	66.5	39.4	26.1	35.1	16.7	-	-	-	-	-
BetaE	41.6	11.8	65.1	25.7	24.7	55.8	66.5	43.9	28.1	40.1	25.2	14.3	14.7	11.5	6.5	12.4
CQD-CO	46.9	-	89.2	25.3	13.4	74.4	78.3	44.1	33.2	41.8	21.9	-	-	-	-	-
CQD-Beam	68.4	-	89.2	65.3	29.7	76.1	79.3	70.6	70.6	72.3	59.4	-	-	-	-	-
ConE	49.8	14.8	73.3	33.8	29.2	64.4	73.7	50.9	35.7	55.7	31.4	17.9	18.7	12.5	9.8	15.1
GNN-QE	72.8	38.6	88.5	69.3	58.7	79.7	83.5	69.9	70.4	74.1	61.0	44.7	41.7	42.0	30.1	34.3
\(CQD^A\)	70.4	42.8	89.2	64.5	57.9	76.1	79.4	70.0	70.6	68.4	57.9	54.7	47.1	37.6	35.3	24.6
FB15K-237
GQE	16.3	-	35.0	7.2	5.3	23.3	34.6	16.5	10.7	8.2	5.7	-	-	-	-	-
Q2B	20.1	-	40.6	9.4	6.8	29.5	42.3	21.2	12.6	11.3	7.6	-	-	-	-	-
BetaE	20.9	5.5	39.0	10.9	10.0	28.8	42.5	22.4	12.6	12.4	9.7	5.1	7.9	7.4	3.5	3.4
CQD-CO	21.8	-	46.7	9.5	6.3	31.2	40.6	23.6	16.0	14.5	8.2	-	-	-	-	-
CQD-Beam	25.3	-	46.7	13.3	7.9	34.4	48.3	27.1	20.4	17.6	11.5	-	-	-	-	-
ConE	23.4	5.9	41.8	12.8	11.0	32.6	47.3	25.5	14.0	14.5	10.8	5.4	8.6	7.8	4.0	3.6
GNN-QE	26.8	10.2	42.8	14.7	11.8	38.3	54.1	31.1	18.9	16.2	13.4	10.0	16.8	9.3	7.2	7.8
\(CQD^A\)	25.7	10.7	46.7	13.6	11.4	34.5	48.3	27.4	20.9	17.6	11.4	13.6	16.8	7.9	8.9	5.8
NELL995
GQE	18.6	-	32.8	11.9	9.6	27.5	35.2	18.4	14.4	8.5	8.8	-	-	-	-	-
Q2B	22.9	-	42.2	14.0	11.2	33.3	44.5	22.4	16.8	11.3	10.3	-	-	-	-	-
BetaE	24.6	5.9	53.0	13.0	11.4	37.6	47.5	24.1	14.3	12.2	8.5	5.1	7.8	10.0	3.1	3.5
CQD-CO	28.8	-	60.4	17.8	12.7	39.3	46.6	30.1	22.0	17.3	13.2	-	-	-	-	-
CQD-Beam	31.8	-	60.4	22.6	13.6	42.6	52.0	31.2	25.6	19.9	16.7	-	-	-	-	-
ConE	27.2	6.4	53.1	16.1	13.9	40.0	50.8	26.3	17.5	15.3	11.3	5.7	8.1	10.8	3.5	3.9
GNN-QE	28.9	9.7	53.3	18.9	14.9	42.4	52.5	30.8	18.9	15.9	12.6	9.9	14.6	11.4	6.3	6.3
\(CQD^A\)	32.3	13.3	60.4	22.9	16.7	43.4	52.6	32.1	26.4	20.0	17.0	15.1	18.6	15.8	10.7	6.5
Table 2: MRR results for FOL queries on the testing sets.  $\mathbf{avg}_p$  designates the averaged results for EPFO queries  $(\wedge, \vee)$ , while  $\mathbf{avg}_n$  pertains to queries including atomic negations  $(\neg)$ . The results for the baselines are from Zhu et al. [2022].
Evaluation Protocol For a fair comparison with prior work, we follow the evaluation scheme in Ren and Leskovec [2020] by separating the answer of each query into easy and hard sets. For test and validation splits, we define hard queries as those that cannot be answered via direct traversal along the edges of the KG and can only be answered by predicting at least one missing link, meaning non-trivial reasoning should be completed. We evaluate the method on non-trivial queries by calculating the rank  $r$  for each hard answer against non-answers and computing the Mean Reciprocal Rank (MRR).
Baselines We compare  $\mathrm{CQD}^{\mathcal{A}}$  with state-of-the-art methods from various solution families in Section 2. In particular, we choose GQE [Hamilton et al., 2018], Query2Box [Ren et al., 2020], BetaE [Ren and Leskovec, 2020] and ConE [Zhang et al., 2021] as strong baselines for query embedding methods. We also compare with methods based on GNNs and fuzzy logic, such as FuzzQE [Chen et al., 2022], GNN-QE [Zhu et al., 2022], and the original CQD [Arakelyan et al., 2021, Minervini et al., 2022], which uses neural link predictors for answering EPFO queries without any fine-tuning on complex queries.
Model Details Our method can be used with any neural link prediction model. Following Arakelyan et al. [2021], Minervini et al. [2022], we use ComplEx-N3 [Lacroix et al., 2018]. We identify the optimal hyper-parameters using the validation MRR. We train for 50,000 steps using Adagrad as an optimiser and 0.1 as the learning rate. The beam-size hyper-parameter  $k$  was selected in  $k \in \{512, 1024, \dots, 8192\}$ , and the loss was selected across 1-vs-all [Lacroix et al., 2018] and binary cross-entropy with one negative sample.
Parameter Efficiency We use the query types  $2i$ ,  $3i$ ,  $2in$ ,  $3in$  for training the calibration module proposed in Section 4. We selected these query types as they do not require variable assignments other than for the answer variable  $A$ , making the training process efficient. As the neural link prediction model is frozen, we only train the adapter layers that have a maximum of  $\mathbf{W} \in \mathbb{R}^{2 \times 2d}$  learnable weights. Compared to previous works, we have  $\approx 10^3$  times fewer trainable parameters, as shown in Table 3, while maintaining competitive results.
Figure 4: Average test MRR score  $(y$ -axis) of  $\mathrm{CQD}^A$  using  $1\%$  and  $100\%$  of the training queries from FB15K-237 throughout the training iterations  $(x$ -axis).
Number of parameters
CQD-A	FB15K	FB15K-237	NELL
\( \underbrace{1.3 \times 10^{7}}_{\text{frozen}} \)	\( \underbrace{1.3 \times 10^{7}}_{\text{frozen}} \)	\( \underbrace{7.5 \times 10^{7}}_{\text{frozen}} \)
\( +4 \times 10^{3} \)	\( +4 \times 10^{3} \)	\( +4 \times 10^{3} \)
BetaE	\( 1.3 \times 10^{7} \)	\( 1.3 \times 10^{7} \)	\( 6 \times 10^{7} \)
Q2B	\( 1.2 \times 10^{7} \)	\( 1.2 \times 10^{7} \)	\( 6 \times 10^{7} \)
GNN-QE	\( 3 \times 10^{6} \)	\( 3 \times 10^{6} \)	\( 3 \times 10^{6} \)
ConE	\( 1.2 \times 10^{7} \)	\( 1.2 \times 10^{7} \)	\( 6 \times 10^{7} \)
GQE	\( 1.5 \times 10^{7} \)	\( 1.5 \times 10^{7} \)	\( 7.5 \times 10^{7} \)
Table 3: Number of parameters used by different complex query answering methods – values for GNN-QE are approximated using the backbone NBFNet [Zhu et al., 2021], while the remaining use their original studies.
5.1 Results
Complex Query Answering Table 2 shows the predictive accuracy of  $\mathrm{CQD}^{\mathcal{A}}$  for answering complex queries compared to the current state-of-the-art methods. Some methods do not support queries that include negations; we leave the corresponding entries blank. We can see that  $\mathrm{CQD}^{\mathcal{A}}$  increases the MRR from 34.4 to 35.1 averaged across all query types and datasets. In particular,  $\mathrm{CQD}^{\mathcal{A}}$  shows the most substantial increase in predictive accuracy on NELL995 by producing more accurate results than all other methods for all query types.  $\mathrm{CQD}^{\mathcal{A}}$  can achieve these results using  $\leq 30\%$  of the complex query types during training while maintaining competitive results across each dataset and query type. For queries including negations,  $\mathrm{CQD}^{\mathcal{A}}$  achieves a relative improvement of  $6.8\%$  to  $37.1\%$ , which can be attributed to the fact that the adaptation is completed with query types  $2in$  and  $3in$  that include negation, which allows for learning an adaptation layer that is robust for these types of queries. In our experiments, we found that calculating the neural adaptation parameters  $\theta$  of the adaptation function  $\rho_{\theta}$  in Equation (3) as a function of the predicate representation yields the most accurate results followed by computing  $\theta$  as a function of the source entity and predicate representation, which is strictly more expressive. In Appendix A, we show the impact of the adaptation layers on the neural link prediction scores.
The adaptation process does not require data-intensive training and allows the model to generalise to query types not observed during training. This prompts us to investigate the minimal amount of data samples and query types required for adaptation.
Data Efficiency To analyse the data efficiency of  $\mathrm{CQD}^A$ , we compare the behaviour of the pre-trained link predictors tuned with  $1\%$  and  $100\%$  of the training complex query examples in FB15K-237, presented in Table 4. For adapting on  $1\%$  of the training complex queries, we used the same hyper-parameters we identified when training on the full dataset. Even when using  $1\%$  of the complex training queries (3290 samples) for tuning, the model still achieves competitive results, with an average MRR difference of 2.2 compared to the model trained using the entire training set.  $\mathrm{CQD}^A$  also produces higher test MRR results than GNN-QE with an average MRR increase of 4.05.
We can also confirm that the adaptation process converges after  $\leq 10\%$  of the training epochs as seen in Figure 4. The convergence rate is not hindered when using only  $1\%$  of the training queries. This shows that  $\mathrm{CQD}^{\mathcal{A}}$  is a scalable method with a fast convergence rate that can be trained in a data-efficient manner.
Out-of-Distribution Generalisation To study the generalisation properties of  $\mathrm{CQD}^{\mathcal{A}}$ , we trained the adaptation layer on all atomic queries and only  $1\%$  of samples for one training query type  $2i$ , one of the simplest complex query types. We see in Table 4 that  $\mathrm{CQD}^{\mathcal{A}}$  can generalise to other types of complex queries not observed during training with an average MRR difference of 2.9 compared to training on all training query types.  $\mathrm{CQD}^{\mathcal{A}}$  also produces significantly higher test MRR results
Dataset	Model	1p	2p	3p	2i	3i	pi	ip	2u	up	2in	3in	inp	pin	pni
FB237, 1%	CQDA	46.7	11.8	11.4	33.6	41.2	24.82	17.81	16.45	8.74	10.8	13.86	5.93	5.38	14.82
GNN-QBEtaE	36.82	8.96	8.13	33.02	49.28	24.58	14.18	10.73	8.47	4.89	12.31	6.74	4.41	4.09
36.80	6.89	5.94	22.84	34.34	17.12	8.72	9.23	5.66	4.44	6.14	5.18	2.54	2.94
FB237 2i, 1%	CQDA	46.7	11.8	11.2	30.35	40.75	23.36	18.28	15.85	8.96	9.36	10.25	5.17	4.46	4.44
GNN-QBEtaE	34.81	5.40	5.17	30.12	48.88	23.06	12.65	9.85	5.26	4.26	12.5	4.43	0.71	1.98
37.99	5.62	4.48	23.73	35.25	15.63	7.96	9.73	4.56	0.15	0.49	0.62	0.10	0.14
Table 4: Comparison of test MRR results for queries on FB15K-237 using the following training sets - FB237,  $1\%$  (resp. FB237 2i,  $1\%$ ) means that, in addition to all 1p (atomic) queries, only  $1\%$  of the complex queries (resp. 2i queries) was used during training. As  $\mathrm{CQD}^{\mathcal{A}}$  uses a pre-trained link predictor, we also include all 1p queries when training GNN-QE for a fair comparison.
Model	2p	2i	3i	pi	ip	2u	up	2in	3in	inp	pin	pni
CQD	13.2	34.5	48.2	26.8	20.3	17.4	10.3	5.4	12.4	6.1	3.2	4.6
CQD_F	9.3	22.8	34.9	19.8	14.5	13.0	7.2	7.4	7.1	4.9	3.9	3.8
CQD_A_F	9.5	23.9	39.0	19.8	14.5	14.2	7.2	8.4	9.7	4.9	4.2	3.6
CQD_C	10.9	33.7	47.3	25.6	18.9	16.4	9.4	7.9	12.2	6.6	4.2	5.0
CQD_R	6.4	22.2	31.0	16.6	11.2	12.5	4.8	4.7	5.9	4.1	2.0	3.5
CQD_A	13.2	35.0	48.5	27.3	20.7	17.6	10.5	13.2	14.9	7.4	7.8	5.5
Table 5: Test MRR results for FOL queries on FB15K-237 using the following CQD extensions: CQD from Arakelyan et al. [2021], Minervini et al. [2022] with the considered normalisation and negations;  $\mathrm{CQD_F}$ , where we fine-tune all neural link predictor parameters in CQD;  $\mathrm{CQD}_F^A$ , where we fine-tune all link predictor parameters in  $\mathrm{CQD}^A$ ;  $\mathrm{CQD_R}$ , where we learn a transformation for the entity and relation embeddings and we use it to replace the initial entity and relation representations; and  $\mathrm{CQD_C}$ , where we learn a transformation for the entity and relation embeddings, and we concatenate it to the initial entity and relation representations.
than GNN-QE, with an average increase of 5.1 MRR. The greatest degradation in predictive accuracy occurs for the queries containing negations, with an average decrease of 2.7. This prompts us to conjecture that being able to answer general EPFO queries is not enough to generalise to the larger set of queries, which include atomic negation. However, our method can generalise on all query types, using only  $1\%$  of the  $2i$  queries, with 1496 overall samples for adaptation.
Fine-Tuning All Model Parameters One of the reasons for the efficiency of  $\mathrm{CQD}^A$  is that the neural link predictor is not fine-tuned for query answering, and only the parameters in the adaptation function are learned. We study the effect of fine-tuning the link predictor using the full training data for CQD and  $\mathrm{CQD}^A$  on FB15K-237. We consider several variants: 1)  $\mathrm{CQD}_{\mathrm{F}}$ , where we Fine-tune all neural link predictor parameters in CQD; 2)  $\mathrm{CQD}_F^A$ , where we fine-tune all link predictor parameters in  $\mathrm{CQD}^A$ , 3)  $\mathrm{CQD}_{\mathrm{R}}$ , where we learn a transformation for the entity and relation embeddings and we use it to Replace the initial entity and relation representations, and 4)  $\mathrm{CQD}_{\mathrm{C}}$ , where we learn a transformation for the entity and relation embeddings, and we Concatenate it to the initial entity and relation representations.
It can be seen from Table 5 that  $\mathrm{CQD}^A$  yields the highest test MRR results across all query types while fine-tuning all the model parameters produces significant degradation along all query types, which we believe is due to catastrophic forgetting [Goodfellow et al., 2013] of the pre-trained link predictor.
In this work, we propose the novel method  $\mathrm{CQD}^{\mathcal{A}}$  for answering complex FOL queries over KGs, which increases the averaged MRR over the previous state-of-the-art from 34.4 to 35.1 while using  $\leq 30\%$  of query types. Our method uses a single adaptation layer over neural link predictors, which allows for training in a data-efficient manner. We show that the method can maintain competitive predictive accuracy even when using  $1\%$  of the training data. Furthermore, our experiments on training on a subset  $(1\%)$  of the training queries from a single query type  $(2i)$  show that it can generalise to
new queries that were not used during training while being data-efficient. Our results provide further evidence for how neural link predictors exhibit a form of compositionality that generalises to the complex structures encountered in the more general problem of query answering.  $\mathrm{CQD}^{\mathcal{A}}$  is a method for improving this compositionality while preserving computational efficiency. As a consequence, rather than designing specialised models trained end-to-end for the query answering task, we can focus our efforts on improving the representations learned by neural link predictors, which would then transfer to query answering via efficient adaptation, as well as other downstream tasks where they have already proved beneficial, such as clustering, entity classification, and information retrieval.
Pasquale was partially funded by the European Union's Horizon 2020 research and innovation programme under grant agreement no. 875160, ELIAI (The Edinburgh Laboratory for Integrated Artificial Intelligence) EPSRC (grant no. EP/W002876/1), an industry grant from Cisco, and a donation from Accenture LLP, and is grateful to NVIDIA for the GPU donations. Daniel and Michael were partially funded by Elsevier's Discovery Lab. Michael was partially funded by the Graph-Massivizer project (Horizon Europe research and innovation program of the European Union under grant agreement 101093202). Erik is partially funded by a DFF Sapere Aude research leader grant under grant agreement No 0171-00034B, as well as by a NEC PhD fellowship, and is supported by the Pioneer Centre for AI, DNRF grant number P1. Isabelle is partially funded by a DFF Sapere Aude research leader grant under grant agreement No 0171-00034B, as well as by the Pioneer Centre for AI, DNRF grant number P1. This work was supported by the Edinburgh International Data Facility (EIDF) and the Data-Driven Innovation Programme at the University of Edinburgh.
S. Amin, S. Varanasi, K. A. Dunfield, and G. Neumann. Lowfer: Low-rank bilinear pooling for link prediction. In International Conference on Machine Learning, pages 257-268. PMLR, 2020.
E. Arakelyan, D. Daza, P. Minervini, and M. Cochez. Complex query answering with neural link predictors. In ICLR. OpenReview.net, 2021.
S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. G. Ives. DBpedia: A nucleus for a web of open data. In ISWC/ASWC, volume 4825 of Lecture Notes in Computer Science, pages 722-735. Springer, 2007.
I. Balăžević, C. Allen, and T. M. Hospedales. Tucker: Tensor factorization for knowledge graph completion. arXiv preprint arXiv:1901.09590, 2019.
O. Bodenreider, R. Cornet, and D. J. Vreeman. Recent developments in clinical terminologies - snomed ct, loinc, and rxnorm. Yearbook of medical informatics, 27:129-139, Aug 2018. ISSN 2364-0502.
K. D. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD Conference, pages 1247-1250. ACM, 2008.
A. Bordes, N. Usunier, A. García-Durán, J. Weston, and O. Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787-2795, 2013.
X. Chen, Z. Hu, and Y. Sun. Fuzzy logic based logical query answering on knowledge graphs. In AAAI, pages 3939-3948. AAAI Press, 2022.
N. Choudhary, N. Rao, S. Katariya, K. Subbian, and C. K. Reddy. Self-supervised hyperboloid representations from logical queries over knowledge graphs. In Proceedings of the Web Conference 2021, pages 1373-1384, 2021.
R. Das, S. Dhuliawala, M. Zaheer, L. Vilnis, I. Durugkar, A. Krishnamurthy, A. Smola, and A. McCallum. Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv preprint arXiv:1711.05851, 2017.
B. A. Davey and H. A. Priestley. Introduction to Lattices and Order, Second Edition. Cambridge University Press, 2002.
A. Sadeghian, M. Armandpour, P. Ding, and D. Z. Wang. Drum: End-to-end differentiable rule mining on knowledge graphs. Advances in Neural Information Processing Systems, 32, 2019.
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. v. d. Berg, I. Titov, and M. Welling. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593-607. Springer, 2018.
F. M. Suchanek, G. Kasneci, and G. Weikum. Yago: a core of semantic knowledge. In WWW, pages 697-706. ACM, 2007.
Z. Sun, Z.-H. Deng, J.-Y. Nie, and J. Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019.
P. Tabacof and L. Costabello. Probability calibration for knowledge graph embedding models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum? id=S1g8K1BFwS.
K. Teru, E. Denis, and W. Hamilton. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning, pages 9448-9457. PMLR, 2020.
K. Toutanova and D. Chen. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pages 57-66, Beijing, China, July 2015. Association for Computational Linguistics. doi: 10.18653/v1/W15-4007. URL https://www.aclweb.org/anthology/W15-4007.
T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, and G. Bouchard. Complex embeddings for simple link prediction. In ICML, volume 48 of JMLR Workshop and Conference Proceedings, pages 2071-2080. JMLR.org, 2016.
S. Vashishth, S. Sanyal, V. Nitin, and P. Talukdar. Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082, 2019.
D. Vrandecic and M. Krötzsch. Wikidata: A free collaborative knowledge base. Communications of the ACM, 57:78-85, 2014. URL http://cacm.acm.org/magazines/2014/10/178785-wikidata/fulltext.
W. Xiong, T. Hoang, and W. Y. Wang. Deeppath: A reinforcement learning method for knowledge graph reasoning. In EMNLP, pages 564-573. Association for Computational Linguistics, 2017.
B. Yang, W.-t. Yih, X. He, J. Gao, and L. Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575, 2014.
F. Yang, Z. Yang, and W. W. Cohen. Differentiable learning of logical rules for knowledge base reasoning. Advances in neural information processing systems, 30, 2017.
Z. Zhang, J. Wang, J. Chen, S. Ji, and F. Wu. Cone: Cone embeddings for multi-hop reasoning over knowledge graphs. Advances in Neural Information Processing Systems, 34:19172-19183, 2021.
Z. Zhu, Z. Zhang, L.-P. Xhonneux, and J. Tang. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 34:29476-29490, 2021.
Z. Zhu, M. Galkin, Z. Zhang, and J. Tang. Neural-symbolic models for logical queries on knowledge graphs. arXiv preprint arXiv:2205.10128, 2022.
Figure 5: The distribution of the scores of the neural link predictor before applying the adaptation layer and after.
Figure 6: Evaluation of  $\mathrm{CQD}^A$  using  $1\%$  and  $100\%$  of the training complex queries during tuning (top) and 2i queries (bottom) from FB15K-237.
A Impact of adaptation