SlowGuess/NAACL_2022_OCR huggingface dataset:
Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. 2015. Aligning knowledge and text embeddings by entity descriptions. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pages 267-272.
A Relation Prediction
Relation prediction results for all the KGE methods are shown in Table 8. As we see, unlike semantic matching-based KGE models, incorporating LDPs into the KG do not improve relation prediction for translational distance-based KGE methods (TransE and RotatE). For KG+ExtractedLDPs embeddings, the performance for with-mention set decreases by 0.045 and 0.012 on average for MRR and H@{10,3,1}, for TransE and RotatE respectively. In-depth analysis for this observation can be conducted in future research.
B Tail Prediction for Relation Categories
Table 9 presents Hits@10 for tail prediction considering 1to1, 1toN, Nto1 and NtoN relation categories. As we see, SuperBorrow embeddings obtain the best results for all KGE methods and all the relation categories.
Method	# Tuples	1to1 192	1toN 1293	Nto1 4289	NtoN 14,696
TransE	KG only	0.547	0.097	0.851	0.574
SuperBorrow	0.943	0.647	0.980	0.907
DistMult	KG only	0.521	0.055	0.774	0.507
SuperBorrow	0.880	0.424	0.898	0.657
ComplEx	KG only	0.500	0.034	0.787	0.518
SuperBorrow	0.869	0.456	0.964	0.753
RotatE	KG only	0.536	0.107	0.855	0.561
SuperBorrow	0.927	0.731	0.983	0.853
Table 9: Hits@10 of tail prediction for different relation categories.
C Training KGE Methods
For reproducability, we list the hyperparameter setting to train KGE methods in Table 10. AdaGrad (Duchi et al., 2011) with 100 batches is used to learn KGEs. Table 11 shows the training time (in hours) to train KGE methods for KG only and SuperBorrow using OpenKE-Pytorch tool (Han et al., 2018).
Model	overall	with-mention	without-mention
MRR	MR	H@10	H@3	H@1	MRR	MR	H@10	H@3	H@1	MRR	MR	H@10	H@3	H@1
TransE (KG only)	0.961	1.6	0.992	0.980	0.940	0.919	1.9	0.988	0.958	0.875	0.967	1.5	0.993	0.983	0.949
KG+ExtractedLDPs	0.934	1.6	0.994	0.967	0.899	0.860	1.7	0.991	0.926	0.789	0.944	1.5	0.994	0.973	0.914
LinkAll	0.932	1.5	0.993	0.955	0.899	0.845	1.9	0.985	0.887	0.778	0.944	1.4	0.994	0.964	0.916
Co-occurrence	0.925	1.6	0.993	0.962	0.887	0.863	1.8	0.990	0.931	0.793	0.933	1.5	0.993	0.967	0.899
NeighbBorrow	0.927	1.5	0.994	0.964	0.888	0.862	1.7	0.993	0.929	0.791	0.936	1.5	0.994	0.969	0.901
SuperBorrow	0.925	1.5	0.993	0.963	0.886	0.868	1.8	0.990	0.926	0.802	0.933	1.5	0.994	0.968	0.897
DistMult (KG only)	0.901	4.1	0.968	0.938	0.856	0.855	4.5	0.959	0.914	0.789	0.907	4.0	0.969	0.942	0.865
KG+ExtractedLDPs	0.918	2.6	0.980	0.955	0.876	0.887	2.4	0.982	0.940	0.826	0.922	2.7	0.980	0.957	0.883
LinkAll	0.825	7.2	0.940	0.887	0.752	0.883	2.1	0.986	0.944	0.813	0.818	7.8	0.934	0.880	0.744
Co-occurrence	0.918	2.4	0.979	0.954	0.875	0.89	2.0	0.980	0.942	0.831	0.921	2.4	0.979	0.956	0.882
NeighbBorrow	0.917	3.0	0.979	0.955	0.874	0.883	2.7	0.976	0.942	0.819	0.921	3.0	0.979	0.956	0.881
SuperBorrow	0.920	2.2	0.984	0.960	0.875	0.885	2.2	0.980	0.943	0.822	0.924	2.2	0.985	0.962	0.882
ComplEx (KG only)	0.929	3.1	0.977	0.954	0.900	0.884	4.8	0.962	0.925	0.835	0.935	2.8	0.980	0.957	0.908
KG+ExtractedLDPs	0.944	1.9	0.987	0.967	0.917	0.921	1.7	0.986	0.962	0.877	0.947	1.9	0.987	0.967	0.922
LinkAll	0.867	4.0	0.955	0.909	0.812	0.906	1.8	0.988	0.960	0.848	0.861	4.3	0.951	0.902	0.808
Co-occurrence	0.944	1.8	0.987	0.967	0.916	0.930	1.7	0.989	0.965	0.892	0.946	1.8	0.987	0.967	0.920
NeighbBorrow	0.948	1.7	0.989	0.973	0.921	0.925	1.9	0.987	0.965	0.884	0.951	1.7	0.989	0.974	0.925
SuperBorrow	0.946	1.7	0.990	0.972	0.917	0.922	1.8	0.987	0.962	0.879	0.949	1.7	0.990	0.973	0.922
RotatE (KG only)	0.972	1.4	0.996	0.990	0.954	0.945	1.3	0.993	0.981	0.910	0.976	1.4	0.997	0.991	0.960
KG+ExtractedLDPs	0.967	1.3	0.995	0.988	0.945	0.933	1.5	0.983	0.974	0.892	0.971	1.2	0.996	0.990	0.952
LinkAll	0.958	1.3	0.995	0.984	0.931	0.923	1.6	0.983	0.964	0.879	0.963	1.3	0.996	0.987	0.938
Co-occurrence	0.964	1.3	0.994	0.985	0.943	0.931	1.4	0.987	0.970	0.892	0.969	1.3	0.995	0.987	0.949
NeighbBorrow	0.964	1.3	0.995	0.985	0.941	0.933	1.5	0.985	0.971	0.894	0.968	1.2	0.996	0.987	0.948
SuperBorrow	0.964	1.2	0.995	0.986	0.941	0.931	1.5	0.985	0.972	0.892	0.968	1.2	0.996	0.988	0.947
Table 8: Relation predictino on FB15K237.
KGE Method	learning rate	embedding dimension	negative samples	loss function	margin	epochs
TransE	1.0	300	25	Margin loss	5.0	1000
DistMult	0.5	300	25	SoftPlus loss	-	1000
ComplEx	0.5	100	25	SoftPlus loss	-	1000
RotatE	2e-5	300	25	SigmoidLoss	9.0	1000
Table 10: The hyperparameter setting for KGE methods on link prediction task.
Method	#Train tuples	Time (h)
TransE	KG only	272,115	0.42
SuperBorrow	1,217,294	1.58
DistMult	KG only	272,115	0.78
SuperBorrow	1,036,904	2.67
ComplEx	KG only	272,115	0.69
SuperBorrow	946,709	2.11
RotatE	KG only	272,115	1.11
SuperBorrow	1,127,099	4.13
Table 11: Training time on FB15K237 in hours.