Table 5: Link prediction results on the test-I, test-II, and test-all sets of FB122 and WN18 (filtered setting).

Test-I || MRR || 0.296

Test-I || MED || 13.0

Test-I || HITS@3 (%) || 36.0

Test-I || HITS@5 (%) || 41.5

Test-I || HITS@10 (%) || 48.1

Test-II || MRR || 0.630

Test-II || MED || 2.0

Test-II || HITS@3 (%) || 77.5

Test-II || HITS@5 (%) || 82.8

Test-II || HITS@10 (%) || 88.4

Test-ALL || MRR || 0.480

Test-ALL || MED || 2.0

Test-ALL || HITS@3 (%) || 58.9

Test-ALL || HITS@5 (%) || 64.2

Test-ALL || HITS@10 (%) || 70.2

FB122 || TransH

0.280

15.0

33.6

39.1

46.4

0.606

2.0

70.1

75.4

82.0

0.460

3.0

53.7

59.1

66.0

FB122 || TransR

0.283

16.0

33.4

39.2

46.0

0.499

2.0

57.0

63.2

70.1

0.401

5.0

46.4

52.4

59.3

FB122 || KALE-Trip

0.299

10.0

36.6

42.9

50.2

0.650

2.0

79.0

83.4

88.7

0.492

2.0

59.9

65.2

71.4

FB122 || KALE-Pre

0.291

11.0

35.8

41.9

49.8

0.713

1.0

82.9

86.1

89.9

0.523

2.0

61.7

66.2

71.8

FB122 || KALE-Joint

0.325

9.0

38.4

44.7

52.2

0.684

1.0

79.7

84.1

89.6

0.523

2.0

61.2

66.4

72.8

WN18 || TransE

0.306

3.0

57.4

72.3

80.1

0.511

2.0

87.5

95.6

98.7

0.453

2.0

79.1

89.1

93.6

WN18 || TransH

0.318

3.0

61.7

72.4

78.2

0.653

2.0

87.1

91.4

94.6

0.560

2.0

80.0

86.1

90.0

WN18 || TransR

0.299

3.0

56.1

66.7

74.5

0.597

2.0

75.0

81.7

88.0

0.514

2.0

69.7

77.5

84.3

WN18 || KALE-Trip

0.322

3.0

61.0

73.9

80.8

0.555

2.0

90.6

96.3

98.8

0.490

2.0

82.3

90.1

93.8

WN18 || KALE-Pre

0.322

3.0

60.6

74.5

81.1

0.612

2.0

96.4

98.6

99.6

0.532

2.0

86.4

91.9

94.4

WN18 || KALE-Joint

0.338

3.0

65.5

76.3

82.1

0.787

1.0

93.3

95.4

97.2

0.662

2.0

85.5

90.1

93.0

Results. Table 5 show the results in the filtered setting. On each dataset we report the metrics on three sets: test-I, test-II, and the whole test set (denoted by test-all). Test-I contains test triples that cannot be directly inferred by performing pure logical inference on the training set, and hence might be intrinsically more difficult for the rules. The remaining test triples (i.e., the directly inferable ones) are included in Test-II. These triples have either been used directly as training instances in KALE-Pre, or encoded explicitly in training ground rules in KALE-Joint, making this set trivial for the rules to some extent. From the results, we can see that in both settings: (i) KALE-Pre and KALE-Joint outperform (or at least perform as well as) the other methods which use triples alone on almost all the test sets, demonstrating the superiority of incorporating logical rules. (ii) On the test-I sets which contain triples beyond the scope of pure logical inference, KALE-Joint performs significantly better than KALE-Pre. On these sets KALE-Joint can still beat all the baselines by a significant margin in most cases, while KALE-Pre can hardly outperform KALE-Trip. It demonstrates the capability of the joint embedding scenario to learn more predictive embeddings, through which we can make better predictions even beyond the scope of pure logical inference. (iii) On the test-II sets which contain directly inferable triples, KALE-Pre can easily beat all the baselines (even KALE-Joint). That means, for triples covered by pure logical inference, it is trivial to improve the performance by directly incorporating them as training instances.