# Convolutional Adaptive Network for Link Prediction in Knowledge Bases
## 3 Proposed Method
### Outline of CANet  
As shown in Figure 2, the CANet is a multi-layer convolutional network consisting of embedding combination, encoding, and scoring components. The vector of the embeddings of \(h\) and \(r\) is extracted by the encoding component. The score function ranks the feature vector and embedding of \(t\) in the scoring part. A summary of the details is given in the following. **Look Up**: We must first define the general embeddings of an input triplet (\(h\), \(r\), \(t\)) in the matrices of relations \(\mathbf{R}\) and entities \(\mathbf{E}\). Such embeddings can be expressed in the following way:  
\[\mathbf{h}=\mathbf{i}_{h}^{\top}\mathbf{E},\mathbf{r}=\mathbf{i}_{r}^{\top} \mathbf{R},\text{ and }\mathbf{t}=\mathbf{i}_{t}^{\top}\mathbf{E} \tag{2}\]  
where \(\mathbf{i}_{h}^{\top}\), \(\mathbf{i}_{r}^{\top}\), and \(\mathbf{i}_{t}^{\top}\) are the one-hot index vectors of \(h\), \(r\), and \(t\). **Interaction and Concatenation**: To generate multiple triplet-specific embeddings for each relation and entity, we present an interaction embedding \(\mathbf{i}_{r}\), which is connected to the relation. An interaction operation is defined to simulate the interaction effect of entities and relations as follows:  
\[\left\{ \begin{array}{l}\mathbf{h}^{i}=\mathbf{i}_{r}\circ\mathbf{h}\\ \mathbf{r}^{i}=\mathbf{h}^{i}\circ\mathbf{r}\end{array} \right. \tag{3}\]  
where \(\circ\) is the element-by-element operator. We call \(\mathbf{h}^{i}\) and \(\mathbf{r}^{i}\) the interaction embeddings of \(h\) and \(r\). They can provide generalization capabilities and rich representations. Then, the input matrices \(\mathbf{M}\) can be obtained as  
\[\mathbf{M}=\mathbf{h}^{i}\oplus\mathbf{r}^{i}\in\mathbb{R}^{2\times d} \tag{4}\]  
where \(\oplus\) denotes the concatenation operation. **Convolutional Operation**: Then, \(N\) separate kernels \(\mathbf{K}=[k_{1},k_{2},\dots,k_{N}]\in\mathbb{R}^{N\times h\times w}\) execute a convolution process on the input layer \(\mathbf{M}\). The feature maps \(\mathbf{F}_{r}=[f_{1},f_{2},\dots,f_{N}]\in\mathbb{R}^{N\times H\times W}\) are generated as follows:  
\[\mathbf{f}_{n}=\mathbf{M}\rightsquigarrow k_{n}\in\mathbb{R}^{H\times W},n=1,2, \dots,N \tag{5}\]  
where denotes the action of convolution. To hold the translational property, we set the height \(h\) of convolutional kernels to 2 in the experiment. **Attention Mechanism**: The local receptive field is then used by each kernel \(\mathbf{k}_{n}\). As a result, outside of the local receptive field, each output function map \(\mathbf{f}_{n}\) is unable to use contextual information. We can compact the global spatial information into a series of channel descriptors to solve this problem. To alleviate this problem, we can compress the global spatial information into a set of channel descriptors \(\mathbf{c}=[c_{1},c_{2},\dots,c_{N}]\in\mathbb{R}^{N}\) as follows:  
\[c_{n}=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}\mathbf{f}_{n}(i,j)\in \mathbb{R},n=1,2,\dots,N \tag{6}\]  
Then, the weights \(\mathbf{w}=[w_{1},w_{2},\dots,w_{N}]\in\mathbb{R}^{N}\) of each channel are learned through the reduction and expansion operation as  
Figure 2: Illustration of the proposed CANet link prediction model. \[\mathbf{w}=\sigma(\mathbf{W}_{a}\cdot\text{ReLU}(\mathbf{W}_{b}\cdot\mathbf{c})) \in\mathbb{R}^{N} \tag{7}\]  
where \(\sigma(x)\) is a sigmoid function, and \(\text{ReLU}(x)\) represents the rectified linear units [26]. \(\mathbf{W}_{a}\in\mathbb{R}^{\frac{N}{\tau}\times N}\) and \(\mathbf{W}_{b}\in\mathbb{R}^{N\times\frac{N}{\tau}}\) are the weights of dimensionality increase and reduction, respectively. The \(\tau\) is the proportion of the dimension reduction. Lastly, the remapping of feature maps \(\mathbf{\tilde{F}}\) with channel interdependence can be identified as  
\[\mathbf{\tilde{F}}=\mathbf{F}\cdot\mathbf{w}=\boldsymbol{f}_{n}\times w_{n}, c=1,2,\dots,N \tag{8}\]  
The design of the attention mechanism is suggested as a solution to the issue of channel interdependence.