# Constructing Uyghur Commonsense Knowledge Base by Knowledge Projection
## 3 Method
### Bilingual Semantic Similarity Scoring Model
#### 3.4.2 Bidimensional Attention Network  
We propose the bidimensional attention network (Figure 2b) to incorporate a multilevel representation of embeddings from RAE into phrase embeddings and further into the semantic similarity of bilingual phrases. We can put vectors from all nodes of a tree into the columns of a matrix of size \((2n-1)\times d\) (\(n_{s},d_{s}\) for source and \(n_{t},d_{t}\) for target), where \(d\) is the dimension of embeddings and \(n\) is the length of phrase (there are \(n-1\) steps in RAE to construction and therefore are \(2n-1\) nodes in total). Let us denote these matrices by \(M_{s}\) and \(M_{t}\) for the source and target tree, respectively. Then we can project all the embeddings into a common attention space by using by a non-linear projection function \(f(Wx+b)\). In this attention space, all the embeddings from the source tree can \"interact\" with all the embeddings from the target tree. We will measure the interaction strength between the \(i\)-th projected source embedding and the \(j\)-th projected target embedding by Equation (7).  
\[A_{s}=f(W^{(3)}M_{s}+b^{A}\ ) \tag{5}\]  
\[A_{t}=f(W^{(4)}M_{t}+b^{A}\ ) \tag{6}\]  
\[B(i,j)=g(A_{s,i}^{T}A_{t,i}\ ) \tag{7}\]  
where \(g(\cdot)\) and \(f(\cdot)\) are non-linear activation functions, e.g., the \(\mathrm{sigmoid}(\cdot)\) and the \(\tan\mathsf{h}(\cdot)\) functions are used in this paper, \(A_{s}\) (Equation (5)) and \(A_{t}\) (Equation (6)) are projections of \(M_{s}\) and \(M_{t}\) to the attention space, \(W^{(3)}\in\ \mathbb{R}^{d_{s}\times d_{s}}\) and \(W^{(4)}\in\ \mathbb{R}^{d_{s}\times d_{t}}\) are transformation matrix, \(b^{A}\in\ \mathbb{R}^{d_{s}}\) is the bias term. We will use the same bias-term force model to learn to encode the attention semantics into transformation matrices, rather than the bias term.  
It can be seen that we define a \((2n_{s}-1)\times(2n_{t}-1)\) matrix \(B\), which is called the bidimensional attention matrix represented by Equation (7). Intuitively, this matrix is a result of handshakes between source and target phrases at a multilevel representation. We can interpret the sum of the \(i\)-th row as the total strength that the \(i\)-th source node has on the semantic similarity between the two considered phrases.  
\[\overline{a}_{s,i}=\sum_{j}B_{i,j}\,\ \overline{a}_{t,j}=\sum_{i}B_{i,j} \tag{8}\]  
where \(\overline{a}_{s}\in\mathbb{R}^{n_{s}}\) and \(\overline{a}_{t}\in\mathbb{R}^{n_{t}}\) are the semantic matching score vectors.  
Because of phrase length uncertainty, we can normalize all these strengths using a \(softmax\) function: \(a_{s}=Softmax(\overline{a}_{s})\), \(a_{t}=Softmax(\overline{a}_{t})\). This forces \(a_{s}\) and \(a_{t}\) to become real-valued distributions in the attention space, known as attention weights. Then, we use them to obtain the final phrase representations by the following Equation (9).  
\[p_{s}=\sum_{i}a_{s,i}M_{s,i},\ \ p_{t}=\sum_{j}a_{t,j}M_{t,j} \tag{9}\]  
where \(p_{s}\in\mathbb{R}^{d_{s}}\), and \(p_{t}\in\mathbb{R}^{d_{t}}\), notice that they still are located in their language-specific vector space.