# Constructing Uyghur Commonsense Knowledge Base by Knowledge Projection
## 3 Method
### Bilingual Semantic Similarity Scoring Model
#### 3.4.1 Learning Multilevel Phrase Embeddings  
We use recursive autoencoders (RAE, Figure 2a) to learn initial embeddings at different levels of phrases. By combining two children vectors from the bottom up recursively, RAE can generate low-dimensional vector representations for variable-sized sequences. The recursion procedure usually consists of two main steps: composition and reconstruction.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Relationship** & **English** & **Chinese** & **Uyghur** \\ \hline DerivedFrom & e2 is derived from e1 & e2\(\#\)\(\#\)\(\#\)\(\#\)\(\#\)\(\#\)\(\#\) & e1 e2 (\(\#\)\  
Composition: Generally, for a list of words in a phrase \((x_{1},x_{2},x_{3})\), each of them will be embedded into a \(d\)-dimensional continuous vector, RAE selects two neighboring children (e.g., \(c_{1}=x_{1}\) and \(c_{2}=x_{2}\)) via some selection criterion, and then composes them into a parent embedding \(y_{1}\), which can be computed by Equation (2).  
\[y_{1}=f(W^{(1)}[c_{1}:c_{2}]+\ b^{(1)}) \tag{2}\]  
where \([c_{1}:c_{2}]\in\ \mathbb{R}^{2d}\) is the concatenation of \(c_{1}\) and \(c_{2}\), \(W^{(1)}\in\ \mathbb{R}^{d\times 2d}\) is a parameter matrix, \(b^{(1)}\in\ \mathbb{R}^{d}\) is a bias term, \(f\) is element-wise activation function such as \(\tan\mathsf{h}\ (\cdot)\), which is used in our experiments.  
Reconstruction: After getting the \(d\)-dimensional representation for parent \(y_{1}\) in the composition step, to measure how well the parent \(y_{1}\) represents its children, we reconstruct the original child nodes via a reconstruction layer formulated in Equation (3).  
\[\left[c_{1}^{\prime}:c_{2}^{\prime}\right]=f(W^{(2)}y_{1}+\ b^{(2)}) \tag{3}\]  
where \(c_{1}^{\prime}\) and \(c_{2}^{\prime}\) are the reconstructed children, \(W^{(2)}\in\ \mathbb{R}^{2d\times d}\) and \(b^{(2)}\in\ \mathbb{R}^{2d}\), The minimum Euclidean distance between \([c_{1}:c_{2}]\), and \(\left[c_{1}^{\prime}:c_{2}^{\prime}\right]\) is usually used as the selection criterion during composition.  
These two steps repeat until the embedding of the entire phrase is generated. While embedding, RAE also constructs a binary tree. The structure of the tree is determined by the used selection criterion in composition. We use a greedy algorithm [27] based on the following reconstruction error, which can be seen as Equation (4).  
\[E_{rec}(\chi)=\ \sum_{y\in T(\chi)}\frac{1}{2}\ \|[c_{1}:c_{2}]_{y}-\left[c_{1} ^{\prime}:c_{2}^{\prime}\right]_{y}\|^{2} \tag{4}\]  
where \(y\) is an intermediate node of the binary tree \(T(x)\), and parameters \(W^{(1)}\) and \(W^{(2)}\) are learned to minimize the sum of reconstruction errors.  
Given a binary tree learned by RAE, the leaf, internal nodes, and root of the tree which represents the representations of words, sub-phrases, and phrases separately, we can use RAE to produce the embeddings of phrases at different levels. As shown in Figure 1, RAE learns representations of the source and target phrases in different semantic spaces, marked as \(d_{s}\) and \(d_{t}\), respectively.