Question: How to build the CapsE ?
Evidence: Let INLINEFORM0 be a collection of valid factual triples in the form of (subject, relation, object) denoted as (s, r, o). Embedding models aim to define a score function giving a score for each triple, such that valid triples receive higher scores than invalid triples.
We denote INLINEFORM0 , INLINEFORM1 and INLINEFORM2 as the INLINEFORM3 -dimensional embeddings of INLINEFORM4 , INLINEFORM5 and INLINEFORM6 , respectively. In our proposed CapsE, we follow BIBREF15 to view each embedding triple [ INLINEFORM7 , INLINEFORM8 , INLINEFORM9 ] as a matrix INLINEFORM10 , and denote INLINEFORM11 as the INLINEFORM12 -th row of INLINEFORM13 . We use a filter INLINEFORM14 operated on the convolution layer. This filter INLINEFORM15 is repeatedly operated over every row of INLINEFORM16 to generate a feature map INLINEFORM17 , in which INLINEFORM18 where INLINEFORM19 denotes a dot product, INLINEFORM20 is a bias term and INLINEFORM21 is a non-linear activation function such as ReLU. Our model uses multiple filters INLINEFORM22 to generate feature maps. We denote INLINEFORM23 as the set of filters and INLINEFORM24 as the number of filters, thus we have INLINEFORM25 INLINEFORM26 -dimensional feature maps, for which each feature map can capture one single characteristic among entries at the same dimension.
We build our CapsE with two single capsule layers for a simplified architecture. In the first layer, we construct INLINEFORM0 capsules, wherein entries at the same dimension from all feature maps are encapsulated into a corresponding capsule. Therefore, each capsule can capture many characteristics among the entries at the corresponding dimension in the embedding triple. These characteristics are generalized into one capsule in the second layer which produces a vector output whose length is used as the score for the triple.
The first capsule layer consists of INLINEFORM0 capsules, for which each capsule INLINEFORM1 has a vector output INLINEFORM2 . Vector outputs INLINEFORM3 are multiplied by weight matrices INLINEFORM4 to produce vectors INLINEFORM5 which are summed to produce a vector input INLINEFORM6 to the capsule in the second layer. The capsule then performs the non-linear squashing function to produce a vector output INLINEFORM7 : DISPLAYFORM0 
where INLINEFORM0 , and INLINEFORM1 are coupling coefficients determined by the routing process as presented in Algorithm SECREF2 . Because there is one capsule in the second layer, we make only one difference in the routing process proposed by BIBREF16 , for which we apply the INLINEFORM2 in a direction from all capsules in the previous layer to each of capsules in the next layer.
[ht] 1.25
all capsule i INLINEFORM0 the first layer INLINEFORM1 0 INLINEFORM2 = 1, 2, ..., m INLINEFORM3 INLINEFORM4 
 INLINEFORM0 
all capsule i INLINEFORM0 the first layer INLINEFORM1 The routing process is extended from BIBREF16 .
We illustrate our proposed model in Figure FIGREF1 where embedding size: INLINEFORM0 , the number of filters: INLINEFORM1 , the number of neurons within the capsules in the first layer is equal to INLINEFORM2 , and the number of neurons within the capsule in the second layer: INLINEFORM3 . The length of the vector output INLINEFORM4 is used as the score for the input triple.
Formally, we define the score function INLINEFORM0 for the triple INLINEFORM1 as follows: DISPLAYFORM0 
where the set of filters INLINEFORM0 is shared parameters in the convolution layer; INLINEFORM1 denotes a convolution operator; and INLINEFORM2 denotes a capsule network operator. We use the Adam optimizer BIBREF19 to train CapsE by minimizing the loss function BIBREF14 , BIBREF15 as follows: DISPLAYFORM0 
 INLINEFORM0 
here INLINEFORM0 and INLINEFORM1 are collections of valid and invalid triples, respectively. INLINEFORM2 is generated by corrupting valid triples in INLINEFORM3 .
We build our CapsE with two single capsule layers for a simplified architecture.
Answer: The authors build the CapsE with two single capsule layers for a simplified architecture.