# A Brief Survey on Deep Learning-Based Temporal Knowledge Graph Completion
## 3 Deep Learning-Based TKGC Methods
### LSTM-Based Methods  
Long short-term memory (LSTM) [50] is a special type of recurrent neural network that can learn long-term dependencies in time series data, such as machine translation [51] and speech recognition [52]. Given the input data at time \(t\), i.e., \(x_{t}\), LSTM performs the following computation:  
\[ \begin{split} i_{t}&=sigmoid(W_{ii}x_{t}+b_{ii}+ W_{hi}h_{t-1}+b_{hi})\\ f_{t}&=sigmoid(W_{if}x_{t}+b_{if}+W_{hf}h_{t-1}+ b_{hf})\\ o_{t}&=sigmoid(W_{io}x_{t}+b_{io}+W_{ho}h_{t-1}+ b_{ho})\\ \widehat{c_{t}}&=tanh(W_{ic}x_{t}+b_{ic}+W_{hc}h_{ t-1}+b_{hc})\\ c_{t}&=f_{t}*c_{t-1}+i_{t}*\widehat{c_{t}}\\ h_{t}&=o_{t}*tanh(c_{t}),\end{split}  \tag{1}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **TKGC Methods** & **Extended from Static KGC Methods** & **Encoder–Decoder** \\ \hline \multirow{3}{*}{LSTM-based methods} & TA-TransE [27] & TransE [17] & ✗ \\  & TA-DISTMULT [27] & DISTMULT [19] & ✗ \\  & TA-TransR [28] & TransR [18] & ✗ \\ \hline \multirow{3}{*}{CNN-based methods} & CATKGC [29] & ✗ & ✗ \\  & ConvTKG [50] & ✗ & ✗ \\  & SANe [31] & ✗ & ✗ \\  & SANe\({}^{+}\)[32] & ✗ & ✗ \\ \hline \multirow{2}{*}{CapsNet-based methods} & TempCaps [33] & CapsE [20] & ✗ \\  & BiQCap [34] & ✗ & ✗ \\ \hline \multirow{2}{*}{Attention-based methods} & RoAN [35] & SimplE [22] & ✗ \\  & HSAE [36] & ✗ & ✗ \\ \hline \multirow{3}{*}{GNN-based methods} & TGAP [37] & ✗ & ✗ \\  & STRGNN [38] & ✗ & ✗ \\  & HyGNet [39] & ✗ & ✗ \\ \hline \multirow{3}{*}{GCN-based methods} & TeMP [40] & TransE [17] & ✗ \\  & MtGCN [41] & ConvTransE [26] & ✗ \\  & TAGCN [42] & ConvE [23] & ✗ \\  & TAL-TKGC [43] & ✗ & ✗ \\  & THOR [44] & ✗ & ✗ \\ \hline \multirow{3}{*}{GAN-based methods} & IAGAT [45] & ✗ & ✗ \\  & DEGAT [46] & ConvKB [25] & ✗ \\  & T-GAE [47] & ConvKB [25] & ✗ \\ \hline \multirow{3}{*}{Language model-based methods} & SToKE [48] & ✗ & ✗ \\  & Llama-2-7b-CoH [49] & ✗ & ✗ \\ \cline{1-1}  & Vicuna-7b-CoH [49] & ✗ & ✗ \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Fine-grained categorization of deep learning-based temporal knowledge graph completion methods. where \(i_{t},f_{t}\), and \(o_{t}\) are the input gate, forget gate and output gate, respectively; \(c_{t}\), \(\tilde{c_{t}}\) are the cell state and candidate cell state, respectively; and \(h_{t}\) is the hidden state. The various \(W\) and \(b\) terms are the weight matrices and bias vectors. [PERSON] and [PERSON] [27] convert quadruples into triples by merging timestamps into relations, i.e., \((s,r,o,t)\) to \((s,r+t,o)\), and apply LSTM to encode the merged relations. Then, they extend two classic static KGC methods, TransE [17] and DISTMULT [19], and propose TA-TransE and TA-DISTMULT, respectively. TA-TransE models merge relations as translation operations into the embeddings of subject and object entities and use a distance-based scoring function \(f(s,r+t,o)=||e_{s}+e_{r+t}-e_{o}||_{2}\) to measure the plausibility of a triple fact, where \(e_{s}\), \(e_{o}\), and \(e_{r+t}\) are the embeddings of subject entity, object entity, and merged relation, respectively. The essential idea of TA-DISTMULT is the same as that of TA-TransE; the only difference is the score function, which is \(f(s,r+t,o)=(e_{s}\otimes e_{o})e_{r+t}^{\top}\), where \(\otimes\) is the element-wise product, and \(\top\) represents the matrix transpose. Since LSTM can handle sequences of varying lengths, a natural advantage is that the merged relation is adaptive to timestamps of different relations, such as points or intervals. Inspired by [27], [PERSON] et al. [28] present TA-TransR, which expands the static KGC method TransR [18] to TKGC, and the scoring function is \(f(s,r+t,o)=||(e_{s}-e_{s}M_{r})+e_{r+t}-(e_{o}-e_{o}M_{r})||_{2}^{2}\), where \(M_{r}\) is the projection matrix corresponding to relation \(r\). Furthermore, they impose three kinds of temporal consistency constraints (i.e., temporal disjointness, temporal ordering, and temporal spans) on TA-TransR, and formulate TKGC task as an integer linear programming optimization problem.