長・短期記憶
覗き穴畳み込みLSTM。 ∗ {\displaystyle *} は畳み込み演算子を示す。
f t = σ g ( W f ∗ x t + U f ∗ h t − 1 + V f ∘ c t − 1 + b f ) i t = σ g ( W i ∗ x t + U i ∗ h t − 1 + V i ∘ c t − 1 + b i ) o t = σ g ( W o ∗ x t + U o ∗ h t − 1 + V o ∘ c t + b o ) c t = f t ∘ c t − 1 + i t ∘ σ c ( W c ∗ x t + U c ∗ h t − 1 + b c ) h t = o t ∘ σ h ( c t ) {\displaystyle {\begin{aligned}f_{t}&=\sigma _{g}(W_{f}*x_{t}+U_{f}*h_{t-1}+V_{f}\circ c_{t-1}+b_{f})\\i_{t}&=\sigma _{g}(W_{i}*x_{t}+U_{i}*h_{t-1}+V_{i}\circ c_{t-1}+b_{i})\\o_{t}&=\sigma _{g}(W_{o}*x_{t}+U_{o}*h_{t-1}+V_{o}\circ c_{t}+b_{o})\\c_{t}&=f_{t}\circ c_{t-1}+i_{t}\circ \sigma _{c}(W_{c}*x_{t}+U_{c}*h_{t-1}+b_{c})\\h_{t}&=o_{t}\circ \sigma _{h}(c_{t})\end{aligned}}}