# Improved Localization and Recognition of Handwritten Digits on MNIST Dataset with ConvGRU
## 3 Methodology
### Proposed Feature Extraction and Attention Mechanism  
In addition to the ConvGRU architecture, our model incorporates a novel feature extraction and attention mechanism to enhance the learning of spatiotemporal dependencies, as shown in Algorithm 1. The feature extraction process consists of two main components: temporal feature extraction and spatial feature extraction. The temporal feature extraction module applies a series of operations to capture the temporal dynamics within the input video frames \(B\). The process can be summarized as Algorithm 2. The spatial feature extraction module focuses on learning the spatial structure within each frame. It applies a similar set of operations as the temporal feature extraction module, with the addition of spatial convolution layers.  
Figure 3: Feature decoupling and processing architecture.  
```
1:Feature Extraction Process
2:Input: Batch of video frames \(hcalB\)
3:Output: Processed features and attention maps
4: 7 Temporal Feature Extraction
5: Extract temporal features:
6: Apply temporal convolution layers
7: Process with batch normalization
8: Apply ReLU activation
9: Store result as \(F_{i}\)
10: // Spatial Feature Extraction
11: Extract spatial features:
12: Apply spatial convolution layers
13: Process with batch normalization
14: Apply ReLU activation
15: Store result as \(F_{i}\)
16: // Attention Computation
17: Compute temporal attention:
18: \(Q_{i},K_{i},V_{i}\leftarrow\text{Linear}(F_{i})\)
19: \(A_{i}\leftarrow\text{Softmax}(Q_{i}K_{i}^{*}/\sqrt{A_{k}})V_{i}\)
20: Compute spatial attention:
21: \(Q_{i},K_{i},V_{i}\leftarrow\text{Linear}(F_{i})\)
22: \(A_{i}\leftarrow\text{Softmax}(Q_{i}K_{i}^{*}/\sqrt{d_{k}})V_{i}\)
23: // Feature Fusion
24: Fuse features:
25: Concatenate \(F_{i}\) and \(F_{i}\)
26: Apply channel-wise attention
27: Apply spatial attention
28: Process with convolution block
29: Output final fused features
30: Return: Fused features \(F_{fused}\), Attention maps \(A_{i}\), \(A_{s}\)
```  
**Algorithm 1** Feature extraction and attention mechanism.  
```
1:Video sequence \(hcalX=\{x_{1},\ldots,x_{T}\}\), Learning rate \(\eta\), Max epochs \(E_{\text{max}}\)
2:Model parameters \(\Theta\), Predictions \(\{\hat{x}_{T+1},\ldots,\hat{x}_{T+k}\}\)
3:Initialize parameters \(\Theta_{\text{ReLU}}\), \(\Theta_{\text{att}}\), \(\Theta_{\text{fusion}}\)
4:for epoch = 10 to \(E_{\text{max}}\)do
5:for each batch \(hcalB\)do
6:\(F_{i}\leftarrow\text{ExtractCompiral}(hcalB)\)
7:\(F_{i}\leftarrow\text{ExtractSpatial}(hcalB)\)
8: Initialize \(h_{0}\)
9:for\(t=1\) to \(T\)do
10:\(r_{t}\gets(v_{T}w_{T}\star x_{t}+W_{w}\star h_{t-1})\)
11:\(z_{t}\leftarrow\sigma(W_{z}\star x_{t}+W_{w}\star h_{t-1})\)
12:\(\hat{h}_{t}\leftarrow\text{tanh}(W_{hh}\star x_{t}+r_{t}\odot(W_{hh}\star h_ {t-1}))\)
13:\(h_{t}\gets z_{t}\odot h_{t-1}+(1-z_{t})\odot\hat{h}_{t}\)
14:endfor
15:\(A_{i}\leftarrow\text{TemporalAttention}(F_{i})\)
16:\(A_{s}\leftarrow\text{SpatialAttention}(F_{s})\)
17:\(F_{fused}\leftarrow\text{Fusion}(A_{i}\odot F_{i},A_{s}\odot F_{s})\)
18:\(\hat{x}_{T+1:T+k}\leftarrow\text{Decoder}(F_{fused})\)
19:\(hcal_{Lrec}\leftarrow\frac{1}{N}\sum_{i=1}^{N}\left\lVert\hat{x}_{i}-\hat{x} _{i}\right\rVert^{2}_{2}\)
20:\(hcal_{Lrec}\leftarrow\frac{1}{N}\sum_{i=1}^{N}\left\lVert\hat{x}_{i}-\hat{x} _{i}\right\rVert^{2}_{2}\)
21:\(hcal_{Lrec}\leftarrow\frac{1}{N}\sum_{i=1}^{N}\left\lVert\hat{x}_{i}-\hat{x} _{i}\right\rVert^{2}_{2}\)
22: Update parameters via Adam
23:endfor
24:if epoch mod 10 = 0 then
25:\(\eta\gets 0.95\eta\)
26:endif
27: Compute validation metrics
28:endfor
```  
**Algorithm 2** ConvGRU-based training process.