# Enhanced Localisation and Handwritten Digit Recognition Using ConvCARU
## 4 Experimental Results and Discussion
### Training Configuration and Strategy  
The proposed method is implemented using the PyTorch [62] framework in Python 3.12.4 and essential libraries on four Nvidia RTX A4000 GPU(s). The training process uses a batch size of 100 and initialises all model parameters, including weights and biases, using a specific strategy. To maintain stability, each layer incorporates group normalisation to ensure independence from variations in batch size. The weight decay factor is configured at \(5\times 10^{-4}\) for the encoder, while the proposed layer is unaffected by weight decay. Training the network takes around 20 h when executed across four Nvidia RTX A4000 GPU(s) in parallel. However, memory limitations arise when the video batch size exceeds \(4\times 16\) GB, which is more than the maximum memory capacity of the GPUs. More configurations have been provided with the following description:  
**Training configuration**: The design and implementation of the model incorporate several crucial aspects to enhance its performance and adaptability. The training process employs the Adam optimiser [63] with a learning rate of 0.001, and sets \(\beta_{1}\) to 0.5 and \(\beta_{2}\) to 0.999. Each GPU is allocated a batch size of 32, and the training extends across 100 epochs. To refine the learning dynamics, a cosine annealing scheduling strategy is applied alongside a warm-up phase to stabilise the initial training stages.
**Model architecture**: The architecture itself incorporates ConvCARU layers with hidden dimensions of 128 and 256, enabling efficient sequential processing. An eight-headed attention mechanism is incorporated to strengthen the model's ability to focus on relevant features. The feature fusion component, structured with 512 channels, also enhances multi-level information integration. With a total of 2.8 million parameters, the model is designed to handle complex pattern recognition tasks effectively.
**Data augmentation**: To further improve robustness, various data augmentation techniques are employed to further improve robustness. Training images undergo random rotation within a range of \(\pm 15^{\circ}\), while scaling variations between 0.9 and 1.1 are applied. Positional transformations introduce random translations of up to 10%, and Gaussian noise with a standard deviation of 0.01 is added to simulate real-world uncertainties. These augmentation strategies enhance the model's ability to adapt to different input conditions, reducing its susceptibility to minor distortions or variations.  
This designed approach ensures a high-performing, stable, and generalised network across diverse environments. Integrating multi-head attention enhances feature selection, and the fusion mechanism improves representation. This leads to greater accuracy and efficiency in practical applications. A warm-up function also helps to mitigate divergence issues encountered in the early stages of training. Further efficiency improvements are achieved through half-precision floating-point computation and Distributed Data Parallel (DDP), which together reduce memory consumption and enhance computational efficiency. To strengthen robustness, the baseline method applies a 30% runtime random modality dropout rate and random video flipping to minimise overfitting concerns. These strategic optimisations significantly enhance the training framework's overall effectiveness, ensuring improved performance and adaptability in real-world applications.