# Lightweight Anomaly Detection in Digit Recognition Using Federated Learning
## 6 Resource Utilization Analysis
### Model and Training Memory Usage  
As discussed earlier, models are designed to run within the ESP32-CAM's available memory, with a preference for internal RAM to maximize speed. When RAM is insufficient, model parameters and training buffers are automatically allocated in external 4 MB PSRAM. Memory usage increases with model complexity as training requires additional space for activations, gradients, temporary buffers, and optimizer states.  
Let the autoencoder have \(k\) layers, indexed from 0 to \(k-1\), with layer sizes \(L=[n_{0},n_{1},\dots,n_{k-1}]\), where \(n_{i}\) is the number of neurons in layer \(i\). The number of parameters \(P\), including weights and biases, is calculated by summing over all pairs of adjacent layers. Therefore, the total number of parameters \(P\) is  
\[P=\sum_{i=0}^{k-2}\underbrace{(n_{i}\times n_{i+1}}_{\text{weights}}+ \underbrace{n_{i+1}}_{\text{biases}})=\sum_{i=0}^{k-2}n_{i+1}(n_{i}+1). \tag{2}\]  
Each parameter is stored as a 32-bit float (4 bytes), so the memory required to store the model is  
\[\text{Inference Memory (bytes)}=4\times P=4\times\sum_{i=0}^{k-2}n_{i+1}(n_{i}+1). \tag{3}\]  
This memory footprint is sufficient for inference-only applications, where no training is performed on the device. However, during training, additional memory allocations are required, including  
* **Activations (_A_)**: Neuron values from the forward pass must be stored for backpropagation: \[A=\sum_{i=0}^{k-1}n_{i}.\] (4)
* **Gradients:** Gradients of the loss are accumulated during backpropagation. This requires the same number of float values as the model parameters: \(P\) floats.
* **Neuron activation function derivative buffers:** During backpropagation, these buffers temporarily store the derivatives of the activation functions for each neuron. Their size matches the number of neurons in each hidden and output layer: \[D=\sum_{i=1}^{k-1}n_{i}.\] (5)
* **Optimizer state (Adam):** Two additional floats per parameter (first and second moment estimates), totaling \(2P\) floats. The total estimated memory for training is therefore Training Memory (bytes) = \(4\times(P+A+P+D+2P)=4\times(4P+A+D)\).  
These equations allow the approximation of RAM consumption for each architecture during training. Table 1 summarizes the memory usage for both inference and training across all tested models. For inference, the smallest model (Model 1) required only 25 KB, while the largest (Model 7) reached nearly 120 KB. Training memory usage varied more significantly due to additional buffers and optimizer overhead, where the smallest model required 104 KB and the largest 482 KB.  
Careful memory tracking was essential to ensure that on-device training could operate within the available DRAM. Out of the 320 KB DRAM, about 20 KB is reserved for static allocations, leaving roughly 300 KB available for dynamic allocation of model parameters and training buffers. According to Table 1, all models fit within DRAM when used for inference only. However, only Models 1, 2, 4, and 5 fit entirely in DRAM during on-device training. Larger models (3, 6, and 7) exceed this capacity and rely on external PSRAM. When PSRAM is used, the system dynamically prioritizes DRAM for smaller allocations while transparently placing larger memory blocks, such as model weights or training buffers, in PSRAM when needed. This flexible allocation strategy enables training and inference of larger models without manual memory management. PSRAM was also used to store the dataset, freeing more DRAM for training operations. Despite resource constraints, the ESP32-CAM successfully deployed this model for on-device training and inference, proving its capability as a reliable and low-power platform for local training and edge-based FL.