## 3 Materials and Methods
### Methodology Framework
#### 3.2.2 Models' Architecture
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Architecture** & **Key Hyperparameters** \\ \hline \multirow{8}{*}{LSTM} & 1. Input: L \(\times\) d sequence & \(\bullet\) u1, u2 \(\in\) {32, 64, 128} \\  & 2. LSTM (units = u1, return\_sequences = True) & \(\bullet\) dr \(\in\) {0.1, 0.2, 0.3} \\  & 3. Dropout (rate = dr) & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\  & 4. LSTM (units = u2) & \\  & 5. Dense (1) & \\ \hline \multirow{8}{*}{BiLSTM} & 1. Input: L \(\times\) d sequence & \(\bullet\) u \(\in\) {32, 64, 128} \\  & 2. Bidirectional LSTM (units = u) & \(\bullet\) dr \(\in\) {0.1, 0.2, 0.3} \\  & 3. Dropout (rate = dr) & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\  & 4. Dense (1) & \\ \hline \multirow{8}{*}{CNN–LSTM} & 1. Input: L \(\times\) d sequence & \(\bullet\) f, u \(\in\) {32, 64, 128} \\  & 2. Conv1D (filters = f, kernel\_size = 3, padding = “causal”, & \(\bullet\) dr \(\in\) {0.1, 0.2, 0.3} \\  & activation = ReLU) & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\  & 3. Dropout (rate = drd) & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\  & 4. LSTM (units = u) & \\  & 5. Dense (1) & \\ \hline \multirow{8}{*}{Transformer} & 1. Input: L \(\times\) d sequence & \(\bullet\) d \(\in\) {32, 64, 128} \\  & 2. Dense to model-dim d + Add pos-enc & \(\bullet\) heads \(\in\) {2, 4} \\  & 3. Repeat n\(\times\): & \(\bullet\) n \(\in\) {1, 2} \\  & 3.1. MultiHeadAttention (heads = h, key\_dim = d/h) \(\rightarrow\) Add & \(\bullet\) dr \(\in\) {0.1, 0.2, 0.3} \\  & and LayerNorm & \\  & 3.2. FFN: Dense(4d)\(\rightarrow\)ReLU\(\rightarrow\)Dense(d) \(\rightarrow\) Add and & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\  & LayerNorm & \\  & 4. GlobalAveragePooling1D & \\  & 5. Dense (1) & \\ \hline \multirow{8}{*}{ConvTransformer} & 1. Input: L \(\times\) d sequence & \(\bullet\) f, d \(\in\) {32, 64, 128} \\  & 2. Conv1D (filters = f, kernel\_size = 3, padding = “same”, & \(\bullet\) heads \(\in\) {2, 4} \\  & activation = ReLU) & \(\bullet\) n \(\in\) {1, 2} \\  & 3. Dense to model-dim d + Add pos-enc & \(\bullet\) dr \(\in\) {0.1, 0.2, 0.3} \\  & 4. Repeat n\(\times\): same TX stack as Transformer & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\  & 5. GlobalAveragePooling1D & \\  & 6. Dense (1) & \\ \hline \multirow{8}{*}{Informer} & 1. Input: L \(\times\) d sequence & \(\bullet\) d \(\in\) {32, 64, 128} \\  & 2. Dense to model-dim d + Add pos-enc & \(\bullet\) heads \(\in\) {2, 4} \\  & 3. Repeat n\(\times\): same TX stack as Transformer & \(\bullet\) n \(\in\) {1, 2} \\ \cline{1-1}  & 4. GlobalAveragePooling1D & \(\bullet\) dr \(\in\) {0.1, 0.2, 0.3} \\ \cline{1-1}  & 5. Dense (1) & \(\bullet\) Learning rate \(\in\) {1 \(\times\) 10\({}^{-4}\), 5 \(\times\) 10\({}^{-4}\), 1 \(\times\) 10\({}^{-3}\)} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Models’ architecture.