## 3 Architectural System Design
### System Architecture
#### 3.8.1 ResRNN-Net and MultiScaleFusion-Net
* **Batch Size:** Evaluated at discrete values of 32, 64, and 128--Number of samples processed before model weight updates. * **Dropout Rate:** [0.3, 0.5]--Prevents overfitting by randomly deactivating neurons. A **random search** approach was used to explore hyperparameter combinations for both models. For each trial, the models were initialized with the **Adam** optimizer and **binary cross-entropy loss** using early stopping (patience: one epoch). The models were evaluated using the F1-score, balancing precision and recall. ResRNN-Net was tested with **ten trials** and MultiScaleFusion-Net with **five trials** due to early convergence. As shown in Table 6, Both models achieved F1-scores above 0.81, showcasing strong prediction performance with balanced precision and recall. The learning rate of 0.0005 ensured smooth convergence, while a batch size of 64 optimized computational efficiency. A 50% dropout rate effectively prevented overfitting without compromising learning capacity. To determine whether the observed performance difference between ResRNN-Net and MultiScaleFusion-Net was statistically significant, a paired \(t\)-test was conducted on the F1-scores across cross-validation folds. The paired \(t\)-test yielded a \(t\)-statistic of **5.9761** with a \(p\)-value of **0.0039**, indicating a statistically significant difference at the 0.01 level. These results suggest that, while both models performed competitively, ResRNN-Net demonstrates statistically superior performance under the \(t\)-test criterion. #### 4.1.4 Hyperparameter Tuning for Multimodal Model

To optimize the performance of the multimodal deep learning model, hyperparameter tuning was conducted as shown in Table 7 to identify the best configurations for the learning rate, batch size, and optimizer selection. The search aimed to enhance the model stability and predictive power across the three modalities: tabular, imaging, and time-series data. The following key hyperparameters were considered for tuning:

* **Learning Rate:** [0.001, 0.0005]. * **Batch Size:** [16; 32]. * **Optimizer:** [Adam, AdamW]. A **grid search** approach was applied across eight hyperparameter configurations, where the model was trained and evaluated on validation accuracy, F1-score, precision, recall, and ROC-AUC. We can obtain better F1-score (83%), accuracy (78.9%), and recall (100%) values by choosing the optimal hyperparameters, which are learning rate = 0.0005, batch size = 16, and Adam as the optimizer. By selecting optimized hyperparameters for cross-validation, the performance of the multimodal model improved. The accuracy increased from 60% to 63%, as shown in Table 8. This highlights the critical impact of hyperparameter tuning in enhancing model robustness and predictive power. \begin{table}
\begin{tabular}{l l l l l} \hline
**Model** & **Learning Rate** & **Batch Size** & **Dropout Rate** & **F1-Score** \\ \hline ResRNN-Net & 0.0005 & 64 & 0.5 & 0.82 \(\pm\) \\ MultiScaleFusion-Net & 0.0005 & 64 & 0.5 & 0.81 \(\pm\) \\  & & & & 0.0018 \\ \hline \end{tabular}
\end{table} 
Table 6: Best hyperparameters found. **Discussion on Multimodal Integration:** The multimodal model represents an important first step toward integrating structured tabular data, medical imaging, and temporal signals for more comprehensive risk prediction. Initially, the model achieved perfect recall, but fine-tuning the hyperparameters was crucial to improving the balance across the other performance metrics, as shown in Table 8. Currently, we use early fusion to combine the learned embeddings from each modality. However, the drop in precision and only moderate accuracy suggest that this approach may not fully harness the power of cross-modal interactions. To address this, future work will explore alternative fusion strategies, such as late fusion, attention-based modality weighting, and hierarchical feature aggregation.