## 5 Conclusions  
We designed a range of attention-based architectures that could handle longitudinal data from EHRs and compared the discriminability and the clinical plausibility of the resulting attention maps. Each approach increased in complexity to accommodate more specific and informative attention mechanisms. The simplest model (time attention) could inform only the contribution of the time periods, serving as the basis for others that could also tell which clinical input features had the highest contribution (domain-specific  
Figure 5: Attention maps on the feature domain for the hierarchical attention architectures (Approach 3). The attention weights shown here are the results of averaging the attention weights of every patient on the test set. The description of each feature can be found on our GitHub [41]. attention), and finally, to one that was able to give the contribution of each feature at each time point (hierarchical attention). The baseline model without attention performed the best in terms of discriminability, slightly decreasing the average performance from 0.98% to 8.7% as the architecture accommodated more informative attention mechanisms. Architectures from Approach 3 were the most informative as they could give the attention weights in two different domains (time and feature) simultaneously. Amongst the feature processing methods in this approach, the attention weights provided by the GRU were the most clinically plausible. In addition, this feature processing method achieved the most consistent results between approaches in terms of top-contributing features. Some limitations should be considered. First, hyperparameter tuning was not performed and that could have modified discriminability. However, the same hyperparameters were used for all the combinations to achieve a fair comparison. Moreover, primary care EHRs contain a detailed and comprehensive picture of patients but lack information from other areas. Older patients might be reallocated to other healthcare facilities like nursing homes, partially losing their follow-up. Incorporating variables from other sources could have improved both discriminability and clinical plausibility. Finally, the evaluated architectures were complex and parameter-heavy and could have benefited from richer data distributions. Future work includes comparing these results with those obtained by other models like transformers and exploring the generalisability of these results to EHRs from different origins, other sizes of observation windows, and other prediction tasks. In addition, it should be noted that determining an \"adequate overall performance\" threshold from a single study may be difficult, as each healthcare application may need to optimize particular metrics over others. To define it, future work should consider performing this study in different use cases and consulting clinicians to establish criteria, perhaps through focus groups, on the determination of this threshold for acceptable discriminability, taking into account the loss or gain of interpretability of the model. In conclusion, models handling longitudinal EHR data using simpler architectures achieved slightly better discriminability than those including more complex attention mechanisms for enhancing transparency. As [PERSON] noted, transparency may be at odds with predictive power [45] and thus the choice of the architecture should align with transparency requirements. In healthcare, prioritizing higher transparency and clinically plausible attention weights may justify choosing a model with a minor decrease in discriminability, provided overall performance remains adequate. Once considering including attention mechanisms, optimal combinations for domain-specific and hierarchical attention architectures were similar, utilizing either GRU or down-sampling CNN for feature processing. Hierarchical attention architectures, capable of highlighting clinical and temporal contributions, may be preferred for more informed decision-making when handling longitudinal EHR data.