## 5 Discussion
The latter involved combining PPG and wrist-worm actigraphy as input to the student model, whereas the teacher model was pre-trained on ECG data acquired during sleep. The results mirrored ours (in the context of AF) for sleep analysis, as it appears empirically that single-channel sensors collected from different populations, captured at different sampling rates, can be adapted for other types of sensors. Specifically, the electric field-based modality of ECG is indeed comparable to the light absorption mechanisms of PPG. **Research implications**: There is an increasing presence of foundational models across different domains, with architectures that are pre-trained on large-scale unlabeled data using representation learning, making them adaptable to a variety of tasks while accommodating increased complexity [86]. Our proposed work considered supervised pre-training on a related modality for better representations due to the scarcity of large, diverse public datasets in this field and tangentially aligns, in principle, with the new paradigms of foundational models. The work by [87] leveraged the Apple Health and Movement study (AHMS) involving 140,000 participants and 3 years of wearable ECG signals to implement a foundational model. This involved self-supervised pre-training with participant-level positive pair selection, a stochastic augmentation module, and a regularized contrastive loss optimized using momentum training. The downstream tasks involved the classification and regression of age, body mass index, and biological gender. Alternatively, [88] proposed a foundational model for general PPG signal analysis with self-supervised contrastive learning using a large semi-private dataset. The premise was that representations of PPG signals from similar physiological states should be similar, and bad-quality PPG signals in the temporal vicinity of a good-quality one from the same person are likely to be from a similar physiological state. Downstream performance was reported for AF detection, blood pressure, heart rate, and respiration rate estimations. Interestingly, [87] advocated for the future use of HRV/PRV indices to infer valuable insights from pre-trained embeddings, which is a direction we have explored. Additionally, [88] achieved generalization across different qualities of data with certain robustness to artifacts and noise but reported limitations in successfully detecting conditions due to the scarcity of annotated PPG signals for diseases. Our work exhibits promise in transferring ECG representations to downstream PPG tasks with high fidelity and contributes to bridging similar gaps in the aforementioned line of research regarding foundational models. **Limitations:** We refer to the findings of [83, 89], who argued that although KD can indeed improve _student_ generalization, there can be discrepancies between the predictive distributions of the _teacher_ and the _student_, even when they match closely, due to the inductive and implicit biases of the dataset considered, the nature of the domain, and the augmentation strategies. For instance, some of the information gained by the _teacher_ is lost during the distillation process, resulting in a slight drop in performance in the _student_ model. We suspect that this is because the morphological characteristics of the signals are not preserved independently during distillation, leading to some misclassifications for borderline instances, where AF and NSR signals have similar temporal features and/or higher signal-to-noise ratios. However, the top-level hierarchical features common to both ECG and PPG, such as the intervals between peaks, the periodic behavior, and the general fluctuations of the waves, appear to be captured. Additionally, measuring biological signals is inherently challenging because of the low quality of data (in naturalistic settings), saturated noise, and artifacts (patient movement and/or sensor displacement), as well as individual physiological variations and inter-device differences [90].