julia-pfarr/HarmoniCA_sleep

HarmoniCA — Sleep Model

Part of the HarmoniCA collection for harmonising psychiatric questionnaire items across studies. This model assigns items from sleep questionnaires to one of four theoretically motivated symptom dimensions.

Dimensions

ID	Label	Description
1	Daytime Sleepiness & Alertness	excessive sleepiness during the day, difficulty staying awake, drowsiness, reduced vigilance
2	Nocturnal Sleep Disturbances	difficulty falling asleep, waking during the night, insomnia, poor sleep continuity or quality
3	REM Sleep Behavior & Dreaming	acting out dreams, vivid or disturbing dreams, movements or talking during sleep
4	Sleep-Disordered Breathing	snoring, sleep apnea, pauses in breathing, breathing difficulties at night

Items rated as not belonging to the sleep construct are assigned dimension -1 (Does not fit).

Model Architecture

• Type: ContrastiveModel (contrastive fine-tuning of a sentence transformer)
• Base encoder: princeton-nlp/sup-simcse-roberta-large

Training Data

• Training items: 180 expert-labeled questionnaire items (151 original + 25 augmented + 4 expert-reviewed)
• Questionnaires: AIS, ASBPD, ESS, ISI, KSS, MDS-UPDRS, MSQ, NMSQuest, NMSS, NPI, PDSS, PDSS-2, PSQI, RBD1Q, RBDSQ, SCOPA-SLEEP, SDQ, plus data augmentation
• Labels: Expert consensus from two rounds of surveys (psychiatrists/neurologists)

Evaluation

Evaluated on 14 held-out items from unseen questionnaires against expert labels.

Cohen's κ	Accuracy
0.468	57.1%

Note: The test set is very small (n=14).

Cross-validation (leave-one-questionnaire-out, 17 folds): mean accuracy 93.4% ± 12.1%.

Limitations

• Trained and validated on Parkinson's disease cohort questionnaires; sleep-disordered breathing items are underrepresented in training.
• Performance is highly variable across questionnaire styles (CV std ±12%). Manual review is especially recommended for questionnaires with atypical phrasing.
• Dimension structure reflects theoretical consensus among a specific expert panel and may not match all theoretical frameworks.

Citation

in preparation