julia-pfarr/HarmoniCA_psychosis

HarmoniCA — Psychosis Model

Part of the HarmoniCA collection for harmonising psychiatric questionnaire items across studies. This model assigns items from psychosis questionnaires to one of two theoretically motivated symptom dimensions.

Dimensions

ID	Label	Description
1	Hallucinations	seeing, hearing, feeling, or smelling things that are not there; perceptual disturbances; illusions
2	Delusions	false beliefs firmly held despite evidence; paranoid ideation; persecution; grandiosity

Items rated as not belonging to the psychosis construct are assigned dimension -1 (Does not fit).

Model Architecture

• Type: ContrastiveModel (contrastive fine-tuning of a sentence transformer)
• Base encoder: hkunlp/instructor-large

Training Data

• Training items: 101 expert-labeled questionnaire items (88 original + 11 augmented + 2 expert-reviewed)
• Questionnaires: CAPE-P15, MDS-UPDRS, NMSQuest, NMSS, NPI, PPRS, PQ-B, SAPS-PD, SCL-90, SCOPA-PC, UM-PDHQ, plus data augmentation
• Labels: Expert consensus from two rounds of surveys (psychiatrists/neurologists)

Evaluation

Evaluated on 18 held-out items from unseen questionnaires against expert labels.

Cohen's κ	Accuracy
0.700	83.3%

Note: The test set is small (n=18), so the confidence interval is wide [0.388, 1.000].

Cross-validation (leave-one-questionnaire-out, 11 folds): mean accuracy 91.4% ± 12.5%.

Limitations

• Trained and validated on Parkinson's disease cohort questionnaires; performance may differ for general-population psychosis scales.
• Dimension structure reflects theoretical consensus among a specific expert panel and may not match all theoretical frameworks.
• Prodromal and subclinical psychosis items are particularly hard to classify; manual review is recommended for low-confidence assignments.

Citation

in preparation