mentBERT / model_performance_metrics.txt
reab5555's picture
Upload 7 files
3ac9698 verified
raw
history blame
No virus
2.22 kB
Training set class counts after balancing:
_Borderline 10398
_Anxiety 10393
_Depression 10400
_Bipolar 10359
_OCD 10413
_ADHD 10412
_Schizophrenia 10447
_Asperger 10470
_PTSD 10489
dtype: object
Validation set class counts after balancing:
_Borderline 1180
_Anxiety 1185
_Depression 1178
_Bipolar 1219
_OCD 1165
_ADHD 1166
_Schizophrenia 1131
_Asperger 1108
_PTSD 1089
dtype: object
Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert/distilbert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Epoch 1, Train Loss: 0.2660, Val Loss: 0.2032
Epoch 2, Train Loss: 0.1891, Val Loss: 0.1873
F1 Score on Validation Set: 0.6356
AUC Score on Validation Set: 0.7643
Classification Report, AUC Score, F1 Score, and Losses
Train Losses: [0.2660091122228647, 0.18910447711386752]
Validation Losses: [0.20317846728614503, 0.18728890073445678]
F1 Score: [0.6355810277640191]
AUC Score: [0.7642971226283637]
Classification Report:
_Borderline:
Precision: 0.7606837606837606
Recall: 0.45254237288135596
F1-score: 0.5674814027630181
_Anxiety:
Precision: 0.7063318777292577
Recall: 0.5459915611814345
F1-score: 0.6158971918134221
_Depression:
Precision: 0.7286096256684492
Recall: 0.4626485568760611
F1-score: 0.5659397715472482
_Bipolar:
Precision: 0.7997076023391813
Recall: 0.44872846595570137
F1-score: 0.5748817656332107
_OCD:
Precision: 0.8222748815165877
Recall: 0.5957081545064378
F1-score: 0.6908909905425585
_ADHD:
Precision: 0.8856382978723404
Recall: 0.5711835334476844
F1-score: 0.6944734098018769
_Schizophrenia:
Precision: 0.7540628385698809
Recall: 0.6153846153846154
F1-score: 0.6777020447906524
_Asperger:
Precision: 0.6743515850144092
Recall: 0.6335740072202166
F1-score: 0.6533271288971615
_PTSD:
Precision: 0.7724687144482366
Recall: 0.6235078053259872
F1-score: 0.6900406504065041