Jorgeutd's picture
Add evaluation results on the default config and test split of emotion (#1)
2ddd6cd
|
raw
history blame
2.58 kB
---
language: en
widget:
- text: I am really upset that I have to call up to three times to the number on the
back of my insurance card for my call to be answer
tags:
- sagemaker
- roberta-base
- text classification
license: apache-2.0
datasets:
- emotion
model-index:
- name: sagemaker-roberta-base-emotion
results:
- task:
name: Multi Class Text Classification
type: text-classification
dataset:
name: emotion
type: emotion
metrics:
- name: Validation Accuracy
type: accuracy
value: 94.1
- name: Validation F1
type: f1
value: 94.13
- task:
type: text-classification
name: Text Classification
dataset:
name: emotion
type: emotion
config: default
split: test
metrics:
- name: Accuracy
type: accuracy
value: 0.931
verified: true
- name: Precision Macro
type: precision
value: 0.8833042147663716
verified: true
- name: Precision Micro
type: precision
value: 0.931
verified: true
- name: Precision Weighted
type: precision
value: 0.9337002742192515
verified: true
- name: Recall Macro
type: recall
value: 0.9087144572668905
verified: true
- name: Recall Micro
type: recall
value: 0.931
verified: true
- name: Recall Weighted
type: recall
value: 0.931
verified: true
- name: F1 Macro
type: f1
value: 0.8949974527433656
verified: true
- name: F1 Micro
type: f1
value: 0.931
verified: true
- name: F1 Weighted
type: f1
value: 0.9318434300647934
verified: true
- name: loss
type: loss
value: 0.17379647493362427
verified: true
---
## roberta-base
This model is a fine-tuned model that was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
- Problem type: Multi Class Text Classification (emotion detection).
It achieves the following results on the evaluation set:
- Loss: 0.1613253802061081
- f1: 0.9413321705151999
## Hyperparameters
```json
{
"epochs": 10,
"train_batch_size": 16,
"learning_rate": 3e-5,
"weight_decay":0.01,
"load_best_model_at_end": true,
"model_name":"roberta-base",
"do_eval": True,
"load_best_model_at_end":True
}
```
## Validation Metrics
| key | value |
| --- | ----- |
| eval_accuracy | 0.941 |
| eval_f1 | 0.9413321705151999 |
| eval_loss | 0.1613253802061081|
| eval_recall | 0.941 |
| eval_precision | 0.9419519436781406 |