|
--- |
|
datasets: |
|
- yirenc/4_ethics_all |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
- f1 |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
--- |
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
This is a fine-tuned version of Meta-Llama-3-8B model on AI Ethics dataset. The usage of this model should comply with the Llama 3 Community License and the Acceptable Use Policy. |
|
|
|
|
|
## Evaluation |
|
|
|
<!-- This section describes the evaluation protocols and provides the results. --> |
|
|
|
|
|
### Original model: Meta-Llama-3-8B |
|
|
|
| Category | F1 Score | Accuracy | Correct Formatting | |
|
|----------|----------|----------|--------------------| |
|
| Deontology | 0.258 | 0.180 | 0.340 | |
|
| Justice | 0.100 | 0.060 | 0.100 | |
|
| Utilitarianism | 0.123 | 0.070 | 0.150 | |
|
| Virtue | 0.057 | 0.030 | 0.040 | |
|
|
|
### Baseline model: Meta-Llama-3-8B on 4_ethics_100 |
|
|
|
| Category | F1 Score | Accuracy | Correct Formatting | |
|
|----------|----------|----------|--------------------| |
|
| Deontology | 0.428 | 0.490 | 1.000 | |
|
| Justice | 0.366 | 0.520 | 1.000 | |
|
| Utilitarianism | 0.449 | 0.540 | 1.000 | |
|
| Virtue | 0.476 | 0.600 | 1.000 | |
|
|
|
### Fine-tuned model: Meta-Llama-3-8B on 4_ethics_all |
|
|
|
| Category | F1 Score | Accuracy | Correct Formatting | |
|
|----------|----------|----------|--------------------| |
|
| Deontology | 0.930 | 0.930 | 1.000 | |
|
| Justice | 0.960 | 0.960 | 1.000 | |
|
| Utilitarianism | 0.750 | 0.750 | 1.000 | |
|
| Virtue | 0.891 | 0.890 | 1.000 | |
|
|
|
|
|
## Model Card Authors |
|
|
|
Yiren Cao |
|
|
|
## Model Card Contact |
|
|
|
yiren.cao@swisscom.com |
|
|
|
yiren.cao@epfl.ch |
|
|
|
yiren.cao@outlook.com |
|
|
