Uploaded model
- Developed by: GuiCas
- License: mit
- Finetuned from model : unsloth/Phi-3-mini-4k-instruct-bnb-4bit
Open Portuguese LLM Leaderboard Evaluation Results
Detailed results can be found here and on the 🚀 Open Portuguese LLM Leaderboard
| Metric | Value |
|---|---|
| Average | 55.66 |
| ENEM Challenge (No Images) | 49.97 |
| BLUEX (No Images) | 43.25 |
| OAB Exams | 38.13 |
| Assin2 RTE | 74.75 |
| Assin2 STS | 71.93 |
| FaQuAD NLI | 43.97 |
| HateBR Binary | 57.34 |
| PT Hate Speech Binary | 60.48 |
| tweetSentBR | 61.11 |
- Downloads last month
- 408
Unable to determine this model’s pipeline type. Check the
docs
.
Quantized from
Dataset used to train GuiCas/Phi-3-Phituguese-3b-q8_0
Evaluation results
- accuracy on ENEM Challenge (No Images)Open Portuguese LLM Leaderboard49.970
- accuracy on BLUEX (No Images)Open Portuguese LLM Leaderboard43.250
- accuracy on OAB ExamsOpen Portuguese LLM Leaderboard38.130
- f1-macro on Assin2 RTEtest set Open Portuguese LLM Leaderboard74.750
- pearson on Assin2 STStest set Open Portuguese LLM Leaderboard71.930
- f1-macro on FaQuAD NLItest set Open Portuguese LLM Leaderboard43.970
- f1-macro on HateBR Binarytest set Open Portuguese LLM Leaderboard57.340
- f1-macro on PT Hate Speech Binarytest set Open Portuguese LLM Leaderboard60.480
- f1-macro on tweetSentBRtest set Open Portuguese LLM Leaderboard61.110