BRisa 7B Instruct
This is an instruction model trained for good performance in Portuguese. The initial base is the Mistral 7B v2 Model (source). We utilized the JJhooww/Mistral-7B-v0.2-Base_ptbr version pre-trained on 1 billion tokens in Portuguese (source).
The base model has good performance in Portuguese but faces significant challenges following instructions. We therefore used the version mistralai/Mistral-7B-Instruct-v0.2 and fine-tuned it for responses in Portuguese, then merged it with the base JJhooww/Mistral-7B-v0.2-Base_ptbr (https://huggingface.co/JJhooww/Mistral-7B-v0.2-Base_ptbr).
Model Sources
- Demo: (Demonstracao da Versão DPO)
Open Portuguese LLM Leaderboard Evaluation Results
Detailed results can be found here and on the 🚀 Open Portuguese LLM Leaderboard
| Metric | Value |
|---|---|
| Average | 66.19 |
| ENEM Challenge (No Images) | 65.08 |
| BLUEX (No Images) | 53.69 |
| OAB Exams | 43.37 |
| Assin2 RTE | 91.50 |
| Assin2 STS | 73.61 |
| FaQuAD NLI | 68.31 |
| HateBR Binary | 74.28 |
| PT Hate Speech Binary | 65.12 |
| tweetSentBR | 60.77 |
- Downloads last month
- 414
Space using J-LAB/BRisa-7B-Instruct-v0.2 1
Evaluation results
- accuracy on ENEM Challenge (No Images)Open Portuguese LLM Leaderboard65.080
- accuracy on BLUEX (No Images)Open Portuguese LLM Leaderboard53.690
- accuracy on OAB ExamsOpen Portuguese LLM Leaderboard43.370
- f1-macro on Assin2 RTEtest set Open Portuguese LLM Leaderboard91.500
- pearson on Assin2 STStest set Open Portuguese LLM Leaderboard73.610
- f1-macro on FaQuAD NLItest set Open Portuguese LLM Leaderboard68.310
- f1-macro on HateBR Binarytest set Open Portuguese LLM Leaderboard74.280
- f1-macro on PT Hate Speech Binarytest set Open Portuguese LLM Leaderboard65.120
- f1-macro on tweetSentBRtest set Open Portuguese LLM Leaderboard60.770