|
Quantization made by Richard Erkhov. |
|
|
|
[Github](https://github.com/RichardErkhov) |
|
|
|
[Discord](https://discord.gg/pvy7H8DZMG) |
|
|
|
[Request more models](https://github.com/RichardErkhov/quant_request) |
|
|
|
|
|
Llama-3-8B-Dolphin-Portuguese - GGUF |
|
- Model creator: https://huggingface.co/adalbertojunior/ |
|
- Original model: https://huggingface.co/adalbertojunior/Llama-3-8B-Dolphin-Portuguese/ |
|
|
|
|
|
| Name | Quant method | Size | |
|
| ---- | ---- | ---- | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q2_K.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q2_K.gguf) | Q2_K | 2.96GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.IQ3_XS.gguf) | IQ3_XS | 3.28GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.IQ3_S.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.IQ3_S.gguf) | IQ3_S | 3.43GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q3_K_S.gguf) | Q3_K_S | 3.41GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.IQ3_M.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.IQ3_M.gguf) | IQ3_M | 3.52GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q3_K.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q3_K.gguf) | Q3_K | 3.74GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q3_K_M.gguf) | Q3_K_M | 3.74GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q3_K_L.gguf) | Q3_K_L | 4.03GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.IQ4_XS.gguf) | IQ4_XS | 4.18GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q4_0.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q4_0.gguf) | Q4_0 | 4.34GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.IQ4_NL.gguf) | IQ4_NL | 4.38GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q4_K_S.gguf) | Q4_K_S | 4.37GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q4_K.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q4_K.gguf) | Q4_K | 4.58GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q4_K_M.gguf) | Q4_K_M | 4.58GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q4_1.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q4_1.gguf) | Q4_1 | 4.78GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q5_0.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q5_0.gguf) | Q5_0 | 5.21GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q5_K_S.gguf) | Q5_K_S | 5.21GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q5_K.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q5_K.gguf) | Q5_K | 5.34GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q5_K_M.gguf) | Q5_K_M | 5.34GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q5_1.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q5_1.gguf) | Q5_1 | 5.65GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q6_K.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q6_K.gguf) | Q6_K | 6.14GB | |
|
| [Llama-3-8B-Dolphin-Portuguese.Q8_0.gguf](https://huggingface.co/RichardErkhov/adalbertojunior_-_Llama-3-8B-Dolphin-Portuguese-gguf/blob/main/Llama-3-8B-Dolphin-Portuguese.Q8_0.gguf) | Q8_0 | 7.95GB | |
|
|
|
|
|
|
|
|
|
Original model description: |
|
|
|
--- |
|
library_name: transformers |
|
datasets: |
|
- adalbertojunior/dolphin_pt_test |
|
language: |
|
- pt |
|
model-index: |
|
- name: Llama-3-8B-Dolphin-Portuguese |
|
results: |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: ENEM Challenge (No Images) |
|
type: eduagarcia/enem_challenge |
|
split: train |
|
args: |
|
num_few_shot: 3 |
|
metrics: |
|
- type: acc |
|
value: 66.83 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: BLUEX (No Images) |
|
type: eduagarcia-temp/BLUEX_without_images |
|
split: train |
|
args: |
|
num_few_shot: 3 |
|
metrics: |
|
- type: acc |
|
value: 53.69 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: OAB Exams |
|
type: eduagarcia/oab_exams |
|
split: train |
|
args: |
|
num_few_shot: 3 |
|
metrics: |
|
- type: acc |
|
value: 45.24 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Assin2 RTE |
|
type: assin2 |
|
split: test |
|
args: |
|
num_few_shot: 15 |
|
metrics: |
|
- type: f1_macro |
|
value: 92.84 |
|
name: f1-macro |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Assin2 STS |
|
type: eduagarcia/portuguese_benchmark |
|
split: test |
|
args: |
|
num_few_shot: 15 |
|
metrics: |
|
- type: pearson |
|
value: 75.92 |
|
name: pearson |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: FaQuAD NLI |
|
type: ruanchaves/faquad-nli |
|
split: test |
|
args: |
|
num_few_shot: 15 |
|
metrics: |
|
- type: f1_macro |
|
value: 79.67 |
|
name: f1-macro |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: HateBR Binary |
|
type: ruanchaves/hatebr |
|
split: test |
|
args: |
|
num_few_shot: 25 |
|
metrics: |
|
- type: f1_macro |
|
value: 88.04 |
|
name: f1-macro |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: PT Hate Speech Binary |
|
type: hate_speech_portuguese |
|
split: test |
|
args: |
|
num_few_shot: 25 |
|
metrics: |
|
- type: f1_macro |
|
value: 58.34 |
|
name: f1-macro |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: tweetSentBR |
|
type: eduagarcia/tweetsentbr_fewshot |
|
split: test |
|
args: |
|
num_few_shot: 25 |
|
metrics: |
|
- type: f1_macro |
|
value: 69.4 |
|
name: f1-macro |
|
source: |
|
url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=adalbertojunior/Llama-3-8B-Dolphin-Portuguese |
|
name: Open Portuguese LLM Leaderboard |
|
--- |
|
|
|
# Model Card for Llama-3-8B-Dolphin-Portuguese |
|
|
|
Model Trained on a translated version of dolphin dataset. |
|
|
|
|
|
## Usage |
|
```python |
|
import transformers |
|
import torch |
|
|
|
model_id = "adalbertojunior/Llama-3-8B-Dolphin-Portuguese" |
|
|
|
pipeline = transformers.pipeline( |
|
"text-generation", |
|
model=model_id, |
|
model_kwargs={"torch_dtype": torch.bfloat16}, |
|
device_map="auto", |
|
) |
|
|
|
messages = [ |
|
{"role": "system", "content": "Você é um robô pirata que sempre responde como um pirata deveria!"}, |
|
{"role": "user", "content": "Quem é você?"}, |
|
] |
|
|
|
prompt = pipeline.tokenizer.apply_chat_template( |
|
messages, |
|
tokenize=False, |
|
add_generation_prompt=True |
|
) |
|
|
|
terminators = [ |
|
pipeline.tokenizer.eos_token_id, |
|
pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>") |
|
] |
|
|
|
outputs = pipeline( |
|
prompt, |
|
max_new_tokens=256, |
|
eos_token_id=terminators, |
|
do_sample=True, |
|
temperature=0.6, |
|
top_p=0.9, |
|
) |
|
print(outputs[0]["generated_text"][len(prompt):]) |
|
``` |
|
|
|
# Open Portuguese LLM Leaderboard Evaluation Results |
|
|
|
Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/adalbertojunior/Llama-3-8B-Dolphin-Portuguese) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard) |
|
|
|
| Metric | Value | |
|
|--------------------------|--------| |
|
|Average |**70.0**| |
|
|ENEM Challenge (No Images)| 66.83| |
|
|BLUEX (No Images) | 53.69| |
|
|OAB Exams | 45.24| |
|
|Assin2 RTE | 92.84| |
|
|Assin2 STS | 75.92| |
|
|FaQuAD NLI | 79.67| |
|
|HateBR Binary | 88.04| |
|
|PT Hate Speech Binary | 58.34| |
|
|tweetSentBR | 69.40| |
|
|
|
|
|
|