Quantization made by Richard Erkhov.

sabia-7b - GGUF

Model creator: https://huggingface.co/maritaca-ai/
Original model: https://huggingface.co/maritaca-ai/sabia-7b/

Name	Quant method	Size
sabia-7b.Q2_K.gguf	Q2_K	2.36GB
sabia-7b.IQ3_XS.gguf	IQ3_XS	2.6GB
sabia-7b.IQ3_S.gguf	IQ3_S	2.75GB
sabia-7b.Q3_K_S.gguf	Q3_K_S	2.75GB
sabia-7b.IQ3_M.gguf	IQ3_M	2.9GB
sabia-7b.Q3_K.gguf	Q3_K	3.07GB
sabia-7b.Q3_K_M.gguf	Q3_K_M	3.07GB
sabia-7b.Q3_K_L.gguf	Q3_K_L	3.35GB
sabia-7b.IQ4_XS.gguf	IQ4_XS	3.4GB
sabia-7b.Q4_0.gguf	Q4_0	3.56GB
sabia-7b.IQ4_NL.gguf	IQ4_NL	3.58GB
sabia-7b.Q4_K_S.gguf	Q4_K_S	3.59GB
sabia-7b.Q4_K.gguf	Q4_K	3.8GB
sabia-7b.Q4_K_M.gguf	Q4_K_M	3.8GB
sabia-7b.Q4_1.gguf	Q4_1	3.95GB
sabia-7b.Q5_0.gguf	Q5_0	4.33GB
sabia-7b.Q5_K_S.gguf	Q5_K_S	4.33GB
sabia-7b.Q5_K.gguf	Q5_K	4.45GB
sabia-7b.Q5_K_M.gguf	Q5_K_M	4.45GB
sabia-7b.Q5_1.gguf	Q5_1	4.72GB
sabia-7b.Q6_K.gguf	Q6_K	5.15GB
sabia-7b.Q8_0.gguf	Q8_0	4.88GB

Original model description:

language: - pt model-index: - name: sabia-7b results: - task: type: text-generation name: Text Generation dataset: name: ENEM Challenge (No Images) type: eduagarcia/enem_challenge split: train args: num_few_shot: 3 metrics: - type: acc value: 55.07 name: accuracy source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BLUEX (No Images) type: eduagarcia-temp/BLUEX_without_images split: train args: num_few_shot: 3 metrics: - type: acc value: 47.71 name: accuracy source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: OAB Exams type: eduagarcia/oab_exams split: train args: num_few_shot: 3 metrics: - type: acc value: 41.41 name: accuracy source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Assin2 RTE type: assin2 split: test args: num_few_shot: 15 metrics: - type: f1_macro value: 46.68 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Assin2 STS type: eduagarcia/portuguese_benchmark split: test args: num_few_shot: 15 metrics: - type: pearson value: 1.89 name: pearson source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: FaQuAD NLI type: ruanchaves/faquad-nli split: test args: num_few_shot: 15 metrics: - type: f1_macro value: 58.34 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HateBR Binary type: ruanchaves/hatebr split: test args: num_few_shot: 25 metrics: - type: f1_macro value: 61.93 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: PT Hate Speech Binary type: hate_speech_portuguese split: test args: num_few_shot: 25 metrics: - type: f1_macro value: 64.13 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: tweetSentBR type: eduagarcia-temp/tweetsentbr split: test args: num_few_shot: 25 metrics: - type: f1_macro value: 46.64 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b name: Open Portuguese LLM Leaderboard

Sabiá-7B is Portuguese language model developed by Maritaca AI.

Input: The model accepts only text input.

Output: The Model generates text only.

Model Architecture: Sabiá-7B is an auto-regressive language model that uses the same architecture of LLaMA-1-7B.

Tokenizer: It uses the same tokenizer as LLaMA-1-7B.

Maximum sequence length: 2048 tokens.

Pretraining data: The model was pretrained on 7 billion tokens from the Portuguese subset of ClueWeb22, starting with the weights of LLaMA-1-7B and further trained for an additional 10 billion tokens, approximately 1.4 epochs of the training dataset.

Data Freshness: The pretraining data has a cutoff of mid-2022.

License: The licensing is the same as LLaMA-1's, restricting the model's use to research purposes only.

Paper: For more details, please refer to our paper: Sabiá: Portuguese Large Language Models

Few-shot Example

Given that Sabiá-7B was trained solely on a language modeling objective without fine-tuning for instruction following, it is recommended for few-shot tasks rather than zero-shot tasks, like in the example below.

import torch
from transformers import LlamaTokenizer, LlamaForCausalLM

tokenizer = LlamaTokenizer.from_pretrained("maritaca-ai/sabia-7b")
model = LlamaForCausalLM.from_pretrained(
    "maritaca-ai/sabia-7b",
    device_map="auto",  # Automatically loads the model in the GPU, if there is one. Requires pip install acelerate
    low_cpu_mem_usage=True,
    torch_dtype=torch.bfloat16   # If your GPU does not support bfloat16, change to torch.float16
)  

prompt = """Classifique a resenha de filme como "positiva" ou "negativa".

Resenha: Gostei muito do filme, é o melhor do ano!
Classe: positiva

Resenha: O filme deixa muito a desejar.
Classe: negativa

Resenha: Apesar de longo, valeu o ingresso.
Classe:"""

input_ids = tokenizer(prompt, return_tensors="pt")

output = model.generate(
    input_ids["input_ids"].to("cuda"),
    max_length=1024,
    eos_token_id=tokenizer.encode("\n"))  # Stop generation when a "\n" token is dectected

# The output contains the input tokens, so we have to skip them.
output = output[0][len(input_ids["input_ids"][0]):]

print(tokenizer.decode(output, skip_special_tokens=True))

If your GPU does not have enough RAM, try using int8 precision. However, expect some degradation in the model output quality when compared to fp16 or bf16.

model = LlamaForCausalLM.from_pretrained(
    "maritaca-ai/sabia-7b",
    device_map="auto",
    low_cpu_mem_usage=True,
    load_in_8bit=True,  # Requires pip install bitsandbytes
)

Results in Portuguese

Below we show the results on the Poeta benchmark, which consists of 14 Portuguese datasets.

For more information on the Normalized Preferred Metric (NPM), please refer to our paper.

Model	NPM
LLaMA-1-7B	33.0
LLaMA-2-7B	43.7
Sabiá-7B	48.5

Results in English

Below we show the average results on 6 English datasets: PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, and OpenBookQA.

Model	NPM
LLaMA-1-7B	50.1
Sabiá-7B	49.0

Citation

Please use the following bibtex to cite our paper:

@InProceedings{10.1007/978-3-031-45392-2_15,
    author="Pires, Ramon
    and Abonizio, Hugo
    and Almeida, Thales Sales
    and Nogueira, Rodrigo",
    editor="Naldi, Murilo C.
    and Bianchi, Reinaldo A. C.",
    title="Sabi{\'a}: Portuguese Large Language Models",
    booktitle="Intelligent Systems",
    year="2023",
    publisher="Springer Nature Switzerland",
    address="Cham",
    pages="226--240",
    isbn="978-3-031-45392-2"
}

Open Portuguese LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Average	47.09
ENEM Challenge (No Images)	55.07
BLUEX (No Images)	47.71
OAB Exams	41.41
Assin2 RTE	46.68
Assin2 STS	1.89
FaQuAD NLI	58.34
HateBR Binary	61.93
PT Hate Speech Binary	64.13
tweetSentBR	46.64