README.md

---
base_model: rhaymison/phi-3-portuguese-tom-cat-4k-instruct
datasets:
- rhaymison/superset
inference: true
language:
- pt
library_name: transformers
license: apache-2.0
model-index:
- name: phi-3-portuguese-tom-cat-4k-instruct
  results:
  - dataset:
      args:
        num_few_shot: 3
      name: ENEM Challenge (No Images)
      split: train
      type: eduagarcia/enem_challenge
    metrics:
    - name: accuracy
      type: acc
      value: 61.58
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 3
      name: BLUEX (No Images)
      split: train
      type: eduagarcia-temp/BLUEX_without_images
    metrics:
    - name: accuracy
      type: acc
      value: 50.63
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 3
      name: OAB Exams
      split: train
      type: eduagarcia/oab_exams
    metrics:
    - name: accuracy
      type: acc
      value: 43.69
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 15
      name: Assin2 RTE
      split: test
      type: assin2
    metrics:
    - name: f1-macro
      type: f1_macro
      value: 91.54
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 15
      name: Assin2 STS
      split: test
      type: eduagarcia/portuguese_benchmark
    metrics:
    - name: pearson
      type: pearson
      value: 75.27
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 15
      name: FaQuAD NLI
      split: test
      type: ruanchaves/faquad-nli
    metrics:
    - name: f1-macro
      type: f1_macro
      value: 47.46
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 25
      name: HateBR Binary
      split: test
      type: ruanchaves/hatebr
    metrics:
    - name: f1-macro
      type: f1_macro
      value: 83.01
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 25
      name: PT Hate Speech Binary
      split: test
      type: hate_speech_portuguese
    metrics:
    - name: f1-macro
      type: f1_macro
      value: 70.19
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
  - dataset:
      args:
        num_few_shot: 25
      name: tweetSentBR
      split: test
      type: eduagarcia/tweetsentbr_fewshot
    metrics:
    - name: f1-macro
      type: f1_macro
      value: 57.78
    source:
      name: Open Portuguese LLM Leaderboard
      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=rhaymison/phi-3-portuguese-tom-cat-4k-instruct
    task:
      name: Text Generation
      type: text-generation
model_creator: rhaymison
model_name: phi-3-portuguese-tom-cat-4k-instruct
pipeline_tag: text-generation
quantized_by: afrideva
tags:
- portugues
- portuguese
- QA
- instruct
- phi
- gguf
- ggml
- quantized
---

# phi-3-portuguese-tom-cat-4k-instruct-GGUF

Quantized GGUF model files for [phi-3-portuguese-tom-cat-4k-instruct](https://huggingface.co/rhaymison/phi-3-portuguese-tom-cat-4k-instruct) from [rhaymison](https://huggingface.co/rhaymison)

## Original Model Card:

# Phi-3-portuguese-tom-cat-4k-instruct

<p align="center">
  <img src="https://raw.githubusercontent.com/rhaymisonbetini/huggphotos/main/tom-cat.webp"  width="50%" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
</p>


This model was trained with a superset of 300,000 instructions in Portuguese. 
The model comes to help fill the gap in models in Portuguese. Tuned from the   microsoft/Phi-3-mini-4k.

# How to use

### FULL MODEL : A100
### HALF MODEL: L4
### 8bit or 4bit : T4 or V100

You can use the model in its normal form up to 4-bit quantization. Below we will use both approaches.
Remember that verbs are important in your prompt. Tell your model how to act or behave so that you can guide them along the path of their response. 
Important points like these help models (even smaller models like 4b) to perform much better.

```python
!pip install -q -U transformers
!pip install -q -U accelerate
!pip install -q -U bitsandbytes

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model = AutoModelForCausalLM.from_pretrained("rhaymison/phi-3-portuguese-tom-cat-4k-instruct", device_map= {"": 0})
tokenizer = AutoTokenizer.from_pretrained("rhaymison/phi-3-portuguese-tom-cat-4k-instruct")
model.eval()

```

You can use with Pipeline.
```python

from transformers import pipeline
pipe = pipeline("text-generation",
                model=model,
                tokenizer=tokenizer,
                do_sample=True,
                max_new_tokens=512,
                num_beams=2,
                temperature=0.3,
                top_k=50,
                top_p=0.95,
                early_stopping=True,
                pad_token_id=tokenizer.eos_token_id,
                )


def format_template(question:str):
    system_prompt = "Abaixo está uma instrução que descreve uma tarefa, juntamente com uma entrada que fornece mais contexto. Escreva uma resposta que complete adequadamente o pedido."
    return f"""<s><|system|>
    { system_prompt }
    <|user|>
    { question }
    <|assistant|>
    """

question = format_template("E possivel ir de Carro dos Estados unidos ate o japão")
pipe(question)
```

If you are having a memory problem such as "CUDA Out of memory", you should use 4-bit or 8-bit quantization.
For the complete model in colab you will need the A100.
If you want to use 4bits or 8bits, T4 or L4 will already solve the problem.

# 4bits example

```python
from transformers import BitsAndBytesConfig
import torch
nb_4bit_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True
)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map={"": 0}
)

```

# Open Portuguese LLM Leaderboard Evaluation Results  

Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/rhaymison/phi-3-portuguese-tom-cat-4k-instruct) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)

|          Metric          |  Value  |
|--------------------------|---------|
|Average                   |**64.57**|
|ENEM Challenge (No Images)|    61.58|
|BLUEX (No Images)         |    50.63|
|OAB Exams                 |    43.69|
|Assin2 RTE                |    91.54|
|Assin2 STS                |    75.27|
|FaQuAD NLI                |    47.46|
|HateBR Binary             |    83.01|
|PT Hate Speech Binary     |    70.19|
|tweetSentBR               |    57.78|


### Comments

Any idea, help or report will always be welcome.

email: rhaymisoncristian@gmail.com

 <div style="display:flex; flex-direction:row; justify-content:left">
    <a href="https://www.linkedin.com/in/rhaymison-cristian-betini-2b3016175/" target="_blank">
    <img src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white">
  </a>
  <a href="https://github.com/rhaymisonbetini" target="_blank">
    <img src="https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white">
  </a>