Edit model card

Training procedure

The following bitsandbytes quantization config was used during training:

  • quant_method: bitsandbytes
  • _load_in_8bit: False
  • _load_in_4bit: True
  • llm_int8_threshold: 6.0
  • llm_int8_skip_modules: None
  • llm_int8_enable_fp32_cpu_offload: False
  • llm_int8_has_fp16_weight: False
  • bnb_4bit_quant_type: nf4
  • bnb_4bit_use_double_quant: True
  • bnb_4bit_compute_dtype: float16
  • bnb_4bit_quant_storage: uint8
  • load_in_4bit: True
  • load_in_8bit: False

Framework versions

  • PEFT 0.5.0

Open Portuguese LLM Leaderboard Evaluation Results

Detailed results can be found here and on the ๐Ÿš€ Open Portuguese LLM Leaderboard

Metric Value
Average 65.16
ENEM Challenge (No Images) 57.03
BLUEX (No Images) 44.92
OAB Exams 39.64
Assin2 RTE 90.68
Assin2 STS 69.97
FaQuAD NLI 65.14
HateBR Binary 83.25
PT Hate Speech Binary 70.36
tweetSentBR 65.45
Downloads last month
3
Inference API
Unable to determine this modelโ€™s pipeline type. Check the docs .

Space using recogna-nlp/zephyr_7b_beta_ultraalpaca 1

Evaluation results