vllm (pretrained=/root/autodl-tmp/output,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto

Tasks Version Filter n-shot Metric Value Stderr
gsm8k 3 flexible-extract 5 exact_match ↑ 0.72 ± 0.0285
strict-match 5 exact_match ↑ 0.72 ± 0.0285

vllm (pretrained=/root/autodl-tmp/Mahou-1.5-mistral-nemo-12B,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto

Tasks Version Filter n-shot Metric Value Stderr
gsm8k 3 flexible-extract 5 exact_match ↑ 0.708 ± 0.0288
strict-match 5 exact_match ↑ 0.708 ± 0.0288
Downloads last month
5
Safetensors
Model size
12.2B params
Tensor type
BF16
·
I8
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for noneUsername/Mahou-1.5-mistral-nemo-12B-W8A8-Dynamic-Per-Token