Model: ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct
Author: ControlLLM
Pipeline: text-generation
Library: transformers
Tags: transformers, safetensors, text-generation, en, dataset:nvidia/OpenMathInstruct-2, arxiv:2501.10979, base_model:meta-llama/Llama-3.1-8B-Instruct, base_model:finetune:meta-llama/Llama-3.1-8B-Instruct, license:llama3.1, model-index, endpoints_compatible, region:us
Base models (finetune): id: meta-llama/Llama-3.1-8B-Instruct
Card metadata: license: llama3.1
datasets: nvidia/OpenMathInstruct-2
language: en
base_model: meta-llama/Llama-3.1-8B-Instruct
model-index: name: Control-LLM-Llama3.1-8B-Math16
results: task: type: math-evaluation
dataset: type: parquet
name: Math, Math Hard, GSM8K
dataset_kwargs: data_files: https://github.com/linkedin/ControlLLM/blob/main/src/controlllm/inference/llm_eval_harness/additional_tasks/math/joined_math.parquet
metrics: name: exact_match,none
type: exact_match
value: 0.6205678398534606
stderr: 0.005249520342473376
verified: no, name: exact_match,none (gsm8k_0shot_instruct)
type: exact_match
value: 0.8968915845337376
stderr: 0.008376436987507811
verified: no, name: exact_match,none (meta_math_0shot_instruct)
type: exact_match
value: 0.6166
stderr: 0.006876797660918556
verified: no, name: exact_match,none (meta_math_hard_0shot_instruct)
type: exact_match
value: 0.36027190332326287
stderr: 0.013198755610252931
verified: no, task: type: original-capability
dataset: type: meta/Llama-3.1-8B-Instruct-evals
name: Llama-3.1-8B-Instruct-evals Dataset
dataset_path: meta-llama/llama-3.1-8_b-instruct-evals
dataset_name: Llama-3.1-8B-Instruct-evals__arc_challenge__details
metrics: name: exact_match,strict-match
type: exact_match
value: 0.6001372485281902
stderr: 0.002821514831773572
verified: no, name: exact_match,strict-match (meta_arc_0shot_instruct)
type: exact_match
value: 0.8248927038626609
stderr: 0.011139722235859526
verified: no, name: exact_match,strict-match (meta_gpqa_0shot_cot_instruct)
type: exact_match
value: 0.3080357142857143
stderr: 0.021836780796366417
verified: no, name: exact_match,strict-match (meta_mmlu_0shot_instruct)
type: exact_match
value: 0.7159948725252813
stderr: 0.00380556397209409
verified: no, name: exact_match,strict-match (meta_mmlu_pro_5shot_instruct)
type: exact_match
value: 0.45403922872340424
stderr: 0.004539171007529716
verified: no
library_name: transformers
pipeline_tag: text-generation
Likes: 0 | All-time downloads: 167
Created: 2025-01-09 09:31:04