Model: ControlLLM/Llama3.1-8B-OpenMath16-Instruct
Author: ControlLLM
Pipeline: text-generation
Library: transformers
Tags: transformers, safetensors, text-generation, en, dataset:nvidia/OpenMathInstruct-2, arxiv:2501.10979, base_model:meta-llama/Llama-3.1-8B-Instruct, base_model:finetune:meta-llama/Llama-3.1-8B-Instruct, license:llama3.1, model-index, endpoints_compatible, region:us
Base models (finetune): id: meta-llama/Llama-3.1-8B-Instruct
Card metadata: license: llama3.1
datasets: nvidia/OpenMathInstruct-2
language: en
metrics: accuracy
base_model: meta-llama/Llama-3.1-8B-Instruct
model-index: name: Control-LLM-Llama3.1-8B-Math16
results: task: type: math-evaluation
dataset: type: parquet
name: Math, Math Hard, GSM8K
dataset_kwargs: data_files: https://github.com/linkedin/ControlLLM/blob/main/src/controlllm/inference/llm_eval_harness/additional_tasks/math/joined_math.parquet
metrics: name: exact_match,none
type: exact_match
value: 0.6327358367133324
stderr: 0.0052245703347459605
verified: no, name: exact_match,none (gsm8k_0shot_instruct)
type: exact_match
value: 0.9052312357846853
stderr: 0.008067791560015407
verified: no, name: exact_match,none (meta_math_0shot_instruct)
type: exact_match
value: 0.6276
stderr: 0.006837616441401548
verified: no, name: exact_match,none (meta_math_hard_0shot_instruct)
type: exact_match
value: 0.3806646525679758
stderr: 0.013349170720370741
verified: no, task: type: original-capability
dataset: type: meta/Llama-3.1-8B-Instruct-evals
name: Llama-3.1-8B-Instruct-evals Dataset
dataset_path: meta-llama/llama-3.1-8_b-instruct-evals
dataset_name: Llama-3.1-8B-Instruct-evals__arc_challenge__details
metrics: name: exact_match,strict-match
type: exact_match
value: 0.5723263625528227
stderr: 0.002858377993520894
verified: no, name: exact_match,strict-match (meta_arc_0shot_instruct)
type: exact_match
value: 0.7974248927038626
stderr: 0.01178043813618557
verified: no, name: exact_match,strict-match (meta_gpqa_0shot_cot_instruct)
type: exact_match
value: 0.25223214285714285
stderr: 0.02054139101648797
verified: no, name: exact_match,strict-match (meta_mmlu_0shot_instruct)
type: exact_match
value: 0.6837345107534539
stderr: 0.0039243761987253515
verified: no, name: exact_match,strict-match (meta_mmlu_pro_5shot_instruct)
type: exact_match
value: 0.4324301861702128
stderr: 0.004516653585262379
verified: no
pipeline_tag: text-generation
library_name: transformers
Likes: 0 | All-time downloads: 245
Created: 2025-01-09 09:30:26