Model: ControlLLM/Llama-3.1-8B-OpenCoder16-Instruct
Author: ControlLLM
Pipeline: text-generation
Library: transformers
Tags: transformers, safetensors, text-generation, en, dataset:nvidia/OpenMathInstruct-2, arxiv:2501.10979, base_model:meta-llama/Llama-3.1-8B-Instruct, base_model:finetune:meta-llama/Llama-3.1-8B-Instruct, license:llama3.1, model-index, endpoints_compatible, region:us
Base models (finetune): id: meta-llama/Llama-3.1-8B-Instruct
Card metadata: license: llama3.1
datasets: nvidia/OpenMathInstruct-2
language: en
base_model: meta-llama/Llama-3.1-8B-Instruct
model-index: name: Control-LLM-Llama3.1-8B-Math16
results: task: type: code-evaluation
dataset: type: mixed
name: Code Evaluation Dataset
metrics: name: pass_at_1,n=1 (code_instruct)
type: pass_at_1
value: 0.7840083073727934
stderr: 0.013257237506304915
verified: no, name: pass_at_1,n=1 (humaneval_greedy_instruct)
type: pass_at_1
value: 0.8170731707317073
stderr: 0.03028135999593353
verified: no, name: pass_at_1,n=1 (humaneval_plus_greedy_instruct)
type: pass_at_1
value: 0.7439024390243902
stderr: 0.03418746588364997
verified: no, name: pass_at_1,n=1 (mbpp_plus_0shot_instruct)
type: pass_at_1
value: 0.8042328042328042
stderr: 0.0204357309715418
verified: no, name: pass_at_1,n=1 (mbpp_sanitized_0shot_instruct)
type: pass_at_1
value: 0.7587548638132295
stderr: 0.02673991635681605
verified: no, task: type: original-capability
dataset: type: meta/Llama-3.1-8B-Instruct-evals
name: Llama-3.1-8B-Instruct-evals Dataset
dataset_path: meta-llama/llama-3.1-8_b-instruct-evals
dataset_name: Llama-3.1-8B-Instruct-evals__arc_challenge__details
metrics: name: exact_match,strict-match (original_capability_instruct)
type: exact_match
value: 0.5630801459168563
stderr: 0.0028483348465514185
verified: no, name: exact_match,strict-match (meta_arc_0shot_instruct)
type: exact_match
value: 0.8248927038626609
stderr: 0.01113972223585952
verified: no, name: exact_match,strict-match (meta_gpqa_0shot_cot_instruct)
type: exact_match
value: 0.296875
stderr: 0.021609729061250887
verified: no, name: exact_match,strict-match (meta_mmlu_0shot_instruct)
type: exact_match
value: 0.6815980629539952
stderr: 0.003931452244804845
verified: no, name: exact_match,strict-match (meta_mmlu_pro_5shot_instruct)
type: exact_match
value: 0.4093251329787234
stderr: 0.004482884901882547
verified: no
library_name: transformers
pipeline_tag: text-generation
Likes: 0 | All-time downloads: 158
Created: 2025-01-09 09:40:59