Model: ControlLLM/Control-LLM-Llama3.1-8B-OpenCoder8-Instruct
Author: ControlLLM
Pipeline: text-generation
Library: transformers
Tags: transformers, safetensors, text-generation, en, dataset:OpenCoder-LLM/opc-sft-stage1, dataset:OpenCoder-LLM/opc-sft-stage2, arxiv:2501.10979, base_model:meta-llama/Llama-3.1-8B-Instruct, base_model:finetune:meta-llama/Llama-3.1-8B-Instruct, license:llama3.1, model-index, endpoints_compatible, region:us
Base models (finetune): id: meta-llama/Llama-3.1-8B-Instruct
Card metadata: license: llama3.1
datasets: OpenCoder-LLM/opc-sft-stage1, OpenCoder-LLM/opc-sft-stage2
language: en
base_model: meta-llama/Llama-3.1-8B-Instruct
model-index: name: Control-LLM-Llama3.1-8B-OpenCoder8
results: task: type: code-evaluation
dataset: type: mixed
name: Code Evaluation Dataset
metrics: name: pass_at_1,n=1 (code_instruct)
type: pass_at_1
value: 0.770508826583593
stderr: 0.013547264970313243
verified: no, name: pass_at_1,n=1 (humaneval_greedy_instruct)
type: pass_at_1
value: 0.823170731707317
stderr: 0.029883277857485988
verified: no, name: pass_at_1,n=1 (humaneval_plus_greedy_instruct)
type: pass_at_1
value: 0.7621951219512195
stderr: 0.033346454086653404
verified: no, name: pass_at_1,n=1 (mbpp_plus_0shot_instruct)
type: pass_at_1
value: 0.7751322751322751
stderr: 0.02150209607822914
verified: no, name: pass_at_1,n=1 (mbpp_sanitized_0shot_instruct)
type: pass_at_1
value: 0.7354085603112841
stderr: 0.027569713464529938
verified: no, task: type: original-capability
dataset: type: meta/Llama-3.1-8B-Instruct-evals
name: Llama-3.1-8B-Instruct-evals Dataset
dataset_path: meta-llama/llama-3.1-8_b-instruct-evals
dataset_name: Llama-3.1-8B-Instruct-evals__arc_challenge__details
metrics: name: exact_match,strict-match (original_capability_instruct)
type: exact_match
value: 0.5599378769819771
stderr: 0.0028491774433443513
verified: no, name: exact_match,strict-match (meta_arc_0shot_instruct)
type: exact_match
value: 0.8094420600858369
stderr: 0.011511446994122106
verified: no, name: exact_match,strict-match (meta_gpqa_0shot_cot_instruct)
type: exact_match
value: 0.32589285714285715
stderr: 0.02216910313464341
verified: no, name: exact_match,strict-match (meta_mmlu_0shot_instruct)
type: exact_match
value: 0.681241988320752
stderr: 0.003932622311434926
verified: no, name: exact_match,strict-match (meta_mmlu_pro_5shot_instruct)
type: exact_match
value: 0.4029255319148936
stderr: 0.004471732136513382
verified: no
pipeline_tag: text-generation
library_name: transformers
Likes: 0 | All-time downloads: 156
Created: 2025-01-09 09:42:27