Model: refactai/starcoderbase-3b
Author: refactai
Pipeline: text-generation
Library: transformers
Tags: transformers, pytorch, gpt_bigcode, text-generation, code, dataset:bigcode/the-stack-dedup, arxiv:1911.02150, arxiv:2205.14135, arxiv:2207.14255, arxiv:2305.06161, license:bigcode-openrail-m, model-index, text-generation-inference, endpoints_compatible, region:us
Card metadata: pipeline_tag: text-generation
inference: yes
widget: text: def print_hello_world():
example_title: Hello world
group: Python
license: bigcode-openrail-m
datasets: bigcode/the-stack-dedup
metrics: code_eval
library_name: transformers
tags: code
model-index: name: StarCoderBase-3B
results: task: type: text-generation
dataset: type: openai_humaneval
name: HumanEval
metrics: name: pass@1
type: pass@1
value: 21.46
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (C++)
metrics: name: pass@1
type: pass@1
value: 19.43
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Java)
metrics: name: pass@1
type: pass@1
value: 19.25
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (JavaScript)
metrics: name: pass@1
type: pass@1
value: 21.32
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (PHP)
metrics: name: pass@1
type: pass@1
value: 18.55
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Lua)
metrics: name: pass@1
type: pass@1
value: 18.04
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Rust)
metrics: name: pass@1
type: pass@1
value: 16.32
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Swift)
metrics: name: pass@1
type: pass@1
value: 9.98
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Julia)
metrics: name: pass@1
type: pass@1
value: 16.1
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (R)
metrics: name: pass@1
type: pass@1
value: 10.1
verified: no
extra_gated_prompt: ## Model License Agreement
Please read the BigCode [OpenRAIL-M license](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) agreement before accepting it.
  
extra_gated_fields: I accept the above license agreement, and will use the Model complying with the set of use restrictions and sharing requirements: checkbox
Likes: 0 | All-time downloads: 1115
Created: 2023-10-04 14:50:27