Model: bigcode/starcoderbase-megatron
Author: bigcode
Pipeline: text-generation
Library: transformers
Tags: transformers, code, text-generation, dataset:bigcode/the-stack-dedup, arxiv:1911.02150, arxiv:2205.14135, arxiv:2207.14255, arxiv:2305.06161, license:bigcode-openrail-m, model-index, endpoints_compatible, region:us
Card metadata: pipeline_tag: text-generation
inference: yes
widget: text: def print_hello_world():
example_title: Hello world
group: Python
license: bigcode-openrail-m
datasets: bigcode/the-stack-dedup
metrics: code_eval
library_name: transformers
tags: code
model-index: name: StarCoderBase
results: task: type: text-generation
dataset: type: openai_humaneval
name: HumanEval
metrics: name: pass@1
type: pass@1
value: 0.304
verified: no, task: type: text-generation
dataset: type: mbpp
name: MBPP
metrics: name: pass@1
type: pass@1
value: 0.49
verified: no, task: type: text-generation
dataset: type: ds1000
name: DS-1000 (Overall Completion)
metrics: name: pass@1
type: pass@1
value: 0.238
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (C++)
metrics: name: pass@1
type: pass@1
value: 0.3056
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (C#)
metrics: name: pass@1
type: pass@1
value: 0.2056
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (D)
metrics: name: pass@1
type: pass@1
value: 0.1001
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Go)
metrics: name: pass@1
type: pass@1
value: 0.2147
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Java)
metrics: name: pass@1
type: pass@1
value: 0.2853
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Julia)
metrics: name: pass@1
type: pass@1
value: 0.2109
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (JavaScript)
metrics: name: pass@1
type: pass@1
value: 0.317
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Lua)
metrics: name: pass@1
type: pass@1
value: 0.2661
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (PHP)
metrics: name: pass@1
type: pass@1
value: 0.2675
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Perl)
metrics: name: pass@1
type: pass@1
value: 0.1632
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Python)
metrics: name: pass@1
type: pass@1
value: 0.3035
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (R)
metrics: name: pass@1
type: pass@1
value: 0.1018
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Ruby)
metrics: name: pass@1
type: pass@1
value: 0.1725
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Racket)
metrics: name: pass@1
type: pass@1
value: 0.1177
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Rust)
metrics: name: pass@1
type: pass@1
value: 0.2446
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Scala)
metrics: name: pass@1
type: pass@1
value: 0.2879
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Bash)
metrics: name: pass@1
type: pass@1
value: 0.1102
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (Swift)
metrics: name: pass@1
type: pass@1
value: 0.1674
verified: no, task: type: text-generation
dataset: type: nuprl/MultiPL-E
name: MultiPL-HumanEval (TypeScript)
metrics: name: pass@1
type: pass@1
value: 0.3215
verified: no
extra_gated_prompt: ## Model License Agreement
Please read the BigCode [OpenRAIL-M license](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) agreement before accepting it.
  
extra_gated_fields: I accept the above license agreement, and will use the Model complying with the set of use restrictions and sharing requirements: checkbox
Likes: 2 | All-time downloads: 0
Created: 2023-04-25 19:04:00