metadata

pipeline_tag: text-generation
license: apache-2.0
tags:
  - text generation
  - Deci AI
  - DeciCoder
programming_language:
  - Java
  - JavaScript
  - Python
  - Rust
  - Go
  - C++
  - C
  - C#
metrics:
  - code_eval
inference: true
widget:
  - text: 'def print_hello_world():'
    example_title: Hello world
    group: Python
model-index:
  - name: DeciCoder-6b
    results:
      - task:
          type: text-generation
        dataset:
          type: nuprl/MultiPL-E
          name: MultiPL-HumanEval (Python)
        metrics:
          - name: pass@1
            type: pass@1
            value: 0.34
            verified: false
      - task:
          type: text-generation
        dataset:
          type: nuprl/MultiPL-E
          name: MultiPL-HumanEval (JavaScript)
        metrics:
          - name: pass@1
            type: pass@1
            value: 0.29
            verified: false
      - task:
          type: text-generation
        dataset:
          type: nuprl/MultiPL-E
          name: MultiPL-HumanEval (Java)
        metrics:
          - name: pass@1
            type: pass@1
            value: 0.3
            verified: false
datasets:
  - bigcode/starcoderdata

Model Card for DeciCoder 6B

DeciCoder 6B is a 6 billion parameter decoder-only code completion model trained on the Python, Java, Javascript, Go, Rust, C++, C, and C# subset of Starcoder Training Dataset.. The model uses variable Grouped Query Attention and has a context window of 4096 tokens. It was trained using a Fill-in-the-Middle training objective. The model's architecture was generated by Deci's proprietary Neural Architecture Search-based technology, AutoNAC.

Model Details

Developed by: Deci
Model type: DeciCoder is an auto-regressive language model based on the transformer decoder architecture, using variable Grouped Query Attention.
Language(s): Python, Java, JavaScript, Go, Rust, C++, C, C#
License: Model checkpoints are licensed under the Apache 2.0

Model Architecture

Parameters	Layers	Heads	Sequence Length	GQA num_key_value_heads	Hidden Size
6B	32	32	4096	Variable	4096

Decoder layer: Variable Grouped Query Attention. Grouped Query Attention was introduced in Ainslie et al., 2023
Position Embeddings: Rotary Position Embeddings Su et al., 2021

Uses

The model is intended to do single/multiline code completion from a context window of up to 4096k tokens. It is not an instruction model and commands like "Write a function that computes the absolute value of an integer," won't yield the desired results. A more effective approach is to frame instructions in the style of source code comments (e.g. # this function calculates the absolute value of an integer) or to present a function signature and docstring, enabling the model to complete the function's body.

How to Use

# pip install -q transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "Deci/DeciCoder-6b"
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, trust_remote_code=True).to(device)

inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

### Attribution

DeciCoder was trained on StarCoder Training Dataset, filtered for
Python, Java, JavaScript, Rust, Go, C++, C, and C#. For additional information, please
refer to [https://huggingface.co/datasets/bigcode/starcoderdata](https://huggingface.co/datasets/bigcode/starcoderdata).

Limitations

The model has undergone training with source code from Python, Java, JavaScript, Go, Rust, C++, C, and C#. While the primary language in the source is English, it does contain other languages. Therefore, the model can produce code snippets given some context. However, there's no assurance that the resulting code will function as expected. It might be suboptimal, contain bugs, or even exploits.

Evaluation

Below are DeciCoder's pass@1 on MultiPL HumanEval scores

Python	JavaScript	Java	C++	C#	Rust	Go	C
33.5%	29.3%	30.3%	29.93%	20.31%	20.5%	77.47%	xx%

Runtime Benchmarks

Inference Tool/Hardware	Qualcomm AI 100 (tokens/sec)
Infery LLM	xxx

Throughput (tokens/sec) - Measured with an optimal batch size of 96

Documentation

Notebook CHANGE
Blog post: Introducing DeciCoder: The New Gold Standard in Efficient and Accurate Code GenerationCHANGE
Questions:Feel free to contact us via our Discord Community!CHANGE

How to Cite

Please cite this model using this format.

@misc{DeciFoundationModels,
title = {DeciCoder},
author = {DeciAI Research Team},
year = {2023}
url={[https://huggingface.co/deci/decicoder-6b](https://huggingface.co/deci/decicoder-6b)},
}