|
--- |
|
license: bsd-3-clause |
|
metrics: |
|
- code_eval |
|
pipeline_tag: text-generation |
|
tags: |
|
- code |
|
model-index: |
|
- name: instruct-codegen-16B |
|
results: |
|
- task: |
|
type: code-generation |
|
dataset: |
|
type: openai_humaneval |
|
name: HumanEval |
|
metrics: |
|
- name: pass@1 |
|
type: pass@1 |
|
value: 0.371 |
|
verified: false |
|
--- |
|
# Model Card for instruct-codegen-16B |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
Instruct-codegen-16B is an instruction following codegen model based on [Salesforce codegen-16B-multi](https://huggingface.co/Salesforce/codegen-16B-multi) , finetuned on a dataset of 250k instruction-following samples in the alpaca format. |
|
|
|
The data was not generated using any commercial LLM api. |
|
|
|
The model achieves a result of 37.1% pass@1 on the HumanEval benchmark. |
|
|
|
## Generation |
|
|
|
```python |
|
# pip install -q transformers |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
checkpoint = "sahil2801/instruct-codegen-16B" |
|
device = "cuda" |
|
tokenizer = AutoTokenizer.from_pretrained(checkpoint) |
|
model = AutoModelForCausalLM.from_pretrained(checkpoint).half().to(device) |
|
instruction = "Write a function to scrape hacker news." |
|
prompt = f"Below is an instruction that describes a task.\n Write a response that appropriately completes the request.\n\n ### Instruction:\n{instruction}\n\n### Response:" |
|
inputs = tokenizer(prompt, return_tensors="pt").to(device) |
|
outputs = model.generate(**inputs,temperature=0.3,do_sample=True,max_new_tokens=256) |
|
print(tokenizer.decode(outputs[0],skip_special_tokens=True)) |
|
``` |
