metadata
datasets:
- vikp/python_code_instructions_filtered
This is code llama 7b finetuned for one epoch on a set of python code and instructions. Scores .512 in humaneval with greedy decoding (matched to code llama pass@1).
To use in inference, you'll need to set trust_remote_code = True to pick up the right rope theta value:
from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("vikp/code_llama_7B_hf")
model = AutoModelForCausalLM.from_pretrained("vikp/instruct_llama_7b", trust_remote_code=True)
text = tokenizer.bos_token + """\
import socket
def ping_exponential_backoff(host: str):""".lstrip()
tokens = tokenizer(text, return_tensors="pt")
output = model.generate(**tokens, max_new_tokens=128, do_sample=True, temperature=.1, top_p=1.0)
print(tokenizer.decode(output[0], skip_special_tokens=True).strip())
You can duplicate benchmark results with the bigcode eval harness:
git clone https://github.com/bigcode-project/bigcode-evaluation-harness.git
cd bigcode-evaluation-harness
pip install -e .
accelerate launch main.py \
--model vikp/instruct_llama_7b \
--tasks humaneval \
--max_length_generation 1024 \
--temperature 0 \
--do_sample False \
--n_samples 1 \
--precision fp16 \
--allow_code_execution \
--save_generations \
--use_auth_token \
--trust_remote_code