CodeParrot-Multi 🦜 (small)
You can load the CodeParrot-Multi model and tokenizer directly in
from transformers import AutoTokenizer, AutoModelWithLMHead tokenizer = AutoTokenizer.from_pretrained("codeparrot/codeparrot-small-multi") model = AutoModelWithLMHead.from_pretrained("codeparrot/codeparrot-small-multi") inputs = tokenizer("def hello_world():", return_tensors="pt") outputs = model(**inputs)
or with a
from transformers import pipeline pipe = pipeline("text-generation", model="codeparrot/codeparrot-small-multi") outputs = pipe("def hello_world():")
The model was trained on the small Github code small after near deduplication, a subset of Github code dataset with the following settings:
The training was executed on 16 x A100 (40GB) GPUs. This setting amounts to roughly 58 billion tokens.
We evaluated the model on OpenAI's HumanEval benchmark which consists of programming challenges:
The pass@k metric tells the probability that at least one out of k generations passes the tests.
- Code: repository
- Downloads last month