File size: 1,503 Bytes
1e77c56
ce77f8f
98b32c1
d97087d
 
 
1e77c56
 
4fa0d0f
98b32c1
 
ce77f8f
005c24f
4fa0d0f
 
 
 
 
 
 
 
 
 
 
 
fda9fad
4fa0d0f
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
[CodeParrot](https://huggingface.co/lvwerra/codeparrot) uses GPT-2 architecture with BPE tokenizer trained on Python code from the training split of the data, and a context length of 1024. This model was released as an educational tool for training large language models from scratch on code, with detailed tutorials and descriptions of the training process. It makes use of 🤗 [`accelerate`](https://huggingface.co/docs/accelerate/index) for distributed training and mixed precision. See this [blog](https://huggingface.co/blog/codeparrot) and [repo](https://github.com/huggingface/transformers/tree/main/examples/research_projects/codeparrot) for more details.

<div align="center">

|Model | # parameters | 
|   -   |   -  | 
| [codeparrot-small](https://huggingface.co/lvwerra/codeparrot-small) | 110M |
| [codeparrot](https://huggingface.co/lvwerra/codeparrot) | 1.5B |

</div>


You can load the model and tokenizer directly from 🤗 [`transformers`](https://huggingface.co/docs/transformers/index):

```python
from transformers import AutoTokenizer, AutoModelWithLMHead
  
tokenizer = AutoTokenizer.from_pretrained("lvwerra/codeparrot")
model = AutoModelWithLMHead.from_pretrained("lvwerra/codeparrot")

inputs = tokenizer("def hello_world():", return_tensors="pt")
outputs = model(**inputs)

```

You can also use `pipeline` to generate code:

```python
from transformers import pipeline

pipe = pipeline("text-generation", model="lvwerra/codeparrot")
outputs = pipe("def hello_world():")
```