File size: 1,147 Bytes
430b9fe eaaa377 430b9fe 12bcf78 5b5f9b2 66fb188 5b5f9b2 30ecc01 5b5f9b2 30ecc01 5b5f9b2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
---
license: mit
datasets:
- wikipedia
- IlyaGusev/gazeta
language:
- ru
library_name: transformers
---
# ruGPT-Neo 1.3B [IN TRANING, NOT FINAL CHECKPOINT]
## Model Description
ruGPT-Neo 1.3B is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. ruGPT-Neo refers to the class of models, while 1.3B represents the number of parameters of this particular pre-trained model.
## Training procedure
This model was trained on the wiki, gazeta summorization, for 38k steps, on 1*v100 gpu, still training . It was trained as a masked autoregressive language model, using cross-entropy loss.
### How to use
You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:
```py
>>> from transformers import pipeline
>>> generator = pipeline('text-generation', model='AlexWortega/rugpt-neo-1.3b')
>>> generator("Как какать? Ответ:", do_sample=True, min_length=50)
[{'generated_text': 'Как какать? Ответ: Cпустите штаны и покакайте, затем воспользуйтесь бумагой'}]
``` |