File size: 845 Bytes
fde4518 bfb82c3 fde4518 bfb82c3 fde4518 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
---
widget:
- text: Em uma bela manhã de
- text: Em uma cidade tão grande como
- text: Maria e Joana são
license: mit
datasets:
- mc4
language:
- pt
metrics:
- perplexity
library_name: transformers
pipeline_tag: text-generation
---
# OPT-125M finetuned Portuguese
Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.
###### Hyper-parameters
- learning_rate = 5e-5
- batch_size = 32
- warmup = 500
- seq_length = 512
- num_train_epochs = 2.0
With an A100 with 40GB of RAM, the training took around 3 hours
**Perplexity:** 9.4
## Sample Use
```python
from transformers import pipeline
generator = pipeline('text-generation', model='Mirelle/opt-125M-pt-br-finetuned', max_length=100, do_sample=True)
generator("Em uma bela manhã de")
``` |