README.md

---
datasets:
- mc4
language:
- pt
metrics:
- perplexity
library_name: transformers
---

This model is a Portuguese fine-tuned version of the [facebook/opt-125m](https://huggingface.co/facebook/opt-125m). It has undergone additional causal language modeling pre-training with a context size of 512, using an extra 300 million tokens in Portuguese (sampled from mc4). The Wandb report is publicly available at [here](https://api.wandb.ai/links/thiagolaitz1/ths2zi4c). The code for training using Colab pro (A100 - 40GB) can be found [here](https://github.com/thiagolaitz/IA368-search-engines/blob/main/Project%2004/opt_125m_pt_finetuning.ipynb). The total cost for training this model was R$17.40 or $3.37 USD (as of March 2023).

Deterministic use:
```python
from transformers import pipeline

generator = pipeline('text-generation', model="thiagolaitz/opt-125m-pt-finetuned", max_length=30)
generator("Eles brincaram o dia inteiro sob o sol quente, mas")
# Output: Eles brincaram o dia inteiro sob o sol quente, mas não se deixaram levar pelo sol.
```

Top-k sampling:
```python
from transformers import pipeline

generator = pipeline('text-generation', model="thiagolaitz/opt-125m-pt-finetuned", do_sample=True, max_length=30)
generator("Eles brincaram o dia inteiro sob o sol quente, mas")
```