File size: 1,911 Bytes
bdfe289 aa716f9 bdfe289 397ecac bdfe289 0bb275d bdfe289 0bc507d bcfc61d bdfe289 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
language: fr
license: mit
tags:
- causal-lm
- fr
datasets:
- gustavecortal/fr_covid_news
---
### GPT-J COVID-19 French News with 8-bit weights
This is a version of Cedille's GPT-J ([fr-boris](https://huggingface.co/gustavecortal/fr-boris-8bit)) with 6 billion parameters fine-tuned on [COVID-19 French News dataset](https://huggingface.co/datasets/gustavecortal/fr_covid_news) to generate French headlines related to COVID-19.
You can generate the model in colab or equivalent desktop gpu (e.g. single 1080Ti) as the model has 8-bit weights. Inspired by [GPT-J 8bit](https://huggingface.co/hivemind/gpt-j-6B-8bit).
Here's how to run it: [](https://colab.research.google.com/drive/1lMja-CPc0vm5_-gXNXAWU-9c0nom7vZ9)
This model can be easily loaded using the `GPTJForCausalLM` functionality:
```python
from transformers import GPTJForCausalLM
model = GPTJForCausalLM.from_pretrained("gustavecortal/gpt-j-fr-covid-news")
```
Remember, you have to Monkey-Patch the model before loading it (see Colab above).
## One thousand AI-generated French headlines related to COVID-19
How not to be disoriented in a pandemic era when faced with an immense flow of information? [This page](https://gustavecortal.com/project/covid) features one thousand AI-generated French headlines related to COVID-19.
## fr-boris
Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the [mesh-transformer-jax](https://github.com/kingoflolz/mesh-transformer-jax) codebase.
Boris was trained on around 78B tokens of French text from the [C4](https://huggingface.co/datasets/c4) dataset.
## Links
* [Gustave Cortal](https://twitter.com/gustavecortal) |