Edit model card

GPT-J COVID-19 French News with 8-bit weights

This is a version of Cedille's GPT-J (fr-boris) with 6 billion parameters fine-tuned on COVID-19 French News dataset to generate French headlines related to COVID-19.

You can generate the model in colab or equivalent desktop gpu (e.g. single 1080Ti) as the model has 8-bit weights. Inspired by GPT-J 8bit.

Here's how to run it: colab

This model can be easily loaded using the GPTJForCausalLM functionality:

from transformers import GPTJForCausalLM
model = GPTJForCausalLM.from_pretrained("gustavecortal/gpt-j-fr-covid-news")

Remember, you have to Monkey-Patch the model before loading it (see Colab above).

One thousand AI-generated French headlines related to COVID-19

How not to be disoriented in a pandemic era when faced with an immense flow of information? This page features one thousand AI-generated French headlines related to COVID-19.


Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the mesh-transformer-jax codebase.

Boris was trained on around 78B tokens of French text from the C4 dataset.


Downloads last month

Dataset used to train gustavecortal/gpt-j-fr-covid-news