Before revision:  In this work, we introduce BARThez, the first BART model for the Frenchlanguage (to the best of our knowledge). BARThez was pretrained on a very large monolingual French corpus from past research that we adapted to suit BART's perturbation schemes. Unlike already existing BERT-based French language models such as CamemBERT and FlauBERT , BARThez is particularly well-suited for generative tasks , since not only its encoder but also its decoder is pretrained.
After revision:  In this work, we introduce BARThez, the first large-scale pretrained seq2seq model for French. Being based on BART , BARThez is particularly well-suited for generative tasks , since not only its encoder but also its decoder is pretrained.