roberta-poetry-sadness-crpo

This model is based on the RoBERTa base model (125 M parameters) fine-tuned for 20 epochs on a poetry dataset of 37 MB. This dataset was extracted from the Gutenberg Poetry Corpus using an automatic classifier for sadness.

The model replaces a masked word, indicated by the <mask> tag, with a word associated with sadness, while preserving fluency. Caution: the emotion (here, sadness) only biases the choice of words with respect to the base model, so do not expect to find only words strongly associated to this emotion.

This model was trained by Teo Ferrari as part of his Bachelor thesis at HEIG-VD, supervised by Andrei Popescu-Belis. The model is described in "GPoeT: a Language Model Trained for Rhyme Generation on Synthetic Data" and is used in the CR-PO system for interactive poem generation, along with several other models for specific topics or emotions.