license: mit
language:
- en
pipeline_tag: fill-mask
library_name: transformers
widget:
- text: This morning, the CEO was <mask>.
example_title: Example 1
- text: Yesterday, all the students were <mask> in the park.
example_title: Example 2
- text: All the children seemed <mask>.
example_title: Example 3
- text: I opened the door and found a <mask> behind it.
example_title: Example 4
- text: We went to see the <mask> movie.
example_title: Example 5
roberta-poetry-happiness-crpo
This model is based on the RoBERTa base model (125 M parameters) fine-tuned for 20 epochs on a poetry dataset of 51 MB (373k lines, 7.8M words). This dataset was extracted from the Gutenberg Poetry Corpus using an automatic classifier for happiness.
The model replaces a masked word, indicated by the <mask> tag, with a word associated with happiness, while preserving fluency.
Caution: the emotion (here, happiness) only biases the choice of words with respect to the base model, but do not expect to find
only words strongly associated to this emotion.
This model was trained by Teo Ferrari as part of his Bachelor thesis at HEIG-VD, supervised by Andrei Popescu-Belis. The model is described in "GPoeT: a Language Model Trained for Rhyme Generation on Synthetic Data" and is used in the CR-PO system for interactive poem generation, along with several other models for specific topics or emotions.