---
license: mit
language:
- en
pipeline_tag: fill-mask
library_name: transformers
widget:
- text: "This morning, the CEO was <mask>."
  example_title: "Example 1"
- text: "Yesterday, all the students were <mask> in the park."
  example_title: "Example 2"
- text: "All the children seemed <mask>."  
  example_title: "Example 3"
- text: "I opened the door and found a <mask> behind it."
  example_title: "Example 4"
- text: "We went to see the <mask> movie."
  example_title: "Example 5"
---
# roberta-poetry-life-crpo

This model is based on the [RoBERTa base model](https://huggingface.co/roberta-base) (125M parameters) 
fine-tuned for 20 epochs on a poetry dataset of 14 MB.  This dataset was extracted from 
the [Gutenberg Poetry Corpus](https://github.com/aparrish/gutenberg-poetry-corpus) using an automatic classifier 
for poems in relation with the topic of **life and death**.

The model replaces a masked word, indicated by the `<mask>` tag, with a word associated with **life and death**, while preserving fluency.
Caution: the topic (here, **life and death**) only biases the choice of words with respect to the base model, but do not expect to find
only words strongly associated to this topic.

This model was trained by [Teo Ferrari](https://www.linkedin.com/in/teo-ferrari-0a4009176/) 
as part of his Bachelor thesis at [HEIG-VD](https://gaps.heig-vd.ch/public/diplome/rapports.php?id=6763), 
supervised by [Andrei Popescu-Belis](http://iict-space.heig-vd.ch/apu/).
The model is described in "[GPoeT: a Language Model Trained for Rhyme Generation on Synthetic Data](https://aclanthology.org/2023.latechclfl-1.2/)"
and is used in the [CR-PO](https://github.com/heig-iict-ida/crpo) system for [interactive poem generation](https://aclanthology.org/2022.lrec-1.377),
along with several other models for specific topics or emotions.