  Writers, poets, singers usually do not create their compositions in just one
breath. Text is revisited, adjusted, modified, rephrased, even multiple times,
in order to better convey meanings, emotions and feelings that the author wants
to express. Amongst the noble written arts, Poetry is probably the one that
needs to be elaborated the most, since the composition has to formally respect
predefined meter and rhyming schemes. In this paper, we propose a framework to
generate poems that are repeatedly revisited and corrected, as humans do, in
order to improve their overall quality. We frame the problem of revising poems
in the context of Reinforcement Learning and, in particular, using Proximal
Policy Optimization. Our model generates poems from scratch and it learns to
progressively adjust the generated text in order to match a target criterion.
We evaluate this approach in the case of matching a rhyming scheme, without
having any information on which words are responsible of creating rhymes and on
how to coherently alter the poem words. The proposed framework is general and,
with an appropriate reward shaping, it can be applied to other text generation
problems.
