  Aiming to generate a set of keyphrases, Keyphrase Generation (KG) is a
classical task for capturing the central idea from a given document. Typically,
traditional KG evaluation metrics are only aware of the exact correctness of
predictions on phrase-level and ignores the semantic similarities between
similar predictions and targets, which inhibits the model from learning deep
linguistic patterns. In this paper, we propose a new fine-grained evaluation
metric that considers different granularity: token-level $F_1$ score, edit
distance, duplication, and prediction quantities. For learning more recessive
linguistic patterns, we use a pre-trained model (e.g., BERT) to compute the
continuous similarity score between predicted keyphrases and target keyphrases.
On the whole, we propose a two-stage Reinforcement Learning (RL) training
framework with two reward functions: our proposed fine-grained evaluation score
and the vanilla $F_1$ score. This framework helps the model identifying some
partial match phrases which can be further optimized as the exact match ones.
Experiments on four KG benchmarks show that our proposed training framework
outperforms the traditional RL training frameworks among all evaluation scores.
In addition, our method can effectively ease the synonym problem and generate a
higher quality prediction.
