  We introduce SummScreen, a summarization dataset comprised of pairs of TV
series transcripts and human written recaps. The dataset provides a challenging
testbed for abstractive summarization for several reasons. Plot details are
often expressed indirectly in character dialogues and may be scattered across
the entirety of the transcript. These details must be found and integrated to
form the succinct plot descriptions in the recaps. Also, TV scripts contain
content that does not directly pertain to the central plot but rather serves to
develop characters or provide comic relief. This information is rarely
contained in recaps. Since characters are fundamental to TV series, we also
propose two entity-centric evaluation metrics. Empirically, we characterize the
dataset by evaluating several methods, including neural models and those based
on nearest neighbors. An oracle extractive approach outperforms all benchmarked
models according to automatic metrics, showing that the neural models are
unable to fully exploit the input transcripts. Human evaluation and qualitative
analysis reveal that our non-oracle models are competitive with their oracle
counterparts in terms of generating faithful plot events and can benefit from
better content selectors. Both oracle and non-oracle models generate unfaithful
facts, suggesting future research directions.
