Before revision:  We show that pre-trained Sentence-BERT embeddings outperform all other approaches, however the benefit over n-gram models is marginal. By closely examining the results of simple models, we also uncover many unexpected properties of pull quotes that should serve as inspiration for future approaches.
After revision:  By examining the contributions of individual features and embedding dimensions from these models, we also uncover many unexpected properties of pull quotes that should serve as inspiration for future approaches.