We explore the use of semantic word embeddings [14, 16, 12] in text segmentation algorithms, including the C99 segmentation algorithm [3, 4] and new algorithms inspired by the distributed word vector representation. By developing a general framework for discussing a class of segmentation objectives, we study the effectiveness of greedy versus exact optimization approaches and suggest a new iterative refinement technique for improving the performance of greedy strategies. We compare our results to known benchmarks [18, 15, 3, 4] , using known metrics [2, 17] . We demonstrate state-of-the-art performance for an untrained method with our Content Vector Segmentation (CVS) on the Choi test set. Finally, we apply the segmentation procedure to an inthe-wild dataset consisting of text extracted from scholarly articles in the arXiv.org database.
REF have proposed the use of semantic word embeddings and a relaxed dynamic programming procedure.
Text Segmentation based on Semantic Word Embeddings
ArXiv
ArXiv
Computer Science