# Using Deep-Learned Vector Representations for Page Stream Segmentation by Agglomerative Clustering
## 1 Introduction  
The task of page stream segmentation (PSS) is concerned with the segmentation of consecutive streams of pages into their original source documents. This task is of great importance to digitalization efforts in many different domains, such as archiving and the digitization of records from financial institutions [1, 2].  
The current state-of-the-art approaches to PSS view segmentation as a classification problem; binary classifiers are trained to detect whether or not a page is the first of a document [1, 2, 3, 4]. However, using this approach, efficiently incorporating information from the surrounding pages is complicated. This information might be highly relevant to the task, for example, for streams of documents where starting pages have very different layouts. Instead, we approach PSS as an agglomerative clustering problem in which each page is compared to its neighbouring pages and pages are represented as either finetuned or pretrained vectors. This way, we leverage the relationships between pages, instead of viewing them as stand-alone objects. We do this under the hypothesis that pages in adocument are more similar to each other than pages from other documents. This method has been used before, by [PERSON] and [PERSON] in 2002 [PERSON] and [PERSON] (2002), but they used a combination of rule-based representations and a classification model to determine similarity. We follow a similar strategy, but instead use representations obtained from deep learning models. Although both text and images are used by the current state-of-the-art approaches ([PERSON] et al., 2019; [PERSON] et al., 2020), early experiments in our research showed almost no improvement when including text and therefore we focus only on images.  
_Research Aim_ Our aim is to improve PSS performance by using restricted connectivity agglomerative clustering and vector representations of pages taken from finetuned and pretrained deep learning models. We will answer the following research questions:  
* Does the clustering performance improve when we use supervision? That is, with vector representations from a finetuned neural model ([PERSON] et al., 2019) instead of from a pretrained classification model (VGG16) ([PERSON] et al., 2020).
* How can we adapt agglomerative clustering to PSS, and does this lead to an improved performance?
* Does treating the number of documents, K, in a stream as given lead to a substantially higher classification performance?  
In our experiments, we found that neither clustering with pretrained models nor finetuned neural page representations outperformed current start of document page classification methods. Clustering with both representations is substantially more effective than the baseline, with finetuned vectors outperforming pretrained vectors. Having the number of documents \(K\) as part of the input, in our use case a realistic assumption, has a surprisingly significant positive effect.  
Although our results were negative, we still feel it has merit for the community to communicate them. We tried the most natural--in our experience, this is how humans do it--approach to PSS: agglomerative clustering. One simply keeps on adding pages to a document as long as they are similar to the pages already collected. However, through an extensive analysis we found that the underlying hypothesis--pages from the same document have higher similarity than pages from different documents--does not hold with state-of-the-art vector representations of pages. Maybe humans use a more versatile representation of pages, choosing and combining on the fly content, structure, layout, style and other clues in their similarity judgements.