# Using Deep-Learned Vector Representations for Page Stream Segmentation by Agglomerative Clustering
## 3 Materials and Methods
### Method  
In this research, we compare our approach based on agglomerative clustering to a binary classification algorithm that uses the VGG16 architecture for image classification [6], both using pretrained vectors from this model, as well as vectors that were extracted after the model had been finetuned on the PSS task. In addition, we also compare our method to a non-learned baseline that uses only information on the mean number of documents in a stream.  
The VGG16 model is an image classification model that has been trained on data from the ImageNet classification task [18]. The model is often used to extract feature vectors from images by feeding images through the network trained on ImageNet and extracting the output from the last layer, where these embeddings can then be used in downstream tasks. As the VGG16 model was not trained on the specific dataset, these will be referred to as _pretrained embeddings_, and the intuition is that these vectors contain general information on the features of an image. However, the model can also be finetuned for a specific classification task, further training the model to extract features specific to the dataset. We will refer to embeddings extracted after this dataset-specific training step as _finetuned embeddings_.  
For both the clustering and the classification methods, streams are represented as binary vectors, where one indicates that the page is the first page of a document and a zero indicates that it is the continuation of a document.  
For the VGG16 model from [PERSON] and [PERSON], we used png images converted to the appropriate format for required the model. Then, we used those images to finetune the model--based on the study of [PERSON] and [PERSON] [2]--for 20 epochs with a batch size of 32 and a learning rate of \(10^{-5}\).  
Finetuned vector representations are taken from the last dense layer of our VGG16 model and pretrained vectors are taken from the last dense layer of the pretrained VGG16 model [6]. Finetuned and pretrained vectors have 256 and 4096 dimensions, respectively. These pretrained and finetuned embeddings will then be used as input to the agglomerative clustering method.  
For the implementation of the agglomerative clustering algorithm, we use the sklearn _AgglomerativeClustering_[19] implementation. We calculate the cosine distance for each page representation and its neighbouring pages, restrict connectivity to only neighbouring pages, normalize the distances and use the _average_ linkage function to perform clustering. Restricting the connectivity to neighbouring pages is necessary, as pages cannot belong to the same document if pages between them are not part of the same documents, given the fact that the pages are represented in a specific order.  
Both the use of cosine similarity and the _average_ linkage function were found to yield the best results in a hyperparameter search. When we performed agglomerative clustering, we assumed that the number of clusters is known and use it in the model.  
Since we assume the number of documents \(K\) in a stream to be known, we differentiate between _normal classification predictions_, in which we use a threshold of 0.5 to assign a page a positive label, and _Top-K classification predictions_, in which we set the \(K\) highest predicted scores in a stream to 1.  
For the experiments in this paper, we split the data into a training and test set, with 70% and 30% of the total streams, respectively.