# Using Deep-Learned Vector Representations for Page Stream Segmentation by Agglomerative Clustering
## 4 Results and Discussion
### RQ2: What Goes Wrong with Clustering and Can We Repair It? Since the finetuned representations are taken from a classifier, these representations reflect its prediction; that is, whether or not a page starts a new document (class 1 or 0). This is supported by Table 2, where we see that the cosine distance between pages of the same class is lower than the cosine distance between pages of the other class. Two of these four scenarios cause a problem: we want (1, 1) to be clustered into different groups, which will not occur given that they have a low distance. Conversely, (1, 0) should be clustered together, as the 0 here represents the second page of a new document. We call this the _similarity problem_ and attempt to solve it by using the \"switch\" cluster method. Here, if a distance between two consecutive pages \(d(p_{i-1},p_{i})\) is higher than a threshold--set to the _Kth_ highest distance in the stream with \(K\) is the number of documents--we set (\"switch\") as the distance between the next pair of pages, \(p_{i}\) and \((p_{i+1})\) to \(2\cdot\mu_{d}-d(p_{i},p_{i+1})\), where \(\mu_{d}\) is the average distance, clipping the distance to zero if the resulting distance is negative. In essence, this formula causes page pairs with a high distance to have a lower distance, and page pairs with a low distance to have a higher distance. The intuition behind this is \begin{table}
\begin{tabular}{l c c} \hline \hline
**Prediction** & **1** & **0** \\ \hline
1 & 0.25 & 0.76 \\
0 & 0.73 & 0.25 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Mean normalized cosine distances of finetuned representations grouped by classifier predictions. Figure 2: GradCam visualization of pretrained and finetuned vectors of a starting (left) and a non-starting (right) page. (**a**) Finetuned vector of starting page. (**b**) Finetuned vector of non-starting page. (**c**) Pretrained vector of the same (starting) page. (**d**) Pretrained vector of the same (non-starting) page. that if the distance between two pages is high, it is either a transition from a boundary page to a non-boundary page or vice versa. After the first page of the stream, we will always encounter the (0, 1) transition first; thus, this transition is indicated by encountering a high similarity. The next transition is then either (1, 0) or (1, 1), the two problematic cases we described above. By switching this distance, we obtain the desired low/high distance in both cases. Figure 3 shows an example of the switch method for an example stream. We can see the effectiveness of this method back in the results in Table 1; the highest score using clustering was obtained with the switch. Clustering still suffers from another problem introduced by using the switch method, in which one mistake is multiplied into many. We call this the _entry problem_. In explanation, Figure 4 contains the binary predictions versus the gold standard of an example from our test set. We only consider the part of the stream indicated by the 0s and 1s. The first two pages are in reality both start pages \((1,1)\), but the switched distance is not enough to classify them apart and thus they are grouped together in the prediction \((1,0)\). What follows after this first mistake is that the cluster model is able to properly identify document boundaries, but because the _entry value_ of this chain was wrong all predictions are exactly the opposite of what they should be (the 40 page long red part in the figure). We were not able to solve both this problem and the _similarity problem_, as we cannot predict where the cluster model is wrong. The impact of this problem on the scores is large. We consider the _entry problem_ to be the case when at least 11 pages with a minimum of four different document boundaries are predicted to be the opposite of the ground truth. With this definition, the entry problem is the cause of 1107 wrongly predicted pages (4.4% of all pages in the test set).