## 2 Segmentation-Enhanced Topic Modeling for Automated Text Annotation
### Text Segmentation with TopicDiff-LDA
In each iteration, \(b\) sets the candidate segment boundary, so that segments \(segA\) and \(segB\) (to the left and right of \(b\), respectively) can be constructed. ## 3 Text Segmentation Experiments

### Data

To evaluate the performance of the proposed topic segmentation algorithm, experiments have been conducted, using the Choi dataset [43]. The set was compiled by the author for benchmarking purposes. It contains 700 synthetic documents created by concatenating sentences from different text samples of the Brown corpus. The Brown corpus comprises 374 \"informative prose\" and 126 \"imaginative prose\" English documents published in 1961. The Choi dataset is structured into four subsets, depending upon the number of sentences in a segment. Segments were created by randomly choosing a document from the Brown corpus, followed by selecting first \(N\) sentences from that document (\(N\) selected at random from a predefined range). Each synthetic document consists of exactly ten segments. Segment length varies, as specified in Table 1. Segments are unlabeled, and only segment boundaries are provided in the set description. ### Experiments

Eleven LDA models, each with a different number of topics \(m\), were trained over 1000 iterations on the Choi dataset. The Tomotopy toolkit was used ([[https://github.com/bab2](https://github.com/bab2) min/tomotopy]([https://github.com/bab2](https://github.com/bab2) min/tomotopy), last accessed on 26 April 2021). The Dirichlet priors were set to \(\alpha=50/m\) and \(\beta=0.01\), as suggested in [44]. With \(m\) being incremented from 20 to 220, the optimal coherence score \(c_{v}^{opt}\) was located at \(m=140\) (see Figure 3). The sliding window width \(h\) was set at 3, corresponding to the shortest known (or estimated) segment length. Through preliminary experiments it was found that selecting an initial seed value \(t_{seed}~{}\sim~{}2\times(1-\mu)\), where \(\mu\) is the average highest topic probability of the generated LDA model, results in an improved performance when \(t_{opt}\) is searched. For segmentation of the Choi data, \(t_{seed}=1.85\) was used. \begin{table}
\begin{tabular}{l c} \hline Number of documents & 700 \\ Number of unique tokens & 5210 \\ Average document length & 83 (1910 tokens) \\ Number of segments per document & 10 \\ Segment length (number of documents) & 3–11 (400), 3–5 (100), 6–8 (100), 9–11 (100) \\ \hline \end{tabular}
\end{table} 
Table 1: Summary of the data used in the segmentation experiments (document and segment lengths are given in sentences). The TopicDiff-LDA algorithm was run on the 700 documents. The obtained segment quality was evaluated using the _Pk_ metric [45]. _Pk_ is a probabilistic measure computed by running a sliding window over sentences in the hypothesis (_hyp_) and reference (_ref_) segments. It counts the number of disagreements in sentences being in the same or different sections. Formally:

\[Pk(ref,hyp)=\frac{1}{n-k}\sum\limits_{i=1}^{n-k}\left(\delta(ref_{i},ref_{i+k}) \bigoplus\delta(hyp,hyp_{i+k})\right)\]

where \(n\) is the number of sentences, and \(k\) sets a fixed window size calculated as half of the average gold-standard segment size. In a single window, the disagreement of _ref_ segment relative to _hyp_ segment is calculated as an exclusive disjunction (denoted with \(\oplus\)) of the sentence separation \(\delta\big{(}s_{i},\;s_{j}\big{)}\):

\[\delta\big{(}s_{i},\;s_{j}\big{)}=\left\{ \begin{array}{ll}1,&\text{if sentences $s_{i}$ and $s_{j}$ are assigned to the same segment;}\\ 0,&\text{otherwise.}\end{array} \right.\]

Table 2 provides _Pk_ obtained for TopicDiff-LDA in comparison with results reported in the literature for the same data (lower scores correspond to better performance). ## 4 Case Study

Experiments described in the previous section have demonstrated that the proposed algorithm outperforms other unsupervised approaches but lags behind the supervised and distant-supervised TS methods when run on the artificially created data.