# Evaluation of the Coherence of Polish Texts Using Neural Network Models
## 2 Materials and Methods
### Coherence Estimation Models for The Polish Language
#### 2.2.2 LSTM-Based Coherence Estimation Model  
According to the performed analysis of state-of-the-art coherence estimation models, LSTM neural networks are commonly used in order to represent either sentences or whole texts. Moreover, consecutive convolution operations are applied for the purpose of revealing main semantic components within input data. It should be mentioned that all considered models were tested on English corpora. As the Polish language belongs to another family group, it is advisable to verify if the usage of LSTM layers is suitable for the processing of Polish texts at all while evaluating output coherence. The key question consists in the order of words' processing, namely, whether it should be taken into account for the coherence evaluation of Polish corpora. Thus, it is suggested to design an LSTM-based coherence estimation model without additional CNN layers in order to estimate the expediency of the analysis of sentences in a word-by-word manner. Let us consider the main components of a suggested coherence estimation model. The input of the model is represented as a text \(T=\{s_{1},s_{2},\ldots,s_{N}\}\), the output value interprets the estimation of the coherence of the text. Let us divide the input text \(T\) into a set of cliques, i.e., ordered groups of sentences where each group incorporates \(L\) sentences. Such a set is created by the sliding of a \"window\" (width of \"window\" is equal to \(L\)) through the text with an ordinary step. The cardinality of the created set of cliques is equal to \(N-L-1\). According to the experimental search for the optimal value of \(L\) that was provided in the paper [20], it is advisable to set the following width of the \"window\": \(L=3\). Such a search was performed by taking different values of \(L\) during the training and inference of a neural network. The further increase of this parameter allows taking into account more sentences simultaneously; however, it does not provide a significant accuracy improvement while solving the sentence ordering task. It should be mentioned that the corresponding experimental selection of \(L\) was performed according to the analysis of an English corpus. However, this feature should not have a severe dependency on a language: despite the different structure of the sentences of English and Polish languages, the way to group sentences around the key idea of a text should remain the same for all persons. The size of a \"window\" may depend on the style of a text (e.g., the long-distance sentences of a step-by-step manual can be more connected than the sentences of some  
Figure 4: The examples of the semantic similarity graphs built by different approaches. fiction that describes nature, environment, etc.). As both English [20] and Polish corpora consist of news reports, it was decided to set the same value \(L=3\) during the designing and experimental verification of the model. The text \(T\) is represented in the following way:  
\[T=\{\langle s_{1},s_{2},s_{3}\rangle,\langle s_{2},s_{3},s_{4}\rangle,\ldots, \langle s_{N-2},s_{N-1},s_{N}\rangle\} \tag{5}\]  
The coherence of the entire text \(T\) is calculated by the equation:  
\[Coh(T)=\prod_{c\in T}p(y_{c}=1), \tag{6}\]  
where \(p(y_{c}=1)\) is the probability of the coherence of an input clique \(c\in T\). Thus, the key task of the model implies the estimation of the probability of coherence for all text's cliques. This task is performed using an LSTM-based neural network. Figure 5 shows the main components of the designed model. Firstly, all input sentences are passed through a preprocessing block. The purpose of the preprocessing block is to convert an input sentence \(s_{i},i=1,2,3\) to a sequence of word vectors \(\{\mathbf{w}_{1}^{i},\mathbf{w}_{2}^{i},\mathbf{w}_{3}^{i}\ldots\}\). Despite the ability to remove stop-words from input texts, it is proposed to use all tokens for the vector representation of a sentence using a pre-trained ELMo embedding.