# The Question of Studying Information Entropy in Poetic Texts
## 3 The Usage of the Information Entropy in the Tasks of Determining the Author's Style  
Upon transmitting information over a certain channel, it is possible to quantify the randomness of the appearance of a particular symbol or a set of symbols. The measure of this randomness is called information entropy and is one of the basic concepts of information theory[PERSON]  
The concept of entropy as a measure of randomness was introduced by [PERSON] in his article \"Mathematical Theory of Communication\" [12]. Entropy, in this context, is understood as a measure of the average amount of information needed to record some event taken from the probability distribution of a random variable. There are the information entropies of different orders, which are calculated through the probabilities of using a certain number of symbols in line with the text. The entropy of the first order denotes the uncertainty of occurrence in the text of a separate unigram; the second is a separate bigram, and so on. The information entropy of any text is calculated using the formula:  
\[S=-\sum_{i=1}^{n}p_{i}\log_{2}p_{i}, \tag{1}\]  
in which \(S\) denotes the information entropy of power \(s\), \(n\) equals the number of unique substrings of length \(s\) contained within the text, \(i\) denotes the ordinal number assigned to the \(i\)-th unique substring of length \(s\), and \(p_{i}\) describes the probability of the \(i\)-th substring to appear within the text; therefore, it is a dimensionless quantity. Information entropy can be both the subject of research and some \"basic\" concept for other statistical methods, in particular, for measuring information gain (Kullback-Leibler divergence or relative entropy) and mutual information, which is used in machine learning, for example, when building decision trees. Another entropy characteristic is cross-entropy, which is a measure of the difference between two probability distributions, which can be interpreted as a measure of the difference between two texts. It is calculated using the following formula:  
\[H=-\sum_{i=1}^{n}p_{i}\log_{2}q_{i}, \tag{2}\]in which \(H\) denotes the cross-entropy value, \(n\) equals the number of unique symbols contained within both texts, \(i\) denotes the ordinal number assigned to the \(i\)-th symbol, and \(p_{i}\) and \(q_{i}\) denote the probability of the \(i\)-th symbol within the first and the second text, respectively. According to [PERSON], metric calculations give the deviation of the text from the \"average\" case, which makes it possible to evaluate the information capabilities of various texts, genres and authors [16]. This postulate is used by [PERSON] and [PERSON] in the monograph \"Methods of statistical analysis of literary texts\" [17], in which, in particular, the deviation of texts by the parameter of information entropy from the \"average\" entropy of the language is considered. According to the authors' conclusions, it seems that the functional S given by form (1) is more effective in the tasks of formal classification of texts than in determining authorship. However, despite the fact that the result of the functional S can be compared to the works of the analyzed authors, one-letter distributions do not allow the genre of a work to be determined. Later, in the same work, the researchers stated: \"the authors have significant differences in the frequency of usage of letter pairs\" (although these differences can rather be used in assessing the general trends). This serves as a confirmation of a postulate from the earlier work by the same authors: \"the selective distribution functions of texts by letters and letter pairs can serve as a tool for grouping works by authors and genres\" [18]. Based on this, the next assumption could be made when using a higher-order functional \(S\), where some general tendencies could be estimated both for authors and for the genres of their works. At the same time, these authors additionally suggested that the second-order functional \(S\) only allowed to clearly separate time epochs from each other.