# A Hierarchical Orthographic Similarity Measure for Interconnected Texts Represented by Graphs
## 4 Representing a Textual Corpus as a Graph
### Graph Representation of a Textual Corpus
Additionally, the subset of epigrams is composed of the epigram groups with identifiers 2148, 2150, 4245, 2326, 3147, 3436, 4152, 4155, 5030, 5248, 6473, 1953, 6475, 1862, 1982, 2225, 3987, 2311, 31190, 3762, and 2054, resulting in a subset consisting of 500 epigrams. Because of the complex orthographic characteristics of Byzantine book epigrams, the original epigrams and verses from the selected datasets need to undergo preprocessing before we use them in our experiments in order to standardise the texts and reduce noise. First, all uppercase letters are converted to their lowercase counterparts. The next step involves the removal of any punctuation or other special characters. All accents and other diacritical marks are then stripped off, because of the frequent occurrence of unstandardised accentuation in these epigrams [41; 50]. In the fourth and final step, all vowels \(\pi,\upsilon,\eta\) and the diphthongs \(\varepsilon,\omega\) are systematically replaced with \(\iota\) in order to tackle with the phenomenon of _itacism_. ### Methods

In the experiments, two methods for text similarity are considered. The first method is the similarity measure proposed in this paper, with different values for various parameters. Before the similarity scores can be determined, the corpus must be transformed into a graph-based representation. The transformation of a set \(T\), representing a corpus of Byzantine book epigrams, into the required graph representation is accomplished throughAlgorithm 1. To successfully execute this algorithm, certain design decisions, related to the given corpus, must first be made. As discussed in the previous section, words, half verses and verses are the key textual elements contributing to the interconnected nature of Byzantine book epigrams. Given that half verses are not stored in the DBBE and no (well-performing) automatic method for their identification exists, the discourse units that make up the hierarchical decomposition of epigrams, and therefore define the structure of the graph, are \(L=[\texttt{Word},\texttt{Verse},\texttt{Epigram}]\). As a last step in defining the graph representation, the procedures for segmenting the text documents into smaller components need to be determined. Given that the DBBE stores epigrams as a list of verses, extracting the verses directly from the database is a straightforward process. Conversely, the identification of words is accomplished by partitioning the verses based on white spaces. Based on this graph representation, we have implemented our approach using two Neo4j ([[https://www.neo4j.org](https://www.neo4j.org)]([https://www.neo4j.org](https://www.neo4j.org)), accessed on 1 February 2024) databases: one for the subset of verses and one for the subset of epigrams. The proposed hierarchical similarity measure for Byzantine book epigrams is implemented as a Neo4j plugin (the Java code for this Neo4j plugin can be found at [[https://github.com/[PERSON]/DBBESimilarity](https://github.com/[PERSON]/DBBESimilarity)]([https://github.com/[PERSON]/DBBESimilarity](https://github.com/[PERSON]/DBBESimilarity)), accessed on 1 February 2024) and can be executed on the graph (database) by providing it with valid cost and threshold values for every level of the hierarchical decomposition. The second method we include in the experiments is a state-of-the-art soft similarity measure that combines the idea of token-based and character-based similarity [15; 16]. More specifically, this method computes the similarity between two texts by first transforming each text into a bag of tokens. Then, each token from the first bag is compared with each token from the second bag by means of a token matcher. The result of this comparison is a similarity score for both tokens. All token similarities are stored in a matrix that is used to compute a leximax-optimal assignment between tokens from both bags. Based on this assignment, a sequence of scores is obtained, and this sequence is aggregated into a final score by using a weighted minimum.