What Makes a Concept Complex? Measuring Conceptual Complexity as a Precursor for Text Simplification (2021)

Abstract: Advancements within the field of text simplification (TS) have primarily been within syntactic or lexical simplification. However, conceptual simplification has previously been identified as another field of TS that has the potential to significantly improve reading comprehension. A first step to measuring conceptual simplification is the classification of concepts as either complex or simple. This research-in-progress paper proposes a new definition of conceptual complexity alongside a simple machine-learning approach that performs a binary classification task to distinguish between simple and complex concepts. It is proposed that this be a first step when developing new text simplification models that operate on a conceptual level.

## introduction text is a constant companion in our daily lives. without the ability to write down and record knowledge, the world and its people could arguably never have evolved to its current state. despite the big impact of text on our daily lives, around one in six adults in the united kingdom has poor literacy skills @xcite . in order to combat this problem, texts have long been simplified to improve understanding for all readers @xcite . with the advancements in the field of natural language processing (nlp), or more specifically, text simplification (ts), true progress has been made especially within the subfields of lexical and syntactic simplifications @xcite @xcite . a text can, however, be difficult to understand not only lexically or syntactically, but also conceptually. as of yet, conceptual text simplification has been paid significantly less attention than its two related subfields, although it has the potential to significantly improve reading comprehension. conceptual complexity is in itself a concept hard to grasp. definitions vary from "the importance of distinguishing between ideas in a text and fact burden of a text" @xcite to the amount of background knowledge a reader needs to in order to understand the concepts mentioned in the text and how they relate to each other @xcite . yet another definition refers to a "need for algorithms that can understand what a user needs to know before they can understand a second concept" @xcite . according to all these definitions, the amount of background knowledge is what essentially makes a text conceptually complex. however, they all fail to take into account reading processes that occur in the brain. while reading a text, a reader constantly has to chunk concepts together into larger order units in order to free up working memory space to avoid displacement @xcite . in order for this to happen, all concepts and their relations that are involved need to be present simultaneously in short-term memory (stm) @xcite . this is where differences between skilled and poor readers become obvious as skilled readers are able to suppress irrelevant information more easily and only store what is truly important, while also being generally better at chunking the relevant information which enables them to comprehend a text more fully than poor readers would @xcite . when reading a text, ideally a reader would chunk as much as possible to avoid displacement as this would mean that important concepts are lost from stm @xcite . it can therefore be argued that a text is conceptually complex if the working memory's capacity is frequently at its limit. we assume that closely related concepts are probably easier to chunk together, which is why relatedness should also play an important role. hence, our working definition of conceptual complexity is as follows: conceptual complexity is the complexity of a concept by itself, referring to level of abstractness and frequency of use in everyday language, as well as the complexity of a concept's context, meaning how complex its relations are to other concepts in its direct vicinity. ## related work one particular challenge within the field of conceptual ts is the estimation of conceptual text complexity. while many studies focus on automatic assessment of lexical and syntactic complexity @xcite @xcite , automatic approaches for measuring conceptual complexity are rare. one of the first automated tools that took conceptual complexity into account was the coh-metrix tool which uses latent semantic analysis semantic features and coreference indices alongside measures of lexical and syntactic complexity @xcite . more recently, the estimation of conceptual complexity has been attempted using knowledge graphs @xcite , as well as a mix of surface-based and shallow semantic features @xcite . recently, the focus of text simplification systems has shifted from simplifying a text to its easiest possible version to simplifications that are tailored to a specific reader's needs @xcite . so far, text simplification systems that take the concept level into account use mainly extractive approaches that eliminate irrelevant information @xcite . some concepts, however, might be essential to a text and cannot be eliminated which is why an abstractive approach needs to be used that generates new and simpler output. to identify which concepts are complex and need to be replaced by simpler substitutes, it is not sufficient to estimate the complexity of an entire text. rather, each individual concept needs to be assigned a complexity score that takes into account the concept's individual complexity, as well as its context complexity. ## methodology the proposed methodology contains several features that take into account a concept's relations with other concepts that occur in the same sentence, as well as how complex the concept is by itself. those features, as well as the model and annotation process are briefly described below. ## results the stacked model was built using all 1687 concepts of which 1373 were used for trainig while 157 were used respectively for valiation and testing. at this point in time, the model built on the subset of the data without the wordnet features (hypernyms, hyponyms, number of senses, path similarity) performs best, achieving a mean squared error of 0.062 and a f1-score of 0.7083. when training the model with a subset not containing the conceptnet features (number of relatives, relatedness), the results achieved were the worst among the three subsets. out of the 157 concepts in the test set, 15 were judged to be conceptually complex by the human annotator. while both the model built on all features, as well as the model built on the subset without conceptnet features found around the same amount of complex concepts, 16 and 14, they included many false positives. in contrast, the model without wordnet features classified a total of 22 concepts as complex, yet included more actually complex concepts. ## discussion as this is a work-in-progress paper and, as of yet, there are no finalised results, the discussion will be primarily focused on the weaknesses of this methodology rather than relating the findings to previous works in the field. so far, due to computational restraints, the dataset used was fairly small. in the future, we plan on increasing the size of the dataset. furthermore, the increased dataset would need to be annotated by more than one annotator in order to decrease the possibility of the one annotator having a bias that is being transferred onto the data. in terms of feature engineering, other knowledge graphs and ontologies should be explored to possibly replace those features based on the wordnet ontology. a big shortcoming of wordnet is that its depth varies in different parts. this leads to some concepts having less hyper -or hyponyms than others and, hence, the possibility of these being falsely classified as simple when they are in fact complex. furthermore, the predictions on the test set show that each of the three models has a slight tendency to falsely classify longer words as more complex. future experiments should experiment with excluding the word-length feature to see whether this can improve results.