Simplification by Lexical Deletion (2023)

Abstract: Lexical simplification traditionally focuses on the replacement of tokens with simpler alternatives. However, in some cases the goal of this task (simplifying the form while preserving the meaning) may be better served by removing a word rather than replacing it. In fact, we show that existing datasets rely heavily on the deletion operation. We propose supervised and unsupervised solutions for lexical deletion based on classification, end-to-end simplification systems and custom language models. We contribute a new silver-standard corpus of lexical deletions (called SimpleDelete), which we mine from simple English Wikipedia edit histories and use to evaluate approaches to detecting superfluous words. The results show that even unsupervised approaches (TerseBERT) can achieve good performance in this new task. Deletion is one part of the wider lexical simplification puzzle, which we show can be isolated and investigated.

## introduction lexical simplification aims to identify words that are too difficult for readers and apply an intervention that will enable them to better understand the term. in almost all lexical simplification studies, the primary intervention is the replacement operation, in which the target word is substituted for a simpler alternative @xcite . some studies have also considered applying an addition operation by adding a definition or explanation of the difficult term @xcite . we propose that a new operation should be considered for lexical simplification, which is that of deletion. lexical deletion is a vital element in making texts easier to understand. a prior analysis of simple english wikipedia showed that 47% of sentence simplifications involved deleting words @xcite . to better understand the role of deletions in modern ts datasets, we performed reference delete add keep turk corpus 26.82 20.19 53.00 pwkp 37.42 23.17 39.41 an analysis of reference datasets in the easse package for text simplification evaluation @xcite . the results in ## related work simplification by deletion has been studied as an emergent property of systems which perform simplification through sentence to sentence translation @xcite @xcite . these systems are trained on parallel datasets that contain a variety of operations including deletion @xcite . it is also possible to force these systems to provide certain types of operations through the use of control tokens @xcite . sentence deletion has also been studied as a means of discourse simplification, where the aim is to drop redundant sentences in a passage @xcite . this is similar to the task of extractive summarisation @xcite where the task is to only retain the relevant sentences. conversely, lexical deletion is similar to sentence compression @xcite , where the goal is to remove all redundant information from a sentence. typical evaluation of simplification has focussed on either matching n-grams @xcite (e.g., bleu-score @xcite ) or analysing the lexical simplification pipeline @xcite . sari score @xcite has become dominant in the evaluation of text simplification, however it is designed for full sentence simplification, and does not explicitly measure text coherence. nonetheless, sari score has been used to measure the ability of a system to perform deletions @xcite . the recent shared task on multilingual lexical simplification at the tsar workshop @xcite popularised several metrics for the evaluation of lexical simplification, including accuracy@k@top1 and mean average precision (map@k). these evaluation methods are appropriate when a ranked list of candidates is produced. our work leverages simple english wikipedia edit histories, drawing on a long line of prior simplification studies to generate corpora using this resource. simple english wikipedia has been shown to contain the type of language that is useful for simplification models @xcite . the edit histories have been used previously to mine examples @xcite and corpora @xcite of complex words. english wikipedia has also previously been used to generate candidate sentences for the complex word identification task @xcite . parallel articles from simple and regular english wikipedia have also been aligned to generate examples of parallel sentences for training text simplification models @xcite . ## corpus development we take an approach similar to our prior work @xcite , by mining simple english wikipedia edit histories. we hypothesise that editors are typically trying to simplify the texts when editing them and so any cases we find of a single word being dropped (with some caveats listed below) are likely to be examples of simplification by deletion. we download the most recent version of the simple wikipedia edit histories as an xml file foot_1 and compare successive revisions of each page using the following pipeline of operations: ## prediction of lexical deletions we test the following four methods for predicting lexical deletion, covering unsupervised and supervised techniques, as well as a sentence simplification system capable of deletions. ## results we evaluate our task and methods in a variety of settings as described below in order to better understand the nature of the lexical deletion problem. ## discussion we introduced the task of lexical deletion in the new context of lexical simplification. this is the first work of which we are aware to explicitly investigate lexical deletion as a simplification operation. we also developed a new silver-standard corpus, simpledelete, mined from simple english wikipedia edit histories and tested our results on it. future work could move our corpus from silver to gold standard by verifying all 18,082 instances either manually or semi-automatically. in our binary classification setting, we demonstrated supervised methods trained on our corpus (svm, bert-large) and unsupervised methods (tersebert, access) for the task of lexical deletion. access gives a low recall, but competitive precision, indicating it is capable of the type of deletions we have identified but does not perform these consistently. tersebert with a thresholded deletion score of 0.27 gave an f1 score of 0.795, which was higher than the svm, but lower than bert-large. as our corpus is silver standard, it is possible that the supervised methods may have also learnt corpus specific factors. our simplifications come directly from simple wikipedia edit histories and we assume that editors remove words to improve the simplicity of the language. the examples in in conclusion, we have introduced and evaluated the capacity of lexical deletion for simplification. as a result, we hope that future works in lexical simplification will also take the deletion operation into account as an alternative to lexical replacement.