AM esure: A Web Platform to Assist the Clear Writing of Administrative Texts (2020)

Abstract: This article presents the AMesure platform, which aims to assist writers of French administrative texts in simplifying their writing. This platform includes a readability formula specialized for administrative texts and it also uses various natural language processing (NLP) tools to analyze texts and highlight a number of linguistic phenomena considered difficult to read. Finally, based on the difficulties identified, it offers pieces of advice coming from official plain language guides to users. This paper describes the different components of the system and reports an evaluation of these components.

## introduction in our current society, written documents play a central role as an information channel, especially in the context of communication between institutions and their target audiences @xcite . unfortunately, although efforts to raise the education level of the population worldwide have increased in recent decades, reports @xcite point out that a significant proportion of citizens still have general reading difficulties. as regards administrative texts, various reading issues have been reported. for instance, @xcite reported that, in a survey carried out in the us, 58% of the respondents admitted to dropping out of an administrative process due to the reading difficulty. administrations have been aware of this issue for decades and have launched various initiatives to address it, the most prominent of which is the plain language movement. plain language aims to increase the accessibility of legal documents for a general audience and has been shown to both reduce costs and please readers @xcite . it has not only been promoted through various campaigns (e.g. plain english campaign in the uk) and writing guides (gouvernement du @xcite ; ministère de la communauté franc ¸aise de @xcite @xcite , but also incorporated in some legal principles. however, its widespread application is still undermined due to, for example, the efforts required to train writers @xcite , or the necessity to persuade writers -especially legal onesto abandon their flowery style, which is seen as a determinant of the image of expertise they project in the reader's mind @xcite . this second reason, however, falls beyond the scope of the current study, which aims to address the first reason, i.e. writers' training. recent research by @xcite revealed that although several plain language guides are available to assist writers of administrative texts in their work, the guidelines provided in these guides are not always followed by writers, mainly because they are too vague and too numerous. to relieve writers from the need to keep all these guidelines in mind, we have designed a web platform, amesure foot_0 , aimed at automatically identifying clear writing issues in administrative texts and providing simple writing advice that is contextually relevant. in its current state, the platform offers the three following functionalities: (1) providing an overall readability score based on a formula specialized to administrative texts; (2) identifying, in a text, linguistic phenomena that are assumed to have a negative effect on the comprehension of the text; (3) for the phenomena detected in step 2, proposing simplification advice found in plain language guides. in the following sections, we first refer to some related work (section 2), before describing the nlp analyses carried out to operate the system (section 3.1). then, we introduce the system and the way suggestions are provided (section 3.2). the paper concludes with a report about the system performance (section 4). ## related work this work stands at the intersection between two very different fields: writing studies -"the interdisciplinary science that studies all the processes and knowledge involved in the production of professional writings and their appropriateness for the addressees" @xcite -and automatic text simplification (ats), a branch of nlp that aims to automatically adapt difficult linguistic structures while preserving the meaning to enhance text accessibility. relevant facts from writing studies have already been covered in the introduction. as regards ats, the last few years have witnessed the publication of numerous interesting studies, reviewed by @xcite , @xcite . in brief, the field has mainly focused on developing algorithms to automatically simplify complex words (lexical simplification) and/or complex syntactic structures (syntactic simplification). it has first relied on rule-based approaches @xcite in which a text is automatically parsed before being applied simplification rules defined by experts. later, ats has been assimilated to a translation task (the original version is translated into a simplified version) and addressed with statistical translation systems @xcite . as neural machine translation has emerged under the impulse of deep learning, the seq2seq model has become prevalent for ats too @xcite . some work has specifically focused on the issue of lexical simplification, which involves different techniques. lexical simplification is generally operated in four steps, the first one being the identification of complex words. some systems choose to consider all words as candidates for substitution @xcite ; others use a list of complex words or machine learning techniques for classification of complex words @xcite . once complex words have been identified, the next step is the generation of simpler synonyms for substitution, either by relying on lexical resources @xcite , getting candidates from corpora @xcite , producing them with embeddings @xcite or, more recently, with bert @xcite . in a next step, the candidates are semantically filtered to fit the context and are ranked according to their difficulty by classifiers using various word characteristics (e.g. frequency, embedding, morphemes, syllabic structures, etc.) @xcite @xcite . although numerous ats systems are described in publications, we have found only four of them that made their way through a web platform tailored to writers' needs. @xcite developed a simplification web platform for portuguese, in which the user is able to either accept or reject simplifications done by the system. similarly, @xcite 's system performs lexical and syntactic simplifications for english and supports human post-editing. more recently, @xcite introduced tecst, which is able to perform simplification at different levels, depending on the user. finally, @xcite implemented a semantic writing aid tool able to suggest context-aware lexical paraphrases to writers. none of these tools, however, have focused on writers of administrative texts, nor on french. amesure could also be related to the family of writing assistants, such as word or libreoffice. however, only a few of them provides writing advice based on specific criteria or plain language guides. there are some examples of these tools available for the general public in french: ## evaluation of the system to assess the performance of the various extraction algorithms included in our platform, three linguists manually annotated, in 24 administrative texts, the following five phenomena: passive structures, relative clauses, object clauses, adverbial clauses, and abbreviations 6 . the work of annotators was supported by guidelines focusing on difficult cases 7 . at the end of the annotation process, the expert agreement was evaluated using fleiss' kappa (see this gold-standard version of the annotation was manually compared to the output of amesure for the 24 texts in the test set. and a recall of 0.62 for the relative clauses and a recall of 0.66 and a precision of 0.94 for infinitive clauses introduced by the particle "to". chinkina and meurers (2016) reached a recall of 0.83 and a precision of 0.71 for relative clauses. however, our system has trouble detecting object clauses, which have a f1-score of only 0.48. investigation of the confusion matrix reveals that 77% of object clauses (37 out of 48) are correctly detected by amesure, but 17 out of 37 are wrongly classified as adverbial clauses. this is a limited issue, as advice can still be provided even if the system gets the type of clause wrong. ## conclusion we have presented the amesure system, which automatically analyzes the readability of french administrative texts based on classic readability metrics, but also on guidelines from plain language books. the system is freely available through a web platform and is aimed to help writers of administrative texts to produce more accessible documents and forms. to that purpose, it offers a global readability score for the texts, 11 readability yardsticks, a detailed diagnosis in which difficult linguistic words and syntactic structures are highlighted, and some plain language advice. we also carried out a manual evaluation of the system based on 24 administrative texts annotated by linguists. performance is satisfactory, except as regards the identification of object clauses. more work is needed on this category, especially to distinguish it from adverbial clauses. we also plan to improve the system providing simpler synonyms by adding a semantic filter based on embedding models. finally, we plan to conduct a study with real writers of administrative texts to measure the perceived usefulness of amesure as a whole, but also the usefulness of each functionality.