Automatic Compilation of Resources for Academic Writing and Evaluating with Informal Word Identification and Paraphrasing System (2020)

Abstract: We present the first approach to automatically building resources for academic writing. The aim is to build a writing aid system that automatically edits a text so that it better adheres to the academic style of writing. On top of existing academic resources, such as the Corpus of Contemporary American English (COCA) academic Word List, the New Academic Word List, and the Academic Collocation List, we also explore how to dynamically build such resources that would be used to automatically identify informal or non-academic words or phrases. The resources are compiled using different generic approaches that can be extended for different domains and languages. We describe the evaluation of resources with a system implementation. The system consists of an informal word identification (IWI), academic candidate paraphrase generation, and paraphrase ranking components. To generate candidates and rank them in context, we have used the PPDB and WordNet paraphrase resources. We use the Concepts in Context (CoInCO) “All-Words” lexical substitution dataset both for the informal word identification and paraphrase generation experiments. Our informal word identification component achieves an F-1 score of 82%, significantly outperforming a stratified classifier baseline. The main contribution of this work is a domain-independent methodology to build targeted resources for writing aids.

## introduction we present the first approach to building resources for an academic writing aid system automatically. academic writing aid systems help in automatically editing a text so that it better adheres to the academic style of writing, particularly by choosing a better academic word in a given domain. in the context of academic paraphrasing tasks, the resources are mainly words or phrases, that are more appropriate to use in an academic writing style. moreover, the academic resources might vary from domain to domain as some words or phrases are extensively used in one domain over the other. the first step in building an academic writing aid tool is to collect resources that determines whether a given phrase follows the style of writing in academia. this involves analyzing a given sentence and determining if the lexemes of the sentences are well-selected academic words and phrases or not. to evaluate the resources compiled, we have to build a system, analogous to the lexical substitution and text simplification tasks, for example, @xcite , that consists of informal word identification, academic candidate generation, and candidate paraphrase ranking components (see figure 1 ). while it is possible to follow the same approaches as the lexical substitution and text simplification approaches for academic text rewriting tasks, the main challenge for the academic paraphrasing task is the collection of resources for academic texts. the following are the main objectives of building academic resources: the informal word identification (iwi) component automatically identifies informal words (see section 4.2) that are going to be replaced with academic paraphrases. the candidate generation and ranking component determine the best academic candidate paraphrase to replace the informal words. the ultimate goal of this research work is to integrate the informal word identification, candidate generation, and paraphrase ranking components into writing aid tools, for example to word processors or text composing software like latex packages, to automatically assist users in academic text composing. in this work, we have targeted the following research questions 1) how to build academic resources (words or phrases), which are used to replace informal or less academic expressions in academic texts? 2) how to build a system that can be used to evaluate the collected resources? in section 2, a brief review of related works is presented. in section 3, we discuss how to build academic resources using reference corpora and evaluate the quality of the resource. in section 4, we present the approaches that are used to build an informal word identification and paraphrasing system for academic rewriting. setups of the academic paraphrasing systems and the experimental results are discussed in section 5. analysis of system results and conclusion of the research are presented in section 6 and section 7 respectively. ## previous work in this section, we review previous work in lexical substitution, a closely related task, and discuss how the academic text rewriting system potentially differs. in essence, our system is similar to lexical substitution (ls) and text simplification tasks, in such a way that both focus on the rewriting of an original text towards a given goal. lexical substitution system mainly focuses on rewriting texts by replacing some of the words or phrases without altering the original meaning @xcite . the work by @xcite targeted text simplification based on the sequence-to-sequence deep neural network model, where its entailment and paraphrasing capabilities are improved via multi-task learning. while the complex word identification (cwi) task focuses on identifying lexical units that pose difficulties to understand the sentence @xcite @xcite , our informal word identification (iwi) component focuses on identifying words that are not fitting or adhering to the academic style of writing. the work by @xcite focuses on the lexical substitution task, particularly for medical documents. they have relied on distributional thesaurus (dt), computed on medical texts to generate synonyms for target words. existing resources for academic writing are limited to a precompiled list of words such as the corpus of contemporary american english (coca) @xcite and the new academic word list 1.0 (nawl) @xcite vocabulary lists. regarding phrases (multiword expressions) for academic writing, the only available resources are the academic bi-grams compiled by pearson foot_0 . however, these resources are 1) limited to a certain domain and target writers (mostly l2 learners and students), 2) their vocabulary is fixed, thus requiring manual work for an extension, and 3) the resources are limited to uni-gram and bi-gram lists. in this work, we build academic resources that are more generic, which can be built from existing reference corpora. in addition to uni-gram and bi-gram resources, we also design a system that can produce resources up to a length of four words (quad-grams). as far as we know, the only system available to academic writing is the work of @xcite , which addresses a different aspect, which is a sentence restructuring based on nominalizing verbal expressions. ## building academic resources in this section, we will first discuss the existing academic resources, how they are built and their limitations. then, we will present our approach that describes the process of building academic resources from different reference corpora. finally, we will discuss the quality of the collected resources against two evaluation measures, namely comparing with the existing resources and manually evaluating the academic fitness of resources. ## analysis of results for the informal word identification task, our models have a slightly lower precision as our dataset is not balanced (we have more informal words than formal words, as shown in from an error analysis, we find out that even if the term is academic in general, its usage in the test dataset is in-clined to be informal. for example, in the sentence "it was last february, after the winter break, that we moved in together.", break is labeled as academic but should be labeled as informal. this issue could be solved by further enhancing the dataset by employing human annotators during the resource compilation process. similarly, some of the errors from the system's prediction are to be attributed to the annotation process of the test set. for example, in the sentence "they included support for marine reserves and money for fisheries management reform.", reserves is annotated as informal while the system identified it as formal. in general, while bootstrapping the academic resource compilation and the informal word identification tasks, a minimal intervention of human annotators would enhance the overall system. furthermore, integration of a bert or other contextualized embedding model @xcite could also help to improve the performance of the system. contextualized word embeddings provide word vector representations based on their context. as the vector representation of words varies as per the context, they implicitly provide a model for word sense disambiguation (wsd). ## conclusion and future direction in the realm of academic text writing, we explored how to compile academic resources, automatically identify informal words (words that are less formal for academic writing), and provide better substitutes. we have used a generic approach to compile the academic resources, which can be easily transferred to domains or languages as it only requires text corpus. the academic text rewriting system, analogous to lexical substitution systems, consists of informal word identification, candidate generation, candidate selection, and ranking components. as far as we know, this is the first experiment towards the development of academic writing support for academia, while there might be commercial cases (for example grammarly foot_7 ) that we do not know how the systems operate. we envision this system to be embedded into open source academic writing aid tools where the academic sources are used to detect informal terms and propose academic substitutes. for the resource compilation process, it would be nice to extend the embedrank approach to extract keyphrases beyond the adjective and noun pos tag patterns, especially to cover verbs used in academic contexts. source code and resources of this the paper are released publicly foot_8 on the github repository under permissive licenses (asl 2.0, cc-by).