Automatic Assistance for Academic Word Usage (2020)

Abstract: This paper describes a writing assistance system that helps students improve their academic writing. Given an input text, the system suggests lexical substitutions that aim to incorporate more academic vocabulary. The substitution candidates are drawn from an academic word list and ranked by a masked language model. Experimental results show that lexical formality analysis can improve the quality of the suggestions, in comparison to a baseline that relies on the masked language model only.

## introduction while most research on automatic writing assistance has focused on grammatical error correction @xcite , there has been increasing attention on analyzing academic writing, with respect to the writing style expected in the genre @xcite . examples of recent efforts include identification of argumentation structure @xcite , automatic assistance for nominalization and sentence restructuring @xcite , and lexical substitution of academic vocabulary @xcite . this paper focuses on the latter, an example of which is shown in lexical substitution (ls) is the subfield of natural language processing that aims to replace target words in a text without changing its meaning @xcite . our task -academic ls -may be viewed as a special form of ls, with the additional requirement that the substitution be typical for academic discourse. we describe and evaluate a writing assistance system for academic word usage that incorporates lexical formality analysis. experimental results show that the incorporation of formality analysis helps raise the performance in identifying suitable substitutions. ## previous work the system reported in yimam et al. (2020), which performs academic lexical substitution (ls) using the typical ls pipeline @xcite , is closely related to ours. in the first step, the system identifies the non-academic target words with a binary classifier. the second step, substitution generation, proposes candidate substitutions for each target word. finally, the system ranks these candidates to determine the optimal substitution. the candidate substitutions were harvested from ppdb @xcite and wordnet @xcite , and ranked with the tf-ranking deep learning model @xcite . automatically mined pairs of informal and formal words @xcite have been shown to be useful for natural language generation @xcite . lexical formality analysis may also be beneficial for academic ls since academic vocabulary tends to be more formal. although informalformal text pairs are now publicly available in large volume @xcite ## data there is no existing dataset for academic lexical substitution (ls). we constructed our evaluation data by compiling an academic word list (section 3.1) and an informal word list (section 3.2), and transforming an ls dataset with these lists (section 3.3). ## approach our approach makes use of bert @xcite , a masked language model that has been shown to attain good performance in lexical substitution @xcite , among many other tasks in natural language processing. we masked each target word in the input sentence, and then retrieved the top-n ranked list of candidates for the masked position from bert. we will refer to this list as the "bert list". we performed evaluation on values of n ranging from 1 to 50. words in the bert list may not be academically appropriate or semantically similar to the target word. we removed non-academic words from the list, and then further filtered it with the following methods: baseline keep all remaining words without further filtering. ppdb include only those words that are paraphrases of the target word in ppdb @xcite . gensim include only those words that are among the 50 words most related to the target word, as estimated by the google news pre-trained gensim model @xcite . lexical formality include only those words that are considered formal equivalents of the target word. the formal equivalents are taken from the informal-formal word pairs from @xcite and the style lexicon from @xcite , including both the manually crafted (4,196 pairs) and automatically generated (654,385 pairs) ones. if no word survives the filter, the system does not attempt substitution. otherwise, it predicts the topranked candidate as the substitution. ## experiments we first used the annotated corpus (section 3.3) to automatically evaluate the proposed methods. then, we applied the optimal method on a user study. ## conclusions we have described a writing assistance system that automatically provides lexical substitution suggestions to incorporate more academic vocabulary. a significant novelty of the system is the use of lexical formality in selecting substitution candidates. experimental results show that the formality analysis led to improved performance over a number of competitive baselines. further, a user study demonstrates that students were able to use the system to improve text quality. in @xcite , and by an applied research grant (#9667151) from city university of hong kong.