L inggle W rite: a Coaching System for Essay Writing (2020)

Abstract: This paper presents LinggleWrite, a writing coach that provides writing suggestions, assesses writing proficiency levels, detects grammatical errors, and offers corrective feedback in response to user’s essay. The method involves extracting grammar patterns, training models for automated essay scoring (AES) and grammatical error detection (GED), and finally retrieving plausible corrections from a n-gram search engine. Experiments on public test sets indicate that both AES and GED models achieve state-of-the-art performance. These results show that LinggleWrite is potentially useful in helping learners improve their writing skills.

## introduction essay writing has been an essential part of language assessments (e.g., toefl, ielts) but a challenging task for most students. to write a good essay not only requires sustained practice, but also demands instructional feedback from teachers. however, pressed with teaching load, teachers can only provide limited corrective feedback on students' essays. this has encouraged the development of computer-assisted writing systems to meet growing needs of automated feedback as a means of writing coaching. computer assisted language learning (call) has been an active field of computational linguistics and pedagogy. some existing computer aided writing systems detect and correct grammatical errors, and give an overall score (e.g., grammarly ( www.grammarly.com ) and pigai ( www.pigai.org) ). instead of directly correcting users' essays, write&improve (writeandimprove.com) only marks highly-likely incorrect words on the grounds that automated grammatical error correction is still very imprecise. recently, researchers have begun to apply neural network models to both automated essay scoring (aes) and grammatical error detection (ged), gaining significant improvement (e.g., @xcite ; @xcite ). however, these web services fall short of providing sufficient "coaching" information (e.g., grammar patterns, collocations, examples) to learners to improve their writing skills. provide writing suggestions as a user types away or during editing is another emerging approach to coaching the learner. for example, writeahead (writeahead.nlpweb.org) provides contextsensitive suggestions, right in the process of writing or self-editing. google recently released smart compose that offers users word or phrase completion suggestions while writing an email @xcite . in line with these systems, we also suggest that feedback on learners' writings could be more effective if a system not only acts as an editor providing direct corrections, but also a coach performing grammatical error detection and offering interactive suggestions @xcite . moreover, illustrating word usage with bilingual examples can better help non-native english learners. this would enhance learners' skills of self-editing and pave the way to lifelong language learning. with that in mind, we developed a web-based system lingglewrite (f.linggle.com) with many assistive writing functions. with lingglewrite users can write or paste their essays and get informative feedback including just-in-time writing suggestions, essay scoring, error detection, and related word usage information retrieved from linggle(linggle.com). ## the lingglewrite system the system consists of 4 components: (1) interactive writing suggestion, (2) essay scoring, (3) grammatical error detection, and (4) corrective feedback. the first component, writing suggestion, will help users with word usage information while writing. the other three components are aimed at providing evaluation and constructive feedback after a user finishes writing. the system is available at f.linggle.com. we'll describe each component as follows. ## method to develop lingglewrite, we extract the most common grammar patterns from a corpus in order to provide writing suggestions. additionally, we develop models for aes and ged based on annotated learner corpora. we retrieve corrective feedback by querying a linguistic search engine according to the predicted edit type of an error. we describe the process in detail in the following subsections. ## evaluation for the aes task, we adopted quadratic weighted kappa (qwk) as our evaluation metric, which 3 https://github.com/google-research/bert#pre-trainedmodels was used in automated student assessment prize (asap) competition and several aes researches @xcite @xcite . for the ged task, we follow the previous research by @xcite and use precision, recall and f 0.5 to evaluate our ged model. model model type avg. qwk score cnn word-level 0.902 lstm word-level 0.927 bi-lstm word-level 0.921 lstm + attention word-level 0.931 cnn-cnn sentence-level 0.934 lstm-lstm sentence-level 0.937 cnn-lstm-att sentence-level 0.952 lstm-lstm-att sentence-level 0.957 ## conclusion and future work in summary, we have presented an writing environment that supports interactive writing suggestions, scoring, error detection and corrective feedback. for the interactive writing task, we provide grammatical suggestions, collocations, and bilingual examples, to guide the user towards writing fluently. for the ged task, we proposed a new label schema, dirc. experiments show that the proposed label schema achieves comparable performance (on binary task) while providing more informative feedback. in addition, we leverage an existing linguistic search engine to provide corrective suggestions for each error type. many avenues exist for future research and improvement of our system. for example, the method for introducing additional training data or generating artificial training data could be implemented to improve the performance. an interesting direction to explore is re-ranking corrective suggestions, so that the suggestion more relevant to the original sentence goes to the top. yet another direction of research would be to detect fine-grained error types. finally, our system currently providing additional chinese translations for english examples. obviously we could easily provide languages trans-