IF ly EA : A C hinese Essay Assessment System with Automated Rating, Review Generation, and Recommendation (2021)

Abstract: Automated Essay Assessment (AEA) aims to judge students’ writing proficiency in an automatic way. This paper presents a Chinese AEA system IFlyEssayAssess (IFlyEA), targeting on evaluating essays written by native Chinese students from primary and junior schools. IFlyEA provides multi-level and multi-dimension analytical modules for essay assessment. It has state-of-the-art grammar level analysis techniques, and also integrates components for rhetoric and discourse level analysis, which are important for evaluating native speakers’ writing ability, but still challenging and less studied in previous work. Based on the comprehensive analysis, IFlyEA provides application services for essay scoring, review generation, recommendation, and explainable analytical visualization. These services can benefit both teachers and students during the process of writing teaching and learning.

## introduction automated essay assessment (aea) is an important educational application @xcite . it aims to reduce the burden of teachers for scoring student essays and give students direct instructions to improve their writing ability. automated essay scoring (aes) is one of the most important modules for aea, which is usually formulated as a supervised learning problem. the early approaches utilized hand-crafted features to predict essay scores @xcite @xcite . recently, deep learning has been applied to aes as well @xcite @xcite . one issue about aes is that its prediction lacks explainability since a single score gives very limited information. many efforts have been paid to expand the boundary of aes, and try to analyze detailed linguistic properties, such as grammatical errors @xcite , coherence @xcite , organization @xcite and so on. several aes systems, such as e-rater @xcite and lingglewrite @xcite , have been successfully applied in the education scenario. however, many of them focus on evaluating second-language learners' writing ability or evaluating basic language usages depending on shallow features, which may be not sufficient for evaluating essays written by native speakers. moreover, most existing platforms mainly target on english, while there are significantly fewer systems working on other languages, such as chinese. in this paper, we introduce the iflyessayassess (iflyea) system, which is a chinese automated essay assessment system, focusing on assessing the quality of essays written by native chinese students from primary and junior schools. iflyea has the following highlights: the target users of iflyea is students from primary and junior schools, in other hands, it is also helpful for teachers to reduce their heavy work. iflyea has been applied in practice and it is being continually improved by learning from user feedback. ## system architecture the main modules of iflyea can be categorized into two types: analytical modules and application modules, as shown in figure 1 . these modules are integrated with visualization and interactive interfaces. the analytical modules involve multi-level and multi-dimension analysis of essay quality, which mainly cover three levels: the system identifies discourse elements for representing and evaluating essay organization, and also has other discourse level analysis such as topic classification and genre classification. the techniques at grammar level are widely used for essay scoring, especially for evaluating secondlanguage learners. the rhetoric and discourse levels are more important for evaluating essays written by native speakers, especially for distinguishing well-written essays from moderate ones. the application modules include: the review generation and recommendation modules depend on the results from the analytical modules and the essay scoring module. in general, the analytical modules are the basis of the application modules, providing evidence and diagnosis, and also improving the explainability for the predictions of application modules. as illustrated in figure 2 , through web page visualization and interfaces, students or teachers can receive rich information and interact with the analytical results. ## analytical modules iflyea has multi-level and multi-dimension quality evaluation to provide comprehensive analytical results. this section will introduce the main analytical modules, which can be roughly categorized into 3 levels: grammar, rhetoric and discourse levels. ## conclusion and future work this paper presented iflyea, a chinese automated essay assessment system. iflyea demonstrates the techniques, that we have developed, could tackle with evaluating the quality of essays written by native chinese students. a demonstrating video is available at https://youtu.be/bujbqfxvx3a . the main advantage of iflyea is its multi-level and multi-dimension analytical modules for essay assessment, especially on several high level skill-ful language usage abilities, which is less studied previously. most of these modules can achieve moderate and above performance. iflyea also provides comprehensive services for rating, review generation and recommendation. together with the visualization and interactive interfaces, teachers and students can get useful feedback and easily understand why the system makes such predictions. iflyea has been applied in practice. in future, we plan to conduct more user studies and continue to improve the system. and how to evaluate the impact of the system on students is another important problem, which is worth exploring.