Automated Evaluation of Written Discourse Coherence Using GPT-4
Year: 2023

Abstract:
The popularization of large language models (LLMs) such as OpenAI’s GPT-3 and GPT-4 have led to numerous innovations in the field of AI in education. With respect to automated writing evaluation (AWE), LLMs have reduced challenges associated with assessing writing quality characteristics that are difficult to identify automatically, such as discourse coherence. In addition, LLMs can provide rationales for their evaluations (ratings) which increases score interpretability and transparency. This paper investigates one approach to producing ratings by training GPT-4 to assess discourse coherence in a manner consistent with expert human raters. The findings of the study suggest that GPT-4 has strong potential to produce discourse coherence ratings that are comparable to human ratings, accompanied by clear rationales. Furthermore, the GPT-4 ratings outperform traditional NLP coherence metrics with respect to agreement with human ratings. These results have implications for advancing AWE technology for learning and assessment.

heading: 1 Introduction
text: Recent advances in large language models (LLMs; Brown et al., 2020), and in particular OpenAI’s GPT-4 model (Eloundo et al., 2023; OpenAI, 2023), have led to a paradigm shift with regard to what machines can generate, such as coherent writing. We are now witnessing the potential power and exponential growth of AI in education, though the impact of LLMs used for educational purposes is still largely unexplored. For instance, applications not intended for educational purposes, such as ChatGPT, are being used in educational contexts – everyone with access to the internet can now ask ChatGPT to complete writing tasks, from generating outlines and ideas, to summarizing documents, to essay writing. With these novel capabilities, we can see immediate advantages, such as leveraging GPT-4 for instructional purposes (e.g., automatic
item generation, see Attali et al., 2022), and disadvantages (e.g., increased plagiarism, see Eliot, 2022). In addition, we are learning about current potential shortcomings of LLMs (e.g., hallucinations or low-quality content generation) due to miscalibrated expectations of what LLMs can do or the pitfalls of non-optimized prompt engineering.
To further our understanding of one innovative application of AI in education, this paper presents an exploratory evaluation of LLMs for automated writing evaluation (AWE). Specifically, it is the first study to our knowledge to examine GPT-4’s ability to provide a rating (score) and rationale for one aspect of writing quality – discourse coherence quality – in test-taker written responses to an online, high-stakes writing assessment item. Discourse coherence is notoriously challenging to satisfactorily assess using AWE, and as such, there is great value in determining whether state-of-the-art AI can be used to improve upon prior options. We believe that the method described in the paper should be generalizable to similar datasets that are publicly available. However, caution in the use of GPT-4 ratings is warranted due to limited reproducibility, the possibility of bias, and limited insight into the underlying processes that determine the ratings.

heading: 2 Background
text: In the field of AI in education, AWE is one of the most widely researched and mature areas. AWE systems evaluate written text quality (Shermis and Burstein, 2003, 2013; Attali and Burstein, 2006) and are widely used for high-stakes writing assessment and instruction. These systems are informed by theoretical writing subconstructs (i.e., factors contributing to writing quality) described in human scoring rubric criteria such as grammatical accuracy, lexical sophistication, relevance, and discourse coherence. These rubric criteria are developed and used by educational testing organizations for scoring purposes and are often informed by
394
education policy (e.g., Common Core Standards, 2010 and Council of Europe, 2020). AWE systems typically provide a holistic score that indicates the overall quality of writing, given a set of rubric criteria. The performance of these scores (accuracy) is then reported through human-system agreement, a well-studied evaluation measure that is typically quite high on modern systems (e.g., Bridgeman, 2013).
In recent years, large language models (and earlier models pretrained on unlabeled text) have been leveraged to good effect in various ways to improve AWE performance through the use of “transformers”, a type of deep learning neural network. For example, Lagakis and Demetriadis (2021) found that the best AWE performance was achieved through a model incorporating linguistic features with the BERT language model (Devlin et al., 2019). More recently, Mizumoto and Eguchi (2023) explored the capabilities of GPT-3 to holistically rate testtaker essays in the TOEFL11 corpus (Blanchard et al., 2013). The researchers showed Human-GPT3 agreement rates to be reasonable (exact agreement 54.33%, adjacent agreement 89.15%). The model’s performance was then further improved by combining GPT ratings and a range of lexical, syntactic, and cohesion features, resulting in substantial Quadratic Weighted Kappa (QWK) of 0.61. Methodologically, it is important note that in their study, the same prompt was used in all conditions, and this prompt did not include examples or ask for rationales for the ratings. To our knowledge, there have been no similar studies with the newer GPT-4 or with comparing different prompt configurations to elicit ratings.
While AWE systems show strong performance for holistic scoring, scores for discourse coherence quality alone have been a challenging area of NLP research (Hearst, 1997; Barzilay and Lapata, 2008; Burstein et al., 2013; Somasundaran et al., 2014; Lai and Tetreault, 2018). Although some discourse features can be considered “surface-based,” for example, pronoun referents and transition terms used in a text, operationalizing aspects of coherence such as the relationship between ideas is less straightforward and involves labor-intensive annotations or less easily interpretable LLM-derived features. In particular, it may be difficult to tell whether LLM-generated “analyses” of a text actually reflect the same aspects of writing that superficially similar human-written analyses describe.
Further complicating coherence assessment is the fact that different disciplines, from linguistics (Halliday and Hasan, 1976) to cognitive psychology (Graesser et al., 2004), to education research (Van den Broek et al., 2009), share slightly different views about how coherence is constructed by readers of a text. However, a common thread is that discourse coherence pertains to the textual continuity or flow of a text, that is, the overall sense of unity and meaning that is conveyed by a text. Within the construct of discourse coherence, assessment rubrics often directly or indirectly refer to subconstructs such as clarity (how easy to understand ideas and purpose; readability; and impact of lexis/grammar on coherence); flow (sequence/progression of ideas; use of linking words; and referencing); structure (appropriacy of paragraphing; introducing/concluding; and connection between topics); and effect on reader (naturalness of cohesion; appropriacy of cohesive features; repetitiveness; and helpfulness to reader for understanding the response).

heading: 3 Methods
text: In this section we describe the dataset of test-taker responses and the processes for evaluating them through human and automated means.

heading: 3.1 DET coherence (DET-Coh) dataset
text: The DET coherence (DET-Coh) dataset contains test-taker written responses from the operational Duolingo English Test (DET). The DET is a highstakes English language test whose primary use is for higher-education admissions. One of the writing tasks, Writing Sample, is an independent writing task in which test takers respond to a prompt requiring them to produce a persuasive or narrative extended piece of writing in five minutes (see Cardwell et al., 2023, for further details). Writing Sample is scored using AWE; the scoring model includes features to assess the writing subconstructs of Content, Discourse coherence, Grammar, and Vocabulary.
In total, there are 500 written responses in the DET-Coh dataset, sampled from the operational DET during a 7-month span in 2022. DET-Coh was deliberately constructed and stratified so that it contains an equal distribution of males and females, as well as an equal distribution of the seven most common first-language groups in the DET test-taker population (Chinese, Arabic, Spanish, Telugu, En-
glish, Bengali, Gujarati). An approximately even distribution of proficiency levels was also ensured based on DET automated scoring models. These levels align with the levels of the Common European Framework of Reference (CEFR; Council of Europe, 2001, 2020), an international standard for describing language ability, ranging from level A1 (basic) to C2 (proficient) on a six-point ordinal scale.

heading: 3.2 Human scoring
text: Test-taker writing responses were scored by four expert raters, each with second language (L2) teaching qualifications, extensive L2 teaching experience, and L2 assessment experience with international proficiency exams. Of the original 500 responses, 20 were double rated collaboratively for standardization, and 80 were rated independently by pairs of raters to assess interrater agreement. The interrater agreement for these 80 items was 0.72 exact agreement and 0.93 QWK, indicating excellent agreement. Having established rater reliability, the remaining 420 responses were rated by a single rater each.
All ratings were based on writing coherence task rubrics created for this study (see Appendix A, Table 2, for full rubric text). The rubric was developed using a 6-point, holistic scale that was based on the six levels/descriptors from the CEFR, other coherence research studies, and publicly-available rubrics from testing organizations. A rating of 0 was also given to blank or bad-faith responses in which the test taker did not attempt to respond to the prompt. In addition, one rater produced paragraph-long rationales for 12 of the ratings (two at each scale point) for the purposes of few-shot prompting (6 responses) and qualitative analysis (6 responses).

heading: 3.3 GPT-4 ratings and rationales
text: To elicit GPT-4 coherence ratings and rationales, we used the OpenAI Python API. The full prompt given to GPT-4 for each student response consisted of the following ordered elements:
• Task – a short paragraph explaining the task of rating the coherence of a written text written by a language learner in response to a prompt
• Rubrics – see Section 3.2 for description
• Guidelines – bullet point guidelines relating to expected terminology and style
• Examples – six training items removed from the dataset (one from each scale point), accompanied by expert ratings and/or rationales (depending on the condition) for the ratings based on the rubrics
• Prompt – the prompt the test taker responded to
• Response – the test taker’s response
Based on these elements, GPT-4 was called to complete three different conditions: 1) rating then rationale (rating-first), 2) rationale then rating (rationale-first), and 3) rating only (rating-only).

heading: 3.4 NLP coherence metrics
text: As a baseline, coherence ratings were predicted using a set of simple NLP features based on CohMetrix (Graesser et al., 2004):
• Binary overlap between sentence pairs: overlap of arguments, nouns, or word stems between two sentences
• Proportional overlap between sentence pairs: overlap of content words as a proportion of all content words in a sentence pair
• Coreference overlap: number of coreferent mentions between two sentences found using a neural coreference model (Lee et al., 2018)
• LSA similarity: measure of the similarity between two sentences calculated using an LSA model trained on a large sample of writing responses
Two versions of each feature were computed, one considering only adjacent sentence pairs (“local”), and one considering all pairs of sentences in a response (“global”). For each response, we fit a linear regression model using the features and human ratings for all other responses, then predicted the rating for the held-out response.

heading: 4 Results

heading: 4.1 Rating comparison
text: Ratings from GPT-4 and the baseline model are compared to the human ratings on all items not included in the prompt (Table 1); for double-rated items the second rating was used. The findings show that the baseline linear regression model is moderately predictive of the human ratings, reaching an adjacent agreement score of 0.82 and Spearman correlation (ρ) of 0.47 despite its simplicity.
All GPT-4 conditions significantly outperform this baseline model, obtaining a correlation of 0.82 with the human rating in the rationale conditions.
Inspired by Mizumoto and Eguchi (2023), we also experimented with a linear regression model that includes the GPT-4 rating as an additional feature along with the baseline features, potentially combining the strengths of the two models. However, unlike that work, we found that the combined model performs almost identically to the GPT-4 ratings on their own and so do not analyze it further.
The rationale-first condition could be interpreted as a form of chain-of-thought (CoT) prompting (Wei et al., 2022) which has been shown to improve performance on reasoning tasks. That work also hypothesized that showing examples with the reasoning after the answer in the prompt could improve performance, by drawing attention to relevant aspects of the tasks, but found it performed similarly to the baseline and worse than CoT prompting. By contrast, we find that GPT-4’s agreement is slightly improved by the use of rationales, regardless of their position. However, there are no significant differences between the agreement rates of any of the GPT-4 configurations, with all versions showing overlapping confidence intervals. These findings suggest that there is not a CoT effect for this task.
We focus on the rating-first condition for error analysis. GPT-4’s ratings have less variance than human ratings (0.37 vs 0.42), especially producing fewer 1, 5, and 6 ratings (most samples rated 1 by
humans are rated 2 by GPT-4). This behavior is actually in-line with a well-documented tendency of human raters, the central tendency effect, in which raters avoid the extremes of rating scales (McNamara et al., 2019). One hypothesis to account for this pattern is that GPT-4 is imitating trends found in its pre-training data. When GPT4’s ratings differ from human ratings (n=143), they are also slightly but significantly lower on average (µ = 3.17 for GPT-4 in the rating-first condition vs µ = 3.41 for the comparable human rating, p=0.04 with Welch’s t-test). In the rating-first condition, GPT-4 mentions “spelling” in 43% of rationales where its rating differs from the human rating, versus only 30% of equally rated rationales. Speculatively, this may indicate an oversensitivity to spelling errors; human raters may be better able to discern the intended word while GPT-4’s tokenbased representation may prevent such recognition.

heading: 4.2 Rationale comparison
text: The six human-generated rationales were compared to GPT-4 rationales in terms of their content and style. Figure 1 provides an example of a response with a 3 rating (CEFR B1; human and GPT-4 rating in agreement), answering a prompt about the advantages and disadvantages of using books, movies, and TV shows to learn about different cultures. Figure 2 shows the accompanying human and GPT-4 rationales. Of note, the trends exemplified in this set of examples hold true for all six pairs of human-
GPT4 rationales we analyzed.
Comparing the content of the two rationales, there is a great deal of consistency, with both addressing the clarity, flow, structure, and effect on the reader. For example, both rationales describe how the writer’s position is initially presented and provide a specific example. The two rationales also note the same main weakness relating to the lack of development of the second point. The two rationales then move on to describe how discourse markers are used to achieve local coherence, even highlighting the same two examples of Firstly and Secondly. Examples of coherence negatively affected by language inaccuracies are then given, though different examples are used to exemplify this point in the two rationales. Finally, both rationales summarize the reason for the overall satisfactory effect on the reader.
Likewise, in terms of style, the GPT-4 rationale has clearly adopted the examples and followed the guidelines from the prompt. The rationales use
terminology such as the writer (rather than the author/student/learner), are written in the 3rd person, and are within the desired length range. The overall format of the rationale is also consistent, starting with an overall statement of coherence, moving to discuss each of the coherence subconstructs in turn, then closing with an overall description of the effect on the reader.
To further illustrate how GPT-4 rationales discuss and incorporate key concepts from the rubrics, we conducted a simple corpus analysis of key words. First, a frequency list was compiled of the most common words (tokens) in the rationales. We restricted this list to content words (nouns, verbs, adjectives, and adverbs) and only counted the first occurrence of each word in each rationale. Of interest, we noted commonly used words related to discourse coherence including ideas (n=509), developed (n=406), impact (n=297), inaccuracies (n=278), and [discourse] markers (n=264). Figure 3 presents a concordance of the first ten oc-
currences of the most frequent of these key words, ideas, to provide the context in which this term is being used. Here we see that ideas are described in a number of ways, for example, relevant, appropriate, basic, and incoherent, all of which are descriptors used in the rubrics. As importantly, these ideas are discussed in terms of how they are presented and arranged in the response, and specific examples of test-taker ideas are listed, that is, there is a focus on content and meaning, not just mechanical use of linguistic features.

heading: 5 Discussion and conclusions
text: This study examined the effectiveness of using GPT-4 for assessing written discourse coherence of test-taker responses on a high-stakes English proficiency test. We found that GPT-4 is able to rate the coherence of writing samples with a good degree of accuracy in terms of agreement with the goldstandard human ratings; regardless of the exact order of the prompt (rating-first or rationale-first), the exact agreement rates were >0.5 and the QWK >0.8. Prompts eliciting rating-only performed slightly worse, though not significantly so. Importantly, all permutations of the GPT-4 prompt greatly outperformed a baseline NLP model composed of traditional coherence features. Human-GPT-4 agreement rates could likely be improved with further tailoring of the prompt; for example, based on the qualitative analysis, we might suggest additional guidelines to lower the weighting that GPT-4 assigns to spelling errors as it may be overvaluing their importance.
Studies such as this one have important implications for the field of AWE. There is often a tension between designing features that are easily interpretable but provide limited signal (e.g., the number of discourse markers) versus features which are less clearly aligned with human rubrics but which may provide more predictive power (e.g., perplexity of
the response under a language model). The promise of ratings based on GPT-4 is that they may bridge this gap by providing quantitative features which seemingly are based on aspects of language of importance to the language assessment community. In the future we therefore expect to see research in a similar vein which looks at further optimizing prompts to elicit ratings and clear, interpretable rationales, especially for subconstructs of writing which have historically been a challenge to measure through automated means. In using LLMs in this manner, we could reduce the “epistemic opacity” of AWE processes (Ferrara and Qunbar, 2022), that is, modern automated assessment could become less of a black box, thereby improving stakeholder confidence in the results. Nevertheless, although these results are encouraging, it is important to recognize that the interpretability promised by generated rationales is limited: GPT-4’s rationales may not accurately reflect the process used to assign the ratings. In particular, rationales may present rationalizations for decisions actually grounded in biasing features, as was found to be true of CoT explanations in Turpin et al. (2023). Rationales should therefore not be treated as offering insight into the process of generating ratings, even when they provide true and relevant information about the response.
The fact that rationales do not reflect a “thought process” by GPT does not, however, reduce their value in all contexts. As suggested in Mizumoto and Eguchi (2023), rationales can support language learning by providing instantaneous feedback. In the context of test takers of the DET, rationales such as the ones in this report are particularly useful because they are based on task- and constructspecific rubrics. For example, test takers completing a practice test would greatly benefit from feedback tailored to the writing subconstructs, such as discourse coherence, that will be assessed under
operational test conditions. GPT-4 could also then be further beneficially exploited by querying it to produce an improved version of the test taker’s own response; in other words, a personalized model answer.
Figure 4 is an example of one such model answer, revising the response from Figure 1. The same prompt as before was used for generating this revision, with the following amendment:
Now, write a revised version of the following response with improved coherence according to the rubric. Stick closely to the original in content, and do not rewrite too extensively; simply improve the organization and complete unfinished ideas.
In this revision, we see that the test taker’s ideas are maintained, for example, the benefits of learning about how other cultures eat and dance. In addition, the appropriate use of some discourse markers from the original are left intact. In contrast, key coherence weaknesses from the original are addressed, most notably the lack of development of disadvantages and the language inaccuracies which impacted clarity. There remains some repetitiveness in the revision of language from the task prompt, but this issue did not prevent the revised response from being independently rated a 5 (CEFR C1) by both GPT-4 and a human rater. As such, this revision would seem a reasonable goal for this particular test taker.
On a broader level, the focus of our study, including the importance of transparency, is in line with the larger field of educational AI application development where responsible AI is a key focus (ATP, 2021; Dignum, 2021; ITC-ATP, 2021; Burstein, 2023; Department for Science, Technology & Innovation, 2023). As novel ideas, applications, and research questions emerge around the use of LLMs for educational purposes, it is essential that research communities investigating the use and impact of AI for education build a research agenda. In light of the need to ensure responsible use of AI in education, researchers need to anticipate and pressure test possible uses of AI for education to ensure fairness.

heading: Acknowledgements
text: We thank the raters for their contribution to the DET-Coh dataset.

heading: A Discourse coherence rubrics

heading: Future Work
text: In the future we therefore expect to see research in a similar vein which looks at further optimizing prompts to elicit ratings and clear, interpretable rationales, especially for subconstructs of writing which have historically been a challenge to measure through automated means. In using LLMs in this manner, we could reduce the “epistemic opacity” of AWE processes (Ferrara and Qunbar, 2022), that is, modern automated assessment could become less of a black box, thereby improving stakeholder confidence in the results. Nevertheless, although these results are encouraging, it is important to recognize that the interpretability promised by generated rationales is limited: GPT-4’s rationales may not accurately reflect the process used to assign the ratings. In particular, rationales may present rationalizations for decisions actually grounded in biasing features, as was found to be true of CoT explanations in Turpin et al. (2023). Rationales should therefore not be treated as offering insight into the process of generating ratings, even when they provide true and relevant information about the response. The fact that rationales do not reflect a “thought process” by GPT does not, however, reduce their value in all contexts. As suggested in Mizumoto and Eguchi (2023), rationales can support language learning by providing instantaneous feedback. In the context of test takers of the DET, rationales such as the ones in this report are particularly useful because they are based on task- and constructspecific rubrics. For example, test takers completing a practice test would greatly benefit from feedback tailored to the writing subconstructs, such as discourse coherence, that will be assessed under
operational test conditions. GPT-4 could also then be further beneficially exploited by querying it to produce an improved version of the test taker’s own response; in other words, a personalized model answer. Figure 4 is an example of one such model answer, revising the response from Figure 1. The same prompt as before was used for generating this revision, with the following amendment:
Now, write a revised version of the following response with improved coherence according to the rubric. Stick closely to the original in content, and do not rewrite too extensively; simply improve the organization and complete unfinished ideas. In this revision, we see that the test taker’s ideas are maintained, for example, the benefits of learning about how other cultures eat and dance. In addition, the appropriate use of some discourse markers from the original are left intact. In contrast, key coherence weaknesses from the original are addressed, most notably the lack of development of disadvantages and the language inaccuracies which impacted clarity. There remains some repetitiveness in the revision of language from the task prompt, but this issue did not prevent the revised response from being independently rated a 5 (CEFR C1) by both GPT-4 and a human rater. As such, this revision would seem a reasonable goal for this particular test taker. On a broader level, the focus of our study, including the importance of transparency, is in line with the larger field of educational AI application development where responsible AI is a key focus (ATP, 2021; Dignum, 2021; ITC-ATP, 2021; Burstein, 2023; Department for Science, Technology & Innovation, 2023). As novel ideas, applications, and research questions emerge around the use of LLMs for educational purposes, it is essential that research communities investigating the use and impact of AI for education build a research agenda. In light of the need to ensure responsible use of AI in education, researchers need to anticipate and pressure test possible uses of AI for education to ensure fairness