Argument Detection in Student Essays under Resource Constraints (2023)

Abstract: Learning to make effective arguments is vital for the development of critical-thinking in students and, hence, for their academic and career success. Detecting argument components is crucial for developing systems that assess students’ ability to develop arguments. Traditionally, supervised learning has been used for this task, but this requires a large corpus of reliable training examples which are often impractical to obtain for student writing. Large language models have also been shown to be effective few-shot learners, making them suitable for low-resource argument detection. However, concerns such as latency, service reliability, and data privacy might hinder their practical applicability. To address these challenges, we present a low-resource classification approach that combines the intrinsic entailment relationship among the argument elements with a parameter-efficient prompt-tuning strategy. Experimental results demonstrate the effectiveness of our method in reducing the data and computation requirements of training an argument detection model without compromising the prediction accuracy. This suggests the practical applicability of our model across a variety of real-world settings, facilitating broader access to argument classification for researchers spanning various domains and problem scenarios.

## introduction in today's educational landscape, the development of critical thinking and persuasive writing skills holds significant importance. the ability to construct compelling arguments is essential for effective communication and argumentative writing enables students to express ideas clearly, present clear evidence, and address counterarguments effectively. these skills are vital for academic success, professional growth, and civic engagement @xcite . therefore, having a system to analyze and detect argumentation in stu-dents' writing would be essential for educators to assess and provide feedback on students' argumentative skills and foster continuous growth in their argumentative writing skills. furthermore, by using the tool to evaluate their writing, students can identify any weaknesses or gaps in their arguments and make necessary revisions independently. this promotes self-reflection and empowers students to take ownership of their learning, improving their critical thinking and communication skills. however, the task of detecting arguments within students' essays poses several challenges due to the nuanced nature of argumentation. constructing an argument involves presenting a "claim" and supporting it with "premises." however, claims can take various forms, ranging from explicit statements to implicit assertions that require inferential reasoning. similarly, premises may be stated explicitly or indirectly implied, further complicating the process of argument detection. traditional supervised models for argument analysis often rely on large amounts of training data to achieve satisfactory performance. collecting and annotating such data can be time-consuming and resource-intensive, making it challenging to build large training datasets that cover the diverse range of argumentative patterns and structures present in student essays. moreover, the practical deployment of large language models such as gpt @xcite and palm @xcite can be hindered by cost, latency, and data privacy concerns. to address these challenges, we introduced an argument classification approach that combines the inherent linguistic characteristics of argumentation with advanced machine learning techniques. we showed the efficacy of exploiting the natural language inference (nli) relationship between argument components to prime a pre-trained language model for the argument detection task. by merging this with a well-suited prompt-tuning strategy, we established a streamlined architecture that effectively reduces the data and computation requirements of training an argument detection model without compromising the prediction accuracy. we evaluate the performance and generalizability of our approach across two scenarios: one characterized by availability of reliable training data, and the other representing a resource-constrained noisy domain more akin to real-world settings. in both cases, our approach yielded competitive results, often surpassing the performance of resourceintensive alternatives in classifying argument components. this suggests the practical viability of our model across a variety of real-world settings. we believe that our approach has the potential to make argument classification accessible to a wider range of researchers and problem domains. ## argumentation as entailment automated argument detection systems have the potential to help teachers and students by offering a consistent and objective means of evaluating students' work, providing them with timely feedback to enhance their critical thinking and argumentative skills. by automating the process of identifying argument components like claims and premises, educators can redirect their efforts toward other crucial aspects of teaching and providing personalized support to students. however, developing a reliable and accurate automatic system poses certain challenges. natural language processing algorithms must be sophisticated enough to comprehend the nuances of human language, including various writing styles and levels of proficiency. the system must also recognize context and cultural differences to avoid misinterpretations. to address these challenges, we propose leveraging semantic relationships between argument elements by framing the argument detection task as natural language inference (nli). nli involves discerning the semantic connection between two sentences, where one sentence logically follows (entails) from the other @xcite . this notion of entailment and contradiction serves as a foundation for enhancing the semantic representation of various natural language understanding (nlu) problems, including parsing, coreference resolution, and reasoning tasks @xcite . similarly, we argue that the nli framework can be effectively extended to capture the semantic relationships be-tween different components within argumentation. for instance, a counter-claim may contradict the main claim of an argument, or a supportive premise might entail the corresponding claim @xcite . we believe that this formulation allows nlp models to leverage their inherent understanding of semantic relationships between logical elements to recognize whether a sentence provides the necessary support or context for a given argument component, and facilitate the development of argument component classification systems, even with a limited volume of training examples. however, employing the entailment paradigm for argument classification requires (a small set of) reliable labeled training data and careful consideration of complex structure of argumentation to ensure accurate and robust results. ## proposed approach given that a primary emphasis of this research lies in addressing the challenges posed by resourcelimited and noisy conditions in student essay argument detection, we naturally lean towards the utilization of zero-shot/few-shot classification methodologies. in section 3.1, we discuss how to leverage the inherent structure of zero-shot classification to improve the performance of argumentdetection models, and in section 3.2, we discuss an approach based on efficiently tuning prompts for argument component classification using a small set of training examples. ## evaluation methodology in order to examine the practical viability of our proposed low-resource argument classification approach in a variety of real-world settings, we evaluate the performance and generalizability of our methods in two different scenarios: (i) a problem domain characterized by an abundance of reliable training data (section 5.1), wherein the availability of the data allows for training traditional supervised models, and (ii) a resource-constrained noisy domain more akin to real-world conditions (section 5.2), wherein llms as an effective lowresource alternative to supervised training, may seem a more suitable option to approach the problem. while we were able to carry out a small annotation project to collect data for the middle school domain, such annotations may not be feasible, especially if we wish to adapt the system to multiple new domains. further information about these problem domains can be found in section 4.1. the detailed overview of the baseline models we established for both the supervised and zero-shot/few-shot llm training approaches, as well as the details of our proposed low-resource argument classification methods, are discussed in section 4.2. ## conclusion in this work, we introduced an argument classification strategy that effectively leverages the logical entailment relationship within argument components, along with a parameter efficient prompttuning technique. our approach demonstrates remarkable efficiency in reducing data and computational requirements for training while maintaining high prediction accuracy. its robust performance across diverse scenarios highlights its practical applicability in real-world settings, making argument classification more accessible to researchers across various domains. notably, the model's ability to achieve competence with a minimal number of examples per class sets it apart from traditional dataintensive supervised alternatives. additionally, unlike expensive and timeintensive llm-based solutions, our proposed approach can reliably operate on smaller foundation models such as bart, offering expedited training and inference, making it a cost-effective and efficient solution suitable for in-house deployment and enjoying the added benefits of data privacy.