Measuring the Groundedness of Legal Question-Answering Systems (2024)

Abstract: In high-stakes domains like legal question-answering, the accuracy and trustworthiness of generative AI systems are of paramount importance. This work presents a comprehensive benchmark of various methods to assess the groundedness of AI-generated responses, aiming to significantly enhance their reliability. Our experiments include similarity-based metrics and natural language inference models to evaluate whether responses are well-founded in the given contexts. We also explore different prompting strategies for large language models to improve the detection of ungrounded responses. We validated the effectiveness of these methods using a newly created grounding classification corpus, designed specifically for legal queries and corresponding responses from retrieval-augmented prompting, focusing on their alignment with source material. Our results indicate potential in groundedness classification of generated responses, with the best method achieving a macro-F1 score of 0.8. Additionally, we evaluated the methods in terms of their latency to determine their suitability for real-world applications, as this step typically follows the generation process. This capability is essential for processes that may trigger additional manual verification or automated response regeneration. In summary, this study demonstrates the potential of various detection methods to improve the trustworthiness of generative AI in legal settings.

## introduction generative ai systems are increasingly employed in high-stakes domains such as legal questionanswering, where accuracy and trust are paramount @xcite @xcite . a significant challenge in these applications is the detection of outputs that are not grounded in the input data (context), which can compromise user trust and diminish the application's value @xcite when does the duty to preserve documents end? focus on sdny case law. the duty to preserve evidence arises when a party reasonably anticipates litigation. this duty generally requires the party to suspend its routine document retention/destruction policy and put in place a 'litigation hold' to ensure the preservation of relevant documents. the duty to preserve evidence ends when the litigation is resolved or when the party no longer reasonably anticipates litigation. the duty to preserve evidence arises when a party reasonably anticipates litigation. this duty generally requires the party to suspend its routine document retention/destruction policy and put in place a 'litigation hold' to ensure the preservation of relevant documents. the duty to preserve evidence ends when the litigation is resolved or when the party files a motion to dismiss. ## grounding definition grounding in legal question-answering systems refers to the extent to which an ai-generated response is firmly rooted in, supported by, and directly attributable to the provided legal source material. it ensures the model's output aligns with and accurately represents the information in the input data, avoiding fabrication, extraneous details, or misleading content. a well-grounded response should adhere closely to the facts, legal principles, and reasoning presented in the source material, without introducing unsupported claims or misrepresenting the legal context @xcite . several key aspects ensure the reliability of aigenerated legal responses. factual alignment and relevance are crucial, ensuring the content reflects the source documents and addresses the legal query accurately. source attribution allows tracing information back to specific input texts, while legal interpretation fidelity ensures conclusions are substantiated by the provided materials. this involves not only accurately conveying factual information but also maintaining the integrity of legal procedures, correctly interpreting the context, and using appropriate legal terminology. the generated responses must adhere to the given context, avoiding unsupported claims or extrapolations, and preserving the nuances and complexities of legal language and concepts @xcite . the assessment of grounding in legal ai responses involves a comprehensive evaluation of how faithfully the generated content aligns with the retrieved legal context. this evaluation considers various aspects of the response, including its factual accuracy, the appropriateness of legal interpretations, the coherence of legal reasoning, and the proper use of legal terminology. grounding is vital in legal applications to maintain the integrity of legal advice, ensure compliance with laws and precedents, and prevent misinformation. by ensuring strong grounding, legal question-answering systems can provide more reliable, trustworthy, and legally sound responses, which is crucial in the high-stakes environment of legal practice and decision-making. ## dataset creation in this section, we will describe and list all the steps involved in creating the groundedness classification dataset used in our benchmarking. ## benchmarking methodologies this section overviews the diverse methodologies employed in our benchmark study for quantifying response grounding, systematically evaluating approaches that assess adherence of generated responses to provided context. ## experimental set-up our benchmarking study aimed to evaluate various methods for classifying llm responses as grounded or ungrounded relative to a given context and query. methodology despite the varied granularity of approaches (response-level vs. sentence-level), we standardized outputs to binary classifications for consistent comparison. we developed each method on the training set, optimized parameters on the development set, and conducted final evaluations on the test set. performance metrics we assessed classification accuracy (including macro-averaged f1, precision, and recall) and computational efficiency through latency measurements. latency was computed as the average processing time across all samples in the development set. these metrics provide insights into each approach's practical applicability. computational resources local approaches utilized amazon ec2 g5 instances (8xlarge) foot_3 . prompting-based methods were executed via azure openai services foot_4 , aws bedrock (anthropic's claude) foot_5 , and anthropic's api directly, ensuring diverse and robust evaluation environments. ## groundedness classification results our benchmark evaluation of groundedness classification approaches revealed insightful performance trade-offs, as shown in tab. 3. the metrics include classification precision, recall, f1-score, and accuracy, providing a comprehensive view of each method's applicability. the multi-stage prompt chaining approach, deepeval claims verify, achieved top classification metrics, but with high latency (26.1 seconds per request). in contrast, direct prompting with gpt-4o achieved the second-highest scores with significantly lower latency (2.2 seconds), as illustrated in fig. 2 . a clear speed-performance trade-off emerged across methods. similarity-based approaches (cos_sim and quip) were fastest but struggled with ungrounded response identification. nli methods showed improved performance at the cost 2 : development set results for our benchmark. we report the f1-scores (y-axis) for each method and the corresponding latency (x-axis) in seconds per response. approach names denoted with * were run on an aws ml.8xlarge instance. unexpectedly, complex prompt chaining approaches like refchecker and selfcheckgpt underperformed, highlighting challenges in developing universally effective methods across diverse contexts. these findings emphasize the importance of balancing task performance and computational efficiency when selecting a groundedness classification approach, with optimal choices depending on specific application requirements and resource constraints. ## error analysis we conducted a detailed investigation into the types of response errors present in our benchmark dataset to gain deeper insights into ungrounded content. through examination of error spans in the training set, we identified six distinct error types. the models were instructed to select from our predefined error types (tab. 5, app. a.2). focusing on the development set, our analysis revealed interesting patterns. the llms achieved exact agreement on the hallucination type in 29% of cases, with at least one overlapping error type for each response. gpt-4o typically predicted a misclassification analysis we conducted a misclassification analysis on our best-performing model, deepeval claims verify, to gain deeper insights into its performance across different error types. as summarized in tab. 4, terminological errors showed the highest misclassification rate (67%), despite their low frequency, followed by factual inaccuracies (20%) and procedural errors (20%). these findings reveal the varying challenges posed by different error categories and highlight areas for potential improvement in groundedness classification models, particularly in handling less common but difficult-to-classify error types. ## conclusion our comprehensive benchmark study on groundedness classification of legal question-answering systems has revealed significant insights into performance and efficiency trade-offs. the multi-stage prompt chaining approach, deepeval claims verify, emerged as the top performer with an f1 score of 0.80, closely followed by direct prompting using gpt-4o at 0.77, which demonstrated lower latency. these results highlight the potential of advanced prompting techniques in achieving high accuracy. similarity-based and natural language inference methods, while less accurate, offered fast processing times. our response error type classification identified factual inaccuracies and reasoning errors as the most prevalent types of ungrounded content, providing direction for future improvements. the study underscores the critical balance between task performance, computational efficiency, and ease of implementation when selecting groundedness classification methods. with topperforming methods achieving f1 scores of 0.80, this benchmark represents a significant advancement in the reliable assessment of ai-generated content across diverse applications.