Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning (2025)

Abstract: Large Language Models (LLMs) have achieved impressive results across numerous domains, yet they experience notable deficiencies in legal question-answering tasks. LLMs often generate generalized responses that lack the logical specificity required for expert legal advice and are prone to hallucination, providing answers that appear correct but are unreliable. Retrieval-Augmented Generation (RAG) techniques offer partial solutions to address this challenge, but existing approaches typically focus only on semantic similarity, neglecting the logical structure essential to legal reasoning. In this paper, we propose the Logical-Semantic Integration Model (LSIM), a novel supervised framework that bridges semantic and logical coherence. LSIM comprises three components: reinforcement learning predicts a structured fact-rule chain for each question, a trainable Deep Structured Semantic Model (DSSM) retrieves the most relevant candidate questions by integrating semantic and logical features, and in-context learning generates the final answer using the retrieved content. Our experiments on a real-world legal QA dataset-validated through both automated metrics and human evaluation-demonstrate that LSIM significantly enhances accuracy and reliability compared to existing methods.

## introduction the rapid advancement of large language models (llms) has provided ordinary individuals with a way to access affordable legal services, substantially expanding their opportunities to obtain legal assistance @xcite . however, given the diversity, complexity, and vague expressions of legal questions, responses generated by llms often lack logical specificity and may include hallucinations, exposing users * corresponding author. to potential legal risks and financial losses @xcite . retrieval-augmented generation (rag) @xcite has emerged as a promising approach to mitigate hallucinations and improve response accuracy by retrieving relevant legal cases and provisions as contextual knowledge sources for llms. nevertheless, existing rag methods primarily focus on semantic similarity and frequently overlook the intricate logical structures and reasoning essential for addressing complex legal issues, limiting their effectiveness in real-world legal applications. as illustrated in fig. 1 , this paper introduces a novel llm framework specifically designed to address complex legal qa tasks. our approach integrates learnable chain-of-thought (cot) reasoning as logical structures with supervised rag, enhanced by the deep structured semantic model (dssm) @xcite . by embedding logical reasoning into the retrieval process, the framework ensures that both semantic relevance and logical coherence are maintained. additionally, incontext learning is leveraged to synthesize highquality answers, utilizing the retrieved auxiliary information to deliver precise, contextually appropriate responses. the contributions of this work are fourfold: ## methodology in this section, we propose our logical-semantic integration model (lsim), as depicted in fig. 2 . ## conclusion this paper addresses the inherent limitations of llms in generating professional legal responses. we propose a novel framework, lsim, designed to enhance the legal llm reasoning by integrating the learnable logical structure and semantic information of legal questions. the lsim framework is composed of three key components: (1) reinforcement learning predicts the fact-rule chain of thought for a given legal query, guiding the reasoning process; we validate the effectiveness of lsim through experiments on a real-world legal qa dataset. results from both automated metrics and manual evaluations demonstrate the superior performance of the proposed framework in delivering accurate and expert-level legal answers. in the future, we plan to extend the lsim framework to other specialized domains, such as healthcare and finance. additionally, we aim to incorporate multi-turn interactions with users, leveraging their real-time feedback to further refine the model's performance and adaptability.