F in M ind- Y -Me at the Regulations Challenge Task: Financial Mind Your Meaning based on TH a LLE (2025)

Abstract: This paper presents our submission to the COLING 2025 regulation challenge, focusing on nine tasks in the regulatory and financial domains. The challenge aims to advance large language models beyond general-purpose capabilities, adapting them for regulatory and financial tasks using a unified framework of task-specific prompts and input templates. We propose a sequential fine-tuning approach that integrates reasoning-based training, tailored system prompts, and Chain-of-Thought (CoT) inference to optimize task-specific performance. This method improves accuracy and reliability across diverse tasks. Notably, CoT inference demonstrates exceptional effectiveness in handling complex scenarios and tasks requiring specific answer patterns, such as named entity recognition and financial calculations. Our model achieved an overall score of 54.801%, ranking 1st among all teams and becoming the top performer in the challenge. These results highlight the effectiveness of sequential fine-tuning, advanced reasoning techniques, and fine-tuned prompts in improving performance and scalability for complex regulatory and financial applications.

## introduction the coling 2025 regulations challenge is a rigorous initiative designed to advance the capabilities of large language models (llms) in understanding and processing complex regulatory and financial documents. this challenge comprises nine carefully crafted tasks that target critical aspects of regulatory text comprehension and practical application, such as deciphering domain-specific acronyms, extracting definitions, identifying named entities, answering intricate regulatory queries, and performing advanced analytics on financial filings. while llms such as gpt @xcite , llama @xcite , gemini @xcite , and qwen @xcite have demon-strated remarkable versatility across general natural language processing tasks, they often falter in specialized domains such as regulation and finance. these fields demand deep reasoning, multistep problem-solving, and precise contextual understanding-capabilities that traditional llms, optimized for straightforward, one-step responses, frequently lack. furthermore, their propensity to hallucinations exacerbates their limitations, particularly when confronted with tasks involving complex calculations, nuanced regulatory language, or sophisticated financial analyses. this paper presents a novel framework that enables a single llm to effectively manage multitasking across various regulatory and financial domains. the framework addresses a range of specialized tasks, these tasks collectively enable the model to navigate the complexities of regulatory and financial domains. collectively, these tasks require the model to demonstrate both the knowledge and capabilities needed to navigate the complexities of regulatory and financial domains, and each task demands precise management of domain-specific contexts and information. our approach integrates unified modeling @xcite with task-specific prompts @xcite and input templates @xcite , tailoring the focus and contextual comprehension of the model for each task to ensure coherent and relevant responses to regulatory and financial challenges. to optimize the learning and performance of the model, we employ sequential fine-tuning @xcite , where the model is progressively trained on tasks in a specific sequence. this approach leverages prior knowledge while minimizing the risk of catastrophic forgetting. to enhance the model's reasoning capabilities, we introduce reasoning-based training, which enables more logical analysis and interpretation of complex datasets by leveraging prior reasoning. during inference, we utilize chain of thought (cot) prompting @xcite , which guides the model through a step-bystep logical reasoning process. this method breaks down complex queries into manageable components, ensuring accurate and contextually relevant responses. by integrating these techniques, our approach significantly improves the performance of llms in handling regulatory and financial tasks, surpassing traditional direct-response methods. this contribution advances llms for specialized applications, opening new avenues for llms in complex and regulated environments. building on this foundation, the main contributions of this paper are as follows: the remainder of the paper is organized as follows: section 3 discusses related works; section 4 presents the methodology; section 5 outlines the experimental setup; section 6 details the results; section 7 addresses the limitations; and section 8 concludes the paper. ## task overview the coling 2025 regulations challenge comprises nine complex tasks aimed at evaluating diverse skills required for processing regulatory and financial texts. the abbreviation recognition task tests a model's ability to identify and expand acronyms prevalent in regulatory documents, emphasizing domain-specific terminology understanding. the definition recognition task involves accurately extracting definitions from dense legal and financial texts, demanding precise contextual comprehension. the named entity recognition (ner) task focuses on identifying and categorizing entities such as organizations, laws, dates, and monetary values, requiring high accuracy in structured data extraction. the question answering task challenges models to provide precise answers to intricate legal questions, testing their ability to interpret both explicit and implicit content. the link retrieval task assesses models' efficiency in locating specific legal documents, necessitating adept navigation through extensive regulatory corpora. the certificate question task evaluates the capability of llms to solve multiple-choice questions from professional financial certification exams, such as the chartered financial analyst (cfa) and certified public accountant (cpa) exams, highlighting their analytical proficiency in meeting global certification standards and achieving examination success. the xbrl analytics task examines a model's ability to extract and analyze financial data from extensible business reporting language (xbrl) filings, showcasing technical expertise in handling financial data formats. the common domain model (cdm) task focuses on understanding the fintech open source foundation's standards for financial industry interoperability. lastly, the model openness framework (mof) licenses task evaluates models on licensing requirements, emphasizing regulatory compliance understanding. collectively, these tasks represent a rigorous challenge, demanding advanced linguistic, analytical, and reasoning skills. ## methodology our methodology leverages four complementary strategies to enhance llms for regulatory and financial tasks: sequential fine-tuning to gradually build domain knowledge, task-specific prompts to align inputs and outputs effectively, reasoningbased training to improve logical problem-solving, and chain-of-thought prompting to ensure precise, template-aligned answers through structured reasoning. sequential fine-tuning is a strategic approach that incrementally enhances a capability of llms by adapting it to a series of tasks in a predefined order. this method builds on knowledge from earlier tasks to improve performance on subsequent tasks, enabling a comprehensive understanding of complex domains such as regulation and finance. in our framework, tasks are grouped by domain relevance and complexity. ## limitations and future work the primary challenge of this research is to develop a single llm capable of effectively multitasking across nine distinct regulatory and financial tasks through fine-tuning while maintaining versatility, domain expertise and efficient knowledge transfer. the llm must perform these tasks simultaneously without any performance degradation, mitigate task interference, and manage specialized terminologies these limitations underscore the need for more comprehensive, diverse and contextually relevant datasets, improved fine-tuning approaches, and the development of advanced reasoning strategies. future research should aim to broaden the range of regulatory and financial tasks to enhance the versatility and scalability of the llm. efforts should also focus on automating prompt engineering to reduce reliance on manual design and explore advanced reasoning methods, such as reinforcement learning with human feedback. furthermore, optimizing task sequences and addressing challenges such as computational resource demands, data dependencies, and processing costs are vital to improving system robustness and adaptability within dynamic regulatory and financial environments. ## conclusion this study presents a unified modeling framework that integrates task-specific prompts, input templates, and sequential fine-tuning to improve performance in regulatory and financial tasks on the coling2025 regulation challange. sequential fine-tuning demonstrates improvements in areas such as financial computations, though its variable impact underscores the importance for tailored strategies. fine-tuned system prompts outperform standard prompts, while reasoning-based training and chain-of-thought prompting further boost performance. our model achieved an overall score of 54.801% across all tasks, the highest among all participants, securing first place in the financial regulation competition and demonstrating excellence across all nine tasks. future work should focus on broadening task coverage, automating prompt engineering, refining sequential fine-tuning, and exploring hybrid models to enhance scalability and adaptability in dynamic regulatory contexts. task input templates system prompt abbreviation "expand the following acronym into its full form: acronym. answer:" you are an expert in abbreviation-expanded-form matching for financial regulation. analyze and expand the following acronym into its official full form. provide the most accurate expansion only. definition "define the following term: regulatory term or phrase. answer:" define the following term while categorizing it into regulatory or financial domains (e.g., federal reserve regulations, accounting). provide the definition clearly and concisely. ner "given the following text, only list the following for each: specific organizations, legislations, dates, monetary values, and statistics: input text." you are an expert in name entity recognition. extract and classify entities such as organizations, legislations, dates, monetary values, and statistics from the given text. return the output in json format with proper labels. qa "provide a concise answer to the following question: detailed question? answer:" you are an expert in regulations and finance. provide precise and accurate answers to detailed questions about regulatory practices or laws based on the provided query. link retrieval "provide a link for ... law, write in the format of ("law: link" or "law: not able to find a link for the law")" you are an expert in link retrieval. provide a link for the specified regulation based on its name and format. ensure the url follows the correct structure (e.g., eur-lex). return only the link or specify if unavailable. cfa "(this context is used for the question that follows: context). please answer the following question with only the letter and associated description of the correct answer choice: question and answer choices. answer:" you are a financial expert tasked with solving a certificate exam question. break down the query logically, analyze each answer choice, and provide the best answer based on regulations or financial principles. xbrl "provide the exact answer to the following question: detailed question? answer:" you are an expert in extensible business reporting language (xbrl). provide precise answers to detailed questions about financial data using extensible business reporting language. address areas such as definitions, calculations, or us gaap tags systematically. cdm "provide a concise answer to the following question related to financial industry operating network's (fino) common domain model (cdm): detailed question? answer:" you are an expert in common domain model (cdm). provide accurate and precise responses to questions related to the cdm within the financial and fintech context. break down terms or processes where applicable. mof "provide a concise answer to the following question about mof's licensing requirements: detailed question? answer:" you are an expert in model openness framework (mof). answer queries about license requirements, osi approval, or abbreviations with precision and clarity. provide only the relevant details. step1: "provide precise answers to detailed questions about financial data extraction and application using xbrl (extensible business reporting language) filings, a standardized digital format for sharing and analyzing financial information. this task covers five areas: defining xbrl terms, domain-specific queries, financial math, numeric queries, and providing the correct us gaap xbrl tags (e.g., us gaap xbrl tag for revenue should be answered asusgaap :revenuefromcontractwithcustomerexcludin-gassessedtax'. ensure responses strictly match the correct answer without additional explanation.when answering questions about xbrl, it's essential to follow a structured approach. here's how to methodically address these types of questions:" step1: "question as question" execution of extraction and application logic using the structured reasoning methodology for contextspecific results (e.g., matching correct us gaap tags). step2: "you are a financial expert tasked with carefully reading, analyzing, and answering the following extensible business reporting language. please follow the steps below:" step2: "your task is to read the extensible business reporting language xbrl question question and find the final answer based on the explanation provided response. provide only the final answer,final answer is ..." cdm addressing cdm inquiries from the fintech open source foundation, applying logical mapping to provide relevant responses for complex financial modeling or structured analysis. step1: "deliver precise responses to questions about the fintech open source foundationś finos common domain model cdm)." step1: "question: question" mof licensing logic for mof compliance focusing on financial license inquiries or compliance context by narrowing domain relevance. step1: "deliver precise responses to questions concerning the requirement of license under the model openness framework." step1: "question: question"