Debt Collection Negotiations with Large Language Models: An Evaluation System and Optimizing Decision Making with Multi-Agent (2025)

Abstract: Debt collection negotiations (DCN) are vital for managing non-performing loans (NPLs) and reducing creditor losses. Traditional methods are labor-intensive, while large language models (LLMs) offer promising automation potential. However, prior systems lacked dynamic negotiation and real-time decision-making capabilities. This paper explores LLMs in automating DCN and proposes a novel evaluation framework with 13 metrics across 4 aspects. Our experiments reveal that LLMs tend to over-concede compared to human negotiators. To address this, we propose the Multi-Agent Debt Negotiation (MADeN) framework, incorporating planning and judging modules to improve decision rationality. We also apply post-training techniques, including DPO with rejection sampling, to optimize performance. Our studies provide valuable insights for practitioners and researchers seeking to enhance efficiency and outcomes in this domain.

## introduction finance, as a negotiation-intensive field, involves the distribution and exchange of financial interests, requiring a higher level of understanding of information and rational decision-making @xcite . due to various personal financial issues, a large volume of non-performing loans (npls) arises each year across banks and financial companies, with debtors often being unable to repay their debts after prolonged overdue periods @xcite . negotiation and mediation are necessary to resolve their credit issues and minimize the losses for financial institutions (creditors) @xcite . traditionally, the debt collection process has been labor-intensive, and data * work done as an intern at ant group. ## data collection our data is primarily divided into two parts, as shown in figure 1 . the basic debt information is known to both the debtor and the creditor, while the debtor's personal financial data is not accessible to the creditor. we now explain how each of these two data components was collected. ## evaluation system different from traditional negotiation evaluations, we argue that the dcn task requires a more comprehensive assessment framework. as illustrated in figure 3 , we developed a evaluation system based on four aspects and extended several average metrics for a comprehensive assessment. ## experiments and results in this section, we report the implementation details and the benchmark performances of several wellknown llms in the dcn task on our dataset. ## method 6.1 a multi-agent framework for dcn to balance the model's attention between debt recovery quality and the debtor's financial health, and to avoid decisions that may harm the creditor's interests in order to reach an agreement, we propose a method to enhance the decision alignment for dcn. inspired by recent advancements in llm-asa-judge frameworks @xcite and llm planning methodologies @xcite , we designed the framework illustrated in figure 4 . the subsequent sections provide a detailed explanation hello, zhang shan. you have a debt of 15,000 yuan that is 39 days overdue. do you have any difficulty in repaying it? hello, my previous investment failed and i don't have much money now. can i take my time? the second category of debtor. his current assets are insufficient but the future increase in assets is relatively normal. i can allow him longer installments but no reductions. ask(@xmath0, ask(@xmath1, ask(@xmath2. i can only accept a 5% reduction. ## conclusion in this paper, we introduced a comprehensive framework for evaluating ai agents in debt collection negotiations (dcn), addressing both the negotiation process and its outcomes. by leveraging synthetic debt data generated through ctgan, we evaluated various ai-driven strategies, focusing on improving debt recovery rate and efficiency. our enhanced llm-based multi-agent framework, which incorporates planning and judging modules, demonstrated significant improvements in negotiation performance. additionally, the application of dpo with reject sampling helped optimize the agents' focus on key objectives, leading to better results on the qwen2.5-7b model. this work provides valuable insights into the use of ai in financial negotiations and lays the groundwork for future advancements in ai-assisted debt collection.