A Diverse and Effective Retrieval-Based Debt Collection System with Expert Knowledge (2025)

Abstract: Designing effective debt collection systems is crucial for improving operational efficiency and reducing costs in the financial industry. However, the challenges of maintaining script diversity, contextual relevance, and coherence make this task particularly difficult. This paper presents a debt collection system based on real debtor-collector data from a major commercial bank. We construct a script library from real-world debt collection conversations, and propose a two-stage retrieval based response system for contextual relevance. Experimental results show that our system improves script diversity, enhances response relevance, and achieves practical deployment efficiency through knowledge distillation. This work offers a scalable and automated solution, providing valuable insights for advancing debt collection practices in real-world applications.

## introduction debt collection plays a crucial role in the financial industry. in practice, outbound calls for debt recovery are typically handled by experienced experts, since negotiating with debtors is often challenging. consequently, large companies must employ substantial number of staff to manage daily debt collection tasks, leading to high operational costs. this has spurred interest in developing systems that assist human experts or automate outbound calls, making it a burgeoning area of research @xcite . recent advancements have demonstrated the feasibility of automatic outbound agents @xcite . currently, many collection chatbots are flow-based systems configured figure 1 : an exemplar between a debtor and a collector, with three candidate responses. the debtor's intent is labeled in red while the strategies in collector's responses are labeled in blue. with rule-based frameworks authored by experts @xcite . in these systems, the chatbot predicts the debtor's intent at each stage and provides predefined responses based on established rules. however, such flow-based systems face notable limitations. they heavily depend on expert-crafted rules, making them difficult to update and scale to different scenarios due to their complexity. additionally, these systems lack response diversity, as the output is fixed for each scenario. to address these limitations, researchers have explored using pretrained language models to generate responses based on dialogue context @xcite @xcite . these methods eliminate the need for predefined rules by fine-tuning models on large-scale debt collection conversations. however, generative models often produce responses that may be ineffective in debt collection. the responses are also difficult to control due to their inherent uncertainty. in view of these problems, retrieval based response system become a better choice in practice, as the response outputs are more controllable. typically, it consists of two stages: script foot_0 generation and response system implementation. as for script generation, current practice remains predominantly a manual process undertaken by experienced experts. however, previously-mentioned challenges still exist. first, achieving script diversity is inherently challenging, as generating distinct responses for a wide range of scenarios demands significant effort. second, updating the system is resourceintensive, requiring expert intervention to craft and integrate new scripts with each revision. to address these issues, automatic script generation from real conversations has become a promising direction. on the other hand, response retrieval in debt collection is particularly challenging due to several factors. in practice, we find that embedding-based methods, while effective in other domains @xcite , struggle here due to the difficulty of distinguishing positive from negative samples without manual annotation. typically, positive samples are selected from actual responses in the dialogue, and negative samples are randomly chosen from other dialogues. however, this random negative sampling often leads to situations where the selected "negative" samples are actually suitable responses for the current dialogue, resulting in "false negatives". this increases the complexity of model training and affects the accuracy of the system, particularly when multiple responses in the script library appear valid during inference. to this end, we propose a two-stage retrieval based response system to select the most effective script from script library. in this work, we propose a comprehensive system for automatic outbound chat-bots that integrates script generation and selection models. leveraging the capabilities of large language models (llms), we first generate diverse and effective scripts based on real-world conversations while incorporating expert knowledge to enhance the quality and naturalness of the dialogues. after that, to ensure the safety and appropriateness of outbound calls, we frame the problem as response selection, where the model must choose the optimal response given the dialogue context. since traditional embedding-based models often struggle to distinguish between similar scripts with identical strategies, to overcome this, we design a two-stage retrieval pipeline: the first stage employs a pretrained model to recall n relevant responses, and the second stage uses llms to evaluate and select the best response based on three aspects: empathetic engagement, effective problem-solving and contextual relevance. the contributions of our work are threefold: ## methodology our proposed framework scores includes two modules: (a) automatic script library construction and (b) retrieval based collection system. ## experiments in this section, we present the experimental settings and results of our proposed methods. ## related work retrieval-based dialogue systems retrievalbased dialogue systems aim to identify the most appropriate response from a set of candidates @xcite . these systems are widely applied in domains such as customer service q&a and forum post interactions @xcite @xcite . modern approaches predominantly leverage pre-trained language models (plms) like bert @xcite , fine-tuned using contrastive learning on domain-specific corpora @xcite @xcite . to enhance semantic relevance and contextual coherence, han et al. @xcite incorporate fine-grained labels during posttraining. su et al. @xcite propose a novel post-training method that improves context embeddings. additionally, han et al. @xcite introduce edhns, which optimizes contrastive learning by focusing on harder-to-distinguish negative examples. automatic outbound chatbots automatic outbound chatbots are designed to engage customers in conversations to achieve specific goals, such as debt collection or advertising. traditional systems often relied on flow-based approaches due to their straightforward logic and ease of implementation @xcite . however, these systems heavily depend on expert-defined rules and are challenging to update. to address these limitations, recent research has shifted towards response generation using plms. jin et al. @xcite propose a persuasion framework that integrates both semantic understanding and strategic considerations. zhang et al. @xcite enhance response generation by incorporating user profiles extracted during conversations. qian et al. @xcite redefine the dialogue process as a sequence-labeling problem, leveraging a dual-path model for joint multi-task learning. ## conclusion in this work, we designed and evaluated a comprehensive system, scores, for automating outbound debt collection, addressing challenges of script diversity, adaptability, and effective response selection. by combining the script generation capabilities of llms with a robust two-stage retrieval framework, we achieved notable improvements in response effectiveness. besides, knowledge distillation enhanced its efficiency for real-world deployment. more importantly, the flexibility of this framework allows it to be adapted to a wide range of domains, such as customer support and telemarketing. future work will focus on further refining script diversity, improving real-time response evaluation, and expanding the framework's applicability to ensure even higher levels of performance and adaptability in diverse settings.