APT : Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training (2025)

Abstract: Large Language Models (LLMs) often require domain-specific fine-tuning to address targeted tasks, which risks degrading their general capabilities. Maintaining a balance between domain-specific enhancements and general model utility is a key challenge. This paper proposes a novel approach named APT (Weakness Case Acquisition and Iterative Preference Training) to enhance domain-specific performance with self-generated dis-preferred weakness data (bad cases and similar cases). APT uniquely focuses on training the model using only those samples where errors occur, alongside a small, similar set of samples retrieved for this purpose. This targeted training minimizes interference with the model’s existing knowledge base, effectively retaining generic capabilities. Experimental results on the LLama-2 and Mistral-V0.3 models across various benchmarks demonstrate that APT ensures no reduction in generic capacity and achieves superior performance on downstream tasks compared to various existing methods. This validates our method as an effective strategy for enhancing domain-specific capabilities without sacrificing the model’s broader applicability.

## introduction large language models (llms) can be applied to multiple domains, such as coding @xcite , mathematics @xcite , general knowledge question answering @xcite . however, substantial differences exist among these domains and tasks, and real-world applications often suffer from limited domain-specific data due to industry barriers. consequently, enhancing domain models with a small amount of domain-specific data combined with general data @xcite has emerged as a prominent research focus. recent work has emphasized self-training to improve task-specific capabilities, * corresponding author figure 1 : similar to how people improve in weaker areas by focusing on their mistakes and practicing similar problems, our method identifies model deficiencies and uses targeted training to enhance performance. in this paper, we propose the apt method, which utilizes a small number of self-generated negative samples for iterative preference training. the dis-preferred data generation process does not require complex task-specific data construction. similar to how humans learn from similar problems shown in figure 1 , our method has the model generate responses to tasks and add incorrect responses ("bad cases") back into this iteration training data. additionally, a retrieval module identifies similar samples ("similar cases") to add more relevant training data. these two types of data are then used for training, which enables the model to learn from its errors and generalize across related tasks, effectively improving its performance on specific tasks. the model training component involves constructing preference data, where model-generated responses are non-preference data and original correct answers are preference data. through iterative preference learning, the model optimizes its output by recognizing and correcting errors while learning from similar samples. this approach strengthens the model's ability to handle challenging cases, ultimately improving performance. through extensive experimental validation, we show that apt is effective in identifying and optimizing model flaws across several domains, such as mathematical reasoning, coding, and instructionfollowing. the results indicate improvements of up to 5.9% in llama2-7b and 6.0% in mistral-7b, outperforming existing methods such as @xcite . additionally, ablation studies show that acquiring bad case data, retrieving similar cases, and optimizing preference data training objectives and iterations all contribute to enhancing model performance. final scaling experiments demonstrate the method's potential for future improvements on stronger models. our contributions are as follows: ## conclusion this work introduces a unified framework apt to enhance domain-specific llm capabilities further. the framework significantly improves performance across multiple areas, including instructionfollowing, mathematical reasoning, and coding. we successfully identified model deficiencies and effectively optimized them by finding similar cases for targeted training, thereby improving the overall model performance. our results demonstrate the potential for further optimization of llms on challenging data like humans, providing a promising solution for enhancing model capabilities.