L o RA - PAR : A Flexible Dual-System L o RA Partitioning Approach to Efficient LLM Fine-Tuning (2025)

Abstract: Large-scale generative models like DeepSeek-R1 and OpenAI-O1 benefit substantially from chain-of-thought (CoT) reasoning, yet pushing their performance typically requires vast data, large model sizes, and full-parameter fine-tuning. While parameter-efficient fine-tuning (PEFT) helps reduce cost, most existing approaches primarily address domain adaptation or layer-wise allocation rather than explicitly tailoring data and parameters to different response demands. Inspired by “Thinking, Fast and Slow,” which characterizes two distinct modes of thought—System 1 (fast, intuitive, often automatic) and System 2 (slower, more deliberative and analytic)—we draw an analogy that different “subregions” of an LLM’s parameters might similarly specialize for tasks that demand quick, intuitive responses versus those requiring multi-step logical reasoning. Therefore, we propose LoRA-PAR, a dual-system LoRA framework that partitions both data and parameters by System 1 or System 2 demands, using fewer yet more focused parameters for each task. Specifically, we classify task data via multi-model role-playing and voting, and partition parameters based on importance scoring, then adopt a two-stage fine-tuning strategy of training System 1 tasks with supervised fine-tuning (SFT) to enhance knowledge and intuition and refine System 2 tasks with reinforcement learning (RL) to reinforce deeper logical deliberation next. Extensive experiments show that the two-stage fine-tuning strategy, SFT and RL, lowers active parameter usage while matching or surpassing SOTA PEFT baselines.

## introduction large language models (llms) such as @xcite and openai-o1 * huangyining1987@gmail.com † b.li2@siat.ac.cn ‡ tkk2012@gmail.com § 523062863@qq.com 1: overview of our proposed workflow. (1) sample splitter: multiple llms role-play the target model and vote to classify the corpus into system 1 and system 2 data. thehumanbrainengagesinfastandslowthinking. inourframework,wemimicthisapproachby classifyingtasksintosystem1andsystem2. tomhas8coins,andeachcoinis eitherapenny(1cent)oranickel(5 cents).iftheirtotalvalueis32cents, howmanynickelsdoestomhave?" datatype2 "whichnumberis larger,10or5?" datatype1 reasoningrl system1 system2 fast intuitive automatic slow deliberative analytic end2endsft llmparameters "tomhas6nickels. " "10islargerthan5. " answer answer question answer learningthe waythat mappingfrom questionto answer directly. question letxbethenumberofnickels.then thereare(8-x)pennies. eachnickelis5cents,eachpennyis1 cent. totalvalueequation:5x+1*(8-x)=32 simplify:5x+8-@xmath0. 2. 3. 4. answer reasoningprocess layer1 layer2 ... layerz partition parameters forsft(system1) partition parameters forrl(system2) of different tasks or reasoning levels. although there have been some recent attempts to design more task-or data-aware peft solutions @xcite , these works largely focus on domain adaptation or layer-wise parameter allocation rather than explicitly targeting more advanced, multi-step reasoning capabilities. meanwhile, inspired by thinking, fast and slow @xcite , we incorporate the dual-system concept into parameter-efficient fine-tuning for llms. specifically, as shown in figure 2 , we draw on the idea that the human brain engages partially distinct neural processes for system 1 versus system 2. recent studies further evidence that large language models can manifest or benefit from distinct "fast" vs. "slow" modes: @xcite exhibit human-like intuitive biases, @xcite propose dynamic decision mechanisms inspired by kahneman's framework, and more general discussions support bridging cognitive dualprocess theories and ai @xcite . by analogy, we posit that an llm's parameters can be partitioned into "subregions" specialized for different response demands. we implement this in three steps: (1) we use multi-expert role-play and voting to classify each training instance into system 1 or system 2 tasks, ensuring quick, direct "fast thinking" problems are kept separate from more deliberative multi-step tasks; 2 related work ## limitations our experiments confirm that selectively activating lora parameters for system 1 and system 2 yields clear benefits in both performance and parameter efficiency. by combining role-play-based data splitting with importance-driven parameter partitioning, we effectively approximate the dual-process paradigm within an llm. nonetheless, there are a few limitations: (1) multi-model annotations. although using multiple teacher llms improves labeling quality, it adds computational overhead and presupposes access to diverse, high-capacity models. ## conclusion we presented a dual-system peft framework inspired by thinking, fast and slow, wherein system 1 and system 2 "subregions" of lora parameters address fast, intuitive tasks vs. slower, multistep reasoning. our pipeline (i) classifies queries via multi-model role-playing and voting, (ii) determines each lora parameter's importance relative to system 1 or system 2, and (iii) fine-tunes in two stages-sft for intuitive response, then rl for deeper logic. across gsm8k, mmlu, and humaneval, we find that focusing updates on topranked parameters not only cuts active parameter usage (often to 40% or less) but also surpasses baseline peft methods that uniformly fine-tune larger parameter sets. by assigning each subregion to a distinct "cognitive" mode, we effectively reconcile fast vs. slow thinking within a single llm. we believe this "subregion specialization" opens new directions for cognitively guided llm adaptations, enabling more efficient models that still excel at both intuitive and methodical reasoning.