Title: CRPA: Curriculum-driven Reinforcement Pre-Alignment for Domain-Adaptive Vision-Language Models
Primary area: foundation or frontier models, including LLMs
Keywords: Vision Lanuage Models, Reforcement Learning, Post training

Abstract: Vision-Language Models (VLMs) demonstrate remarkable general-purpose capabilities but often fall short in specialized domains such as medical imaging or geometric problem-solving. Supervised Fine-Tuning (SFT) can enhance performance within a target domain, but it typically causes catastrophic forgetting, limiting its utility as a general AI agent. The central challenge, therefore, is to adapt VLMs to new domains while preserving their general-purpose capabilities.
Continual pretraining is effective for expanding knowledge in Large Language Models (LLMs), but it is less feasible for VLMs due to prohibitive computational costs and the unavailability of pretraining data for most open-source models. This necessitates efficient post-training adaptation methods. Reinforcement learning (RL)–based approaches such as Group Relative Policy Optimization (GRPO) have shown promise in preserving general abilities, yet they often fail in domain adaptation scenarios where the model initially lacks sufficient domain knowledge, leading to optimization collapse.
To bridge this gap, we propose Curriculum-driven Reinforcement Pre-Alignment (CRPA), a novel post-training paradigm that introduces a curriculum-aware progressive modulation mechanism. In the early phase, CRPA applies partial output constraints to safely expose the model to new domain concepts. As the model’s domain familiarity increases, training gradually transitions to full generation optimization, refining responses and aligning them with domain-specific preferences. This staged adaptation balances domain knowledge acquisition with the preservation of general multimodal capabilities.
Extensive experiments across specialized domains (e.g., OpenI for medical imaging and Geo170K for geometry) and general benchmarks (e.g., COCO Captions) validate the effectiveness of CRPA. Results show that CRPA achieves domain-specific performance competitive with SFT while significantly outperforming SFT in retaining general multimodal understanding, establishing a practical pathway toward building high-performing, domain-adaptive VLMs.