Can One Domain Help Others? A Data-Centric Study on Multi-Domain
  Reasoning via Reinforcement Learning
Authors: name: Yu Li, name: Zhuoshi Pan, name: Honglin Lin, name: Mengyuan Sun, name: Conghui He, name: Lijun Wu

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a
powerful paradigm for enhancing the reasoning capabilities of LLMs. Existing
research has predominantly concentrated on isolated reasoning domains such as
mathematical problem-solving, coding tasks, or logical reasoning. However, real
world reasoning scenarios inherently demand an integrated application of
multiple cognitive skills. Despite this, the interplay among these reasoning
skills under reinforcement learning remains poorly understood. To bridge this
gap, we present a systematic investigation of multi-domain reasoning within the
RLVR framework, explicitly focusing on three primary domains: mathematical
reasoning, code generation, and logical puzzle solving. We conduct a
comprehensive study comprising four key components: (1) Leveraging the GRPO
algorithm and the Qwen-2.5-7B model family, our study thoroughly evaluates the
models' in-domain improvements and cross-domain generalization capabilities
when trained on single-domain datasets. (2) Additionally, we examine the
intricate interactions including mutual enhancements and conflicts that emerge
during combined cross-domain training. (3) To further understand the influence
of SFT on RL, we also analyze and compare performance differences between base
and instruct models under identical RL configurations. (4) Furthermore, we
delve into critical RL training details, systematically exploring the impacts
of curriculum learning strategies, variations in reward design, and
language-specific factors. Through extensive experiments, our results offer
significant insights into the dynamics governing domain interactions, revealing
key factors influencing both specialized and generalizable reasoning
performance. These findings provide valuable guidance for optimizing RL
methodologies to foster comprehensive, multi-domain reasoning capabilities in
LLMs.