RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format
Authors: Zhehao Huang
Yuhang Liu
Baijiong Lin
Yixin Lou
Zhengbao He
Hanling Tian
Tao Li
Xiaolin Huang
Venue: ICLR 2026 Oral (2026)
Keywords: Large Reasoning Model
Instruction Following
Model Merging
Null-Space
Primary area: foundation or frontier models, including LLMs

Abstract: Large reasoning models (LRMs) excel at a long chain of reasoning but often fail to faithfully follow instructions regarding output format, constraints, or specific requirements. We investigate whether this gap can be closed by integrating an instruction-tuned model (ITM) into an LRM. Analyzing their differences in parameter space, namely task vectors, we find that their principal subspaces are nearly orthogonal across key modules, suggesting a lightweight merging with minimal interference. However, we also demonstrate that naïve merges are fragile because they overlook the output format mismatch between LRMs (with explicit *thinking* and *response* segments) and ITMs (answers-only). We introduce **RAIN-Merging** (Reasoning-Aware Instruction-attention guided Null-space projection Merging), a gradient-free method that integrates instruction following while preserving thinking format and reasoning performance. First, with a small reasoning calibration set, we project the ITM task vector onto the null space of forward features at thinking special tokens, which preserves the LRM's structured reasoning mechanisms. Second, using a small instruction calibration set, we estimate instruction attention to derive module-specific scaling that amplifies instruction-relevant components and suppresses leakage. Across four instruction-following benchmarks and nine reasoning & general capability benchmarks, RAIN-Merging substantially improves instruction adherence while maintaining reasoning quality. The gains are consistent across model scales and architectures, translating to improved performance in agent settings.

Published as a conference paper at ICLR 2026
RAIN-MERGINE A GRADIENT-FREE METHOD TO EN- HANCE INSTRUCTION FOLLOWING IN LARGE REASON- ING MODELS WITH PRESERVED THINKING FORMAT
Zhehao Huang- Yuhang Liu- Baijiong Lin
Yixin Lou Zhengbao He* Hanling Tian
Tao Li Xiaolin Huang 1.3
Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing,
Shanghai Jiao Tong University
'hhe Hong Kong University of Science and Technology (Guangzhou)
*MoB Key Laboratory System Control and Information Processing (Shanghai)
(kinght_h, yuhangliu, wwwloo_ooooooe 00201710__07701_11111111110 1er,,ssteet lstefanie, hanlingtian Esjtu.edii 1oulou_0722
(1i tai, www.aiiaiiiiie wawpolliinnnnnnssttaaaa edu. en
(blin241) wec.neect..ksstteeee....... edu...eeddcccmmm en
ABSTRACT
Large reasoning models (LRMs) excel at long chain of reasoning but often fail faithfully follow instructions regarding output format, constraints, or spe- cific requirements. We investigate whether this gap can be closed by integrat- ing an instruction-tuned model (ITM) into an LRM. Analyzing their differences in parameter space, namely task vectors, we find that their principal subspaces are nearly orthogonal across key modules, suggesting lightweight merging with minimal interference. However, also demonstrate that naive merges are fragile because they overlook the output format mismatch between LRMs (with explicit thinking and response segments) and ITMs (answers-only). Instruction-atteniin
We introduce RAIN-Merging (Reasoning-Awant Null-space projection Merging), gradient-free method that integrates instruc- tion following while preserving thinking format and reasoning performance, First, with small reasoning calibration set, project the ITM task vector onto the null space of forward features at thinking special tokens, which preserves the LRM's structured reasoning mechanisms. Second, using a small instruction cal- ibration set, we estimate instruction attention to derive module-specifi scaling that amplifies instruction-relevant components and suppresses leakage. Across four instruction- foolooiin benchmarks and nine reasoning & general capability benchmarks, RAIN Merring substantially improves instruction adherence while maintaining reasoning quality. The gains consistent across model scales and architectures, translating to improved performance in agentic scenarios. Code is available https: /ttpttgtttt/ con/ Kinght/ RRAN- Merging.
guided
INTRODUCTION
In the current boom of research, Large Reasoning Models (LRMs, like OpenAl-ol (Jaech al., 2024), DeepSeek-RI -1 (Guo et al., 2025)) have shown strong potential on tasks that require rigorous multi-step reasoning (Wei et al., 2022), such as mathematical derivation (Shao al., 2024) and program synthesis (Guo et al., 2024). However, discouraging paradox has emerged: although LRMs settings, they lag instruction following (Fu al., 2025a;
perform well in purely reasoning-orient Li et al., 2025a). They often generate lengthy logical derivations yet ignore user-specified formats, constraints, or specific operational requirements in the final response. This inconsistency undermines LRM practicality and reliability in real- wolld applications (Chkirbene al., 2024), especially in agent (Qi et al., 2025) and professional tool deployments (Zhao et al., 2024).
A straightforward remedy is continue training LRMs with supervised fine-tuning (SFT) to strengthen instruction following. However, building high-quality supervision datasets for tasks that require generating long chains of thought entails substantial annotation and computational re-

---

Published as a conference paper at ICLR 2026
Query:
[Quastion]
[Insbruction 1]
[Instruction 2]
Campute the let derimaa dighs.ol 7(7")
Outtaut wwr chasartor hesadecinal sring. Inct 2 Pyrhon sertly the resuh
Output LRM
THi, 2003
<thinks
Bipto eteo
"Compute the last decimal digits of SN 7-4777755 Let's compute: 7-7 823543, So exponent 823543.... </thinks
User Experience
We wish to compure the ast 8 decimal diglis SN 7-17747166 Since $747 6235435, this s equivelen: to comouiing
[Thinking]
N Vimos 10-8 = 74/87 1543) Vomood 100030030 M
[Correc Answer]
This can be effiiennlly done using moduler exponentiatio. Final Ánswer is 1bx3xd170132333..
[Not output hax!]
Merilication with Pytiior Corke
[More than 2 lines!]
R RAIN-MERGING
Reasoning Null-space Projection (Stage 1)
9. U2
Lange
Margad
Instruction intruction-attention Guided Merging Coefficients (Stage 2)
Output Instruction Following Enhanced LRM by Merging
powe?, 7+*7, 10+M0
cthinko
forsstin, 'BX')1
'Compute the last decimal digits of 74:777755 compute 267 623543, exponent 823548
</thinks
Experience [Thinking]
We wish to compute the ast decimal dighns of SN 7-/177005 Since $747 = 89 8735435, this equivolent to computigg
N/N ibmmad 13+8 - 7-1523543) lemod 10000000 V
This efficienlly done using modular exponentiation, direct computation
701 70132343.
[Corract Answer]
Convert the decimal number 323/3333 to an B-digit string in hexodecimal final anower is (bcxed(011222277 The fullzwing Pythom code compoutes the resull and formatt it as an dedigt uppercase hax string:
[Output hax]
[Code in 2 Lines]
the case, the LRM arrives at the correct solution but ignores the required format and specific code. To preserve the reasoning structure, we perform training- -fee merging by combining task vector projected onto the null space of the thinking format with instruction-attentin guided coefficients. The merged model remains correct while satisfying the specified constraints. See Sec. 3 for details.
Figure 1:
An overview of RAIN-Merging,
sources (Qin et al., 2025). Moreover, these post-training methods often induce capability regressions, with degradation in generality and in responses to unseen instructions (Shenfeld et al., 2025). In contrast, a training-free free and compute-ligh alternative is model merging. which extracts parameter differences between fine-tuned and pre-trained models (namely the task vector), then combines these task vectors to create a unified model that preserves pre-trained knowledge while incorporating capabilities from multiple tasks (Ilharco et al., 2023). This motivates central question: whether we can merge the LRM and the Instruction-tumed Model (ITM) to enhance the instruction fontructionfollovinng while preserving its reasoning capability.
We begin with a parameter-space analysis of the task vectors from the LRM and the Instruction-tuned Model (ITM) relative to their shared base. We find that their principal subspaces are nearly orthogonal across key modules, which indicates minimal interference between the two capabilities and suggests that merging a promising lightweight way to enhance the LRM's instruction following (Ortiz- Jiménez et al., 2023). However, direct merging carries risks. LRMs and ITMs differ fundamentally in output structure: the former explicitly separates "thinking" and "response" with special markers (e.g.. RI R1-style <think>.... </think>), whereas the latter provides only final answer. Traditional data- free merging (Ilharco et al., 2023; Goddard et al., 2024) prunes or scales the task vector purely from murameter-internal statistics to balance domain performance, thereby ignoring output-distribution mismatches and disrupting the LRM's structured reasoning. Recent work (Nobari et al., 2025; Yao al., 2025; Chopra et al., 2025) has tried to guide merging with forward activations using small calibration sets. Although this introduces data-driven constraints, the lack of an explicit notion of the output mismatch between the two types still makes it difficult to achieve stable and effective balance between preserving reasoning structure and improving instruction following.
To this end, we propose two-stage merging strategy that enhances instruction-followe without sacrificing the thinking format and reasoning performance of the LRM. First, leveraging task-vector orthogonality between the LRM and ITM, we preserve reasoning ability and enforce thinking-forne format invariance by projecting the ITM task vector into the null space derived from forward features at thinking tokens on a small reasoning-calibration calibration set. This keeps the merged model's s reasoning representations aligned with the original LRM and retains structured outputs. Second, while keeping these invariances fixed, we aim to enhance possible. We improve instruction adherence by estimating per-module importance based on attention outputs over instruction- coefficients then assigned to strengthen instruction-relest two-stage approach as Reasoning- Awwae Instruction -guided Null-space projection Merging (RAIN- Merging) in Fig. 1, which effectively synergizes reasoning and instruction-followii performance.
capability
cidtuuction-oollwwinn performance as much as
spans from a small set of instruction examples. Attention Attention-guided behaviors. We refer to the overall
2

---

Published as a conference paper at ICLR 2026
We conduct systematic evaluation of our proposed method four instruction-following benchmarks and on nine evaluation benchmarks that cover mathematics, code, STEM, and creative- writing capabilities. The results show that RAIN-Merging instraction-following ability but also maintains reasoning and general capability. Moreover, our method exhibits consistent stability across different model sizes and architectures, and demonstrates enhanced performance in agentic scenarios.
only substantially improves the LRM's
2 PRELIMINARY AND OBSERVATIONS
Task Vector. A task vector (Ilharco et al., 2023) characterizes the parameter delta from base model to task-specific one. A straightforward way combine capabilities is task arithmetic, which linearly adds such deltas to a base model to obtain a multi-skilled model. This simple approach can work when tasks are compatible. However, for distinct abilities such reasoning and instruction- following that impose different output structures (Yadav et al., 2023), naive linear addition may cause capability interference and disrupt the representations essential each domain.
Orthogonality between Reason & Instruction Task Vectors. To examine whether capability interference arises when merging ITM By into LRM Rg. we take the shared base model 0g as reference and define the LRM task vector R = OR 0B and the ITM task vector A O, 0R. We perform singular value decomposition (SVD) within the main forward modules, e.g. attention and FFN, for these two task vectors and evaluate the principal subspace cosine similarity of their principal subspaces. As shown in Fig. 2, AI, A2, the are nearly orthogonal since their similarities are all < 0.1. Prior studies (Ortiz-Jiménes et al., 2023) indicate that this phenomenon reflects low degree of coupling between reasoning ability and instruction following in parameter space, which suggests that lightweight task-vector merging strategies can enhance instruction following while preserving the original reasoning performance. More details are Appendix E.I.
0.08
SOMAAAA Layers
0.06
Q
K
Gate Up Down
Modules
Figure 2: Principal subspace co- sine similarity between LRM and ITM task vectors for each layer and submodule. The similarities sistently low 0.1).
Risks in Thinking Format During Merging, However, orthog- onality in parameter space is not sufficient guarantee that the merged model will retain the LRM's 's structured output behavior, since this behavior is determined by downstream propagation and forward features (see Appendix E. for proof). In particular, the LRM relies special tokens such as <think> and < </think> to explicitly separate the reasoning segment from the answer segment, and these tokens are crucial in instruction- following tasks. For example, if the model fails to generate the terminator correctly after merging (as Fig. 3), it may conflate the reasoning content with the instruction -complaan response, which can violate constraints such as limits output length. Therefore, although task-vector orthog- onality suggests minimal capability interference, we still need to explicitly constrain the distributional shift of the output structure during merging preserve the integrity of the reasoning process.
OUR RAIN-MERGING METHOD
For notational convenience in later derivations, we flatten model submodules by layer and head with index 1,...,
Notation.
K + W == [vec(W/1) ..... rec(WK) where ④ denotes the block-wise concatenation that assembles disjoint parameter blocks into a single coordinate vector. More details of the forward mechanism in Transformer (Vaswani al., 2017) are in Appendix G.I. Let hf denote the forward input vector at the k-th submodule and the f-th sampled token position. The corresponding linear map of this submodule admits the Kronecker-vectorization form (Koning et al., 1991) with Kronecker product &, identity matrix diag(1) (1), and vectorization operator vec(-), (-), as & k = ((h)) @ diag(1)) veo( (e)... Stacking all sampled positions row-wise yields the forward feature operator & and outputs for the k-th submodule:
i [( To diag(1), (1...,(1)) s diag(1)), W <Bi vec(W*).
(1)
3

---

Published as a conference paper at ICLR 2026
Optimization Objective. To preserve the original reasoning performance of the LRM as much as possible, we take the reasoning model parameters ²R as the anchor. We transform the ITM task vector A through merging function f to obtain A f(Ar), and form the merged model 0 = On A. Our goal is to enhance instruction followwing without damaging the LRM's 's thinking format and reasoning performance. We therefore formulate constrained optimization problem: over the instruction data distribution Dr, maximize the surrogate objective for instruction folllwing, [0) _ Ex=D, Eyr-nst-lz) [IF y)]. while, over the reasoning data distribution DR. constraining the deviation between the model's output distribution within the segment of thinking special tokens Qthink and the reference policy of the original reasoning model dR. This constraint is quantified by aggregating the per-step KL divergence within the segment:
Lahink (0) A EE-DR Byrnao ( (11x)
Σ
KL(TOO- 1 xr Me))
|| (m (- xt (e))))))
(2)
(ERaua()))
The overall objective with tolerance d is then:
max Jr(OR+ + A) s.t. Lahnn + A) S 6.
(3)
A
Noting that Jr is a surrogate objective for instruction following, referring to class of functions IF that evaluate instruction alignment. In later we instantiate with metrics based on instruction- attention alignment or leakage. In addition, motivated by the orthogonality between the LRM and ITM task vectors discussed earlier, we constrain only the conditional distribution in the segment of thinking special tokens and do not restrict the content generated in the other thinking or response segments, which preserves flexibility for improving cnstruction-following performance.
Reasoning-aware Null-space Projection (Stage 1). To satisfy the KI. constraint on the segment of thinking special tokens, we try to seek a parameter subspace that preserves the thinking format. Intuitively, if we view the forward inputs at the thinking positions as "measurement" of the reasoning style, then any parameter perturbation that is unresponsive under this measurement will not change the model's thinking pattern. This idea corresponds to projecting the perturbation onto the null space (Wang et al., 2021) of the forward feature operator & blkdiag ".b...,,......................
...., **) (blkdiag
denotes the block-diagonal matrix), namely (())- = {u tv = 0}. as illustrated in Fig. 3 (a). Such null space projection keeps the token-level forward features at the thinking positions invariant.
Formally, for each submodule k. we construct the least-squares orthogonal projector P-(-) (.) using the
forward feature operator T Raisk built from thinking special token indexs Qtiink to form the null space:
p ( diag(1) - & &
T(
7)
Daink
where (.)+ denotes Moore-Penrose pscudoinverse. And then project the ITM submodule task vector A by submodule-wise and stack them to form the overall projected task vector to satisfy the null space constraint:
K A &
vec
(A) -P p
| L.k
(amm vec(A))
Pran vec vec(A/) = 0, where A/
(5)
This projection keeps the merged model's intermediate representations and even the final logits at the thinking special tokens close to those of the anchor model. To verify its effectiveness in preserving the thinking format, we analyze a second-ordes expansion of the softmax KL divergence and show that the task vector after null-space projection satisfies the KL constraint on the special token output distribution in Eq. (2). This yields the following Prop. (proof is in Appendix E.2):
Proposition 1. Let the logits of sample x at thinking special tokens E (x)) be zo 2o(x, t), and let To(- 2. Vc+) = softmax 2g .)). By second-orde order approximation of the softmax- KL divergence with a uniform upper bound, for any perturbation u,
(sofsmmmaxx(= -i) softmax(2)) < ||uu1/2 + ((1u1)))
(6)
Assuming the model's 's intermediate representations are Lipschitz continuous and bounded, there exist constants C₁ C2 such that for uxx t) =0 (x,, z0 (xr t), we have:
|1 1)]l2 S C1 | vec(A))l2 + C2 ||
4

---

Published as a conference paper at ICLR 2026
Heranzing arrad Nall-spuce Precciio Slage
Iuzacismmttttiii Gunded Mariig Coctliciona (Seas
Thinking Foraal
Quargt
Instrccion Calbrrtio
Attentior Curpor with Mid
Intrecton snverton Soor
Namilin
"Nilee 2irg sbouu stepp
Risna
Piinding
Why
@ Null sullo
Prorecion
A (+**)
Irpui
Queyyh cite de come
Ores
Thinking Format KL Almrat No shift
Ourput othinco
Ckiyy so the wse has alled
Crack
The 152 post Sleaci booding....
Saziely longth
Effle Uiwraud
Agecciaaae
Figure 3: Two stages of our RAIN-Merging pipeline. (a) For each submodule, the ITM task vector is projected onto the null space preventing shifts in thinking format. (b) Given the instruction calibration set, we compute the instruction attention score from attention outputs to obtain merging coefficients.
Substituting the projected vector A = ①
k=1 AT" and the condition vec( ( = O yields:
Laiine (O + A7) 8 E
(1 I + (E,,, 11 ( (2) R (8)
Therefore, null-space projection in Eq. (5) approximately removes the thinking format constraint in the original objective and reduces the original optimization objective Eq. (3)
max Ji(Ox A where A f(
(9)
A
With the thinking-forman constraint relaxed, we next focus on strengthening the task vector's effect on instruction following.
Instruction-atteetion Guided Merging Coefficients (Stage 2). To enhance the performance gain of the ITM task vector during merging, we seek suitable gradient-free surrogate objective instantiate Jr. Prior studies (Guardiciro et al., 2025) suggest that failures in instruction following often stem from insufficient conditioning on the instruction span during decoding: attention does not sufficiently focus on instruction- relevartett tokens and instead leaks to unrelated regions. A simple remedy is to amplify attention outputs on the instruction span decoding time, which can remarkably improve instruction following. This approach, however, requires pre- rre-ideniffying the instruction span, and excessive amplification may cause the model to ignore other necessary content. Motivated by this, we hypothesize that different layers and heads exhibit heterogeneous response behavior to instructions.
Consequently, the null-space- projecced task vector A we introduce per-module scaling coefficients a = {a*} E R and reparameterize the merged model as f(a) = AR + θ K to instantiate merging function f. Given that attention outputs directly coupled to the self-attention mechanism, we first focus on the merging coefficients of these submodules, as â {a*} E R
k=1 a* A
1.k
where denotes the self-attention submodule index. Our central intuition is that an ideal merge should yield stronger attention responses on instruction-releve spans (high alignment) while maintaining low attention activation on inttucttio-irrelevaan content (low leakage). To translate this intuition into measurable quantities, we formalize the model's forward computation as follows and Fig.
(b). Let Att k (ar, [)(t, r] denote the attention output of the merged model with â at head i from token position to r. For an instruction- folluwiinnnooooinngg...oo sample x ~ Dr, we define the per-sample normalized alignment a and leakage u metrics for head k:
Att (x. (t,, [Tx)] [(())
Att (r. @)t, [(x))
8 (x, = Σ
u (2,
Σ
(10)
alignment
(ER(z) TEI(()
leakage
(EU(() TEI(x)
where I(x) C (1...., T} represents the index set of instruction tokens that encodes the task description, formatting rules, constraints, and any examples the query span. Likewise, R(x) denotes the set of output tokens whose content is directly constrained by the instruction in the response span, and ((x) the set of output tokens unrelated the instruction. Taking expectations over instruction- following samples D, and heads & yields averaged alignment a(ā) = Σ3 Ea-D, (x, a)] and averaged leakage u(ā) = Σi Ez=DD,u (2r ā)]. We seek merging coefficients that achieve high alignment and low leakage. Accordingly, combine the two metrics into a single instruction- attention score J.Pooyy with trade-off hyperparameter p 0, instantiating the surrogate objective in
5

---

Published as a conference paper at ICLR 2026
the reduced problem Eq. (9) then yields:
max Jt Piony
a()) pā(ā))
(11)
Quadratic Approximation of Instruction-attention Score. Although this objective differentiable and could be optimized by gradient descent, adopt forward-pass pass approximation to reduce
computation. Initialize at the directly merged point after projection, a(o) 1. Perform a second-
order Taylor expansion of Jj Proxy
( around (o):
Jprony (ā) 2 Prixy (ā(o)(((())) + V &&Jp Praxy (á(o)) (aaaaaaaa (ā-ā 22 a())) H (ā - a()), (12)
where V2J J.Poxyy (ā)()) is the Hessian. Writing g = VaJtpoo (ā)())) and ignoring the constant term in Eq. (11), we obtain the quadratic surrogate:
JH (a) = g (ā a())) + 1 (
(ā (()))
(13)
① For first-order term if we restrict á to small deviations near (()) and adopt a linear approximation of alignment and leakage on merging coefficients, the per-head gradient can estimated as:
aJPPoNy (a)
dā(ā)
Oū(ā) P dāk EEWDD [a a,0)) - pu (
g x
(14)
0ā
Oā
(Q())
which replaces partial derivatives with the current metric values. In practice, this approximately scales the contribution of each head to instruction versus non-instruction attention mass, consistent with the intuition behind attention amplification.
@ For second-orde term H, to avoid the cost of computing the Hessian for large models, we adopt diagonal approximation that limits the step size, H diag(1) + Ez-D, [u aroy)], where the second term imposes a stronger quadratic penalty on heads with higher leakage, thereby limiting their amplification. Substituting the approximations into the quadratic objective, dropping (c(u) for
simplicity, and constraining ā E [ān, ā.. K to prevent scaling instability, we obtain a closed-form solution to the convex quadratic program:
g² H
max &
[la-jaTial
a cliplanaa
(15)
where A = diag (rk) and cl () (-) clips to the interval (a, b). Thus, by second-order expansion with engincering approximations and using only forward attention statistics a gradient-free frre manner, approximate the optimal merging coefficients ã, of celf-attention submodules that increase instruction alignment while controlling attention leakage to non-instruction content. For modules shared across attention heads, such as the feed-forward network (FFN), coefficient to the average over heads. Aggregating the coefficients for all submodules yields the complete instruction attention guided merging coefficients a. {
the layer-wise
Combined to Our Two-stage Merging Method. We chain "Reasoning-aware Null-space Projection (Stage 1)" with "Instruction-atteniion Guided Merging Coefficients (Stage 2)" to propose fully gradient-free merging pipeline, termed Reasoning- Aware Instruction-atten -nteenttooontntnnitonnn guided Null-space projection Merging (RAIN-Merging)
Fig. 3. Our method addresses the challenge the original optimization problem of Eq. (3), improving instruction following while preserving the reasoning structure after merging. The final merged model is:
K
8. = 0R +
+
h A L.k
(16)
where a global scaling coefficient that controls the merging strength. The entire procedure only relies on forward-feature extraction and attention statistics, and does not require gradient-based based updates. EAIN-Mergin offers a low-cost, interpretable path to strengthen instruction following in LRMss, filling the gap left by costly SFT.

---

Published as a conference paper at ICLR 2026
Table 1: Comprehensive comparison of instruction following and reasoning & general capabilities. merge Qwen2.5-7BB BB-Instruct (TTM) into DeepSeek- RI-Distilll -welll--eee-B (LRM) and compare our RAIN- Miriing against multiple merging methods as well as SFT trained on the same calibration data, "Avg." denotes the average over all subsets. "RT"" reports the run-time for merging training in minutes. The hest and second-hest results highlighted in bold and underlined, respectively.
Instruction Following
Reasoning General
Method
RT
IFEva
CELLO
Info Bench
Complex Bench
Avg.
Math
GPQA
Aider
Arena- Hard-v2
Avg.
ITM
70.43
19.15
78.49
43.63
52.92
47.27
29.80
33.33
62.86
43.32
LRM
55.45
16.59
71.73
32.72
44.12
64.75
44.44
29.63
65.29
$1.03
SFT
62.48
17.11
68.58
32.15
45.08
62.57
41,92
28.89
64.67
49.51
1201,2
Data-free Merging
Task Arithmetic
60.44
16.97
73.07
33.34
45.96
64.22
42.93
26.67
64.53
49.59
0.93
SLERP
58.96
17.56
72.18
34.93
45.95
63.82
31.85
65.29
Karcher
62.11
17.99
73.16
34.06
46.83
64.85
48.99
30.37
66.13
$2.58
1,20
TIES
58.60
18.48
73.91
34.40
46.35
65.44
46.46
32.59
63.47
$1.99
1.18
DARE-TIES
60.81
17.88
73.33
33.49
46.38
64.26
47.98
29.63
64.17
51.51
2.21
Deta-dependent Merging
ACM-TIES
59.33
16.45
72.44
33,75
45.50
64.57
45,96
32.59
62,00
51.28
12.45
LEWIS-/IESS
60.44
17.41
72.67
34.40
46.23
62.07
48.99
31.11
64.80
51.74
16.60
AIM-TIES
62.78
17.93
73.11
34.28
47.02
65.92
33.33
63.64
RAIN-Merging
63.22
19.03
74.53
35.66
48.11
68.75
54.55
33,33
65.73
55,59
20.96
Implementation details. To balance compute and storage efficiency, we merge only the core modules that are most sensitive to attention outputs, namely the Q. K, V, O, and FFN parameters. In Stage 1, we sample 150 examples from the Mixture-of- Thoughts (Face, 2025) dataset distilled from DeepSeek- R1 (Guo et al., 2025) from to form the reasoning calibration set. In Stage 2, we instruction calibration set obtained by distilling DeepSeek-RI on IFEval (Zhou et al., 2023b), followed by LLM-as-Judge filtering and manual screening, resulting total of 365 samples. More details of implementation, complete algorithm pseudocode, calibration set construction, and ablation studies are provided in Appendix G. F, H, and J.3.
EXPERIMENTS
In this section, we empirically investigate three research questions:
RQ1 (Effectiveness and Efficiency). Compared with baseline methods, can RAIN-Merging
improve instruction-follooiing while maintaining reasoning capabilities, maintaining the computa- tional and memory efficiency characteristic of gradient-free approaches? (Tab. 1 and Fig. 4)
RQ2 (Scalability). How well does RAIN-Merging scale across models of varying sizes and architectures, and does it perform effectively in interactive agentic scenarios? (Tab. 2, 3)
RQ3 (Ablation). What roles do the two stages of RAIN- play? Specifically, does Stage 1 preserve the format of thinking segments and the output distribution, and does Stage 2 enhance iastruction-attention scores? (Tab. 4 and Fig. 5, Fig. 6)
RQ4 (Investigation). How does augmenting the instruction following capabilities of LRMs influence their inherent reasoning efficacy? Specifically, how does this evolution in reasoning ability manifest in specific cases? (Tab. 5, Sec. J.9, J.10)
4.1 EXPERIMENTAL SETUP
We begin with brief overview of the benchmarks, models, and baselines used in our experiments. Additional details on experimental settings, benchmarks and evaluation metrics, and hyperparameters provided in Appendix I.
Benchmarks. To comprehensively assess instruction following, we use four mainstream bench- marks: 1FEval (Zhou et al., 2023b), CELLO (He et al., 2024), InfoBench (Qin al., 2024), and ComplexBench (Wen et al., 2024). To comprehensively evaluate reasoning and general capabili- ties, nine benchmarks: Mathematical reasoning is evaluated by aggregating results from six benchmarks, as Math. We also measure performance on code editing (Aider (Aider, 2024)), STEM (GPQA (Rein al., 2024)), and creative writing (Arena-Hard-v2 (Li al., 2024)) reflect general
7

---

Published as a conference paper at ICLR 2026
Table 2: Merging performance and relative gains RAIN- MArg-ng across model three scales and two architectures. We merge the corresponding ITM into the LRM with base models: Llama-3.1 1.nna-3.1-991 1-B. "Avg." denotes the average over all subsets. For each scale, the subsequent "(relative gain)" row reports the relative improvement of our method over the I.RM, highlighted in green.
and
Instruction Following
Reusening & Generul
Model
IFEval
CELLO
Bench
Complex Bench
Avg.
Math
GPQA
Aider
Hard-v2
Arena-
Avg.
Qwea2,5-115B Instruct
61.76
27.83
31.77
16.30
38.15
28.07
DeepSeek-R1 RI DDssill-Quenn-
39.00
16.03
55.29
21.54
32,97
41.62
29.29
14.07
39.73
31.18
Qwen2,5-1.58 1...5...---AAN-Mergpih
41.59
16.51
58.18
23.62
34,97
45.87
33.33
14,81
40.93
33.74
(relative gain)
+2.98%
+9.63%
+10.21%
+13.79%
+5.26%
+3.02%
+8.20%
Llama-3. 3.1-a-3..----BB- Instruct
68.58
27.21
78.67
38.47
53.23
35.59
25.25
34.07
72.23
41.79
DeepSeck R1--Disill-Lamma
58.41
17.78
73.33
38.38
46.97
60.21
38.38
27.41
71.93
49.48
63.77
18.84
77.38
38.93
49,73
61,95
40.37
77.07
53.33
(relative gain)
+9.18
+5.99%
+5.52%
+1.42%
+5.86%
+2.89%
+14,47%
+10.81%
+7.15%
+7.78%
Qwes2.5.144. Instructt
44.19
50.29
DeepSeck-R1 -1
71:35
18.71
81.33
41.68
53.02
72.31
$7.077
33.33
80.67
(1) 85
Qwea2.5-14BB- 1-----RA---MA RA-N--MMrggigggg
76.71
19.57
84.13
44.63
56,26
74.58
57,58
40,00
$6.25
64, 60.60
(relative gain)
+7.51%%
+9.69%
+6.11%
+3.1346
+0.88%
+20.00%
+6.92%%
+6.17%
Owen2.5-322D Instruct
78.56
18.59
84.40
16.91
57.11
52.35
36.87
$7.78
$1.90
57.22
DeepSeek-R1 Distill-Qwen-333
76.52
19.69
83.56
44.44
56.05
68.00
60.10
54,81
82.00
66.23
77.26
19.96
84,76
45.74
56,93
75.67
61.62
54.07
83,70
68.77
(relative gain)
+0.97%
+1.39%
+1.44%
+2.93%
+1.57%
+11.28%
+2.52%
-1.35%
+2.07%
+3.83%
and reasoning capabilities. For agentic scenarios, we use ALFWorld (Shridhar et al., 2021) and WebShop (Yao et al., 2022), two realistic multi-turn interactive tasks, to evaluate how well the model integrates reasoning and instruction following to solve complex problems.
Models. We evaluate RAIN-Merging on models of different sizes and architectures: DeepSeek- RI-Distill-Qwen-I 1.tp:////BB//B/1///2222 (Guo al., 2025) (LRM) and Swwe2.551..BB77 BBBB/BB/144///21 /14B/32B- Instruct (Yang et al., 2025) (ITM), as well as the Llama family (Dubey et al., 2024) using DeepSeek- RI-Distill-Llama-B8 (LRM), its instruction-tune counterpart Llama-3.1
(ITM).
Baselines. We include several data-free, task-vector based merging methods: Task Arithmetic (II- harco et al., 2023), SLERP (Biship. 2007; Goddard al., 2024), Karcher (Nielsen & Bhatia, 2013; Goddard et al., 2024), TIES (Yadav et al., 2023), and DARE (Yu et al., 2024). We also compare with data-dependent, activation-based merging approaches that leverage small calibration sets, including ACM (Yao et al., 2025), LEWIS (Chopra et al., 2025), and AIM (Nobari et al., 2025). To strengthen baseline performance, we apply TIES on top of other merging baselines in previous work (Wu et al., 2025). In addition, we report a training baseline using SFT on the same calibration data.
4.2 RESULTS
Performance Comparison with Baseline Methods. (RQ1) As shown in Tab. 1, MerN-M achieves overall gains across both instruction-followinng and reasoning & general capability evalua- tions, outperforming all merging baselines. While Task Arithmetic and SFT can improve instruction following to some extent, they typically do so at the cost of reasoning and general capabilities. In contrast, our method consistently surpasses all baselines on eestructionfollowinnn reasoning, and general-capability benchmarks. Our merged LRM trails the ITM slightly instruction following, indicating room for further improvement. Interstingly, the merged model exhibits stable gains in reasoning and general ability. We hypothesize that stronger instruction adherence improves the quality of the model's internal chain of thought, which yields better reasoning performance. Overall, RAIN-Merging substantially enhances instruction following without sacrificing the LRM's reasoning and general capabilities.
mathematical
Run-time and Memory Analysis. (RQ1) Our method achieves favorable efficiency trade-off. Its runtime, though slightly above activation-based merging baselines due to null-space computation, far below SFT (RT in Tab. 1). Similarly, while storing hidden features increases memory use compared to other merging methods, its footprint remains much smaller than SFT's (Fig. 4). This demonstrates our approach as a highly practical, training-free alternative for enhancing LRMs.
Although DeepScek-Ri. RI-Distill Qwen- are trained from Qwen2.5-Mathe 1...555-at---5B7777 2024a), we find that Qwen2.5-Math-1.55 B5B7/BB-nstttt do not outperform the distilled LRMs in instruction following. We therefore select the stronger instruction followers, Qween.551..B/77BInnsme as ITMs.
1.5B/7B (Yang et al.,
8

---

Published as a conference paper at ICLR 2026
Memory (GB)
112.6
Table 3:
Performance of
Table 4: Performance of ablation
100-
RAIN-Merging in agent set-
Stage and Stage under the same setup as Tab. 1. "T Avg." and "R Avg." denote the average performance on mnstruction-follwiin and reasoning & general benchmarks.
tings.
We merge Qwen2.5-7Bt Instruct (lTM) into DeepSeck-
16.7
16.8
18.2
22.1
RI-Distill- 11-Distil--wwen-71 (LRM).
TWS
LEW9B-TES
ABM- TES
SPT
Model
Worl WebShop
Method
Avg.
R Avg.
Figure 4: GPU memory usage com-
17.50
10.45
RAIN- Merging wis Stage 2
46.58
54.92
parison between different methods der the same configuration as Tab.
LRM
22.00
26.63
RAIN Merging Stage
47.62
52.44
RAIN-Merging
25.00
29.42
Marging
55.59
Text Arthank
RAIN MAgS-Megiia
Instructionatteenn attentioo Score
7.5
LRM
Task Arithmetic
RAIN-Merging
0.1224
Propuer (Sem of riisolng (%)
6.4%
0.125
2.0-
Lehink
0,100
2.5
0.075
2.0-
0.090
0.5
0.025
0L09655
0.0%
0.000
14 15
Methods
Figure 5: Caink Eq. (2) (left) on the reasoning calibration validation set, and the proportion of generations missing the clos- ing </think> token (right) on IFEval un- der the same configuration Tab.
Figure 6: Layer-wise instruction attention score (alignment Icakage), Per-layer scores on IFEval instruction examples; higher is better. We compare the unmerged LRM, Task Arithmetic, and RAIN-Merging when merging Qwen2.5--BB -een2.55-BBInstrruc (TTM) into DeepSeek-RI DeepSeek-R1--istill--eee D-iiisiill-Qeen-7B (LRM)..
Performance on Models of Different Sizes and Architectures. (RQ2) To evaluate the scalability of several configurations,
our method across model sizes and architectures, we conduct experiments including the Qwen2.5 family distilled from DeepSeek-RR R1 II at 1.5B. 14B, and 32B parameters, and the 8B model built on the Llama 3.1 architecture. As reported in Tab. 2, our method consistently enhances nnstruction-folloviin and reasoning performance, achieving average improvements from 1.57% to 8.20% on LRMs. These results confirm that RAIN- Mergnng robustly strengthens instruction adherence and preserves complex reasoning across diverse model sizes and architectures.
Performance in Agentic Scenarios. (RQ2) To further assess the practical benefits of improved instruction following, we evaluate the merged model on two representative agentic scenarios, Web- Shop and AlfWorld. As shown in Tab. 3, the merged model achieves better performance than the original LRM and ITM in these scenarios, indicating that enhanced instruction understanding and reasoning effectively support multi-tume -urn interaction and complex decision making. These results also demonstrate that our gradient-free approach is effective for increasing the real-world utility of LRMs.
Ablation Study of Stage and Stage 2. (RQ3) We investigate the contribution the two components in RAIN- RAIN-Merging, shown in Tab. 4. When without Stage 2, the merged model retains reasoning and general capabilities while achieving competitive instruction-following performance. Conversely, when without Stage cestruction-follwiiin performance improves further but at noticeable cost to reasoning and general ability, as it lacks protection of the thinking format. Incorporating both stages yields the best trade-off: Stage ensures reasoning performance is maintained while Stage boosts cnitructionfollowwiii performance. Thereby, both stages play critical and complementary roles.
Effectiveness of Null-space Projection. (RQ3) To assess how our null-space projection in Stage 1 preserves thinking formats, we evaluate its impact on thinking special token distributions and resulting generation outputs. We measure the KL divergence near thinking tokens as in Eq. (2) and the rate of missing </think> tokens. Results Fig. 5 show that Task Arithmetic substantially alters the distribution (Lehink = 0. 1224) and results in 6.4% missing </thi nk> tokens, violating the output format. Our approach, in contrast, only induces minimal change (Cthink 0.0065) and ensures missing tokens (0.0%). These findings indicate that null-space projection successfully protects thinking formats.
Effectiveness of Merging Coefficients. (RQ3) To validate the merging coefficients, we compare the instruction-Attention Score in Eq. (11) across layers before and after merging under different methods. As shown in Fig. 6, instruction-attens attention guided coefficients in Stage enable RAIN-Merging to consistently outperform both the LRM and Task Arithmetic, exhibiting a higher aligmment and lower leakage. This finding suggests that our weighted reparameterization of merging submodules enhances

---

Published as a conference paper at ICLR 2026
Table 5: Merging performance of RAIN-
Table 6: Evaluation of reasoning and answer traces under the same configuration as Tab. 1. We report Reasoning Internal Coherence (RIC) and Reasoning- Answer Alignment (RAA) on IFEval, AIME25, and GPQA (0-5 scale). The subsequent "(relative gain)"
Merging on MathlF under the same config- uration as Tab. 1. IF Acc. and Math Acc.
are the accuracy of instruction constraints
and math answers, respectively. Both Acc.
represents both constraints and math an- swers are correct.
row reports the relative improvement of our method
over the LRM, highlighted in green.
Model
IF Acc.
Math Acc.
Both Acc.
Model
IFEval
AIME25
GPQA
Avg.
ITM
48.81
40,95
19.76
RIC
RAA
RIC
RAA
RIC
RAA
LRM
25.86
53.81
12.62
LRM
4.58
4.41
1.50
3.60
3,53
3.76
4,06
RAIN Merging
35.
54.76
20.48
RAIN Merging
4.61
4.51
4.30
4.10
3,56
4.26
4,26
(relative gain)
+35.73%
+1.77%
+62.26%
(relutive gain)
40.7756
+2.26%
0.00%
+1389%
+0.86% +1331% +4.78%
activation along instruction-awann pathways while slightly suppressing leakage, which improves instruction following without altering the original reasoning pattern.
Joint Evaluation of Reasoning and Instruction-Followinng (RQ4) To more directly assess whether RAIN-Merging can jointly maintain strong reasoning and strict instruction-following within a single task, we further evaluate verifiable constraints and reports both constraint satisfaction and math correctness, as well as a joint metric requiring both. As shown in Tab. 5, RAIN- SAIN-Merging markedly improves MathIF instruction hard accuracy over the LRM while leaving math correctness essentially unchanged. Crucially, on Both Acc, the merged model rises from 12.62% 62% 20. 48% (+62.26% relative), outperforming both the LRM and the ITM, demonstrating a substantial gain on the core goal of being both correct and follow within a single benchmark.
Math/F (Fu al., 2025b). MathIF augments math problems with
Reasoning and Answer Traces Evaluation. (RQ4) To complement our reasoning and answer quality evaluations, we adopt the framework of Wang et al. (2025) and perform a process- level analysis of reasoning traces and answers. We measure two chain-of-thought level metrics by LLM--a----uddge: ① Reasoning Internal Coherence (RIC) assesses how logically consistent and self-contained the reasoning trace is. @ Reasoning- semantically supports the final answer. For each sample we provide the question, the ground-truth answer, the full reasoning trace, and the model's 's final answer response. The judge then assigns 0-5 scores for RIC and RAA. Tab. 6 shows that RAIN-Merging yields slight improvements in both RIC and RAA on IFEval. For the reasoning- focused benchmarks AIME25 and GPQA, it maintains RIC scores comparable to the LRM, while achieving a marked increase in RAA, notably by over 13% in both cases. In other words, the internal coherence of the reasoning is preserved, with the connection between the reasoning and the final decision becoming noticeably tighter.
Alignment (RAA) measures how well the reasoning trace
Case Study on Instruction Following and Reasoning. (RQ4) To provide granular understanding of how RAIN- RAIN-Merging enhances model performance, we conducted series of case studies focusing on instruction following and complex reasoning in Appendix J.9 and J. In instruction-follwwii scenarios, R.AIN-erggne effectively rectifies common failure modes of baseline LRMs, such as the violation of explicit formatting rules. On reasoning tasks, the merged model demonstrates superior internal consistency. It not only corrects localized numerical slips through more deliberate and step-by-step computations but also successfully bridges the "knowing-doin gap" (Schmied et al., 2025) where the bascline LRM's final selection contradicts its correct internal derivation. These qualitative observations suggest that RAIN-Merging achieves more than just preserving reasoning chains, it actively aligns the model's decision-making process with its logical reasoning.
5 CONCLUSION
We propose RAIN-Merging a gradient-free method to enhance instruction following in LRMs while preserving their structured reasoning outputs. By projecting the instruction task vector onto the null space of the thinking format and scaling it by instruction- achieves a balance between instruction following and reasoning structure preservation. The method is evaluated on cnstruction-folloiinn reasoning & general capability, agentic benchmarks, showing that RAIN- MAIN-Merging not only substantially improves the LRM's instruction followiinn ability but also brings gains in reasoning and general capability across several settings.
guided coefficients, RAIN- RAIN-Merging
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors would like to thank the anonymous reviewers for their insightful comments.
The author would also like to express his gratitude to Yucheng Bai for his assistance in identifying and correcting typographical errors within the formulas.
The resaerch leading to these results has received funding from the Al for Science Program, Shanghai Municipal Commission of Economy and Informatization (No. S-A.GG/---G----BB----------- and the National Natural Science Foundation of China (No. 62376155).
REFERENCES
Aider. ol tops aider's new polyglot leaderboard, 2024.
Mislav Balunovié, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. Matharena: Evaluating iims on uncontaminated math competitions. arXiv preprint wrXiv:2505.2328. 2025.
Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, and Joydeep Ghosh. Clustering with bregman divergences. Journal of Machine Learning Research (JMLR), 2005.
Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran E1- El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp, Ehud Karpas, Ran Zilberstein, Jiaqi Zeng. Soumye Singhal, Alexander Bukharin, Yian Zhang, Tugrul Konuk, Gerald Shen, Ameya Sumil Mahabaleshwarkar, Bilal Kartal, Yoshi Suhara, Olivier Delallcau, Zijia Chen, Zhilin Wang. David Mosallanezhad, Adi Renduchintala, Haifeng Qian, Dima Rekesh, Fei Jia, Somshubra Majumdar, Vahid Noroozi, Wasi Uddin Ahmad, Sean Narenthiran, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Siddhartha Jain, Igor Gitman, Ivan Moshkov, Wei Du, Shubham Toshniwal, George Armstrong, Branislav Kisacanin, Matvei Novikov, Daria Gitman, Evelina Bakhturina, Jane Polak Scowcroft, John Kamalu, Dan Su, Kezhi Kong, Markus Kliegl, Rabeeh Karimi, Ying Lin, Sanjeev Satheesh, Jupinder Parmar, Pritam Gundecha, Brandon Norick, Joseph Jennings, Shrimai Prabhumoye, Syeda Nahida Akter. Mostofa Patwary, Abhinav Khattar, Deepak Narayanan, Roger Waleffe, Jimmy Zhang, Bor- Yiing Su, Guyue Huang. Terry Kong. Parth Chadha, Sahil Jain, Christine Harvey, Elad Segal, Jining Huang, Sergey Kashirsky, Robert McQueen, Izzy Putterman, George Lam, Arun Venkatesan, Sherry Wu, Vinh Nguyen, Manoj Kilaru, Andrew Wang, Anna Warno, Abhilash Somasamudramath, Sandip Bhaskar, Maka Dong, Nave Assaf, Shahar Mor, Omer Ullman Argov, Scot Junkin, Oleksandr Romanenko, Pedro Larroy, Monika Katariya, Marco Rovinelli, Viji Balas, Nicholas Edelman, Anahita thiwandiwalle Muthu Subramaniam, Smita Ithape. Karthik Ranamoorthy, Yuting Wu, Suguna Varshini Velury, Omri Almog, Joyjit Daw, Denys Fridman, Erick Galinkin, Michael Evans, Katherine Luna, Leon Derczynski, Nikki Pope, Fileen Long, Seth Schneider, Guillermo Siman, Tomasz Grzegorzek, Pablo Ribalta, Monika Katariya, Joey Conway, Trisha Saar, Ann Guan, Krzysztof Pawelec, Shyamala Prayaga, Oleksii Kuchaiev, Boris Ginsburg, Oluwatobi Olabiyi, Kari Briski, Jonathan Cohen, Bryan Catanzaro, Jonah Alben, Yonatan Geifman, Eric Chung, and Chris Alexiuk. Llama-nemotron: Efficient reasoning models, 2025.
Christopher Biship. Pattern recognition and machine learning (information science and statistics). Springer, 2007.
Dankmar Böhning. Multinomial logistic regression algorithm. Annals of the institute of Statistical Mathematics (AISM), 1992.
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Lang Cao, Chao Peng, Renhong Chen, Wu Ning, Yingtian Zou, and Yitong Li. Step guided reasoning: Improving mathematical reasoning using guidance generation and step reasoning. arXiv preprint arXiv:2410.19817 2024.
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program of thoughts prompt- ing: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint 2r11v:2211.12288. 2022.
11

---

Published as a conference paper at ICLR 2026
Zina Chkirbene, Ridha Hamila, Ala Gouissem, and Unal Devrim. Large language models (Ilm) in industry: A survey of applications, challenges, and trends. In IEEE International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and loT (HONET), 2024.
Hetarth Chopra, Vidhi Rambhia, and Vikram Adve. Lewis (layer wise sparsity)-a training free guided model merging approach. arXiv preprint arXiv:2503. 03874, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichir Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 1Xiv:2110.14166 2021.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The Ilama 3 herd of models. arXiv preprint arXiv:2407, 2024.
Hugging Face. Open rl: A fully open reproduction of deepseek-rl, 2025.
Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphacdit: Null-space constrained knowledge editing for language models. In International Conference Learning Representations (ICLR), 2025.
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George J. Pappas. Efficient and accurate estimation of lipschitz constants for deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Tingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, and Yu Cheng. Scaling reasoning, losing control: Evaluating instruction following in large reasoning models. arXiv preprint arXiv:2505.14810, 2025a.
Tingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, and Yu Cheng. Scaling reasoning, losing control: Evaluating instruction following in large reasoning models. arXiv preprint arXiv:2505.148100 2025b.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: program-aided language models. In International Conference Machine Learning (ICML), 2023.
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's mergekit: A toolkit for merging large language models. arXiv preprint arXiv:2403. 13257, 2024.
Vitoria Guardieiro, Adam Stein, Avishree Khare, and Eric Wong. Instruction following by boosting attention of large language models. arXiv preprint arXiv:2506. 13734, 2025.
Daya Guo, Qihao Zhu, Dejian Yang. Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu. YK Li. et al. Deepseck-coder When the large language model meets programming rise of code intelligence. arXiv preprint arXiv:2401. /4/96, 2024.
Ae: Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang,
12

---

Published as a conference paper at ICLR 2026
Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang. Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shamhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu. Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang. Wenjun Gao, Wenqin Yu, Wentao Zhang, W. I.. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He. Yishi Piao. Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yonggiang Guo, Yuan Ou, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma. Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang, and Yanghua Xiao. Can large language models understand real-world complex instructions? In Association for the Advancement of Artificial Intelligence (AAAI), 2024.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.0388 2021.
Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannanch Hajishirzi. and Ali Farhadi. Editing models with task arithmetic. In International Conference on Learning Representations (ICLR), 2023.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameyy, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Ruud H Koning, Heinz Neudecker, and Tom Wansbeek. Block kronecker products and the veeb operator. Linear Algebra and Its Applications, 1991.
Bohdan Kovalevskyi. Ifeval-extended: Enhancing instruction-folloiing evaluation in large language models through dynamic prompt generation. Journal of Artificial Intelligence General science (JAIGS), 2024.
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph Gonzalez, and lon Stoica. From crowdsourced data to high-quality benchmarks: Arena-hann hard and benchbuilder pipeline. arXiv preprint arXiv:2406. 1939, 2024.
Xiaomin Li, Zhou Yu, Zhiwei Zhang, Xupeng Chen, Ziji Zhang, Yingying Zhuang, Narayanan Sadagopan, and Anurag Beniwal. When thinking fails: The pitfalls of reasoning for instruction- following in llms. arXiv preprint arXiv:2505.11423 2025a.
Zhenyu Li, Kehai Chen, Yunfei Long. Xuefeng Bai, Yaoyin Zhang, Xuchen Wei, Juntao Li, and Min Zhang. Xifbench: Evaluating large language models multilingual instruction following. Advances in Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2025b.
13

---

Published as a conference paper at ICLR 2026
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In International Conference Learning Representations (ICLR), 2024.
Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Guilherme Penedo, Edward Beeching, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra. Openr1-math-2211 2025.
Yurii Nesterov. Introductory lectures convex optimization: A basic course. Springer, 2013.
Frank Nielsen and Rajendra Bhatia. Matrix information geometry. Springer, 2013.
Amin Heyrani Nobari, Kaveh Alimohammadi, Ali ArjomandBigdeli, Akash Srivastava, Faez Ahmed, and Navid Azizan. Activation-informmed merging of large language models. arXiv preprini arXiv:2502.02421, 2025.
Guillermo Ortiz-Jiménez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Guilherme Penedo, Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Edward Beeching, Agustín Piqueres Lajarín, Quentin Gallouédee, Nathan Habib, Lewis Tunstall, and Leandro von Werra. Codeforces cots, 2025.
Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, and Hannanch Hajishirzi. Generalizing verifiable instruction following. Advances in Neural Information Processing Systems (NeurIPS) Track Datasets and Benchmarks, 2025.
Yunjia Qi, Hao Peng, Xiaozii Wang, Amy Xin, Youfeng Liu, Bin Xu, Lei Hou, and Juanzi Li. Agentif: Benchmarking instruction following of large language models in agentic scenarios. arXiv preprint arXiv:2505. 16944, 2025.
Yiwei Qin, Kaiqiang Song, Yehowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang. Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. Infobench: Evaluating instruction following ability in large language models. In Proceedings of the Association for Computational Linguistics (ACL Findings), 2024.
Yulei Qin, Gang Li, Zongyi Li, Zihan Xu, Yuchen Shi, Zhekai Lin, Xiao Cui, Ke Li, and Xing Sun. Incentivizing reasoning for advanced followii of large language models. arXiv preprint ar arXiv:2506.01413 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel Bowman. Gpqa: graduate-level google- proof q&a benchmark. In Conference on Language Modeling (COLM), 2024.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In International Conference on Learning Representations (ICLR), 2021.
Thomas Schmied, Jörg Bornschein, Jordi Grau-Moya, Markus Wulfmeier, and Razvan Pascanu. Lims are greedy agents: Effects of rl fine-tuning on decision makinng abilities. arXiv preprint arXiv:2504.16078, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseckmath: Pushing the limits of mathemat- ical reasoning open language models. arXiv preprint arXiv:2402.033000 2024.
Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. RI's razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04255) 2025.
14

---

Published as a conference paper at ICLR 2026
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Mohit Shridhar, Xingdi Yuan, Alexande Côté. Yonatan Bisk, Adam Trischler, and Matthew J. Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. In International Conference Learning Representations (ICLR), 2021.
Pengwei Tang, Yong Liu, Dongjie Zhang, Xing Wu, and Debing Zhang. Lora-null: Low-rank adaptation via null space for large language models. arXiv preprint Sidi::2503.0265555 2025.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Martin J. Wainwright and Michael I. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning. 2008
Changyue Wang, Weihang Su, Qingyao Ai, and Yiqun Liu. Joint evaluation of answer and reasoning consistency for ballucination detection in large reasoning models. arXiv preprint arXiv:2506.0453 04832, 2025.
Shipeng Wang. Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. senfsconnistency improves chain of thought reasoning language models. In International Conference on Learning Representations (ICLR), 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxing Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. Benchmarking complex instruction-followinn with multiple constraints composition. In Advances in Neural Information Processing Systems (NeurIPS). 2024.
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rehecca Roelofs, Raphael Gontijo Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning (ICML), 2022.
Han Wu, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. Unlocking efficient long-to-short Ilm reasoning with model merging. arXiv preprint arXiv:2503.20641, 2025.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409. 12122, 2024a.
Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in llms, milms, and beyond: Methods, theories, applications and opportunities. arXiv preprint 2408 076666, 2024b.
15

---

Published as a conference paper at ICLR 2026
Qwen: An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei 7hang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. In Advances Neural Information Processing Systems (NeurIPS), 2022.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances Neural Information Processing Systems (NeurIPS), 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023b.
Yuxuan Yao, Shuqi Liu, Zehua Liu, Qintong Li, Mingyang Liu, Xiongwei Han, Zhijiang Guo, Han Wu, and Linqi Song. Activation-guided consensus merging for large language models. arXiv preprint arXiv:2505, 14009, 2025.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as free lunch. In International Conference on Machine Learning (ICML), 2024.
Haobo Zhang and Jiayu Zhou. Unraveling lora interference: Orthogonal subspaces for robust model merging. In Proceedings of the Association for Computational Linguistics (ACL), 2025.
Huaqin Zhao, Zhengliang Liu, Zihao Wu, Yiwei Li, Tianze Yang, Peng Shu, Shaochen Xu, Haixing Dai, Lin Zhao, Gengchen Mai, et al. Revolutionizing finance with Ilms: An overview of applications and insights. arXiv preprint arXiv:2401. 16641, 2024.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and H. Chi. Least-to-most prompting enables complex reasoning in large language models. In International Conference on Learning Representations (ICLR), 2023a.
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. instuuction-folooiing evaluation for large language models. arXiv preprint arX19:2311.07911, 2023b.
Tao Zou, Xinghua Zhang, Haiyang Yu, Minzheng Wang, Fei Huang, and Yongbin Li. Eifbench: Extremely complex instruction following benchmark for large language models. Proceedings of Empiriaal Methods in Natural Language Processing (EMNLP), 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A ETHICS STATEMENT
This research adheres to the licenses and applicable laws governing upstream open-source models and datasets. RAIN-Merging is developed using publicly available weights and data that permit derivation and redistribution.
Safety. Model merging can introduce "capability or safety drift," such as new biases, jailbreak risks, shifts in hallucination patterns while improving instruction following. The merged model may produce inaccurate, biased, or inappropriate content. It must not be used directly in high-risk decision-making contexts such as medicine, law, or finance. Any production deployment should include human oversight, operation logging, rate limiting, and compliance review procedures.
Dataset use. We rely only on data authorized for academic reproducibility. During data cleaning, we make every effort to remove personally identifiable information and sensitive content. We also disclose potential dataset biases, coverage gaps, and risks benchmark contamination.
Societal impact. We caution that generative models may exacerbate information asymmetries, reinforce stereotypes, or be applied to produce misleading content. We firmly oppose misuse and will work with the community to address any identified negative impacts.
B REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, we provide the following resources and documentation: all algorithm implementations and experiment scripts will be released anonymously with the sup- plementary materials, accompanied by documentation of key functions and the project directory structure. The calibration datasets used in our experiments will be made available alongside the appendix. Public links are included for all open-source models and datasets used in this work.
C LLM USAGE STATEMENT
We used large language models (LLMs) in the following stages and disclose their roles follows:
Writing Stage. LLMs (both closed- and open-source) were used only for copyediting and grammar checking, including terminology normalization, syntactic polishing, and formatting. They were not used to generate claims, collect evidence, or construct results.
Benchmark Evaluation. When benchmark's 's original paper community practice requires closed-source LLM (for example, as a judge or as baseline), we strictly follow the prescribed protocol and disclose the exact model versions.
Calibration Set Construction. We adopt an LLM-as-Judge procedure for automated filtering and scoring of candidate samples an initial pass (producing only scores labels; generated text used as a training target). A human second-pas pass review follows to ensure data quality and compliance. All third third-party data and models are used within their licenses, with source links and permission details provided.
D RELATED WORK
Large Reasoning Model. Early studies show that prompting models to explicitly produce inter- mediate steps during reasoning can substantially improve complex reasoning performance, as in Chain-of- Thoughh (Wei et al., 2022) and Prce-of-Though (Yao et al., 2023a). Building on this insight, a new generation of LRMs has shifted toward training paradigms that directly incentivize reasoning with reinforcement learning; for example, OpenAl's ol series and DeepSeek-RI report marked advances on tasks in mathematics and code that require extended reasoning (Jaech et al., 2024; Guo et al., 2025). These models typically generate structured "thought processes" "thinking formats," yet in real applications they often exhibit tension with strict instruction following. Beyond explicit intermediate reasoning such CoT and ToT, subsequent work further improves reasoning
17

---

Published as a conference paper at ICLR 2026
quality and stability: Self-Consistency samples multiple solution paths and uses majority voting to increase reliability; Least-to-Moot decomposes complex problems into subgoals ordered from easy to hard; oferamm-of-Thughhn and PAL externalize the reasoning into executable programs, decoupling computation from reasoning and substantially reducing arithmetic and procedural errors (Wang et al., 2023; Zhou et al., 2023a; Chen et al., 2022; Gao et al., 2023). In the "reasoning plus acting" paradigm, ReAct interleaves thought traces with tool interactions to mitigate hallucinations, while Reflexion employs language-based self-reflection and memory to iteratively refine policies over multi-turn interactions (Yao et al., 2023b; Shinn et al., 2023). In parallel, LRMs are trained with process-level feedback and reinforcement learning to directly encourage thinking before answering: the ol system emphasizes large-scale RL and thinking-fiee first training and alignment strategies, and DeepSeek-RI shows that under weak or no supervision, pure RL (e.g., GRPO) can induce longer and more stable chains of thought (Jaech et al., 2024; Guo et al., 2025). Moreover, process supervision and process reward models (PRMs) provide finer-grained step-level feedback that, compared with outcome supervision of final answers, better cultivates verifiable reasoning chains and test-time expansion (Lightman et al., 2024). RAIN-Merging is complementary to this trajectory: instead of retraining the LRM, we preserve the thinking format merge time and use structured coefficients to selectively enhance instruction responses, thereby striking a balance between fidelity to the reasoning structure and improved instruction following.
Instruction Following. In the alignment paradigm, InstructGPT systematically improved the stability of "following user intent" through reinforcement learning from human feedback (RLHF), and showed that small instruction-tunee models can achieve strong human preference scores, establishing a foundation for subsequent research on instruction following (Ouyang et al., 2022). For objective evaluation, IFEval targets programmatically verifiable rules, for example, length limits, keywords, formatting, to reduce subjective scoring noise and facilitate reproducibility and fair comparison (Zhou et al., 2023b). CELLO abstracts multi-dimensional attributes from real-world complex instructions, such as multi-step dependencies, format or quantity constraints, and semantic consistency, to charac- terize where LLMs struggle with complex instruction understanding (He et al., 2024). InfoBench proposes a decomposed metric that breaks a complex instruction into checkable sub-requirements, en- abling finer-grained measurement of compliance and error sources (Qin et al., 2024). ConplexBench emphasizes the compositional challenge of multiple simultaneous constraints, systematically testing robustness and trade-off's when many constraints co-occur (Wen et al., 2024). Building on these merging strategy:
mainstream benchmarks, this work introduces an instruction-aateniionnnniiin during merging, we quantitatively constrain and amplify the model's 's responsiveness to instruction- relevant spans while maintaining the stability of its long-chain reasoning format, thereby balancing compliance and an interpretable process.
Model Merging, Parameter-space merging offers a training-freee or low-data path for integrating capabilities. Model Soup averages weights from multiple fine-tuned checkpoints to improve out-of- distribution robustness and overall performance (Wortsman et al., 2022). Task vectors implement additive editing and compositionality by linear arithmetic on weight differences, enabling positive and negative edits well as multi-task synthesis (Ilharco et al., 2023). TIES-Merging explicitly addresses interfering factors such as resetting parameters with negligible updates and resolving sign confficts, which mitigates performance degradation caused by parameter-leve interference when merging multiple models (Yadav et al., 2023). Community tools and practice are also maturing. MergeKit consolidates and engineers diverse merging algorithms, supporting large-model merging and recipe reproduction under resource constraints, which facilitates methodological comparison and reproducibility (Goddard et al., 2024). Systematic surveys have begun organize theoretical perspectives, method taxonomies, and application boundaries for merging, providing references for unified terminology, evaluation settings, and future research agendas (Yang et al., 2024b). However, most existing methods focus on average multi-task performance and out-of outto-ddistriuttion robustness, with limited attention the fidelity of fine-gras ane-grained functional structures such as the reasoning format, for example, explicit thought traces and process markers. RAIN-Merging targets this gap: during parameter fusion it introduces subspace constraints tied to the "thinking format," and allocates merging coefficients at the per-layer and per-head levels using instruction attention, thereby strengthening instruction following while suppressing structural drift of the original reasoning patterns.
Null Space Projection. Constraint ideas centered on orthogonality and null spaces have been re- peatedly validated in continual learning and knowledge editing. OGD projects gradients for new tasks onto the orthogonal complement of the subspace of old tasks, explicitly constraining update
18

---

Published as a conference paper at ICLR 2026
directions mitigate forgetting (Farajtabar al., 2020). GPM extracts and maintains "important gradient subspaces" via singular value decomposition, then performs laser-wise orthogonal projec- tion of new gradients reduce interference across tasks (Saha al., 2021). For LLM knowledge editing, AlphaEdit projects edit perturbations into the null space of "preserved knowledge and provides theoretical guarantees output preservation, which markedly reduces cumulative damage in sequential edits (Fang et al., 2025). In parameter-efficien and mergeable settings, L-RA-Null initializes or constrains the LoRA adaptation subspace using the null space of pretrained representa- tions, alleviating forgetting and improving parallelism and mergeability with other updates (Tang al.. 2025). For multi-task and multi-LoRA model merging, OSRM imposes orthogonalization constraints on task-specific LoRA subspaces before fine-tuning, reducing mutual interference at merge time and improving compatibility (Zhang Zhou, 2025). Following this line of work, we construct null-space projection on features tied to the "reasoning format," and combine it with ietiiion-natttiionnnue
coefficients. The merged model thus preserves structured reasoning outputs while improving adherence verifiable constraints such as format, length, and enumeration.
E
PROOF
6050
Gue
Down
Figure A1: Principal subspace cosine similarity between Qwen2..511.5B-IIstruc (TTM) task vectors for each layer and submodule.
(LRM) and
3058
Down
Layers
Figure A2: Principal subspace cosine similarity between DeepSeek-RR SeepSeek-RI--istee RI--istill RI-Distill-Qwen Swelll--ee--lle 14B (LRM) and Qwen2.5.
14B-Instruct (ITM) task vectors for each layer and submodule.
E. PROOF OF WHY ORTHOGONAL. IN PARAMETERS & INVARIANT IN OUTPUTS
We first describe how we compute orthogonality between principal parameter subspaces. Let the two sources be the LRM task vector weight difference, denoted by R, and the IT'M counterpart, denoted by For each layer and each linear submodule W Be x we take the top singular directions (default S = 16 in our experiments) and perform SVD:
W
U E (V
(A1)
Write Uk.s E x5 for the top-S left singular vectors of U (similarly Vh.s E Rd and analogously Uk.s, Vl.s for source T.
Principal subspace cosine similarity. We focus on the left (output-side) principal subspaces and define the alignment matrix
A = (Uhs) Uf, E R SxS
(A2)
Let of.... o E [0, 1] be the singular values A They equal the cosines of the principal angles between the two subspaces: of = cos Sb. We define the principal subspace cosine similarity as the mean cosine of principal angles:
cos (UR U/.s) 5
Σ
(A3)
Smaller values indicate stronger orthogonality between the sources that (layer, module) cell.
19

---

Published as a conference paper at ICLR 2026
Empirical observation. Across model sizes and all layers/modules in Qwen2.5-1.5B/7BBe (Fig. A1, A2), we observe cos O < 0.1 (with only a few exceptions), indicating that LRM and ITM task vectors are largely orthogonal in parameter principal directions. However, the theory below shows, such parameter-space orthogonality does not imply invariance in the output space (i.e., unchanged logits the thinking format), and thus cannot replace the null-space projection constraint used in our method.
Why orthogonal in parameters * invariant in outputs. We formalize this issue as the following Prop. and give proof with dimension argument.
Proposition (Insufficiency of parameter-space orthogonality for output invariance). For each submodule k, let U4 s: Vi.s .. be the S-dimensional instruction-side principal left/right subspaces and define the admissible low-rank parumeter perturbation space
T -ii @ Vř.s span (pan(vec(uu") a Uf v Vi.s Ti T
(A4)
Let J be the Jacobian of the logits on the protected thinking tokens Q at the anchor 2R. with total parameter dimension D df and rank r = rank(J). Then, in generic position.
dim(Tr (Nul((J) I max(0, -r)
(A5)
In particular, ifr KS one has Tr Null(J) = {0} and hence Tr & Null(J). Even when r SS, the inclusion Tr Null(J) requires measure-zero alignment and thus almost never holds. Consequently, there exists a nonzero A E TT with J A + O, implying
Cinin (OR + A)
A (B[JTFJ) + o(1A)))) > 0,
(A6)
where F is the Fisher matrix of the softmax.
Each module contributes an S-dimensional left subspace and an S-dimensional right subspace; their Kronecker product yields
Proof of Prop. 1.
dim T S = 5² in generie position,
(A7)
so, ignoring accidental cross-module dependencies,
K
dim Ti
dim T KSS.
(A8)
By the rank-nullity theorem for E RmXD
dim Null(J) = D -r
(A9)
For two subspaces A,B RDD, standard upper bound on the intersection dimension states
dim(AnB) I max(0, dim A + dim B - D}.
(A10)
Setting A 1= Tr and B [J) gives
din(Ti n Nu)l(J)) maxfo, KS2 + (D-r) = max KS -r..
(A1I)
Hence. if KSS the intersection is trivial and T₁ S. Null(J) is impossible. Even when < KSS the full inclusion would require not only dim Ti S dim (J) but also a non gen-generic containment (measure-zeru alignment) between the two subspaces; thus it almost never holds in generic position.
Finally, since F > and M = [E FJ m 0, any nonzero E Ti with JA * 0 satisfies A MMMA > 0, yielding
Lahink (0n A2)
A A M (11A))3 > 0.
(A12)
L
In words, orthogonality of principal parameter subspaces does not guarantee fuaranteefiistt-ne first-order invariance of outputs on the thinking format. This is precisely why our Stage imposes a null-space projection constraint (i.e., D veo( --) 0) cancel first-order effects.
20

---

Published as a conference paper at ICLR 2026
E.2 PROOF OF PROP. 1
Proof of Prop. 1. Let p = softmax(2) E A and = softmax(z + u), where z E RV is the logits vector at a thinking position / E (1)) and RV is the perturbation induced by the parameter change.
Step 1: KL as a Bregman divergence of Ise and a uniform quadratic bound. Let Ise(z)= V e ex so that V Ise(z) = softmax(=) = and V Ise(z) = diag(p) pp". For the multinomial exponential family, the KL divergence equals the Bregman divergence of the log partition function (Banerjee et al., 2005; Wainwright & Jordan, 2008):
LogE
KL (Lsftmmaxx2 + u) || softmax(2)) (z + u, 2) = Ise(= + u) - Ise(z) - (VVss(()) u). (A13)
Using the integral form of the Bregman remainder for a twice differentiable convex f. D (x h h, = (1 8) h T V + sh)) h ds, and the fact that for all the Hessian satisfies || V Ise(=)|122 S by positive semidefinite covariance form as in Lemma 1 (Boyd & Vandenherghe, 2004; Böhning, 1992), we obtain
(L(sfttmax(2 + u) softmax(2)) = =/[ (1 5) V>>sss( + su) us
< /. (1- 8) 11ll1 ds / |u||
(A14)
Equivalently, the second-order Taylor expansion with third-order remainder yields (L(ssftmax(z + u) || softmax(22) 1/2 V2 Ise(z) + 011)u1))) < =1lu122 (A15)
Lemma 1 (Hessian bound for Ise). For any E RY with p softmax(2).
Ise(2) = diag(p) - pp" T = O,
V [se(s)
||2 7
(A16)
Step 2: Bounding the logits perturbation via linearization and Lipschitz regularity. Let (1r () RV V xd be the Jacobian of (0 t) w.r.t. 0 at 0 = 0g. By the mean-value theorem and Taylor expansion with Lipschitz gradient (Nesterov, 2013),
u(x, t) = zen+a (z, t) - zea (I, t) = (x, vec(A) + t), || - 1/4 || (A17)
where is a local Lipschitz constant of Vezg(a, t) around ÖR. Let + bikdiag ( ..... Φ be the block- bock--diggona] forward feature operator that maps vee(A) to the linearized token-level feature change collected at thinking positions (per submodule k). Under bounded intermediate activations and operator norms, which are standard in local linearization of deep nets (Fazlyab et al., 2019), there exists C₁ > such that [| 1) veo( ())l S C. 11 vec(A)]l2. Combining with Eq. (A17),
||[[ l)
C 1| vec(A)]l2 + C2 11
C2 :
(A18)
Step 3: Enforcing the null-space constraint and aggregating into Lihink. Apply the submodule- wise null-space projection (see Eq. (5) in the main text):
vec (a p
(
(amn vec( (A))))
A
L 1
(A19)
k=1
so that by construction P vec( ()) O. Plugging this into Eq. (A18) yields for all t Quiik (z):
||u(z, f)1l < C2 || 12.
(A20)
Combining with Eq. (A14) and summing/averaging over (x, f) in the definition of Lihink (Eq. (2)) gives
Lihink (OR + 1 = Ex
Σ
KL (LTTTt ( x. ve)))) |l T I x, Beer))
tEQ (2)
< 8 En, [Il u(x 1) + +0 + (E,, 11))
( (1t)
< 0 ||| / 12) ~ 0.
(A21)
This completes the proof.
21

---

Published as a conference paper at ICLR 2026
Algorithm 1: RAIN-Merging! Geasoning-Awart Instruction-attentin guided Null-space projec- tion Merging
Input :LRM PR: ITM By; base model Py: reasoning calibration set DR with thinking indices Quink: instruction calibration set Dr with spans (I, R,U); hyperparameters p; 0t Cu, X.
Output :Merged model 0..
Stage 0: Task vector and objective.
Ar
- -0
// instruction task vector
Stage 1: Reasoning-aware Null-space Projection (satisfy Eq. (2)).
for <- 1 to K // iterate over submodules (per-Layer Wq, Wk, Wr, WFN) do
$ PeatureOperator (On: Dn, Qhink, k) // forward feature extraction
thinking tokens
Pr +- -ddag(1) (()()))) + diag(1)) 2 //
orthogonal
projector
submodule projection per Eq. (5) // direct merge after Stage Stage 2: Instruction-atteenion Guided Merging Coefficients (optimize Eq. (11)).
vec(At Pk vec(At)
8 OR K A7
Initialize head wise coefficients a² +-1 for all attention heads k.
for each attention head i do
12
f- Bu-DD
EiER(E) ErEr(e) Atth (x) [ZG) ())))
u 5- Ez-Di
[ [((a)) Ensu(e) E-Er(H) ( (
13
14
for each attention head & do
15
& + a -p
11 first-order term for Eq. (11)
H² t 1-1 k
// diagonal Hessian approx
17
ah clippar and
9
per-head optimal scaling
1TK
18 at + Aggregate (aa5) (lof)Eemmann Eemodale k)
// mean over heads for FFN
Output (Two-stage Merge).
20 return 0 + 0g + A D == a A
// final model in Eq. (16)
F ALGORITHM
Following Alg. is the algorithm of our RAIN-Merging.
G METHOD IMPLEMENTATION DETAILS
G.1 FORWARD MECHANISM IN TRANSFORMER
A standard Transformer layer consists of multi-head self-attention and feed-forward network (FFN).
In layer the hidden state of the token position i, denoted i, (E-11) Rdd linearly projected to
queries, keys, and values: 4i (( h (8-1) A w(())
(8- 1) . = -w(0) =w(O (6-1)
For head h, softmax, (1a) ((a(th)) / which
the head attention weights are Att (f.A)
( (x) 7] (x)
represent the probability that the token at position attends to position The corresponding
head output is of (f,h) =, Att ((.t) (x) 7] (.f.M) After concatenating the outputs from all heads
and applying w:(+) we obtain (.) The FFN then computes A((") (W: n, + 6 A
(-(1) ((() ((0)) The tayelayer hidden state is mapped vocabulary logits (z. t), which transformed by a softmax into the conditional distribution Tp(- æ, V..)). We follow the notation and the scaled dot-produce attention definition of Vaswani al. (2017) to align with prior work.
22

---

Published as a conference paper at ICLR 2026
Table Al: Reasoning calibration set construction from Mfxture-ofTThoughee We uniformly sample
50 examples per domain for calibration and 50 for validation. Raw sizes are taken from the official dataset composition page.
Domain
Raw size
Calibration
Validation
Math
93,700
50
50
Code
83,100
50
50
Science
173,000
50
Total
349,800
150
G.2 IMPLEMENTATION DETAILS IN MERGING
To balance computational efficiency and memory usage, all model-merging experiments adopt layer-wise merging strategy. During parameter fusion, we compute in FP64 precision ensure numerical stability, and we store the final models in BF16. Our experiments show that higher compute precision yields consistent but modest improvements for this merging procedure.
H CALIBRATION SET CONSTRUCTION
H.1 REASONING CALIBRATION SET
We employ the Fiiture--ff-Thogiie (Face, 2025) dataset as the source for reasoning-style calibration. This dataset contains validated R1 1-stlel reasoning traces spanning three domains: math, code, and science, with a total size of approximately 350k samples. Its official data composition page clearly specifies the sample sizes and origins for each sub-domain: math samples are sourced from OpenR I Math (Lozhkov et al., 2025), code from CodeForces Coo's (Penedo et al., 2025), and science from the science subset of the Nemotron post-training set (Bercovich al., 2025). From each domain, we randomly sample 50 instances form the calibration set (150 in total), and an additional 50 instances per domain are randomly sampled to serve as the validation set (150 total). Tab. Al shows the detailed numbers of samples in each domain.
Thinking Special Token Set Construction. To apply preservation constraints on "thinking format", extract the thinking tokens, specifically <think> and </think> in the model output-based on the RI-style chat template and tokenizer. The procedure involves rendering messages using the chat template provided by LRM. RI RI-family models prefill <think> in reasoning mode and insert < / think> the context, while some templates may omit the visible output of the initial <think> enforce thinking mode. We then obtain token positions of <think> and </think> in Qtink
H.2 INSTRUCTION CALIBRATION SET
We construct high-quality instruction calibration from neri-ierii prompts through four auto- mated and auditable steps. The pipeline produces span-based samples (x ~ Dr; I(x), R(x), U(x)) for computing the instruction-attention score proxy in Stage of RAIN-Merging. We choose to distill from IFFyva--tyyle instructions for ease of implementation and to test generalization on out-of-domain eestruction-oolowimnn datasets. The final size of the instruction calibration set 365. The full workflow is:
Instruction selection. We select rule-verifiable instruction prompts from IFEvl (Zhou et al., 2023b) as queries. Each record contains a natural-language instruction and machine-checkable constraints.
Step 1: Response generation by LRM. For each instruction query, we invoke an R1-style reasoning model (deepseek-reaseee response. This step yields fisllocttonnfollooiing samples generated by a reasoning decoder that reflect realistic decoding behavior.
SeepSeck-R1-0522 to produce a format-explic
http://hwggiiaaaaoottt/ts//oo//r Thouuhhss hhtps:/app..ecpsseekccom
23

---

Published as a conference paper at ICLR 2026
Instruction opan prompt)
Riaated span (responne matcoee instruction) Uimoekttdd spen (neeponss content not Hed inseruction)
Query (Prompt)
Given sentence unclesr of this
children", the sentiment regative? (The last
fesponse should there anything eise can help
Reason
asking about sentiment anslysl for specific The sertenne uncertainty about financial allocation children, which inherently carress negarive comnotation to implied lock of transparency or potential misoirection funds.
The phrasing it unclear" suggests doubt, and the focus on children as beneficiar adde an emotional weight to cribciim. This just about money-it's about athical reeporsbiiity making the negatisity pronounced
whould clariyy wny tt's negatios without cveecamplicet then and with the required claxing phaase instrrutedd The needs conzioe justifiec.
Response
The sentiment the ""negatire"t
The doubt misallocation
that funds intended children reaching
transparency,
there anything cise can help with?
Figure A3: sample illustration in instruction calibration
Table A2: Instruction-followii benchmarks. We list dataset size, constraint taxonomy, composition types, verification, and aggregation strategy.
Benchmark
Size
Construint
Composition Type
Verificttion
Evaluation
Taxonomy
Code Ludgas
Aggregation
IFEval
541
25
priic_ppompp_eoc Jevel_accuracy
average
InfoBench
500
5
DRTR
ComplexRench
1,150
depenlency-awnnc DRFR
Step 2: Rule evaluation and filtering. We evaluate the outputs Step with Iomvallco checker and relain only passing samples that satisfy all constraints. This removes cases that clearly fail the requirements.
Step 3: Striet span extraction (LLMMas-Judde)) We use nigh-peeformance instruction-tuned DeepSeek- V precisely extract instruction-relevant spans: I(x) (tokens in the prompt corresponding to the instruction) and R(x) (tokens in the response that are governed by the instruction). The unrelated span (1()) is then implicitly defined as the remainder of the response. See Fig. A3 for example.
LLM (deepseek-che)
Step 4: Tokenizer-level verification. We verify the extracted spans on the target tokenizer (aligned with our anchor LRM), ensuring that boundaries lic on token edges and can be deterministically reconstructed. Samples that fail alignment are discarded.
Step 5: Human review and ethical screening. To ensure data quality and compliance with safety and ethics standards, we introduce a manual review stage. Researchers verify the accuracy of the LLM-extracted spans I(x) and R(x,, and conduct ethics audit of the responses based on content-safety guidelines, removing any samples that contain biased, harmful, or inappropriate content. This step further enhances the reliability and ethical soundness of the calibration set.
This calibration pipeline readily transferable and can be extended to additional instruction- followwin datasets to further improve merging effectiveness by enriching the calibration set. The reasoning distillation model and the LLM--s-Judge can be updated over time to continually enhance the quality of the instruction calibration data.
24

---

Published as a conference paper at ICLR 2026
Table A3: evaluation.
Test set sizes of the six math benchmarks used in our mathematical reasoning (Math)
AIME2025
AIME2024
AMC23
GSM8K
Math500
MinervaMath
# Test samples
30
30
40
1,319
500
272
I DETAILED EXPERIMENTAL SETUP
1.1 BENCIMARRKS
Instruction-following Benchmarks. We evaluate instruction compliance on four widely used, programmatically verifiable benchmarks. The size and constraint types are summarized in Tab. A2.
IFEval (Zhou et al., 2023b). IFEval provides four accuracy metrics: (1) prompt-level strict accuracy and (2) instruction-level strict accuracy, plus (3) prompt-level loose and (4) instruction-level loose variants. The strict metrics require exact satisfaction (all constraints per prompt for prompt-level strict; per-constraint averaging across prompts for instruction-level strict). The loose metrics first normalize model outputs (e.g., strip Markdown, boilerplate intros/outros) to reduce false negatives. unless otherwise noted.
We report the official strict_prompt prttppprropteeepp_p_peee
CELLO (He et al., 2024). CELLO uses a code-based verifier that scores four granular aspects: (i) count limit (word/sentencc//saplle counts), (ii) answer format (parsability, keywords), (iii) task-prescribed phrases (mandatory phrases covered), and (iv) input-dependent query (presence of key phrases from the input), with a penalty discourage verbatim copying, We follow the benchmark's s practice and average these checks to produce the final score.
InfoBench (Qin et al., 2024). InfoBench adopts the Decomposed Requirements Following Ra- tio (DRFR): each instruction is split into scoring questions that are judged by LLMM-as-a-Jdde with binary YES/ / NO labels; the final score is the mean over all questions, enabling fine-grained interpretability. We evaluate by G GPT-5-mini. We report the official DRER.
ComplexBench (Wen et al., 2024). ComplexBench also evaluates via decomposed scoring ques- tions with YES/NO judgments and aggregates them into DRFR, but crucially uses a dependency- aware scheme: if any prerequisite constraint fails, all dependent (downstream) constraints are automatically marked as failed. This better reflects multi-constraint compositions. We evaluate by GPT-4o-mini. We report the dependency-aware DRFR.
Reasoning & General Benchmarks. We evaluate reasoning and general capabilities on the follow- ing benchmarks:
Math (Mathematical reasoning). We aggregate Pass@ 1/accuracy over six common math bench-
marks: AIME2025 (Balunović al., 2025): AIME2024 (Balunović et al., 2025); AMC23 (Cao et al., 2024); GSM8K (Cobbe et al., 1021); Math500 (Hendrycks et al., 2021); Minerva- Math (Lewkowyez al., 2022). The size of each math benchmark shown in Tab. A3. We report the averaged accuracy over all benchmarks.
Aider (Aider, 2024) (Code editing). Aider-Edit assesses code editing capability under a minimal- edit paradigm. It contains 133 small Python coding exercises sourced from Exercism, where the model is provided with a natural- language edit instruction and the existing code, and must generate a correct patch. The generated patch is required to apply successfully to the codebase and pass compilation and associated tests. Performance is measured by the Pass @ Edit Success Rate.
GPQA (Rein et al., 2024) (STEM). A curated, expert-level subset of GPQA comprising 198 four-option multiple-choice questions across biology, chemistry, and physics. Items are selected to be "Google-proof" and unambiguous: both expert validators must answer correctly while at most of three skilled non-experts succeeds, yielding a particularly hard split. We follow common practice and report accuracy (strict single-choice).
Arena- -enn--Hard-22 (Li et al., 2024) (Creative writing). A hard, open-ended benchmark constructed to maximize model separability and align with human preferences.
curate ~500 challenging prompts covering difficult real-world tasks including creative writing, scoring follows the pairwise battle paradigm with human or LLM-as-a a-Jdge assessments, and results are commonly summarized as win rate transformed Elo scores. For reproducibility, we fix
25

---

Published as a conference paper at ICLR 2026
Owen2. 5-7B the reference baseline and use GPT-5-mini the judging LLM: report the resulting win rate of each model against this baseline on the official Arena-Hard-v2 prompt set.
Agentic Scenarios. We evaluate the ability of the model to interact with the environment and complete tasks by two agentic benchmarks: ALFWorld (Shridhar et al., 2021) is text-interactive household-task environment involving multi-step planning and execution. We evaluate on 100 tasks. Metrics Success Rate of goal completion. WebShop (Yao al., 2022) is web shopping agent task (search, click, compare), with reported Normalized Reward to capture both path efficiency and goal matching. We evaluate on 100 tasks.
1.2 BASELINES
We compare our method with the following merging baselines:
Task Arithmetic (Ilharco et al.. 2023). The simplest linear composition injects the task vector additively near the anchor,
On + À Ar sealar À [0, 1] usually controls the strength; using the same À per layer or per block is also common. Its advantages are zero data and negligible compute; its drawback is that conflicts across submodules are hard to disentangle.
SLERP (Biship, 2007; Goddard et al., 2024). In SLERP (Spherical Linear Interpolation), weights are Ez-normalized to the unit sphere and interpolated along the geodesic to preserve norm and angular geometry. Let Ω arccos((wRR w1))- Then slerp(wR, wy; sin((I-1)()) WR sin
sin((() slngt W7, L E [0, 1]. During merging, we apply SLERP to each tensor and rescale by the original norm. This reduces norm drift compared with linear interpolation.
Karcher (Nielsen Bhatia, 2013; Goddard et al., 2024). On chosen manifold like the unit sphere Stiefel manifold, compute the Fréchet mean by minimizing the sum of squared geodesic
distances: min E. (1) ws). The iterative update is (1+1)) = Exp Σ, Loga (w;)).
TIES (Yadav et al., 2023). TIES is data-free method that explicitly prunes and sparsifies to handle parameter-level conflicts. For each layer's edit vector it applies (i) a sign-consistency mask (retain entries aligned with the dominant direction to reduce cancellation). (ii) magnitude thresholding or Top-k truncation (keep high-contribution entries and zero out the rest), and (iii) optional rescaling to match target norm. We can stack TIES as post-processing step on top of feasible baselines to improve robustness.
DARE (Yu et al., 2024). DARE uses first-order sensitivities on a small calibration set (for example, gradient norms of labeled loss, log-likelihood changes, Fisher approximations of the output distribution) to learn a per-layer or per-tensor coefficient a (or a diagonal preconditioner), yielding 0 = 02 D a A y- It can be viewed as data data-aware recalibration that reduces the bias introduced by naive addition with very low compute.
ACM (Yao et al., 2025). ACM (Activation-Guite sistency. On small calibration set it measures, before and after injecting the task vector, how each layer or head changes its response on instruction-relevan spans and its leakage on irrelevant spans. It then solves for per-submodule coefficients at optionally with cross-sample consensus regularization to improve generalization.
Consensus Merging) targets activation con-
LEWIS (Chopra et al., 2025). LEWIS (LayEr WIse Sparsity) is a merge with layer-wise sparsity allocation. Based on sensitivity indicators per layer (such as edit-vector magnitude, activation gradients. or Fisher approximations), it sets budget and merges only the Top-s parameters of that layer, leaving the anchor weights elsewhere unchanged. It can be combined with Task Arithmetic, SLERP, or Karcher as the base, and }sk} are determined by heuristics or grid search on a small calibration set.
AIM (Nobari et al., 2025). AIM (Activation- based Importance Merging) weights the task vector by activation importance (for example, the effect of Value or FFN outputs on downstream logits, or the instruction-aligned component of attention weights), performing element-wise or block-wise reweighting: Weights the task vector by activation importance, for example, the effect of Value or FFN outputs on downstream logits, or the instruction- aligned component of attention weights, performing element- wise or block-wise reweighting: 0 0R Dg W @ Af, where W is obtained from a single forward pass on the calibration set. Intuitively. this preserves edits that meaningfully change useful representations and suppresses noisy updates.
26

---

Published as a conference paper at ICLR 2026
Table A4: The hyperparameters of various merging methods in Tab. means the global scaling coefficient in merging. & denotes the trim ratio in TIES-Merging, p means the drop rate in DARE merging. r is sharpness the ACM, p is the pruning ratio LEWIS. means the balance factor in AIM.
Hyper-parameters
Method
7B
Task Arithmetic
1.0
SLERP
= 1.0
Karcher
À 1.0
TIES
0.8, 0
DARE-TIES
p 0.3, 0.5, =
T 1.0, = 0.5, x= 1.1
LEWIS-TIES TIES
p == 0.5, k = 0.5, X = 1.1
AIM-TIES
0.4, À 0.5, X 1.0
Table A5: The hyperparameters of RAIN-Merging in different model sizes. À means the global scaling coefficient in RAIN-Merging.
Hyper-parameters
Method
1.5B
7B
8B
14B
RAIN-Merging
À = 1.0 == 1.0
À 0.9
À = 1.0
Unless otherwise specified, following common practice in previous work (Wu al., 2025), we apply TIES post-pron poosterocsssing (sign consistency and magnitude truncation) on the outputs of DARE, ACM, LEWIS, and AIM, in order improve comparability across baselines.
1.3 HYPERPARAMETERS
For SFT, we use batch size of 16 with the Adam optimizer (Kingma & Ba, 2014), learning rate of 2 x 10-5, weight decay 0.05, and train for 20 epochs.
For all model-merging methods (including the proposed RAIN-Merging and all baselines). merge only the task vectors extracted from the ITM's WK/V/O/FF modules. The specific hyperparameter settings for each baseline used in Tab. are listed in Tab. A4.
In RAIN-Merging, we set the leakage penalty to p = 10 and bound the attention-head coefficients by [āg, a.] [0.0, 1.0). The global scalar À selected via grid search over [0.0, 1.5] with step size of 0.1; the chosen values for different model families are provided in Tab. A5. An ablation study of the global scalar À is included in Appendix J.2.
J ADDITIONAL EXPERIMENTS
J.1 DETAILED MATH BENCHMARK RESULTS
Tab. A6 and Tab. A7 report detailed results on the mathematics benchmarks. RAIN-Merging consistently preserves the mathematical reasoning ability of LRMs across different model sizes and architectures. In some cases, improving instruction following also correlates with better mathematical performance, suggesting that enhanced adherence can support clearer intermediate reasoning and more reliable final answers.
J.2 ABLATION STUDY OF THE GLOBAL SCALAR
We conduct a sensitivity analysis of the global scalar À (Fig. A4). Across wide range around our chosen value near 1.0, the merged model maintains strong follloii performance. As
27

---

Published as a conference paper at ICLR 2026
Table A6: Math benchmark results under the samee configuration in Tab. "Avg." denotes the average over all math benchmarks. The best and second-best results are highlighted in bold and underlined, respectively.
Method
AIME2025
ATME2024
AMC22
GSM8K
Math500
Minerva
Avg.
ITM
10.00
10.00
67.50
86.66
73.80
35.66
47.27
LRM
30.00
50.00
80.00
91.36
89.00
48.16
64.75
SFT
33.33
43.33
75.00
90.75
87.40
45.59
62.57
Task Arithmetic
30.00
50.00
80.00
90.75
89.00
45.59
64.22
SLERP
30.00
46.67
77.50
91.05
89.20
48.53
63.82
Karcher
30.00
50.00
80.00
90.98
89.20
48.90
64.85
TIES
33.33
46,67
82.50
91.36
90.60
48. 16
65.44
DARE-TIES
36.67
40.00
82.50
90.98
90.20
45.22
64.26
TIMSTIES
33.33
50,00
77.50
91.36
88.20
47.06
64.57
LEWIS
30.00
33.33
80.00
90.27
88.80
50.00
62.07
AIM-TIES
33.33
50.00
85.00
89.76
89.60
47.79
65.92
MAr--Megging 36.67
60.00
85.00
92.12
90.20
48.53
68.75
Table A7: Math benchmarks results under the same configuration as Tab. 2. "Avg." denotes the average over all math benchmarks.
Method
AIME2025
AIME2024
AMC23
GSM8K
Math500
Minerva
Average
Qwen2.5-1.58 1-B.5---1,BB--strree
3,33
0.00
30.00
75.44
59.40
22.43
31.77
DeepScck-RIIDDe -i--ek---l----illllll--
20.00
13.33
42.50
73.77
71.80
28.31
41.62
Qwen2.5-1.58 1...--1..B-RAAI-MM.. MiN-Mngggng
20.00
16.67
60.00
76.36
72.40
29.78
45.87
Llama-3.1-82 1-8---111--snsttcc
3.33
6.67
20.00
81.65
67.60
34.26
35.39
DeepSeck-RI Mol--isill-Laama..
30.00
40.00
75.00
90.52
80.40
45.37
60.21
30.00
43.33
77.50
90.30
82.80
47.79
61.95
Qwen2.5-14BB 115-stuutt
20.00
13.33
65.00
93.18
80.00
44.85
$2.73
DeepSeek-RI R11-istill-Qwent
50.00
56.67
92.50
93.22
$2.49
72.31
B0em2.5-14B-RAN--M--e
50.00
63.33
92.50
94.37
91.00
56.25
74.58
increases, reasoning ability improves slowly at first but then drops sharply beyond 1.0, indicating that overly large merge strength can still harm reasoning.
J.3 ABLATION STUDY OF REASONING CALIBRATION SET SIZE
Fig. A5 presents an ablation over the size of the reasoning calibration set. As the set grows, preservation of reasoning improves; however, costucttio-foolooinn performance degrades gradually. We hypothesize that overly strict preservation the thinking format can limit gains in instruction adherence and also increase computation. Balancing performance and resource usage, we select calibration size of 150.
J.4 VISUALIZATION OF MERGING COFFFICIENTS IN STAGE 2
As shown Fig. A6, the heatmap of merging coefficients for CrepSeck-Rl-Diseee RI-Dissill--Qenn----- Qwen- exhibits clear layer-wise differences, indicating that different layers respond instruction focus to different degrees. Notably, the earliest layers show the strongest response, with coefficients reaching the upper bound, and this pattern is consistent with the observations in Fig. 6.
J.5 ABLATION STUDY INSTRUCTION CALIBRATION SET GENERALIZATION
While we use instruction calibration set from IFEval in the main experiments due to its cleanly separable instruction spans and rule-based labels, the Stage proxy is not restricted to such data. construct additional instruction calibration set from InfoBench. InfoBench focuses open- ended constraints such as tone, style and content focus. We follow the same filtering pipeline as in Appendix H.2 and perform manual screening to ensure that the selected spans correspond to
28

---

Published as a conference paper at ICLR 2026
56
Instruction Following Avg.
55.59
54
Reasoning General Avg.
Average Score
52
50
48.11
48
0.0
0.3
0.4
0.5
0.7
0.8
0.9
1.1
1.2
1.3
1.4
Scaling Factor
Figure A4: Instruction following and reasoning & generage performance of our RAIN-Merging using different global scalar A. The configuration the in Tab. 2. The performance is measured by the average of the instruction following and reasoning general capability benchmarks. The marked result is our choice in experiments.
56
55,59
54
Average Score
52
50
48.11
48
Instruction Following Avg.
46
Reasoning & General Avg.
1500
15000
Reasoning Calibration Set Size
Figure A5: Instruction following and reasoning & generage performance of RAIN-Merging using diffcrent reasoning calibration set sizes. The x axis represents the size the reasoning calibration set with exponential scale. The configuration is the same as in Tab, 2. The performance is measured by the average of the instruction following and reasoning & general capability benchmarks. The marked result is our choice in the experiments.
instructional constraints rather than problem content, resulting in a total of 260 samples. We refer the calibration set of 365 rule-verifiable instructions from IFEval as Rule and the new set from InfoBench Open. We also consider mixed variant Rule+Opeen obtained by simply concatenating the two sets. Tab. A8 reports the performance of RAIN-Merging under these three calibration variants. Using the Open calibration set alone yields instruetion-following performance that is comparable to the original Rule setting, with slightly higher accuracy on the more open-ended benchmarks (InfoBench and ComplexBenehs This indicates that the instruction-attentie guided coefficients can still identify effective modules even when calibrated exclusively on open-ended instructions, and are restricted to IFEval-style rule-verifiable patterns. The mixed Rule+Open calibration consistently benchmarks. Combining rule-verifiabl and open-ended
improves all four instruction-followe instructions therefore produces more general proxy that better captures diverse instruction types. But carefully, we observe that both the purely Open and the Rule+Open variants incur modest drop reasoning & general performance compared to the Rule baseline. This suggests that while open-ended calibration further enhance instruction- following, may also slightly interfere with the preservation of reasoning. We view designing cleaner open-ended calibration sets and more explicitly modelling instruction-reasooning entanglement promising directions for future work.
instruction-oolloiingg
29

---

Published as a conference paper at ICLR 2026
Laya
13
Attention Head
Figure A6: Heatmap merging coefficients by our Stage each layer and attention DeepSeek-RI- Distill- Seiill-Qwen-7B..
Table A8: Performance of RAIN-Merging with different instruction calibration sets in Stage 2. Rule is our original instruction calibration from IFEval, Open is the new from InforBench, Rule+Open simply concatenates both sets. We merge Qwen2.5-7B - (ITM) into DeepSeek- RI-Distill-Qwen -weenl- (LRM) under the same configuration as Tab.
configurat
Instruction Following
Reasoning & General
Info Bench
Complex Bench
Arena-
IFEval
CELLO
Math
GPQA
Aider
Avg.
Hard-v2
LRM
55.45
16.59
71.73
32.72
44.12
64.75
44.44
29.63
65.29
51.03
RAIN-Merging (Rulc)
63.22
19.03
74.53
35.66
48.11
68.75
54.55
33.33
65,73
55.59
RAIN- Merging (Open)
62.92
19.24
74.89
35.67
48.15
65.14
49.49
31.11
64.67
52.59
RAIN-Mergiin (Rule+Open)
64.03
19.63
75.64
36.70
49,00
67.43
53.03
35.56
65.29
55.32
GENERALIZATION TO UNSEEN COSTRUCTION-FOLLOWWWR BENCHMARKS
To mitigate risks of data contamination from established benchmarks, we evaluate our method on three recently proposed cntructiio-followiin benchmarks that could not be used for calibration or training: IFBench (Pyatkin al., 2025), XIFBench (Li et al., 2025b), and E.FBBench (Zou et al., 2025).
Bench targets precise, verifiable output constraint and tests generalization to diverse out-of- domain constraint templates.
XIFBench evaluates multilingual instruction-follovii under fine-grained constraints across six languages, covering five categories such as content, style, format, situation and numerical require- ments.
EIFBench focuses extremely complex cestucttionfoolowinne scenarios, where models must execute multi-task workflows under multiple interacting constraints, closer to real-world product use-cases.
These datasets introduce new instruction formats, domains, and evaluation protocols, and are therefore suitable for testing robustness distribution shifts in instraction-followingg
30

---

Published as a conference paper at ICLR 2026
Table A9: Merging performance and relative gains of RAIN-Merging on three new instruction- following benchmarks. We merge the Qwen2.5-7B family under the same configuration as Tab. 1. The subsequent "(relative gain)" row reports the relative improvement our method over the LRM, highlighted in green.
Model
IFBench
XIFBench
EIFBench
Average
Qwen2. 5-
27.89
83.35
55.62
55.62
DeepSeek-R Iiii---DDDiiiil--e -ween-B
17.69
72.93
45.31
45.31
Qwen2.5-7B-Re T-B-RAIN-Merrae
19.39
76.32
47.85
47.85
(relative gain)
+9.62%
+4.65%
+5.62%
+5.62%
DeepSeok RI-Dietll Qwen--t
Qwen2.5 7B RAIN Merge
1.0
0.8
Accuracy
0.6
0.4
repoome Jangugg
camma
replish loveraee
queentnn
letter fieguney
capiaal wood Eopumer
forhaddan wonls
lite
mamber highttghtet cection
deepenyy
munbiee plawdbiddrr
repert proupt
muhiipee Sootfors
nunthrr ansteters
uit puraapaph, fiot wore
countaatd copostor
munber parogappi
posterrp
englihh captta
foop fermur
nambbe, baller lits
and shosker
Instruction Type
Figure A7: Instruction-type accuracy comparison between the LRM
and RAIN-Merging ( on IFEval. The results highlight the largest improvements on instructions like response laapoonseelaguaeeeeeeeeaa number_words, and no_comma.
The results are reported in Tab. A9. Across all three benchmarks, RAIN-Merging consistently outperforms the baseline LRM, with relative gains ranging from +4.65% +9.62% on individual datasets and +5.62% on the average. These improvements suggest that our method can generalize to new, previously unseen instruction- -ooluction-folooiing tasks.
J.7 INSTRUCTION-TYPE BREAKDOWN ON IFEVAL
To provide a deeper understanding of how RAIN-Merging improves instruction-followingg we perform per-instruction-type analysis on IFEval, which includes 20+ distinct instruction categories in the original [FEval span. Fig. A7 reports the instruction-lype accuracy comparison results. The analysis reveals that RAIN-Merging achieves the most substantial performance gains instruction types, such as response lanquage, number __rdds and no comma. For the majority of other types, the merged model either matches modestly surpasses the LRM's performance. These results complement the aggregate accuracy metrics reported in the main paper and offer further insight into the model's instruction-follae followwig capabilities.
31

---

Published as a conference paper at ICLR 2026
Table A10: Robustness to paraphrased instructions on IFEval. We merge the Qwen2.5-7B family under the same configuration as Tab. 1. HAcc is hard accuracy (%). Robustness is defined as (prrapaaaphrase) HAcc(original)
Model
IFEval(200) HAce
paraallparaprrase HAce
Robustness
Swen2.5-7B-Instteee
65.71
64,16
0.98
DeepSeek-R Baiiece-RR--iisss I-Distill-Weens
57.86
50.85
0.88
Qwen2.5-7B- R-1-2----BBRR----- RAIN-Merging
61.29
61.81
1.01
J.8 SEMANTIC ROBUSTNESS TO PARAPHRASED INSTRUCTIONS
Our main instruction forlowiin evaluation already includes benchmarks with non-trivial semantic components. In particular, InfoBench and ComplexBench explicitly evaluate whether the content of the response aligns with the prompt (e.g.. style, tone, key information coverage), rather than relying solely surface-level pattern matching. Thus certain degree of semantic evaluation is already baked into our instruction-follovin metrics. To more directly assess whether RAIN- Merging improves semantic instruction following, rather than simply adapting to specific phrasings, we additionally follow IFEval-extendee
(Kovalevskyi, 2024) and construct paraphrased version of IFEval. Concretely, we select 200 valid IFEval examples and use GPT-4o to generate three paraphrases for each instruction, yielding 600 phrased Instructionssdenoted as paraaa-para) We then evaluate: (i) hard accuracy on the original 200 IFEval prompts, and (ii) hard accuracy on the 600 paraphrased prompts. We also define a simple robustness metric, Robustness = (Aacoparuphrause) which measures how well performance is preserved under paraphrasing.
HAcc(onggina))
Tab, A10 reports the robustness paraphrased instructions on IFEval. We observe that LRM's performance degrades notably under paraphrasing. RAIN-Merging not only improves over the LRM on the original prompts, it also maintains slightly higher accuracy on the paraphrased prompts, achieving robustness of 1.01 that surpasses both the LRM and the ITM. These results indicate that RAIN-Merging enhances semantic easructioo-followinng capability, as performance improvements remain consistent even when instructions are substantially paraphrased. This demonstrates that the gains are not merely due to overfitting to specific phrasings or template-bar bastd patterns. The paraphrase- rrouutres-robustness experiment complements aggregate instruction-folloinnn evaluations and supports the claim that the merged model better captures the intended meaning of user instructions.
J.9 CASE STUDY ON IFEVAL
We provide two case studies to illustrate the effectiveness of RAIN-Merging compared with DeepSeek- R1 RI--istill-Qwen--B as the baseline LRM on IFEval.
IFEval Example 1: The baseline LRM violates the explicit rule to first echo the request verbatim and further duplicates its poem, yielding "following: False" outcome. In contrast, RAIN-Merging correctly repeats the request word-for-word, includes the required keywords ("intern," "grow"), and produces coherent, father-pleasinn limerick ("following: True").
The baseline LRM introduces capitalized section headers and markup (e.g., "Verse 1"), breaking the "all lowercase" constraint ("following: False")). RAIN-Merging delivers fully lowercase lyrics with clear structure and consistent semantics ("following: True").
IFEval Example 2:
J.10 CASE STUDY ON GPQA
To better understand why RAIN-Merging improves performance on GPQA, we go beyond final- answer accuracy and analyze concrete examples where the merged model corrects the LRM's mistakes.
GPQA Example 1: In the first case, the LRM constructs mostly correct physical derivation but makes a numerical slip in the computation as marked in red. The merged model follows essentially the same derivation but corrects the arithmetic step and therefore reaches the correct option. Notably, the merged model employs a step-by-step computational approach rather than relying single direct calculation in the LRM, reflecting its stronger logical coherence. This
32

---

Published as a conference paper at ICLR 2026
suggests that method goes beyond faithfully maintaining the structure of the reasoning chains and can also enhance the reliability of the LRM's step-by step-by-teep logical and numerical reasoning.
GPQA Example 2: In the second case, the LRM correctly analyzes each option and explicitly concludes that option (C) violates both isospin and parity conditions, while other options satisfy the constraints. However, in the final answer, the LRM unexpectedly outputs (A) as the best choice, contradicting its own reasoning. This is a classic tnowing-doing gap (Schmied et al., 2025), where the model's s decision is misaligned with its internal chain of thought. The merged model largely reuses the same analysis but now selects (C) as the final answer, aligning its decision with its reasoning. This is also the behavior captured by the improved RAA scores in previous Tab. 6 to explain the performance gains on GPQA.
33

---

Published as a conference paper at ICLR 2026
Instruction- followiin Example in IFEval
Prompt
Write a limerick about Hannah, a college student, doing an internship at a coffee company. Make sure that her father would love the limerick. Include the words "intern" and "grow". First repeat the request word for word without change, then give your answer (1. do not say any words characters before repeating the request; 2. the request you need to repeat does not include this sentence)
DeepSeek- RR- Diistill-Qwen------- Reason
Okay, so the user has given me query where they want limerick about Hannah, a college student doing an internship at a coffee company. They also want me to make sure her father would love it, include the words "intern" and "grow".
First, I need to understand what limerick is. It's five-line poem with AABBA rhyme scheme, usually humorous. So, I should structure it that way.
The main characters here are Hannah, who's 's an intern, and her father. I need to highlight something that would please her father. Since he's looking back on her, maybe something positive, like growth or learning. The user mentioned "grow," so that's 's a keyword must include. I'll start the first line with "There once Next, she's doing an intersship at coffee company, so the second line can mention that. For the third line, need to introduce her father. Maybe something like "Her dad would be tickled," which fits the rhyme and the idea that he's proud or finds it amusing.
college student named Hannah," which sets the scene.
In the fourth line, have to include "grow," perhaps about Hannah growing into her role or learning something. Something like "she grew as an intern." That works because it's common phrase and fits the rhyme.
Finally, the last line should be a nice, positive ending. Maybe the coffee company has grown as a result of her hard work. "And the JavaHouse grew from her intern!" That ties the "grow" into the company's s success.
I should make sure the rhyme scheme is consistent and the lines flow well. Iet me check each line for the AABBA pattern: the first and third lines have "Hannah," the second and fourth have "intern" and "grow." and the fifth is the punchline. The rhymes at the end of each line should be "annah," "inn," "dad." "un. and "or." Wait, that doesn't It quite fit perfectly. Let me adjust the fifth line to "and JavaHouse grew from her intern." " 'hat way, the last two lines rhyme with "grow" and "grew," which is close enough for limerick's loose rhyme.
I think this captures Hannah's 's experience, shows her learning, and how her father is proud. It also tics the coffee company's 's growth to her internship. which is a nice touch. should make sure the language is playful and fits the humorous tonc of limericks.
DeepSeek- B1ak--R---issss RE1-Disiil--wen-TTT Qwen-7B Response (following: False)
There once was a college student named Hannah, Whose intern days JavaHouse she would swap for much. Her dad would be tickled, I'm sure, If he heard she grew as an intern.
And JavaHouse grew from her intern!
There once a college student named Hannah, Whose intern days at JavaHouse she would swap for much.
Her dad would be tickled, I' I'm sure, If he heard she grew as an intern. And JavaHouse grew from her intern!
RAIN-Merging Reason
Okay, the has given doing an internship a coffee company. They also want to include the words "intern" and "grow". Plus, the limerick should make my father happy. need to make sure that the poem not only fits the structure of a limerick but also touches on what a father might appreciate.
query where they want limerick about Hannah, college student
First, recall the structure of limerick (AABBA, typically anapestic). Incorporate internship details and the required keywords. Emphasize traits father would love: responsibility. learning, contribution. Draft lines that meet the rhyme scheme and include "intern" " and "grow," aiming for playful, positive tone that showeases Hannah's 's growth and impact at the coffee company.
RAIN-Merging Response (following: True)
Write a limerick about Hannah, a college student, doing an internship at a coffee company. Make sure that her father would love the limerick. Include the words "intern" and "grow".
There
college student named Hannah.
An intern at coffee company she was.
34

---

Published as a conference paper at ICLR 2026
With her skills, both sharp and bright, She's learning every day,
She's growing taller, her skills grow bright.
Her dad would surely give her a smile, For Hannah's heart and Hannah's skill. She's balancing tasks, balancing time, Turning great coffee into something divine.
A limerick full of sunny pride!
Instructionnfollowinng Example IFEval
Prompt
Compose song lyrics about a cocio-cconomic problem. The song should be English and in all lowercase letters.
DeepSeck-R1-DDe
Reason
The user asked for song lyrics about socio-economic problem, in English and all lowercase. Choose relatable issue (e.g., inequality, unemployment), structure with verses/chorus/bridge maintain empa- thetic tone, vivid imagery, and ensure every character is lowercase. Aim for accessibility, repetition in chorus, and subtle sense of resilience or hope.
DeepSeek- -1--Seek-----e R21-Distill-Que Qwen-7B Response (following: False)
Verse
every penny's s a struggle like fuel for car that won't start the stars are bright but we don't see them
the ladder's 's always broken
Chorus
every penny's 's a struggle a weight in your chest
the world's game but you're 're just piece no matter how hard you try every penny's a struggle
Verse
the roads wet but the rain falls cold
the wind's whisper but it's got hold we're dancing to a tune we can't 't control but the beat's 's the same
Chorus
every penny's a struggle a weight in your chest the world's is game but you'te 're just piece no matter how hard you try every penny's struggle
Bridge
look beyond the pain is there spark in your soul? can you see the stars night? maybe there's 's something more
Chorus
every penny's struggle a weight in your chest
the world's game but you're just a piece no matter how hard you try
every penny's struggle
Outro
a penny's 's worth of love or light the struggle's 's real but we gotta fight we'll find the way through the cold the world's game but you're just piece
35

---

Published as a conference paper at ICLR 2026
RAIN-Merging Reason
Generate lyrics that address socio-cconomic issue with clear imagery and structure (verses/chorus/bridee)) Conform strictly to the all-lowercase constraint. Keep language simple and emotive, focusing on systemic barriers and lived experience, while maintaining flow and singability.
RAIN-Merging Response (following: True)
"the gap in the system"
(verse !)
sce the faces in the lines, faces searching
ticket out of here.
factories grinding, bodies quiet, world that promises better, but i' I'm still waiting for the breath.
there's crack in the system, fault (chorus)
law,
half the planet's frozen in time. we're 'r crafted from the same mold, same broken spell, hut the world won't let climb the hill.
(verse 2)
the streets are empty, the shops tiled over,
the only sound is the shuttle's old tunc. we're 'e drowning the waves, but the shores lie, the wind.
the promises fade like
(chorus)
there's crack in the system, fault in the law, half the planet's frozen in time. we're re're crafted from the same mold, same broken spell, but the world won't let us climb the hill.
(bridge)
oh, it's not just the lack, it's the hroken chain, we're all stuck in the margins, never quite seen.
hut the fights don't t stop, they never do, we're trapped in the loop, chained to the ground.
(chorus)
there's crack in the system, a fault in the law, half the planct's frozen in time. we're crafted from the same mold, same broken spell, but the world won't let climb the hill.
(outro)
so here's 's to the night shifts and the cold nights, the endless loop where the kids don't 't get the break. we'r all fragments on the ground, trying to find a way the ground.
36

---

Published as a conference paper at ICLR 2026
Reasoning Example I in GPQA
Question (Answer: C)
a meson resonance. What is mean decay distance? Knowing that the production energy Ex = 8GeV, the mass vix = 1.2GeV, and the width I'x 320MeV.
A. 4.06555 x 10 m
B. B.5.0223 X 10 10-18m
C. 4.0655 X 10-1m
D. 5.0223 x 10 1011m
DeepSeck-RI-Dise RR1-Distll--ween Reason
Okay, so have this physics problem here about I'm a bit rusty on partiele physics, but let me try think this through step by step. First, the question gives me some data: the production energy Ex 8 GeV, the mass mxx 1.2 GeV, and the width I'x 320 MeV. need to find the mean decay distance from these values. The options are given in meters, so guess need to calculate the decay length.
resonance and its decay distance. Hmm,
(We omit the lengthy reasoning process for clarity.)
Wait, h 6.582 10-25 GeV-s. So 0.32 GeV
26 2.056875 10-25,
Now, the mean decay length L 7/Ser. Let's plug the numbers. x 10 m/s.
6.6667, B 0.9887,
So L = 6.6667 x 0.9887 x 3 10 m/s 2.056875 x 10 10-25, s Let me compute this step by step. First, 6.6667 x 0.9887 zs 6.6049. Then, 3 x 10 x 2.056875 x 10-25 = 6.1 6.170 x m. Now multiply that by 6.6049: 6.6049 x 6. 70625 10-17 se 4.07 x 10-16 m.
Looking the options, option is 4.0655 10 m, which very close my calculation. Option 5.0223 10 10-16, which higher. So think the correct answer is
Response (Answer; A, Wrong)
The mean decay distance L for particle is given by L = rySer, where y is the Lorentz factor, / is the velocity in terms of the speed of light, e the speed of light, and is the proper lifetime. The proper lifetime is related the total decay width I by h/T, where à the reduced Planck constant.
Given:
Ex = 8 GeV,
-mx 1.2 GeV,
Tx 320 McV = 0.32 GeV.
First, calculate
Ex /xx 8/1.2 6.6667.
Next, 8 /1 (1/72)
/1 (1 (166.6677)) 0.9887.
The proper lifetiiee is 7 = h/T x, where h = 6.582 x 10-25 GeV-s.
Thus, T= (6.582 x 10 1 / x 2.056875 x 10
Finally, the mean decay length is: 78e7 6.6667 0.9887 x (3 x 10 2.056875 10-25
4.0655 106
Comparing with the options, the closest match is A 4.0655 x 10 16 m.
Answer: A
37

---

Published as a conference paper at ICLR 2026
RAIN-Merging Reason
Okay, so I've got this physics problem here about meson resonance and its mean decay distance.
Hmm, I'm a bit rusty on particle physics, but let's try to think this through.
The question gives me Ex = 8 GeV, mxx = 1.2 GeV, and I'x 320 McV. need to find the mean decay distance. The options are all in meters, so guess need to calculate the decay length.
Wait, decay length... remember that the mean decay length (L) related to the particle's ' lifetime (r) and its velocity (3). The formula is == Byee, where ? is the Lorentz factor. But wait, sometimes also expressed using the width because the lifetime is related to I' by r = A/T. Oh right, because is the total decay width, which is inversely proportional to the lifetime.
So first. need to find the lifetime I' is given in MeV, so should convert that GeV for consisteney with Ex and x- 320 McV 0.32 GeV.
Now, using
R/T. But wait, what's the value h GeV-s? I think h approximately 6.582 10- Ge Let me double-check that. Yeah, think that's 's correct.
So 7 = (6.582 10 GeV (0.30 GeV) = 6.582 x 10 25 /0.32 =u 2.057 x 10 24 seconds.
(We omit the lengthy reasoning process for clarity.)
Wait, let me check the exact values.
First, R/T. h is 6.582 10-25 G-V-s. I is 0.32 GeV. S 6.582 x 10 25 /0.32 = let's compute that exactly.
6.582/0.32 = 20.56875, so 20.56875 x 10 25 2.036875 x 10
(We omit the lengthy reasoning process for clarity.)
RAIN-Merging Response (Answer: C, Correct)
To calculate the mean decay distance of the meson resonance, we the formula: L BYT,
where:
-c = 3 x 10 m/s (speed of light),
-3
E the velocity units of
- E is the Lorentz factor,
TT= is the proper lifetime, with h = 6.582 x 10 26 GeVis.
Step 1: Calculate
Given F'x 320 MeV = 0.32 GeV, the proper lifetime is: 7 Tx
6.582x10-25 0.32
& 2.056875 X
10-24 Step 2: Calculate 3 and
The energy is Ex = GeV, and the mass is mx = 1.2 GeV/e. The Lorentz factor is: y= Ex
6.6667
The velocity factor B is: 3
1.22 82
0.0225 /0.9775 0.9887
Thus, Br 0.9887988888 6.6667 R2 6.5911.
Step 3: Calculate L Substituting into the formula for T: I 8 10 1
66.591 2.056875 x 10
First, calculate fr: 3 10 6.5911 1.97733
10" m/s
Now multiply by T: = 1.97733 x 10 2.056875 x 10
24
4.067 x 10 m
Step 4: Match with the closest option
The calculated value L x 4.067 x 10 15 is closest to option C (4.0655 x 10 10 m).
ANSWER: C
38

---

Published as a conference paper at ICLR 2026
Reasoning Example 2 in GPQA
Question (Answer: C)
Consider a bound system of two nucleons (NN) a 1S0 state which emit particle X of intrinsic parity -1. Pauli statistics require that the isospin, angular momentum, and spin quantum numbers of the final NN state satisfy T(NN) S(NN) + L(NN) + (mod 2).
Suppose the final NN state has T(NN) Which of the following partial waves is not permitted, where the lowercase letter denotes the angular momentum state of Make use of the conservation of both angular momentum and parity.
Note that the term symbol (e.g. 1S0) is in general defined (2S+1)L(J), where is the total spin, L the total orbital angular momentum written in spectroscopic notation, and J the total angular momentum.
A. ISO 3D3 f
B. ISO - 3S1 +p
C. IS0 - 3PO 5
D. 1S0 - DDDI +p
Reason
Okay, I'm trying figure out this problem about mucleon-nucleon bound systems and the partial waves involved when they emit a particle. Let me start by breaking down the question and given information.
(We omit the lengthy reasoning process for clarity.)
But I'm not entirely confident. think the correct answer is (A) because doesn't 't satisfy the isospin condition, but I' m not 100% sure.
DeepSeek-R1-o
Ri-i-itil----------- Rii-iisilll--een-T Response (Answer: A, Wrong)
The problem involves bound system of two nucleons (NN) in 1SO state emitting particle X with intrinsic parity -1. The isospin, angular momentum, and spin quantum numbers of the final NN satisfy the condition T(NN) S(NN) + L(NN) + mod 2, where T(NN) = 0 for the final NN state. The task is to determine which partial wave ()) of X is not permitted, considering both angular momentum and parity conservation.
(We omit the lengthy reasoning process for clarity.)
##tf Option-by-Option Analysis:
(A) 1S0 ~> 3D3 +
NN term symbol 3D3: $ 1, = =3
T(NN) (1 + 2 1) mod 2 4 mod 2 (satisfies the isospin condition).
Parity of NN state: (-1)L+ S (-1)2 + = (satisfies the parity condition). Conclusion: This partial wave is allowed.
(B) 180 - 3S1 +p
NN term symbol 3S1: S =. J=
T(NN) (1 0 + 1) mod 2 2 mod 2 0 (satisfies the isospin condition).
Parity of NN state: (-1) SS (-1)
-1 (satisfies the parity condition). Conclusion: This partial wave is allowed.
(C) 1S0 ~> 3PO +
NN term symbol 3PO: S L=
T(NN)
mod 3 mod (does not satisfy the isospin condition). Parity NN state: (-1) + +S (-1)î = +1 (does not satisfy the parity condition). Conclusion: This partial wave not allowed.
(D)
1S0 > 7D1 +p
NN term symbol 7D1: S -=2,-=1
T(NN) = (3 2 1) mod 2 mod 0 (satisfies the isospin condition).
Parity of NN state: (-1) + (-1) =1 (satisfies the parity condition).
Conclusion: This partial wave allowed.
39

---

Published as a conference paper at ICLR 2026
#i# Final Answer:
Options (A) and (C) are permitted because they fail either the isospin or parity condition. However, since the question asks for the single best answer, (A) chosen as the answer because it the first invalid option encountered in the analysis.
ANSWER: A
RAIN-Merging Reason
Okay, so I'm trying to figure out which partial wave isn allowed in this nuclcar physics problem. Let me start by understanding the question step by step.
(We omit the lengthy reasoning process for clarity.)
So the only option that doesn't satisfy the isospin condition (C). Therefore, answer should be (C).
RAIN-Merging Response (Answer: C, Correct)
To determine which partial wave is not permitted, we analyze the problem using the given conditions:
1. Initial State: The initial NN system is in the 180 state, which has total spin 5 0, orbital angular momentum
0, and isospin T 0. The parity of the initial state is +1 since
P= (-1-))
+1.
Final State Conditions:
TTee final NN state has T = 0, so the isospin condition is T S 8 L+ mod 2. Substituting
T= 0, we get S L
0 mod 2, which simplifies to S + == mod 2. This means S and must have opposite parity (one even, one odd).
The parity of the final state must conserve parity, The emitted partiele X has intrinsic parity - so the parity of the final NN state is Fax = PNK Px = (+1) (-1) -1. Since Fax = (-1) _ this
implics (-1)2 1, so must be odd.
Analysis of Each Option:
(A) 1S0 3D3 The NN state 3D3, S (since 25 3) and T. 2.
1 3, which satisfics $ + T. = mod 2.
is even, which violates the parity condition L must be odd.
Allowed.
(B) 1S0 - 3S1 p: The NN state is 351, so S = 1 and L, = 0.
-. 1, which satisfies + T. = mod 2. is even, which violates parity condition must odd.
Allowed.
(C) 180 - 3PO s: The NN state is 3PO, S = and L = 1.
- -S+L=14 1+1 2, which does not satisfy S + L (mod 2).
= odd, which satisfies the parity condition. Not allowed.
(D) 1S0 - 1D1+ p: The NN state is 7D1, S = 3 (since 25 + 7) and L 2.
- 5, which satisfies + L = mod 2. = is even, which violates parity condition must odd.
Allowed.
4. Conclusion:
All options except (C) satisfy the parity condition must odd. However, the isospin condition S L = mod 2 violated in (C), making it the only permitted partial wave that is not allowed.
ANSWER: C
40

---

Published as a conference paper at ICLR 2026
K IMPLICATIONS
The potential impacts of this work span both academic and applied dimensions.
On the positive side, RAIN-Merging offers interpretable, low-compute pathway for capability integration. It enables researchers and resource-constrained labs to inject instruction-following competence into LRMs without additional training. By enforcing null-space constraint the thinking segment (<think>.. ./thnkkkk.t/tink>) the method preserves the model's structured reasoning format, which helps maintain reliability in reasoning. This direction may catalyze systematic studies of the relationship between task-vector orthogonality and thinking-format stability, and it encourages reproducible evaluation practices (for example, public evaluation scripts, calibration sets, and hyperparameter configurations) and greater standardization of community benchmarks. In agent applications such as WebShop and ALFWorld, Rerg-nggging can lower the barrier to integrating multiple capabilities and improve the practicality of tool use and structured outputs.
On the risk side, parameter merging can introduce capability drift or safety drift. For example, while improving instruction following, it may alter jailbreak sensitivity, amplify biases present in training data or in LLM-at LLM--s-jude pipelines, or induce hallucinations tied to specific output formats. Instruction attention proxy metric may also encourage myopic optimization for format matching, which not equivalent to value-aligned safety. Moreover, increased model usability can be misused for mass generation of misleading content, evasion of platform policies, or automated spam. The current method also depends on RI-style special markers and prompting templates; its cross-model and cross-modal generalization remains be established.
L LIMITATIONS AND FUTURE WORK
Our method has the following limitations. (i) The method relies R1-style templates and tokeniza- tion to extract <think>... </think>> for constructing the null space. If a model hides its reasoning (for example, implicit CoT) or adopts different templates, the constraint may weaken fail. (ii) The instruction and reasoning calibration sets are limited in size and include noise from LLM-aa-judge auto-annotation. Distribution shifts across languages or task domains may affect the generalization of the merging coefficients. (iii) Although the KL constraint the thinking segment helps preserve the reasoning format, non-thinking content and safety-relevant behaviors may still drift, and there currently no formal safety guarantee. (iv) Experiments focus primarily on the Qwen/DeepSeek families. Applicability to multimodal LLMs, tool use, code- oode-generation settings, and multilingual scenarios requires systematic evaluation.
41


Reviews:
[{"review_id": "9jMlLvFdm7", "reviewer": "Reviewer_Yy8R", "summary": "The paper proposes RAIN-Merging, a training-free (gradient-free) technique to improve instruction following in Large Reasoning Models (LRMs) without degrading their explicit “thinking→answer” format. The method has two stages: (1) Reasoning-aware null-space projection that projects the instruction-tuned model’s task vector into the null space of forward features at special thinking tokens (e.g., <think> … </think>), thereby preserving the LRM’s thinking distribution (formalized via a KL constraint). (2) Instruction-attention guided scaling, which uses forward attention statistics from a small instruction calibration set to compute per-module coefficients that increase attention alignment to instruction spans while penalizing leakage to unrelated spans. Experiments across Qwen/Llama backbones and sizes show instruction-following gains with maintained or improved reasoning and lower compute than SFT; ablations support both stages’ roles and show the method keeps the <think> format intact (no missing terminator).", "questions": "How does Stage 1 handle LRMs whose “thinking” is not demarcated by explicit tokens, or that interleave tool calls and thoughts (e.g., ReAct-style)? \n\nGeneralization of the proxy. If the calibration set used open-ended instructions (no machine-checkable rules), would Stage 2 still pick effective heads? Any results using other instruction corpora?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Clear problem & neat insight. The paper pinpoints a real pain point: LRMs reason well, but violate format/constraints. The idea to protect the thinking segment explicitly while injecting instruction-following behavior is crisp and well-motivated.\n\nStrong empirical results. On the headline 7B setting, RAIN-Merging improves instruction-following average (48.11 vs. 44.12 LRM; +4 points absolute) while also improving reasoning/general (55.59 vs. 51.03) and beating task-arithmetic, SLERP, Karcher, TIES, DARE-TIES, and activation-based methods (AIM/ACM/LEWIS combined with TIES) \n\nEfficiency. Minutes to merge (20.96 min reported) vs. SFT’s 120+ min; GPU memory also far below SFT (22.1 GB vs. 112.6 GB in their config)", "weaknesses": "Calibration-set specificity. The instruction calibration set is distilled from IFEval-style instructions (365 samples). This may bias the proxy to rule-verifiable patterns and possibly underrepresent open-ended or tool-use instructions.\n\nReliance on explicit thinking markers. Stage 1 presumes accessible special tokens and feature extraction around them. It is unclear how well this transfers to LRMs with different templates (or hidden/implicit thinking) or to models without consistent <think> tags, using ReAct format for thinking tool use.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fJp4UTRKW0", "reviewer": "Reviewer_snDt", "summary": "This paper presents RAIN-Merging, a novel two-stage, gradient-free method for merging Large Reasoning Models (LRMs) with Instruction-Tuned Models (ITMs) to enhance instruction-following capability while preserving structured reasoning outputs. The approach leverages task-vector orthogonality and introduces a reasoning-aware null-space projection to maintain thinking formats, combined with instruction-attention guided coefficients to improve instruction adherence. Extensive experiments across multiple benchmarks and model scales demonstrate that RAIN-Merging significantly improves instruction-following performance without compromising reasoning or general capabilities. The method offers a computationally efficient and interpretable alternative to supervised fine-tuning, making it highly relevant for real-world applications of LRMs.", "questions": "1.  Overall, I think you have done very meaningful work. My question is: Will your work be open-sourced as a toolkit in the future? I am very much looking forward to using the methods proposed in your paper.\n2.  Have you considered validating your method on larger model sizes (e.g., above 30B parameters) to further verify its effectiveness?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.   Novel Research Problem: The work addresses an important and underexplored challenge—balancing instruction-following and reasoning capabilities in LRMs—through model merging, a lightweight and training-free approach.\n2.  Effective Methodology: The two-stage RAIN-Merging framework is well-motivated, combining null-space projection to preserve reasoning structure with attention-guided scaling to enhance instruction alignment, all without gradient updates.\n3.   Comprehensive Experiments: The paper provides extensive evaluations across multiple instruction-following, reasoning, and agentic benchmarks, with consistent improvements shown across different model sizes and architectures.\n4.  Clear and Well-Structured Writing: The paper is clearly written, with a logical flow, detailed derivations, and accessible explanations of both the motivation and technical contributions.", "weaknesses": "While the method is evaluated on several model families (Qwen, Llama), further validation on a wider range of architectures and modalities (e.g., multimodal or multilingual models) would strengthen the generalizability claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Tlb8bqsR5Z", "reviewer": "Reviewer_i2ye", "summary": "This paper introduces RAIN-Merging, a gradient-free method for enhancing instruction following in Large Reasoning Models while preserving their structured thinking format. The approach projects instruction task vectors onto the null space of thinking token features, then applies instruction-attention guided scaling coefficients. The method demonstrates improvements across instruction-following and reasoning benchmarks without requiring gradient-based training.", "questions": "* What is your hypothesis for why reasoning ability and instruction-following have low coupling in parameter space? Is this due to structural differences (thinking tokens vs. output format) rather than semantic content differences?\n* How can you verify that Stage 1 preserves actual reasoning ability (content) and not just the habit of using thinking tokens? What metrics or analyses distinguish these scenarios?\n* In Stage 2, how do attention-guided coefficients handle tokens where reasoning and instruction-following are deeply entangled (e.g., instructions about reasoning strategy, logical constraints, or structured argumentation)? Can you provide examples and analysis?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Merging ITM and LRM is an interesting and practical problem.\n2. The finding that task vectors' principal subspaces are nearly orthogonal across key modules provides an interesting understanding about the parameter space structure of these capabilities.\n3. The evaluation spans multiple model families and sizes.\n4. The gradient-free nature makes this a practical, accessible alternative to SFT.\n5. Using four instruction-following benchmarks and multiple reasoning datasets provides reasonable empirical breadth.", "weaknesses": "### 1. Data Contamination / Generalization Concerns\nFor example, Qwen2.5-7B-Instruct is trained on IFEval, InfoBench, and ComplexBench as calibration data, and this paper evaluates RAIN-Merging on the same benchmarks. Results may not generalize to unseen instruction-following or reasoning scenarios. Maybe the null-space projection and coefficients are optimized on the same distribution they're tested on.\n\n### 2. Data\nThe paper evaluates instruction-following and reasoning on separate benchmark datasets. While the paper attempts to evaluate integrated capabilities using agentic scenarios in Table 3, these tasks may not simultaneously stress complex reasoning and strict, arbitrary instruction-following to the same degree as the benchmarks. The main evaluation in Table 1 still separates these two skills, leaving a gap in the core claim.\n\n### 3. Metric\nThe paper relies primarily on accuracy metrics across all benchmarks. More metrics would be valuable to answer such questions: Is the thinking process coherent, or just structurally preserved (only thinking tokens)? Which types of instructions improve most/least? Are outputs semantically following instructions, or just superficially (evaluation on phrased instructions)?\n \n### 4. Method\nIn stage 1, how do we know null-space projection preserves reasoning ability rather than just token usage? Does the model still perform meaningful reasoning in <think> blocks, or just maintain the format while reasoning quality degrades? If the reasoning benchmark results show preservation, but is this because the reasoning content is truly preserved, or the model learned to use thinking tokens without meaningful reasoning?\n\nIn stage 2, what happens when reasoning and instruction-following are highly entangled? How are attention-guided coefficients computed when both reasoning and instruction-following are active in the same tokens?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dwNwWZLWeV", "reviewer": "Reviewer_2pjo", "summary": "This paper uses task vectors to alleviate the issue of reasoning models where they are struggling with instruction following. Authors proposed a gradient-free method to address this problem, called Reasoning-Aware Instruction-attention guided Null-space projection Merging (RAIN-Merging). At the first stage, ITM task vector is projected on the null space of forward features at thinking tokens, and at the next stage, an instruction attention is estimated to further amplify instruction-relevant components.", "questions": "See Weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, and figures and tables help better understand the complicated method that authors proposed.  \n2. Reasoning models are bad at instruction-following is a well-known phenomenon, and the proposed method uses simple task vectors to boost the instruction-following ability while maintaining the original reasoning performance.  \n3. The results are reported in multi-dimensions according to the research questions in Section 4. Specifically, reporting a performance in agentic scenarios is a very good experimental evidence that the proposed method works well in the real world.  \n4. The proposed method is theoretically well-grounded.", "weaknesses": "1. In Table 1, there are some cases where RAIN-Merging outperforms the performance of the original LRM. Authors hypothesize that stronger instruction adherence improves CoT quality. -- I suggest to prove this hypothesis (by manually checking randomly sampled predictions). This phenomenon seems very interesting to me since my intuition says the opposite. Specifically, the performance is increased by ~10% in GPQA. How is it possible?  \n2. In the ablation study, only stage 2 is ablated. Therefore, the paragraph's name should be scoped down. Also, could you ablate stage 1 as well to prove its effectiveness?\n\nDespite some of these questions, I believe this paper would contribute to the community significantly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)