Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Authors: Xu Wang
Yan Hu
Wenyu Du
Reynold Cheng
Benyou Wang
Difan Zou
Venue: ICML 2025 poster (2025)
Keywords: Circuit Analysis
Fine-Tuning
Mechanistic Interpretability
Primary area: deep_learning

Abstract: Fine-tuning significantly improves the performance of Large Language Models (LLMs), yet its underlying mechanisms remain poorly understood. This paper aims to provide an in-depth interpretation of the fine-tuning process through circuit analysis, a popular tool in *Mechanistic Interpretability (MI)*. Unlike previous studies (Prakash et al. 2024, Chhabra et al. 2024) that focus on tasks where pre-trained models already perform well, we develop a set of mathematical tasks where fine-tuning yields substantial performance gains, bringing the setup closer to real-world scenarios. In our experiments, we identify circuits at various checkpoints during fine-tuning and examine the interplay between circuit analysis, fine-tuning methods, and task complexities. First, we find that while circuits maintain high node similarity before and after fine-tuning, their edges undergo significant changes, contrasting with previous work (Prakash et al. 2024, Chhabra et al. 2024) that reported only small circuit additions after fine-tuning. Based on these observations, we develop a **circuit-aware Low-Rank Adaptation (LoRA)** method that assigns ranks to layers according to edge changes in the circuits. Experimental results demonstrate that our circuit-based LoRA achieves an average improvement of 2.46% over standard LoRA with comparable parameter sizes. Furthermore, we explore how combining circuits from subtasks can enhance fine-tuning in compositional tasks, offering new insights into task design and deepening our understanding of circuit dynamics and fine-tuning mechanisms.

Towards Understanding Fine- Tinn-Tnning,, Mechanisms of LLMs via Circuit Analysis
Xu Wang 12 Yan Hu? 2 Wenyu Du
Reynold Cheng'
Benyou Wang? Difan Zou'
Abstract
1. Introduction
Fine-tuning significantly improves the perfor- mance of Large Language Models (LLMs), yet its underlying mechanisms remain poorly under- stood. This paper aims to provide an in-depth in- terpretation of the fine-tuning process through cir- cuit analysis, a popular tool in Mechanistic Inter- pretability (MI). Unlike previous studies (Prakash et al., 2024; Chhabra et al., 2024) that focus on tasks where pre-trained models already perform well, we develop set of mathematical tasks where fine-tuning yields substantial performance gains, which are closer to the practical setting. In our experiments, we identify circuits at vari- ous checkpoints during fine-tune fine-tuning and examine the interplay between circuit analysis, fine-tuning methods, and task complexities. First, we find that while circuits maintain high node similarity before and after fine-tuning, their edges undergo significant changes, which is in contrast to the pre- vious work (Prakash et al., 2024; Chhabra et al., 2024) that show circuits only add some additional components after fine-tuning. Based on these ob- servations, we develop a circuit-aware Low-Rank Adaptation (LoRA) method, which assigns ranks to layers based edge changes in the circuits. Experimental results demonstrate that our circuit- based LoRA algorithm achieves an average per- formance improvement of 2.46% over standard LoRA with similar parameter sizes. Furthermore, we explore how combining cireuits from subtasks can enhance fine-tuning in compositional tasks, providing new insights into the design of such tasks and deepening the understanding of circuit dynamics and fine-tuning mechanisms.
Mechanistic Interpretability (MI) has become a powerful approach for exploring the inner workings of machine learn- ing models, particularly Large Language Models (LLMM) (Rai et al., 2024). It provides valuable insights into how information flows and transforms across different layers (Ferrando et al.. 2024). One of the most critical aspects of deploying LLMs in real-world scenarios is fine-tuning (Chung et al., 2024). However, the interpretability of how pre-trained models improve during fine-tuning remains lim- ited, and the underlying mechanisms enabling their success across tasks require further investigation.
Many studies in MI regard models as computational graphs (Geiger et al., 2021), where circuits are specific subgraphs that perform identifiable functions (Wang et al., 2022). No- tably, this framework has been successfully applied to vari- ous LLMs, revealing emergent behaviors within attention heads and Multi-Layer Perceptrons (MLPs) (Heimersheim & Janiak, 2023; Burns et al., 2023; Hanna al., 2023; Gould et al., 2023). Moreover, circuits have recently been leveraged to investigate the fine-tuning process of language models, seeking to understand the mechanisms behind fine- tuning (Prakash et al., 2024; Chhabra et al., 2024; Jain et al., 2024). However, these studies often focus on tasks where pre-trained models already perform well (e.g.. GPT-2 (Rad- ford et al., 2019) achieves around 98% accuracy on the IOI task), they use general data for fine-tuning rather than domain -specff datasets (Prakash et al., 2024). Under such conditions, fine-tuning mainly enhances existing mecha- nisms (c.g., by adding some attention heads). Consequently, their arguments may not be applicable in more practical fine-tuning scenarios where models initially perform poorly and require fine-tuning on domain data.
To better understand fine-tuning mechanisms in practical settings, it is crucial to focus on tasks where fine-tuning leads to performance improvements. In this work, we de- sign a class of mathematical tasks on which pre-trained large language models initially perform poorly with low accuracy, yet demonstrates a performance boost after fine-tuned. We employ the Edge Attribution Patching with Integrated Gra- dients (EAP-IG) (Hanna et al., 2024) method to identify circuits within both pre-trained and fine-tuned models. Sur- prisingly, we observe that this approach consistently finds
'School of Computing and Data Science, The University of Hong Kong "School of Data Science. The Chinese University of Hong Kong. Shenzhen. This work is done when Xu Wang working at The Chinese University of Hong Kong, Shenzhen supervised by Dr. Yan Hu. Correspondence to: Difan Zou <dzou@cs.hku.he. hk..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
circuits with high faithfulness, even though the two models differ markedly in performance (see $3). To further vali- date the stability of the discovered circuits, we introduce another circuit metric, robustness, which measures the sta- bility of identified circuits by assessing their edge similarity under different perturbation ratios of the dataset. We show that when compared with a randomly initialized transformer model, the pre-trained model, despite exhibiting very low prediction accuracy, can still achieve substantially higher robustness. This finding further supports the validity of the circuits discovered during the fine-tuning process, irrespec- tive of their prediction performance.
circuits, enhancing the model's 's performance on the com- bination task during fine-tuning.
2. Related work
2.1. Mechanistic Interpretability
Mechanistic Interpretability investigates how components in large language models process and represent informa- tion (Wang al., 2024). At present, many MI studies have been applied in various fields of Al Safety. For in- stance, oversimplified probes risk (Friedman et al., 2024), unlearning fabricated knowledge (Sun et al., 2024), reducing toxicity via alignment (Lee et al., 2024), mitigating hallu- cinations by editing representations (Zhang et al., 2024), and generating truthful outputs through inference-time in- terventions (Li et al., 2023). Other studies explore how local model edits propagate across tasks (Cohen et al., 2024; Meng al., 2023), Multi-Head Attention in-context learn- ing (Chen et al., 2024; Chen & Zou, 2024) and enhance geluence-funcciinn sampling (Koh et al.). Specifically, our study examines how circuits evolve during fine-tuning for mathematical tasks, focusing on node and edge changes to reveal mechanisms behind performance improvements.
Our Main Findings. Based on the circuits analysis tech- niques and tasks introduced in $3, we provide a com- prehensive interpretation of the key factors in the fine- tuning process. Specifically, we focus on three central research questions and summarize our main observations as follows. The code and data are available at tttps: / //github. com/ homm/xu0615..0 wttp////iinuunnniroecc
1. (84) How do circuits evolve during the fine-tuning process? We use pythia-1.45 1ythia-1.4B-dedee 1.4B-deduped (Biderman et al., 2023), gpt-neo-2.7B (Black al., 2021), opt- 6.7B (Zhang et al., 2022) to fine-tune five math tasks. By extracting the circuits at each stage of the model during fine-tuning and analyzing these circuits, the cir- cuits identified by EAP-IG demonstrate high fidelity in both pre-trained and fine-tuned models, despite signif- icant performance differences. We observe that during fine-tuning. circuits gradually converge as modifications to nodes and edges decrease. Meanwhile, new circuits emerge after fine-tuning, with edge changes playing a more significant role in this process.
2.2. Circuit Analysis and Fine- Tine-Tuning
One direction of Circuit Analysis focuses on building com- plete circuits. Early work localizes factual associations in mid-layer modules (Meng al., 2022) and uses causal me- diation uncover biases (Vig et al., 2020; Hase al., 2023). Automated methods like Automated Circuit Discovery iden- tify significant units (Conmy et al., 2023), while techniques like attribution patching, and refine circuit extraction by handling near-zero gradients (Syed et al., 2023; Hamna et al., 2024). Edge pruning (Bhaskar et al., 2024) provide insights into building the edge of the eircuit. Another line of research investigates the functional roles of circuit components, such as Attention heads (Wu et al., 2024; McDougall et al., 2023; Olsson et al., 2022; Gould et al., 2023; Cabannes et al., 2024) and Feed Forward Networks (FFNs) MLPs (Geva et al., 2021; 2022; Bhattacharya & Bojar, 2024). Addition- ally, circuits have been used to analyze specific tasks, such as factual knowledge retrieval (Geva al., 2023), arithmetic computation (Stolfo et al., 2023), Greater Than task (Hanna et al., 2023), and circuit recognition in Indirect Object Iden- tification (Wang al., 2022). Unlike these analyses, which focus on smaller-ccale tasks and models, our work offers a new lens on how circuits evolve specifically during fine- tuning on mathematical tasks, revealing crucial roles of edge changes.
2. ($5) Can circuit insights enhance the fine-tuning pro- cess? We develop a circuit-aware Low-Rank Adaptation (LoRA) method, which assigns higher ranks to layers that have more edge changes in the circuits. We demon- strate across five different mathematical tasks that using circuit insights to optimize the fine-tuning algorithm is effective, significantly improving LoRA's accuracy and parameter efficiency. Our experiments highlight how Mechanistic Interpretubility enhances fine-tuning effi- ciency, improving performance fewer parameters using circuit change insights.
3. ($6) How capable is the Union Circuit in performing compositional tasks? To validate our hypothesis, we design a two-step compositional task, such as "(61 45) * 45 =". This compositional task was decomposed into an addittion/aubraactinn task and a muliipiicationn/dvviiion task and we use the union of the circuits from these sub- tasks to approximate the circuit for the compositional task. Our results indicate that the circuit for the combi- nation task can be approximated by the union of subtask
As pre-trained language models scale, fine-tuning methods have emerged, optimizing only small subset of param- eters (Ding et al., 2023), Parameter-efficient fine-tuning
2

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
(PEFT) methods, such as LoRA (Hu et al., 2021), reduce computational costs while preserving functionality (Ding et al., 2023). Advances in LoRA, including pruning (Zhou et al., 2024) and adaptive budget allocation (Zhang et al., 2023; Liu et al., 2022; Lialin et al., 2024), further improve efficiency. In our study, we introduce a circuit-aware LoRA approach that adaptively assigns higher ranks to layers with more edge changes, boosting efficiency and accuracy in mathematical tasks, and further illustrates how combining circuits from subtasks can enhance performance in compo- sitional tasks during fine-tuning.
where F(M) represents the performance of the full model M and F(C) represents the performance of the circuit C.
Robustness. To evaluate the stability of the identified cir- cuit, we propose robustness score based on its robustness under dataset perturbations. Taking addition and subtraction tasks as an example, perturbations include numeric noise (e.g., changing 7 + 12 to 7 15), and operator noise (e.g., replacing 12 + with 12 7). And we conduct robustness calculations on these perturbed datasets, applying noise at varying levels to create noisy datasets.
The robustness score is computed using the Jaccard Sim- ilarity (Jaccard, 1912) between the initial circuit Go and perturbed circuits Gp. The formula is:
3. Circuit Discovery and Task Design
3.1. Circuit Discovery: EAP-IG
Robustness(p) = J5 Gp);
Aurriuttoon patching is technique for identifying circuits using two forward passes and one backward pass (Syed et al., 2023). In our experiments, we use Edge Attribution Patching with Integrated Gradients (EAP-IG) (Hanna al., 2024), which addresses computational inefficiency in large models and resolves zero-gradient issues with KL diver- gence. EAP-IG computes importance scores by integrating gradients along the path between clean and corrupted acti- vations, making it our method of choice. The formula for scoring each edge is:
where Ja(Go, Gp) represents the Jaccard Similarity for edges between the initial circuit Go and the perturbed circuit Gp. and denotes the perturbation level.
This modification focuses on edge similarity, as it better re- flects structural integrity. A high robustness score indicates that the perturbed circuits maintain similar edge structure to the original, with score closer to I reflecting robust circuit structure.
3.3. Tasks Design
1
To examine the effect of fine-tuning on circuit dynamics, we construct a suite of challenging mathematical tasks in which pre-trained models initially perform poorly. As shown in Figure 1, these tasks help reveal the underlying fine-tuning mechanisms that drive significant performance gains during the process.
AL(EE R (Ceur
Celemn)
Ve, eN L(x)
where Gcleun and Geor denote the activations in the circuit under the clean and corrupted inputs, respectively. m is the total number of interpolation steps, and k represents the index of a specific step. (z) denotes the gradient of the loss function [xx with respect to the interpolated activations eg.
Addition and Subtraction (Add/Sub). This task evaluates the model's s ability to perform basic addition and subtraction operations. Corrupted data involves altering the arithmetic operation. The task includes five subtasks categorized by the range of numbers involved within 100, 200, 300, 400, and 500. Each subtask contains 5,000 instances.
In this study, we choose 5 based on Hanna et al.'s (2024) recommendations (Hanna et al., 2024).
3.2. Circuit Evalutaion: Faithfulness and Robustness
Multiplication and Division (Mul/Div). This task assesses the model's capability to handle multiplication and division accurately. Corrupted data involves changing the opera- tion between multiplication and division. A total of 2,000 instances are included in this task.
Faithfulness. Faithfulness serves key metric to evaluate the reliability of circuits discovered in MI and it quantifies how closely a circuit replicates the behavior of the original model (Wang et al., 2022; Chhabra et al., 2024; Prakash et al., 2024). We adopt Kullback-Leibler divergence (KL- divergence) as the metric, following Conmy al. (Conmy al., 2023). Let M denote the model and C' the discovered circuit. Faithfulness is defined as the percentage of the model's performance captured by the circuit. The formula for faithfulness is:
Arithmetic and Geometric Sequence (Sequence). This task measures the model's ability recognize and extend arithmetic or geometric sequences. Corrupted data involves altering one term in the sequence. The dataset for this task contains 5,000 instances.
Least Common Multiple (LCM). This task tests the model's ability to calculate the Least Common Multiple (I.CM) of two integers. Corrupted data involves changing
Faithfulness (1
[F(M) (T(M)-F(CC) F(())) 100%,
F(M)
3

---

Towards Understanding Fine- Tine-Tuning Mechanisms of LLMs via Circuit Analysis
Tasks
Add/Sutx 96- 60
Mul/Div: 26 15
Pre-trained Model
Sequence: 9, 18, 27, 36
Fine-tuned Mode
Low accuracy
Fine-turing
LCME LCM(3, 66)
Fundtion: y=5x+74 (0vm
Understand
Q
Find Circuit
Initial Circuit
Final Circuit
Find
Circuit
Differene
Figure The workflow of understanding fine-tuning mechanisms using circuits. The initial pre-trained model shows low accuracy on tasks. The corresponding circuits were found in both the pre-trained model and the fine-tuned model, and the fine-tuning mechanism was understood by comparing changes in the circuits before and after.
the input numbers the conditions of the LCM calculation. The task includes 2,500 instances.
of the model on five mathematical tasks during fine-tuning, We track the model's accuracy at various training stages across different tasks, revealing consistent improvements in performance throughout the fine-tuning process.
Function Evaluation (Function). This task focuses on model's ability to compute values for linear functions, typically of the form y mæ + Corrupted data involves altering the constant term in the function. The dataset tains 5,000 instances.
Next, explore the faithfulness of the circuits found at each stage of fine-tuning. Prior work (Hanna et al., 2024) achieved over 85% faithfulness by selecting 1-2% of edges. Given our more complex tasks and larger model, we se- lect 5% of edges to ensure reliable circuits (faithfulness > 70%).. As shown in the middle panel of Figure 2, circuit faithfulness consistently exceeds 80% across most tasks, both before fine-tuning (Checkpoint 0) and throughout fine- tuning (Checkpoints 1-10). The only exception is Add/Sub task, where faithfulness is 77.52% before fine-tuning. These results confirm high circuit faithfulness in both pre-trained and fine-tuned models across all tasks.
For each task, we ensure a strict separation between the dataset used for fine-tuning and the dataset used for circuit analysis. Specifically, 80% of the dataset allocated for fine-tuning, and the remaining 20% reserved for identify- ing circuits and evaluating the model's and circuit's accura- cies. This separation guarantees that performance evaluation is conducted on data unseen during fine-tuning.
4. How Do Circuits Evolve During the Fine-Tuning Process?
Finally, we conduct robustmess analysis on the circuits iden- tified by EAP-IG. We evaluate the robustness circuits in the pre-trained model, the fine-tuned model, and a randomly initialized model. this section, we present the robustness analysis for the Add/Sub (100), with analysis for other tasks provided in Appendix C. As discussed in Section 3.2, we perturb the original dataset by 10% to 90% and identify the circuit three models in perturbed datasets with varying noise levels. Then, we compute the robustness score of Fine- Fine-tuned, Pre-trained, and Random models under differ- ent perturbation levels. Results in the right part of Figure
4.1. Model Accuracy, Circuit Faithfulness, and Robustness Analysis
To analyze circuit evolution, we first evaluate model accu- racy across fine-tuning checkpoints. We use LoRA (Hu et al., 2021) to fine-tune the Pythia-1 1itiia-1.45. model (Biderman et al., 2023) on five different mathematical tasks. The exper- imental settings for fine-tuning are shown Appendix A. The left panel of Figure 2 depicts the accuracy dynamics

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Model Accuracy (%)
Robustness Score
0.9
80
Faithfulness (%)
0.8
60
0.7
Mul/Dvv
Sdd
Add/Sub MuliDiv
Fine-Eunae Modal
Pre-truined Model Model
Sequence
Sequence
LCM
LCM
20
Function
Function
2
3
5 Checkpoints
4
0
1
2
Checkpoints 10
10
20 30
60 70 80
90
Perturbation Level (%)
Figure 2. Circuit Accuracy, Faithfulness, and Robustness during Fine-tuning. Left: Training progress of the model accuracy across different mathematical tasks, showing continuous improvement over checkpoints. Middle: Evolution of faithfulness metrics during training, demonstrating consistently high faithfulness across five mathematical tasks. Right: Robustness analysis for the Add/Sub task. Robustness evaluation under different perturbation levels, comparing Fine-tuned, Pre-trained, and Random models.
reveal that circuits identified by EAP-IG demonstrate high fidelity in both pre-trained and fine-tuned models, despite significant performance differences.
change from checkpoint checkpoint 1+1,, and So denotes the total number of nodes or edges in the initial circuit.
As shown in Figure 3, fine-tuning induces structural changes, with As (Edge) consistently exceeding As (Node) by factor of 2-3 across three tasks. This underscores the pivotal role of edges as the primary drivers of structural adaptation during fine-tuning. For the other tasks, the change rates of nodes and edges in the circuit also shown in Appendix D,
Key Observation Circuits can be identified in both pre-trained and fine-tuned models with high faithfulness and robustness, regardless of their sig- nificant performance differences.
Key Observation 2: Fine-tuning performs more sig- nificant edge modifications than node modifications.
4.2. Circuit is Converging During Fine-Tuning
We conjecture that the model's accuracy on the task tinues to improve, the model's 's internal circuits should con- tinue to stabilize. To verify our hypothesis, we analyze the change of nodes and edges across consecutive checkpoints.
4.3. Reorganizing Circuit Edges to Form a New Circuit
As discussed Section 3.1, each edge's score is computed as the dot product of the averaged loss gradients and acti- vation difference, quantifying its influence on model pre- dictions. To examine structural changes in circuits during fine-tuning, we use the 95th percentile of edge scores dynamic threshold. Edges in the initial and final circuits ex- ceeding this threshold are retained, yielding sparser circuits that capture the model's 's core information flow. Experimen- tal results for all other tasks are provided Appendix E.
First, we analyze node and edge changes across checkpoints. The top right of Figure illustrates three mathematical tasks, corresponding the model's increasing accuracy during fine-tuning. By tracking the number of node and edge modifications between different checkpoints, we assess whether circuit changes diminish over time and tend toward convergence as the accuracy of the model improves. Details for the remaining tasks are provided in Appendix D. As shown in Figure 3, the number node/edge state changes decreases consistently over time, indicating stabilization and convergence of the circuit.
The distribution of added and deleted nodes and edges fol- lows a distinct pattern. As illustrated in the left part of Figure 3, added nodes predominantly located in the middle and later layers of the circuit, whereas added and deleted edges are concentrated in the middle layers. The shallow layers exhibit minimal changes, providing stable foundation for task specific adaptations.
Subsequently, we propose new metric to measure the de- gree of change of nodes and edges during fine-tuning. To quantify the changes in edges and nodes during fine-tuning across n checkpoints, we define a unified change rate:
In order to prove our conclusions, we conduct investiga- tions into how the circuit evolves under different fine-tuning regimes. Specifically, Appendix F examines the circuit mod- ifications resulting from various PEFT strategies, while Ap-
Astatt
As
100%.
n
So
where As denotes the number of nodes or edges that
5

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
70
3500
Differences in Add/Sub Circuit Checkpoint 0->10
Add/Sub Sequence
Add/Sub
60
3000
Sequence
Node Changes
Edge Changes
LCM
50
2500
40
2000
30
1500
20
1000
10
500
0
2
4
6 8 10
1 2 6
Checkpoint Transition
Checkpoint Transition
LCM
Evolution of Circults
Quantitative Change
Sequence
Added node
Removed node
& Change Rate
Unchanged node
Added edge Removed edge
Add/Sub
4, (Node)
(Edge)
10 15
Change Rate (%)
Figure 3. The structural differences of Add/Sub(100) circuit (24 layers) and evolution of circuits across checkpoints in terms node and edge changes. Left: Differences Add/Sub(100) circuit before and after fine-tuning. The layout organizes nodes hierarchically from input output (logits). Nodes and edges color-coded based their status: added (blue). removed (red), unchanged (grey/white). Darker grey nodes indicate higher degrees. Top Right: Node and edge changes between checkpoints during fine-tuning three tasks. The left chart depicts the number node changes per transition, while the right chart focuses on edge changes. Bottom Right: Change rate of Add/Sub, Sequence, LCM three tasks. Bars with diagonal lines represent the change rate of edges.
pendix G focuses on the changes induced by full-parann call-parameter fine-tuning and LoRA. Finally, Appendix provides com- parison of circuit changes observed under different LLMs.
analysis to enhance fine-tuning efficiency and performance. CircuitLoRA operates two phases: first, the edges with the largest score changes are analyzed identify Criti- cal Layers; second, higher-rank LoRA modules are assigned to layers with more edge changes, while standard-rank mod- ules are applied to other layers. The complete procedure is detailed in Algorithm 1.
5. Can Circuit Insights Enhance the Fine-tuning Process?
In the previous section. we observe that while the nodes in the model's circuit exhibit minimal changes during fine- tuning, the edges undergo significant modifications. This observation raises an intriguing question: Can LoRA be improved by fine-tuning the edges that change the most? We would like to improve the fine-tuning algorithm from the perspective of Mechanistic Interpretability.
Our hypothesis that this improved fine-tuning algorithm, which leverages cireuit-based analysis, can make better use of the fine-tuning mechanism. In the subsequent section, we investigate this hypothesis, designing experiments across different mathmatical tasks to compare our strategy against full parameter fine-tuning and LoRA baseline.
5.2. Improving Fine- Tuning Efficiency and Accuracy by Circuit Insights
5.1. Applying Circuit Edge Changes into LoRA Fine-Tuning
To verify our hypothesis, we perform experiments on range of arithmetic and mathematical reasoning tasks. The experimental results of Circuit are summarized in two tables. our experiments, 5 Criticul Layers are se- lected. We compare CircuitLoRe against control groups including LoRA and RandomLoRA (5 Critical Layers randomly selected). For each method in the experiment, we report the final accuracy as the mean of five runs with
Based on the score of edges and the result of section 3.1, we assume that the most "active" edges play key role in the fine-tuning process. Also, considering that LoRA is fine-tuned in layers of the model, we want to focus the layers where the most "active" edges are located.
We propose CircuitLoRa, a circuit-aware Low-Rank Adaptation (LoRA) method that incorporates circuit-level
6

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Algorithm 1 CircuitLoRs Improve LoRA Using
6.1. Compositional Tasks, Compositional Circuits and Union Circuits
Circuit- Based Critical Layers Identification
Input: Pre-trained model M, Pre-finetuning circuit Chefore: Post- fost-finetuning circuit Cafter, LoRA ranks Fox
In the beginning, we first introduce a series of definitions regarding the composition of tasks and circuits.
Te: Scaling factors a, Qeritical
Phase I: Critical Layers Identification
Compositional Tasks. A compositional task consists of a sequence or combination of two or more simpler subtasks, where the output of one subtask often serves as the input to the next. For example, computing (61 45) x 45 first requires solving the subtraction (61 45), then using its result in a multiplication. By breaking complex reasoning into these interrelated steps, can isolate and analyze each module's 's contribution to overall performance.
Compute edge differences
Ae between Cbefore and Aggregate De to layer scores A and select critical layers
Cafter
Ceritical
Phase 2: Module Replacement
for each layer 1 M do
if / E Laritical then
Replace I with EnhancedloRALinea using and Ceritical
Compositional Circuits. A Compositional Circuit is the end-to-end subnetwork of the model that directly imple- ments a compositional task. It captures both the intra- subtask pathways and the cross-step dependencies that arise when information must flow from one operation (e.g. sub- traction) into the next (e.g. multiplication). Extracting this circuit requires running the discovery pipeline on the full compositional task.
else
Replace I with LoRALi using To and a end if
end for
Return: Updated model M
Union Circuits. A Union Circuit formed by taking the edge-union of individual subtask circuits without re- extracting dedicated compositional cireuit. By merging the critical edges and nodes from each primitive operation's circuit- while preserving edge counts for fair compari- son-the the Union Circuit approximates the full Compositional Circuit at a fraction of the discovery cost.
different random seeds.
As shown in Table 1, CircuitLoRA consistently outper- forms baseline methods, including LoRA and Random- LoRA, across all five tasks. For instance, in the "within 300" task, CircuitLoRA (ro 8, 32) achieves an accuracy of 82.70% with fewer training parameters, surpassing RadomLoRA and LoRA. When configured as CircuitLoRA (r.- = 32, re = 64) reaches 83.10% 1,,0 out- performing RandomLoRA and LoRA. CircuitLoRA ex- periments on other tasks refer to the Appendix I.
To design the compositional tasks, we consider the two-step operation, which involves the calculation of two different types of mathematical problem, such as addittion/sbbraacionn and multiplicattion/diisiioo For instance, the compositional task "(61 - 45) x 45 =" involves two mathematical oper- ations: (1) (Additiion/ubrractionn): "61 - 45 ="; and (2) (Muttipiictioo//Dvvssoo))) "16 x 45 ="" More examples of compositional tasks can be found in Appendix J.
By focusing the edges with the highest score during fine-tuning, CircuitLoRA demonstrates significant im- provements in both accuracy and parameter efficiency across various mathematical tasks. The experimental results pre- sented in this study provide a compelling answer to the question posed in Section 5. This approach leverages in- sights from Mechanistic Interpretability, identifying and prioritizing Critical Layers where critical changes occur.
Our intuition is that if the circuits can represent the mini- mum calculation block for one tasks, then it is conjectured that the Union Circuits of the two subtasks can exhibit the power to represent the circuits for the compositional task. In the following, we will investigate the conjecture through two approaches: (I) we compare the similarities between the Union Circuits and the Compositional Circuits; (2) we use the Union Circuits to develop the CircuitLoRd algo- rithm and evaluate whether the performance of the composi- tional task can also be improved.
Key Observation 3: Circuits can in turn improve fine-tuning with higher accuracy and parameter effi- ciency across various mathematical tasks.
6. How Capable is the Union Circuit in Performing Compositional Tasks?
6.2. Efficient Single-Phase Fine- Fine-Tuning Compositional Task with Union Circuit
In this section, we further explore the behavior of circuits in compositional tasks, aiming to investigate whether these tasks can be interpreted through the combination of circuits.
We conduct overlap analysis and fine-tuning experiments on the two-step operation combination task. For circuit
7

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Table Performance metrics for Add/Sub (within 300) and four other math tasks: Mul/Div, Sequence, LCM, and Function across different configurations. The control groups of CircuitLoRA (r. = 8, re 32) are LoRA (r, 16) and RandomLoRA (r. 8, re 32), and the control groups of Circuit (r, 32, re 64) are LoRA (r. 32) and RandomLoRA (r. 32, Te 64). Here, ro and represent the ranks used in Ci CircuitLoRk where is the rank for critical layer modules, and ro is the rank for non-critical layer modules. Model Accuracy is expressed as percentages.
Method
Parameter Ratio
Add/Sub(300)
Mul/Div
Sequence
LCM
Function
Pre-trained
0%
18.30
39.75
15.70
18.80
32.00
Full FT
100%
79.20
95.75
91.50
91.40
100,00
LoRA (ro, 2)
0.1111%
72.60
90.00
67.10
86.40
84.10
LoRA (r, =8)
0.4428%
78.30
94.25
79.60
91.20
96.80
LoRA (ro = 16)
0.8816%
78.40
95.50
83.40
91.20
97.30
LoRA (ro = 32)
1.7479%
80.50
96.25
92.70
92.80
98.60
CircuitLORA (T' = 8, = 32)
0.7175%
82.70
96.00
92.20
92.60
99.40
RandomLoRA (r, = 8, T. = 32)
0.7175%
77.50
95.50
81.70
90.40
97.70
CircuitLoRA (To - 16, T. = 64)
1.4248%
83.10
97.00
94.60
93.00
99.50
RandomLoRA (r'- 16, re 64)
1.4248%
79.10
95.75
92.10
92.00
98.50
Table 2. Overlap for Different Values of l: in Circuit Compar- isons. The table presents Overlap, hetween Union Circuit and the Combination Circuit. Additionally, circuits from the Add/Sub task are compared with those from the Mul/Div and Sequence tasks control groups.
Table 3. Performance metries for Two-Step Operations Task. CircuitLRRA represents using Compositional Circuit for Crit- ical Layer Identification and Circui tLoRAa represents using Union Circuit. Model Accuracy all expressed as percentages.
Method
Parameter Ratio
M.Acc.
Circuit Comparison
Overlap, (C1,C2)
Pre-trained
0.90
A == 100
k = 500
= 1000
LoRA (r, = =2)
0.1111%
59.60
LoRA (ro = -8)
0,4428%
60.50
Union vs Compositional
69
259
470
LoRA 16)
0.8816%
61.10
Add/Sub vs Mul/Div
51
187
357
LoRA (r. =32)
1.7479%
64.70
Add/Sub vs Sequence
42
156
286
CircuitLoRAc (r., - -8, 32)
0.7175%
67.20
CircuitLoRA, (r., 8, = 32)
0.7175%
65.50
RandomL =8, 32)
0.7175%
62.30
C, we define Top, (C) metric quantify how many of the top-k edges, ranked by their scores, shared between two circuits. Then we define the Overlap metric as follows:
proximation ability"" of Union Circuit. Table 3 summa- rizes the performance of CircuitLoRA and LoRA on the two-step operations task. Specifically, CircuitLoRA with Compositional Circuit achieves the highest accuracy of 67.20%. Surprisingly, when using the Union Circuit for Critical Layer identification, CircuitLoRd achieves 65.50% % still exceeding the performance of LoRA except the Compositional Circuit configuration.
Overlapı CC,, = (Ci) n (C2)
First, we calculate the Union Circuit and Combination Cir- cuit under the two-step operation combination task.
Through overlap analysis, we prove the efficiency of Union Circuit to a certain extent. Table 2 analyzes the overlap for different values of k to evaluate the efficiency of the Union Circuit. The results show that, regardless of the value of k. the overlap between the Union Circuit and the Compositional Circuit is consistently the highest. Compar- isons are made between the addition/subrraction circuit and circuits from control tasks, such as mullipiicationddiissoon and arithmetic/geometric sequences. The overlaps in these cases are notably lower. These findings demonstrate that the Union Circuit provides an approximate representation of the Compositional Circuit.
This demonstrates we can use Union Circuit single-phase fine-tune. This means that for fine-tuning of the combination task, if we want to use CircuitLoRA we do not need to find its combinational circuit first, but can replace it with the union of the circuits of the subtasks that have been discovered to some extent.
Key Observation 4: The composition of the circuits can effectively represent the circuits of the composi- tional task.
Then, we use Union Circuit and Compositional Circuit to identify the Critical Layers to further explore the "ap-
8

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
7. Conclusion and Future Work
consequences of our work, none which we feel must be specifically highlighted here.
In this paper, we build on circuit analysis to deepen our understanding of fine-tuning and better leverage learned mechanisms. Our findings show that fine-tuning primarily modifying edges rather than merely introducing new com- ponents to form new circuits. Building on this insight, we develop circuit-aware LoRA method. Across multiple tasks, our results demonstrate that incorporating this MI perspective enhances fine-tuning efficiency. Additionally, we show that the composition of subtask circuits effectively represents the circuit of compositional task.
References
Bhaskar, A., Wettig, A., Friedman, D., and Chen, D. Finding transformer circuits with edge pruning. In Proceedings of the 38th Conference on Neural Information Process- ing Systems (NeurIPS), Spotlight, 2024. URL https: www.iv.orr/bbs/24000 Spotlight.
NeurlPS 2024
Bhattacharya, S. and Bojar, O. Understanding the role of ffns in driving multilingual behaviour in Ilms, 2024. URL https: htt//rrxvvvv/// www.awwwwi....aaaaaaaaaaaoaeeeee www.oorgggbb///eeeee hbtp///ss///4/..////// 240.2.444.........
Moving forward, will explore the following directions. Although our work focused on math tasks, applying circuit- based methods to more tasks would further validate the generality of our insights. Additionally, while our com- positional experiments only explore two-step arithmetic, extending this analysis to multi-step or more compositional tasks could provide deeper insights into eircuit interactions, enhancing interpretability and fine-tuning efficiency.
Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., BrBri,n, K., Hallahan, E.,, Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. Pythia: A suite for ana- lyzing large language models across training and scaling, International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Acknowledgements
Black, S., Gao, L., Wang, P., Leahy, C., and Biderman, Gpt-neo: Large scale autoregressive language modeling with mesh-tensorflow. If you use this software, please cite it using these metadata, 58(2), 2021.
This work was supported by the Shenzhen Doctoral Startup Funding 01BB520022100003330066 Shenzhen Science and Technology Program (JCYJ202208 8 1103000102)) Tianyuan Fund for Mathematics of National Natural Science Foundation of China (NSFC) (12326608), Shenzhen Key Laboratory of Cross-Modal Cognitive Computing (grant number ZDSYS20230626091 3320066), and Shenzhen Sta- bility Science Program 2023. Difan Zou acknowledges the support from NSFC 62306252, Hong Kong ECS award 27309624, Guangdong NSF 2024A1515012444 and the central fund from HKU IDS. Reynold Cheng and Wenyu Du are supported by the Hong Kong Jockey Club Charities Trust (Project 260920140) (0)), the University of Hong Kong (Project 2409 100399), the HKU Outstanding Research Stu- dent Supervisor Award 2022-23, and the HKU Faculty Ex- change Award 2024 (Faculty of Engineering).
Burns, C., Ye, H., Klein, D., and Steinhardt, J. Discov- ering latent knowledge in language models without su- pervision. In Proceedings of the International Confer- ence on Learning Representatiins (ICLR), 2023. URL https: pettp::/arxiv..rree wwwp:ii..rr//bb/////e12 03827 ICLR 2023.
3020000)
Cabannes, V., Arnal, C., Bouaziz, W., Yang, A., Charton, F., and Kempe, J. Iteration head: A mechanistic study of chain-of-thougeee 2024. URL wtttts:://tttiiiee abs / 02..6 02128.
Chen, X. and Zou, D. What can transformer learn with vary- ing depth? case studies on sequence learning tasks. In Forty-first International Conference on Machine Learn- ing. 2024.
Impact Statement
Our work provides concrete insights for advancing Mech- anistic Interpretability. This deeper understanding of the internal processes guiding model updates paves the way for more efficient, accurate, and trustworthy AI systems. We hope these findings inspire new methods and applica- tions that take advantage of circuit-based analysis unlock greater transparency, reliability, and performance in LLMs development, and to make better use of the learned mecha- nisms in these models.
Chen, X., Zhao, L., and Zou, D. How transformers utilize multi-heas attention in in-context learning? a case study sparse linear regression. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
Chhabra, V. K., Zhu, D., and Khalili, M. M. Neuroplasticity
and corruption in model mechanisms: A case study of indirect object identification. In Proceedings of the ICML 2024 Workshop on Mechanistic Interpretability. 2024. ICML, 2024 Workshop on Mechanistic Interpretability.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fe- dus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-fineturm language models. Journal of Machine Learning Research, 25(70):1 2024.
Language Processing (EMNLP), 2022. URL https: / //aaxiv.org//aaaaaaeee 2203. 14..003..48800 14680. EMNLP 2022.
Geva, M., Bastings, J., Filippova, K., and Globerson, A. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. URL https: http://xxvvvrrrr/ abs/ 2304.14782 EMNLP 2023.
Cohen, R., Biran, E., Yoran, O., Globerson, A., and Geva, M. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283-299 2024.
Conmy, A., Mavor-Parker, A., Lynch, A.. Heimersheim, S., and Garriga-Alonso, A. Towards automated circuit discovery for mechanistic interpretability. In Oh, A., Nau- mann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Pro- cessing Systems, volume 36, pp. 16318-16352. Curran Associates, Inc., 2023.
Gould, R., Ong, E., Ogden, G., and Conmy, A. Succes- sor heads: Recurring, interpretable attention heads in the wild, 2023. URL https: httpss//axxvvvvo.e /rb//bs///////aoooo//oao/aaaeaaa 2312 09230.
Hanna, M., Liu, O., and Variengien, A. How does gpt-2 compute greater-than?: Interpreting mathematical abili- ties in a ppe-trained language model. In Oh, A., Naumann, T.,, Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Sys- lems, volume 36, pp. 76033-76060. Curran Associates, Inc., 2023.
Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. Parameter- efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence. 5(3):220-235, 2023.
Hanna, M., Pezzelle, S., and Belinkov, Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. In Proceedings of the Conference on Learming Mechanisms (COLM), 2024. URL https: / // //aaxiiv.coo wewxivv.rg/aatee abs /203 403. 17806. COLM 2024.
Ferrando, J., Sarti, G., Bisazza, A., and Costa-jussà, M. R. A primer on the inner workings of transformer-based language models, 2024. htLhttp::// http://aavvvvo////ee 2o05/ 00208.
Friedman, D., Lampinen, A., Dixon, I.., Chen, D., and Ghandeharioun, A. Interpretability illusions in the gen- eralization simplified models. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. URL https: /wtxiv.vvvv//////////...////aav/ org/ abs/ 2312. 03656.
Hase, P., Bansal, M., Kim, B., and Ghandeharioun, A. Does localization inform editing? surprising differences in causality- localization vs. knowledge editing in lan- guage models. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 17643-17668. Curran Associates, Inc., 2023.
ICML 2024.
Geiger, A., Lu, H., Icard, T., and Potts, C. Causal abstrac- tions of neural networks. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp.
Heimersheim, S. and Janiak, J. A circuit for python docstrings in a 4-layer attention-only transformer. URL: https://www. aligumentforum.
Curran Associates, Inc., 2021. URL http::::s///oceeediiee meu.riiipssnnnnnnoo...omom.. cc cc/paper_files//peeeeeppeeeeeeee httpe//feee/eppprr2/////////////
orgpostsuoKKKMMMEFXXWWWWW. 2023.
Paper..
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang. L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprinl arXiv:2106.0960 2021.
pdf.
Geva, M., Schuster, R., Berant, J., and Levy, O. Trans- former feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021. URL. https: / arxia. org orggabs////22../// 14913. EMNLP 2021.
Jaccard, P. The distribution of the flora in the alpine zone. 1. New phytologist, 11(2):37-50, 1912.
Jain, S., Kirk, R., Lubana, E. S., Dick, R. P., Tanaka, H., Grefenstette, E., Rocktäschel, T., and Krueger, D. S. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks, 2024. URL https:// e arxiv. brxiv.org/al www.obsg.s....oooooooo abb////11111111/////// 22111 12786.
Geva, M., Caciularu, A., Wang. K. R., and Goldberg, Y. Transformer feed-forward layers build predictions by pro- moting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods Natural
10

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Koh, J., Lyu, H., Jang, J., and Yang, J. Faithful and fast influence function via advanced sampling. In ICMI. 2024 Workshop Mechanistic Interpretability.
Rai, D., Zhou, Y., Feng, S., Saparov, A., and Yao, Z. A practical review of mechanistic interpretability for transformer-based language models, 2024. URL https: http://///ab///..0222266666
/
Lee, A., Bai, X., Pres, I., Wattenberg. M., Kummerfeld, J. K., and Mihalcea, R. A mechanistic understanding of alignment algorithms: case study on dpo and toxic- ity, 2024. URL https:/ ptsttpa://axii...o/ //tpxxxxivcco/ www//og////a...oooo.o..........a 2401. 01967.
Stolfo, A., Belinkov, Y., and Sachan, M. mechanistic interpretation of arithmetic reasoning in language mod- els using causal mediation analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. URL https: / warxiv.orrglo.........oo...o..oe abs 2305. 15054. EMNLP 2023.
Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-tine intervention: Eliciting truthful answers from language model. In Oh, A., Naumann, T., Glober- son, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, vol- ume 36, pp. 41451-41530
Sun, C., Miller, N. A., Zhmoginov, A., Vladymyrov, M., and Sandler, M. Learning and unlearning of fabri- cated knowledge in language models. In Proceedings of the ICML, 2024 Workshop on Mechanistic Interpretabil- ity, 2024. URL https:/ /tas:///axivv org/ abs/ 2410. 21750. ICML 2024 Workshop on Mechanistic Inter- pretability.
Curran Associates, Inc., 2023.
Lialin, V., Deshpande, V., Yao, X., and Rumshisky, A. Scal- ing down to scale up: guide to parameter-efficien fine-tuning, 2024. BRLhttt:://axxt org. org/ / 2303. 2303.15647.
Syed, A., Rager, C., and Conmy, A. Attribution patching outperforms automated circuit discovery. In Proceedings of the NeurlPS 2023 ATTRIB Workshop, 2023. URL https: /arxivv.vvvv..////////a////aaav/ wtgg/bs///// /2100. 1 10348. NeurlPS 2023 ATTRIB Workshop.
Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang. T., Bansal, M., and Raffel, C. A. Few-shot parameter-efficient fine- tuning is better and cheaper than in-context learning. Ad- vances in Neural Information Processing Systems, 35: 1950-1965, 2022.
Vig. J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang. J., Singer, Y., and Shieber, S. Causal mediation analysis for interpreting neural nip: The case of gender bias, 2020. URL https: /ttss///axxivi //ariixvv.og////// abs /2//2200.122222
McDougall, C., Conmy, A., Rushing, C., McGrath, T., and Nanda, N. Copy suppression: Comprehensively understanding an attention head, 2023. URL https: //arxiv....////////// org/ abs/ 22100.30000000 04625.
Meng, K. Bau, D., Andonian, A., and Belinkov, Y. Locat- ing and editing factual associations in gpt. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Pro- cessing Systems, volume 35, pp. 17359-17372. Curran Associates, Inc., 2022.
Wang. K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, Interpretabilityy the wild: circuit for indirect object identification in gpt-2 small, 2022. URL https: arxiv. .rg. abs/ 221/2221....2 2111000099.
Wang. M., Yao, Y., Xu, Z., Qiao, S., Deng. S., Wang. P., Chen, X., Gu, J.-C., Jiang, Y., Xie, P., Huang, F., Chen, H., and Zhang, N. Knowledge mechanisms in large language models: A survey and perspective. In Proceedings of EMNLP 2024 Findings, pp- 1-39, 2024. URL https: //arxiv org/ abs 0.407.15017. EMNLP 2024 Findings; 39 pages (v4).
Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., and Bau, D. Mass-editing memory in transformer, 2023.
URL
org/ abs/ 2abs/2220 2210.221.... 07229.
Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T.. Mann, B., Askell, A., Bai, Y., Chen, A., et al. In-context learning and induction heads. arXiv preprint arXiv:2209. 1895, 2022.
Wu, W., Wang, Y., Xiao, G., Peng, H., and Fu, Y. Re- trieval head mechanistically explains long- long-context factual- ity, 2024. URL 15574.
Prakash, N., Shaham, T. R., Haklay, T., Belinkov, Y., and Bau, D. Fine-tuning enhances existing mechanisms: A study entity tracking. In Proceedings of the International Conference Learning Representations (ICLR), 2024. URL https: http://axivvv..////i//eee//..a./ org/ abs/ 2402. 4811. ICLR 2024.
org/ abs/2404.
Zhang, Q., Chen, M., Bukharin, A., Karampatziakis, N., He, P., Cheng, Y., Chen, W., and Zhao, T. Adalora: Adaptive budget allocation for parameter-efficient fine- tuning. In Proceedings of the 11th International Confer- ence on Learning Representations (ICLR), 2023. URL https: /alss//aazee /aaaivivvv///////////../////aaia wttp:////p//.........o 2023.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
ICLR

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., Mi- haylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P. S., Sridhar, A., Wang, T., and Zettlemoyer, I. Opt: Open pre-trained transformer language mod- cls, 2022. URL https: /ttpa//////vvvv////aa/aaaeaaraar org/ abzgg/abb22255. 01068.
Zhang, S., Yu, T., and Feng, Y. Truthx: Alleviating halluci- nations by editing large language models in truthful space. In Proceedings of the 62nd Annual Meeting of the Associ- ation for Computational Linguistics (ACL), 2024, URL ht https: /arivv.vvv.v..////vv//.a.//aavvv org/ abs//bs//// 2402 17811. ACL 2024 Main Conference.
Zhou, H., Lu, X., Xu, W., Zhu, C., Zhao, T., and Yang, M. Lora-drop: Efficient lora parameter pruning based on output evaluation, 2024. URL https: //arxi.//rxiv.. org/abs/2442..070 077211
12

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
A. Experimental Setup of Fine-Tuning
Fine-tuning experiments were conducted across various arithmetic tasks, with configurations tailored to each. All tasks were trained with batch size of 8, gradient accumulation steps and a warmup of 50 steps, using a weight decay of 0.01.
Addition and Subtraction (Add/Sub) task, which includes subtasks with ranges of 100, 200, 300, 400, and 500, each subtask consists of 5,000 samples. The 100-range subtask was trained for 2 epochs, while others were trained for 4 epochs, LoRA experiments were performed with ranks r 2, 8, 16, 32, using learning rate of 3e-4, except for the 400-range (r 32, 1r=2e-4). Full Parameter Fine- Fine-Tuning (FPFT) used learning rates of 8c-6 (100-range), 6e-6 (200-range), 5e-6 (400-range), and 4e-6 (500-range). CircuitLoRA applied higher learning rates (4e-4 or 5e-4) for Critical Layers and 3e-4 for non-Critical Layers.
Multiplication and Division (Mul/Div) task contains 2,000 samples and was trained for epochs. LoRA used learning rate of 3,- FPFT used 4e-6, and Circuit used 2e-4 for Critical Layers and 3e-4 for non-Critical Layers.
Arithmetic and Geometric Sequence (Sequence) task includes 5,000 samples, trained for 4 epochs. LoRA experiments used a learning rate of 3e-4, FPFT used 8e-6, and Circuit applied 6e-4 (r 32) and 5e-4 (r 64) for Critical Layers, with 3e-4 for non-Critical Layers.
Least Common Multiple (L.CM) task, which contains 2,500 samples and was trained for 2 epochs, LoRA used learning rates of 3e-4 (r 2), 4e-4 (r 16), and 2e-4 (r 32). FPFT used 4e-6, and CircuitLoRA used 4e-4 (r 32) and 6e-5 (r 64) for Critical Layers, with 3e-4 for non-Crinn Critical Layers.
Function Evaluation (Function) task, with 5,000 samples trained for 2 epochs, used consistent LoRA learning rates of 3e-4 (r 2, 8, 16, 32), FPFT with 8e-6, and Circuit LORA with 4e-4 for Critical Layers and 3e-4 for non Critical Layers.
B. Model Accuracy and Circuit Faithfulness on Other Tasks
Model Accuracy
Faithfulness
100
0.9
Model Accuracy (M. Acc.)
0.8
0.7
Faithfulness (%)
80
0.6
0.5
70
0.4
60
0.3
Add/Sub (200)
Add/Sub (200)
Add/Sub (300)
Add/Sub (300)
0.2
50
Add/Sub (400)
Add/Sub (400)
0.1
Add/Sub (500)
Add/Sub (500)
40
1
2
3
4
7
8
9
10
0
1
2
3
5
6
7
8
10
Checkpoints
Checkpoints
Figure Performance analysis Add/Sub tasks across different checkpoints. Left: Model Accuracy (M.Acc.) shows the performance trends of four Iadks-Add/Sub (200), Add/Sub (300), Add/Sub (400), and Add/Sub (500) across checkpoints. Right: Faithfulness scores measure the reliability of predictions for the same tasks across the same checkpoints.
The left part of Figure 4 presents the model's accuracy for four task and the results indicate consistent improvement in accuracy across all tasks.
The right part of Figure illustrates the faithfulness of circuits for the same tasks. Faithfulness scores remain above 70% for all tasks. These results highlight both the accuracy improvements and the reliability of circuits throughout the fine-tuning process.
13

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
C. Robustness Analysis Experiments on Other Tasks
Building on the results reported in the main text, this appendix details our additional robustness experiments conducted across multiple arithmetic tasks. Following the methodology presented in Section 3.2, we systematically apply imput perturbations Muliipiiationn/ivisioon, Arithmetic/Geometic Sequence, Least Common Multiple, and Function Evaluation tasks. Our findings further corroborate the consistency and fidelity of circuits identified by EAP-IG, demonstrating their ability adapt under varying perturbation conditions while preserving core computational relationships.
Multiplication and Division Tasks Data perturbation in multiplication and division tasks involves altering of the operands within specified range while maintaining the validity of the operation. This introduces variability without disrupting the fundamental arithmetic relationship.
Example:
Original: Calculate the result of the following arithmetic expression and provide only the final answer: 26 15
Perturbed: Calculate the result of the following arithmetic expression and provide only the final answer: 26 20
Arithmetic and Geometric Sequence Tasks For arithmetic sequences, perturbation is achieved by uniformly shifting each term by fixed integer. In geometric sequences, the first term is adjusted, and subsequent terms recalculated using the original common ratio to preserve the sequence's ' structure.
Example:
Original:
Derive the following sequence: 26, 66, 106, 146,
Perturbed:
Derive the following sequence: 21, 61, 101, 141,
Least Common Multiple (L.CM) Tasks Data perturbation for LCM tasks involves regenerating the last LCM expression using one of three strategies: generating multiples, coprimes, or pairs with common factors that multiples. This ensures diversity and prevents redundancy in the dataset.
Example:
Original: Calculate the least common multiple (ICM) of two numbers. LCM(189) 84) 756, LCM(200 400)
Perturbed: Caleulate the least common multiple (LCM) of two numbers. (CM9, 84) = 756, (75, 120)
Function Evaluation Tasks function evaluation tasks, perturbation involves modifying the constant term b in linear function y = az b by value within a specified range. The corresponding y-values are recalculated reflect the change, ensuring the functional relationship remains intact.
Example:
Original:
There is function y=5x+201. Given x=1,2,3, 3,4, y-206, 211, 216,
Perturbed:
There is function 5x5x11 Given 2, 3,4, 4, y-156, 161, 166,
In line with the observations for addition and subtraction, our experiments on LCM, Sequence, Multiplication/Diviiooo and Function Evaluation tasks demonstrate that circuits can be identified both pre-trained and fine-tuned models with high
faithfulness and robustness. This finding holds true despite the significant performance gap between the two model states, underscoring the reliability and stability of the discovered circuits across diverse arithmetic tasks.
14

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Robustness analysis (LCM)
Robustness analysis (Sequence)
0.90
0.90
Confidence Score
Confidence Score
0.85
0.80
0.80
0.75
0.75
0.70
Fine-tuned Model
0.70
Fine-tuned Model
0.65
Pre-trained Model
0.65
Pre-trained Model
Random Model
Random Model
0.60
10
10
20
Perturbation Level (%)
Perturbation Level (%)
Robustness analysis (Mul_Div)
Robustness analysis (Function)
0.90
0.95
Confidence Score
0.85
Confidence Score
0.80
0.85
0.75
0.80
0.70
0.75
0.65
Fine-tuned Model
0.70
Fine-tuned Model
Pre-trained Model
Pre-trained Model
0.60
Random Model
0.55
0.60
10
20
40
10
20
30
40
Perturbation Level (%)
Perturbation Level (%)
Figure 5. Robustness analysis across four additional tasks: LCM, Sequence, Mul/Div, and Function. Despite the varying perturbation levels, both the pre-trained and fine-tuned models exhibit consistently high confidence scores compared to the randomly initialized model.
D. Node, Edge, and Change Rate Analysis Other Tasks
This section combines the analysis of node and edge changes with the change rates for various tasks. By evaluating these metrics together, we provide comprehensive view of the structural and dynamic adjustments observed across different tasks. The node and edge changes reflect the structural variations in the underlying data models, while the change rate quantifies the intensity of these changes, offering deeper insights into task-specific behaviors.
The Figure 6 presents an analysis of node and edge dynamics during fine-tuning across six tasks: Within 200, Within 300, Within 400, Within 500, MultipiiatioonDDivsioon, and Function Evaluation. It highlights how the interplay between node, edge, and change rate metrics contributes to the overall task dynamics, ensuring a holistic understanding of the transformations involved each scenario.
E. Changes in Circuits Before and After Fine-Tuning in Other Tasks
In this section, we compare the circuits before and after fine-tuning for tasks involving addition and subtraction within the ranges of 200, 300, 400, and 500, as well as tasks on Multiplication and Division, Arithmetic and Geometric Sequence, Least Common Multiple, and Function Evaluation. Please refer to Figures 7, 8, 9, and 10 for the comparison results of all tasks before and after fine-tuning.
Common Observations: The distribution added and deleted nodes and edges follows a distinct pattern. Our analysis reveals similarities with the findings from addition and subtraction tasks within the range of 100. Same as Figure 3 in Section 4.3, added nodes predominantly appear in the middle and later layers of the circuit. Similarly, added and deleted edges are concentrated in the middle layers. In contrast, the shallow layers exhibit minimal changes, serving stable foundation for spec-fe adaptations.
Trends with Increasing Number Ranges in Addition and Subtraction: We observe a distinct trend in the circuits fine-tuned for addition and subtraction tasks as the number range increases. The edges in the fine-tuned circuits become more concentrated, reflecting a refined structure to handle the broader numeric range efficiently.
15

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Within 200
Within 200
Within 300
Within 300
Function
50
Within 400
2500
Within 400
Node Changes
Within 500
Edge Changes
Within 500
Mul/Div
Mul/Div Function
Mul/Div
Function
Within 500
30
1500
Within 400
20
1000
Within 300
10
500
(Node)
Within
(Edge)
2
10
2
4
6
8
10
5
10
15
Checkpoint Transition
Checkpoint Transition
Change Rate (%)
Figure 6. Analysis of Node and Edge Changes Across Add/Sub Tasks, including Within 200, Within 300, Within 400, Within 500, MultiplicationDiissiion, and Function Evaluation. Left: Node changes during fine-tuning. Middle: Edge changes during fine-tuning. Right: Change rates for nodes and edges across tasks.
Comparative Observations Across Tasks: Tasks such Multiplication and Division, Arithmetic and Geometric Sequence. Least Common Multiple, and Function Evaluation exhibit different circuit adaptations compared to addition and subtraction tasks. As illustrated in Figures 9 and 10, these tasks utilize circuits that focus more the later layers. This difference indicates a shift in computational emphasis, highlighting the tailored adaptations of the circuit for distinct task requirements.
F. Circuit Changes During Fine- Tinn-Tuning: A Comparison Across PEFT Methods
In this uppendix, we compare three different PEFT methods AdaLoRA, and IA3 across various mathematical tasks (e.g., addittion/subraaction with 200, multiplicatioon/diission Sequence, LCM, and Function). Figures 11 through 15 illustrate the model accuracy, faithfulness, and the evolution of nodes and edges at each checkpoint, well the corresponding change rates. By examining these metrics, we can observe how the internal circuits of each model evolve during fine-tuning until they converge.
The three PEFT methods each create new circuits after fine-tuning on different mathematical tasks. Additionally, we draw the following conclusions:
1. Across different PEFT methods (LoRA, AdaLoRA, IA3) and diverse math tasks, as model accuracy improves, the circuits converge while edges undergo more significant modifications than nodes, consistent with previous observations.
2. Moreover, IA3 has lower node and edge change rates compared to the other two PEFT methods, which can be attributed to its smaller number of trainable parameters.
Hence, the choice among these three PEFT methods does not alter our primary conclusions that edges exhibit greater changes than nodes, and that the circuits ultimately converge as accuracy increases.
G. Circuit Changes During Fine-Tuiie Tuning: Full Parameter Fine-Tuning vs. LoRA
This appendix compares the circuit changes during fine-tuning between Full Parameter Fine-Tuning (Full-FT) and LoRA. Figure 16 presents the model accuracy and faithfulness at different checkpoints, along with the node and edge modifications and their respective change rates. By examining these metrics, we can observe the similarities and differences in circuit evolution under these two fine-tuning approaches.
Full parameter fine-tuning and LoRA exhibit highly similar convergence trends in circuit evolution. Therefore, both full parameter fine-tume tuning and parameter-effiee efffccieeeefii finettuiin can create new circuit structures. In full parameter fine-tuning, the changes in edges are significantly greater than those nodes, which is consistent with the previous findings using LoRA.
16

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Graph Differences: Within 200
Graph Differences: Within 300
Added
Removed node
Unchanged node
Added edge
Removed edge
Figure 7. Differences of Add/Sub(200) and Add/Sub(300) circuit (24 layers). Darker gray indicates a higher node degree.
H. Circuit Changes During Fine- Tine-Tunnng: A Comparison Across Different Large Language Models
In this appendix, we compare circuit changes across different large language models (LLMs), including pythia-I.4B- deduped, gpt-neo-2.7B, and opt-6. 7B. Figure 17 illustrates their accuracy and faithfulness during fine-tuning, well the modifications of nodes and edges through the checkpoints and the corresponding change rates. By examining these metrics, we can gain insights into how the internal circuits evolve under different model sizes and architectures.
Circuits in the additionn/ubbracttion task converge across all models, with edge change rates consistently exceeding node change rates. Fine-tuning in each model creates new circuits through significant reconfiguration connections among components.
Larger models exhibit higher node and edge change rates. As shown Figure 17, the opt-6.7B model demonstrates the largest change rates, while the faithfulness of its discovered circuit remains stable throughout fine-tuning.
Therefore, using different model architectures and sizes, we consistently observe the conclusions presented in Section 4.2 and Section 4.3: Circuit Convergence During Fine-Tuning and Reorganization of Circuit Edges to Form a New Circuit.
I. CircuitLoRA Performance on Other Tasks
In this appendix, we extend our investigation of CircuitLoRA additional tasks beyond those discussed in the main text. These tasks include variety of numerical operations, such addition and subtraction with varying ranges, to further examine the performance and robustness of our circuit-aware fine-tuning approach. By testing Circuit on these additional benchmarks, we aim to provide more comprehensive evaluation, highlighting how incorporating circuit-based insights can yield consistent gains across broader set of mathematical tasks.
In summary, the results presented in Table demonstrate that CircuitLoRA maintains its advantage over both LoRA and CandomLoRR4 baselines across multiple configurations and numerical ranges. Even when the parameter ratio constrained.
17

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Graph Differences: Within 400
Graph
Added node
Removed node
Unchanged node
Added edge
edge
Figure 8. Differences of Add/Sub(400) and Add/Sub(500) circuit (24 layers). Darker gray indicates higher node degree.
Table 4. Performance metrics for addition and subtraction across various configurations in different numerical ranges. The control
groups CircuitIoRa (ro 8, re 32) are LoRA (r. 16) RandomLoRA (r. re 32), control groups
of CircuitLoRA (r. 32, 64) are LoRA (r.. = 32) and RandomLoRA (r,, = 32, 64). Here, and re represent the ranks used CircuitLoRA, where r. is rank critical layer modules, and r. the rank for non-critical layer modules. Model
Accuracy is expressed as percentages.
Method
Parameter Ratio
Add/Sub(100)
Add/Sub(200)
Add/Sub(400)
Add/Sub(500)
Pre-trained
0%
46.90
24.90
12.20
9,90
Full FT
100%
96.80
90.50
75.30
63.60
LoRA (ro 2)
0.1111%
94.40
82.90
68.90
55.60
LoRA (re =8)
0,4428%
95.40
86.40
73.10
64.30
LoRA (ro 16)
0.8816%
96.70
87.80
77.90
68.30
LoRA (ro 32)
1.7479%
97.40
90.30
78.20
69.70
CircuitLoRA (r, 8, T. = 32)
0.7175%
96.90
90.40
77.90
70.60
RandomLoRA (ro =8,r. 32)
0.7175%
95.70
87.30
73.30
63.70
CircuitLoRA (ro 32, r, 64)
1.4248%
97.90
91.00
78.20
73.00
RandomLoRA (ro 32, r. 64)
1.4248%
97.00
89.60
77.70
64.20
Circuitt LORA effectively identifies and prioritizes Critical Layers, ensuring superior accuracy compared to methods that allocate ranks uniformly randomly. These findings further validate the effectiveness of fircuit-based analysis in enhancing fine-tuning efficiency and performance, reinforcing Key Observation 3: Circuits can improve fine-tuning by achieving higher accuracy and parameter efficiency across various mathematical tasks. In the addition and subtraction task, we can see that after using CircuitLoRA (ro = 8, re = 32), we can achieve almost the same accuracy or even higher with half the training parameters of LoRA (r, 32).
18

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Graph Differences: Mul/Div
Graph Differences: Sequence
Added node
Removed node
Unchanged node
Added edge
Removed edge
Figure 9. Differences of Mul/Div and Sequence circuit (24 layers). Darker gray indicates a higher node degree.
J. Examples of Compositional Task
Our compositional task involve two-step arithmetic operations, requiring reasoning across different mathematical operations. This task requires the model to perform addition and subtraction operations first, and then multiplication and division operations. The following examples demonstrate a diverse set of arithmetic problems designed for this purpose.
Example:
Clean: Calculate the result of the following arithmetic expression and provide only the final answer: (43 7) 21
Corrupted:
Calculate the result of the following arithmetic expression and provide only the final answer: (43 7) 88
Example:
Clean: Calculate the result of the following arithmetic expression and provide only the final answer: (82 43) / 13
Corrupted: Calculate the result of the following arithmetic expression and provide only the final answer: (82 43) / 3
These tasks demonstrate each example in this task can be divided into two subtasks: addition and subtraction tasks and multiplication and division tasks. The purpose of this task is to see whether the circuit in the combination task can be approximately replaced by the union of the circuits of the two subtasks. This provides ideas and experimental basis for exploring more complex combination tasks.
19

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Differences: LCM
Differences: Function
Added node
Removed node
Unchanged node
Added edge
Removed edge
Figure 10. Differences of LCM and Function circuit (24 layers). Darker gray indicates a higher node degree.
K. Structural Dynamics of Nodes and Edges during Fine-Tuning
To achieve a fairer comparison, We quantify circuit evolution by tracking, for each task, the normalized change rates of nodes and edges over all fine-tuning checkpoints:
AS AS. x 100%,
AS. x 100%.
node
odge
So
So
Here, No, Eg are the initial counts of nodes and edges, and AN AE are their changes between consecutive checkpoints.
Table 5 reports these rates for five mathematical tasks.
Table 5. Normalized change rates of nodes and edges (As) during fine-tuning.
Task
A node
(%)
A edge
(%)
Add/Sub (100)
17.4
69.5
Mul/Div
23.2
76.9
Sequence
24.2
67.9
L.CM
23.9
80.4
Function
15.4
65.8
In every task, edge change rates exceed node change rates by approximately 2-4x, indicating that fine-tuning predominantly restructures edges rather than adding or removing nodes.
20

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Model Accuracy
Falthfulness
100
0.9
Add/Sub (200)
Model Accuracy (%
0.8
Falthfulness (%)
Comparison of Circuit
0.6
70
0.5
60
(LoRA, AdaLoRA, IA3)
0.4
0.3
LoRA (200)
AdeLoRA (200)
50
LoRA (200)
AdaLonn (200)
(200)
(A3 (200)
Checkpoints
Checkpoints
70
3500
LoRA(200)
LoRA(200)
As (Node)
60
AdaLoRA(200)
3000
AdaLoRA(200)
(A3(200)
As (Edge)
Node Changes
50
IA3(200)
Edge Changes
2500
(A3(200)
40
2000
AdaLoRA(200)
30
1500
20
1000
10
500
LoRA(200)
0 2 Checkpoint Transition
90
0 2 3 4 6 90 Checkpoint Transition
15 Change Rate (%)
10
20
Figure 11. Comparison of Add/Sub (200) circuits during fine-tuning with LoRA, AdaLoRA, and IA3. Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Node edge changes across checkpoint transitions. Bottom Right: Change rate nodes and edges during fine-tuning.
L. Comparison with Other PEFT Methods
To demonstrate the practical benefits of our circuit-aware adapter allocation, we compare Circui against the adaptive PEFT method A under similar parameter budgets. Table 6 shows final model accuracies on five tasks.
Table 6. Accuracy comparison between AdaLoRA and Circuit (r. 16, r. 64).
Method
Param Ratio
Add/Sub(300)
Mul/Div
Sequence
LCM
Function
AdaLoRA
1. 7.7481 %
76.70
92.75
90.10
89.80
98.20
CircuitLoRA (ro 16, 64)
1.4248%
83.10
97.00
94.60
93.00
99.50
Conclusion:
Despite using smaller parameter budget, CircuitLoRe significantly outperforms AdaLoRA on every task, highlighting the value of circuit-driven adapter rank allocation.
M. Effectiveness of Union Circuits in Compositional Tasks
We measure how well the Union Circuit-former by merging top-scoring edges from each subtask captuure- the structure of the true Compositional Circuit by evaluating faithfulness across varying edge thresholds. Table reports the percentage of model behavior recovered by the Union Circuit when using the top p% of edges.
Table 7. Faithfulness of the Union Circuit vs. percentage of top edges used.
Top Edges Used
1%
2%
3%
4%
5%
6%
8%
10%
Faithfulness (%)
67.4
79.4
83.7
87.2
89.2
89.4
89.6
89.7
Key Observation: Even with only 5% of edges, the Union Circuit recovers 89.2% of the model's behavior, demonstrating
21

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Model Accuracy
Falthfulness
100
Mul/Div
0.9
90
Faithfulness (%)
Model Accuracy
80
Comparison of Circuit Changes
0.7
(LoRA, AdaLoRA, IA3)
(MulDDv)
50
(Hul, Div)
(Mul Div)
(Mut piv)
0
1
3
4
8
9
10
40
1
2
5
6
9 10
Checkpoints
Checkpoints
LoRA(Mul/DIV)
3000
LoRA(Mul/Div)
4, (Node)
AdaLoRA(Mul(Div)
AdaLoRA(Mu//DN)
IA3 Div)
As (Edge)
Node Changes
80
(A3(Mul/Div)
Edge Changes
2500
(A3(Mul/Div)
60
2000
1500
AdaLoRA(Mul_Do Div)
1000
20
500
LoRA(Mul Div)
1 2 4 8 10 Checkpoint Transition
2 4 6 8 9 Checkpoint Transition
10 Change (%)
20
Figure 12. Comparison of Mul/Div circuits during fine-tuning with LoRA, AdaLoRA, and LA3. Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Node and edge changes across checkpoint transitions. Boftom Right: Change rate of nodes and edges during fine-tuning.
its ability to approximate the Compositional Circuit without additional line-tuning
N. Top-5 Critical Layers Across Tasks
Table 8 lists the five layers with the largest aggregate edge-score changes (A₂) for each task, identified by Circuit LoRA.
Table 8. Top-5 critical layers & per task (by descending At).
Task
f
f
l4
Add/Sub (100-500)
0
4
Mul/Div
0
3
11
13
Sequence
0
7
10
11
LCM
0
3
4
11
13
Function Evaluation
0
3
13
14
Insight: While layers 3, 4 recur across tasks, each task also has unique critical layers, suggesting both shared and task-specific adaptation locations.
22

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Model Accuracy
Faithfulness
100
0.9
Sequence
Model Accuracy -
2.8
Falthfulness (%)
0.7
Comparison of Circuit Changes
(LoRA, AdaLoRA, IA3)
0.3
LoRA (Sequence) (Sequence) (Sequence)
(Soquence)
0.2
(Sequence)
10
Checkpoints
Checkpoints
80
LoRA(Sequence)
LolA((Sequence)
(Node)
Node Changes
Adal (euuuueencc)) (43(Seqe)
Changes
2000
AdaLoRA(Sequence)
(3(Sequince)
IA3(Seqeence)
(Edge)
50
1500
40
AdaLoRA(Sequence)
30
1000
LoRA(Sequence)
%
3 Checkpoint Transition
2
20
Checkpoint
Figure Comparison of Sequence circuits during fine-tuning with LoRA, AdaLoRA, and IA3. Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Nodc and edge changes across checkpoint transitions. Bottom Right: Change rate of nodes and edges during fine-tuning.
Model Accuracy
Faithfulness
0.9
LCM
0.0
Model Accuracy c
0.7
80
Comparison of Circuit C Changes
0.6
70
(LoRA, AdaLoRA, IA3)
0.4
60
0.3
LORA (LEM)
AdaLoRA
50
LoRA (LCM)
AdaLoRA (LCM)
(LCM)
(LCM)
40
Checkpolnts
Checkpoints
80
LoRA[LCM]
AdaLoRA(LCM)
3000
LORAILCM) AdaLoRA(LCM)
(A3(LCM)
8552
b, (Edge)
A. (Node)
Node Changes
(A3(LCM)
Edge Change
(A3(LCM)
2500
2000
AdaLORA(LCM)
1500
1000
LORA(LCM)
Checkpoint Transition 10
4
Checkpoint Transition 10
10 15
20
Change Rate (%)
Figure 14. Comparison of L.C circuits during fine-tuning with LoRA, AdaLoRA, and IA3. Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Node and edge changes across checkpoint transitions. Bottom Right: Change rate of nodes and edges during fine-tuning.
23

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Model Accuracy
Faithfulness
100
Function
Falthfulness (%)
Modal Accuracy
Comparison of Circuit Changes
(LoRA, AdaLoRA, IA3)
(Function)
LoRk (function)
AdaLoRA (Function)
AdaLoRA (Function)
(Function)
(Punction)
3
Checkpoints
Checkpoints
LoRA|Function]
3000
LoRA(Function)
As (Node)
AdaLoRR/Function)
AdaloRAlFunction)
(Edge)
Node Changes
60 50
(A3(Function)
Edge Changes
(A3iFunction)
2000
40
1500
AdaLoRA(Function)
LoRA(Function)
10
500
Checkpoint Transition
2
10
Checkpoint Transition
Change Rate (%)
Figure 15. Comparison of Function circuits during fine-tuning with LoRA, AdaLoRA, and IA3. Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Node and edge changes across checkpoint transitions. Bottom Right: Change rate of nodes and edges during fine-tuning.
Accuracy
100
0.9
Sequence
g0.8
Comparison Circuit Changes
Model Accuracy
Palthfulness
80
0.6
70
0.5
0.4
Full fine-tuning -LoRA
0.3
50
PulL
0.2
LORA
LORA
1
2
4
6
9
10
40
9 10
Checkpoints
Checkpoints
70
Full
2500
Full
LoRA
LORA
(Edge)
Node Changes
Edge Changes
LoRA
1500
30
20
1000
Full FT
0
10 Change Rate (%)
Checkpoint Transition
Checkpoint Transition
Figure 16. Comparison of circuit changes during fine-tuning between Full Parameter Fine-Tunine (Full-FT) and LoRA. Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Node and edge changes across checkpoint transitions. Bottom Right: Change rate of nodes and edges during fine-tuning.
24

---

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Model Accuracy
Falthfulness
Add/Sub(100)
1.0
100
Model Accuracy 190
Comparison of Circuit Changes
Falthfulness (%)
0.6
70
pythia-1.44-den -eduedd gpt-neo-2.78
0.5
deduded
Lythi--L-a-dediuped
opt-6.7B
gpt-aep2.77
gpt-nee-2.77
0.2
opt-6.78
opt-6.78
2
5
7
10
40
0
5
9 10
Checkpoints
Checkpoints
200
pythie-1.48-deen deddduuee 2.78
pythia-1 1.hhl-1.440a deduped
gpt-neo-2.78
opt-6.78
Node Changes
Edge Changes
opt-6.78
150
4000
3000
gpt-neo-2.78
100
50
1000
Pythia-1.48-deduge
200 (Edge)
(Noda)
%
10
10
10
Checkpoint Transition
Checkpoint Transition
Change (%)
Figure 17. Comparison of circuit fine-tuning changes between different LLMs 1.ythia-14B--eduupp gpt-neo-2.7B, opt-6.7B).
Top: Model accuracy and faithfulness across checkpoints. Bottom Left: Node and edge changes across checkpoint transitions. Bottom Right: Change rate of nodes and edges during fine-tuning.
25


Reviews:
[{"review_id": "vxJrxQS4BS", "reviewer": "Reviewer_42ZJ", "summary": "The paper studies the dynamics of fine-tuning a LLM on mathematical tasks that the model initially can't perform. This is studied through the lens of circuits identified in an automated manner using edge attributions derived via integrated gradients (where the gradients are presumably derived from the ground-truth task labels). Specifically, a circuit for the task is computed in this way throughout the fine-tuning process, and these circuits are compared.\n\nThe main findings of interest are:\n- the fine-tuning mostly re-uses early nodes in the circuit and adds new nodes in later layers\n- LoRA can be improved by concentrating more parameters in layers that see more change with fine-tuning\n- using the union of circuits for 2 tasks can be helpful as an approximation for the circuit of a task that composes these tasks.\n\n## Update after rebuttal\nReading the rebuttal has mildly updated me upwards on the merits of the paper,", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and helpful suggestions!\n>Q1: However, this method based on ratios is not a priori an apples-to-apples comparison...the observed higher fraction of edges may be an artifact of these scaling dynamics ...\n\nTo further distinguish whether it is the natural expansion of the structure or the influence of the fine-tuning mechanism, we supplement experiments on our tasks: **the difference between the estimated edge changes caused by node changes and the actual observed edge changes.**\n\nIn all tasks, the actual edge changes substantially exceeded the upper bound implied by node changes — often by a factor of **2.6x to 3.1x.** This gap indicates that these additional edge changes are not just caused by nodes, but are actively adjusted during the fine-tuning process. This finding is consistent with our original conclusion.\n>Q2: It is shown that the circuit nodes in early layers largely do not change…simply because almost all nodes in early layers happen to already be in the circuit?\n\nCircuit node changes involve **both additions and removals**, not just additions. As shown in Figure 3 (left), a considerable number of nodes in the middle layers are present in the initial circuit but are removed after fine-tuning. This demonstrates that circuit evolution is bidirectional and the relative stability in early layers cannot be solely explained by initial saturation.\n>Q3: I don't understand what the robustness metric…the specifics of the corruption process..Some questions I had while reading...\n\n1. Motivation. We consider robustness experiments because the model performs poorly on the task before fine-tuning. It is important to ensure that the circuits identified in this low-performance are still robust. Earlier studies ignored this point. Robustness is defined as the Jaccard similarity between edge sets of circuits extracted on clean vs. perturbed data for the same task. \n\n2. Corruption strategy. We use this strategy follow the **Symmetric Token Replacement (STR)** principle—minimally altering input while preserving task type.\n- `+` to `–` does not mean testing addition vs. subtraction circuits. Instead, we assess whether specific edges are sensitive to small, task-relevant input changes.\n- Changing one term in a sequence may break the exact pattern intentionally. This tests whether circuits remain structurally stable under small semantic shifts, not whether the model still solves the task correctly.\n\n>Q4: the Δₛ metric is dependent on the checkpoint schedule… this is unnatural as far as the metric aims to quantify the overall change in nodes and edges.\n\nWe intentionally designed $Δ_S$ to capture the cumulative dynamic changes of nodes and edges throughout the fine-tuning process. Specifically, certain nodes/edges may be added and later removed (or vice versa) during fine-tuning. These transient changes are meaningful and important intermediate behaviors are possibly lost if we only consider the final vs. initial states. $Δ_S$ aims to provide an approximate measure of the dynamic changes in structure evolution, which we believe is important.\n\n>Q5: It should be noted that…Thus the value of the finding is conceptual rather than practical.\n\nOur primary motivation for designing CircuitLoRA was to validate insights gained from our analysis in Section 4. We answer your doubts from two aspects of the experiment:\n- **Memory efficiency**: In the first stage of identifying critical layers, we find that using LoRA with `rank=2` is sufficient, which uses significantly fewer parameters than the base LoRA setup `rank=32` . This highlights the lightweight nature of our approach.\n- **Time efficiency**: In a 4-epoch training setup, critical layers identified after just 1 epoch were already consistent with those from the final model, indicating that full fine-tuning is not required to extract critical layers.\nThese suggest that our method offering a degree of practical applicability alongside the conceptual value.\n\n>Q6: When studying the compositional task..as opposed to the harder-to-interpret metric of overlap?\n\nIn this paper, we chose to use Overlap as a **structural metric** . To explore the faithfulness of the union circuit, we conducted additional experiments on the compositional task. We evaluated the faithfulness of the Union Circuit and found it to be 89.18%, compared to 96.86% for the Compositional Circuit. This faithfulness result, together with the structural overlap, supports our claim in paper.\n>Q7: In Figure 3, the legend does not describe all the node colors - what do the various shades of grey represent?\n\nThe various shades of grey in the figure indicate the total degree of each node in the final circuit. Darker grey represents nodes with higher connectivity.\n>Q8: how is \"change rate\" defined in Figure 3 (bottom right)?...What is the \"compositional circuit\" in 6.2.?\n\nWe sincerely apologize for the confusion caused during a first read. Your understanding of these two concepts is completely correct.", "claims_and_evidence": "- A problematic main finding advertised by the paper is: \"Meanwhile, new circuits emerge after fine-tuning, with edge changes playing a more significant role in this process.\" (line 83), also see \"Key Observation 2\" (line 244, right column). \n\t- This is evaluated by measuring the ratio between (roughly speaking) new nodes/edges divided by initial node/edges, respectively. See line 267 and surrounding paragraphs for discussion.\n\t- However, this method based on ratios is **not a priori an apples-to-apples comparison** as it does not account for the different asymptotics of nodes vs edges in the graph of the model. Since edges increase quadratically in the number of nodes (because any two attention heads/MLP blocks can connect, not just ones in consecutive layers), increasing the number of nodes by e.g. a factor of $2$ will generally increase the number of edges by a factor of $4$ absent any special structure (e.g., if done randomly). As such, the observed higher fraction of edges may be an artifact of these scaling dynamics and not a fundamental property of the fine-tuning process as claimed.\n- it is shown that the circuit nodes in early layers largely do not change, while the fine-tuning introduces new nodes in later layers (figure 3). However, could this be simply because almost all nodes in early layers happen to already be in the circuit? This is particularly plausible seeing as the number of nodes in early layers of the circuit is roughly constant. This should be addressed.\n- the claims about Circuit LoRA are well-motivated, interesting, and well-supported by the experimental evidence.", "methods_and_evaluation_criteria": "- I don't understand what the robustness metric & associated experiments bring to the story of the paper. Also, the description of how the robustness metric is calculated was not sufficiently clear.\n- the $\\Delta_S$ metric (Line 267, left column) is dependent on the checkpoint schedule of the fine-tuning. In particular, a denser checkpoint schedule will lead to a value at least as high as a coarser schedule (unless the $\\Delta s_{t\\to t+1}$ quantity is not signed, but in this case the entire sum would telescope to final - initial). This is unnatural as far as the metric aims to quantify the overall change in nodes an edges in fine-tuning. \n- When studying the compositional task in section 6, it would be far more natural to report the performance obtained by the union circuit on the task (when e.g. the rest of the network is mean-ablated, or under other interventions; the original IOI paper provides many different ways to measure the faithfulness/completeness of a circuit), as opposed to the harder-to-interpret metric of overlap?", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "- In Figure 3, the legend does not describe all the node colors - what do the various shades of grey represent?", "relation_to_broader_scientific_literature": "The paper is motivated by an important and timely question in the mechanistic interpretability literature, and closes some small gaps in our mechanistic understanding of fine-tuning.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well-written and easy to follow and understand\n\nWeaknesses:\n- It should be noted that even though the paper shows that LoRA can be improved using mechanistic knowledge of the network, this knowledge itself is derived by actually performing full fine-tuning on the network. In this sense, the paper does not constitute a practical improvement upon LoRA, but rather a conceptual proof of concept that post-hoc knowledge of a fine-tuned model can be funneled through some mechanistic metrics into a succinct summary that can in turn improve LoRA fine-tuning. Thus the value of the finding is conceptual rather than practical.", "other_comments_or_suggestions": "- the specifics of the corruption process should be explained earlier in the text, as it is central to the meaning of the robustness metric. Some questions I had while reading:\n\t- does corrupting the sign from + to - mean that we find a circuit using only addition problems and then we check how similar this circuit is to a circuit derived from subtraction problems? \n\t- what does it mean to corrupt an arithmetic/geometric sequence problem by \"changing one term in the sequence\"? Wouldn't the sequence fail to be an arithmetic/geometric sequence after the change?\n- in general, when there is a reference to the appendix, it is good to describe what the results there demonstrate; as a reminder, reviewers are not required to read the supplementary material.", "questions_for_authors": "- how is \"change rate\" defined in Figure 3 (bottom right)? OK, I guess it is the thing denoted by $\\Delta_S$ in the main text. Could help to clarify, it was confusing on a first read.\n- What is the \"compositional circuit\" in 6.2.? I assume it is the circuit identified for the compositional task (e.g. $(a+b)*c$)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cW4FX5GTzU", "reviewer": "Reviewer_4WDA", "summary": "The paper studies how fine-tuning works in LLM using the circuit analysis method. It presents a set of mathematical tasks that show clear performance improvements during fine-tuning, unlike previous work that focused on already well-performing pre-trained models. The authors find that fine-tuning mainly changes the connections (edges) in the model while keeping most of the internal similarities (nodes), which goes against the idea that fine-tuning only adds new components. Based on this, they introduce a circuit-aware Low-Rank Adaptation (LoRA) algorithm that ranks circuit layers by how much their connections change, resulting in a improvement in performance compared to standard methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and helpful suggestions!\n>Q1: This indicates that circuit dynamics play a crucial role…Actaully, I cannot agree with it. The total num of edges is much greater then total number of nodes. The ratio of changement can deliever more information.\n\nTo account for this, we use a **normalized change metric $ \\Delta S$**, which measures the change rate of nodes and edges relative to their initial quantities. This metric is introduced in Section 4 and visualized in Figure 3 (bottom right). \n\nUsing this metric, we observe across all tasks that edge change rates consistently exceed node change rates, by a factor of **2–3x**. To further distinguish whether it is the natural expansion of the structure or the influence of the fine-tuning mechanism, we supplement experiments on our tasks: **the difference between the estimated edge changes caused by node changes and the actual observed edge changes.** In all tasks, the actual edge changes substantially exceeded the upper bound implied by node changes — often by a factor of **2.6x to 3.1x**. This further confirms our previous experimental conclusions.\n\n>Q2: However, in Section 4 the authors use LoRA with the Pythia-1.4B model for fine-tuning, which I believe is not good. … Consider using full fine-tuning (FT) for small models and PEFT for large models.\n\nWe have conducted both LoRA and full-parameter fine-tuning (FT) experiments with the Pythia-1.4B model for comparison. The results are provided in Appendix G. We mention this in the last paragraph of section 4.3.\n\n**The reason we chose to present LoRA-based results in the main text (Section 4) is due to the logical structure of the paper:** in Section 5, we introduce CircuitLoRA. Since our CircuitLoRA is a response to LoRA-based insights from Section 4, presenting LoRA results earlier improves narrative consistency — i.e., we derive insights under LoRA, then use them to improve LoRA.\n\n>Q3: The study focuses on a specific set of mathematical tasks,...Broaden the range of mathematical tasks to assess the generalizability of the findings.\n\n1, Motivation. We consider these tasks because many recent works in Mechanistic Interpretability are based on tasks like IOI or greater-than where pre-trained models already achieve high accuracy on these tasks (e.g., 98% for GPT-2 on IOI), which is not practical for understanding fine-tuning. So we choose to explore scenarios where models start with low performance and improve significantly after fine-tuning — allowing us to observe meaningful structural changes in circuits.\n\n2, To follow your suggestion, we extended our experiments to both new mathmatical tasks and two natural language tasks. Compared to their tasks, our designed tasks are more challenging, involving more complex reasoning patterns. \n- Comparison Task: Is 121 > 112? Answer:\n- Complex IOI Task: Robert needed brush, Kelly wanted pan, Daniel handed brush to\n- Complex Capital-Country Task: If Abkhazia corresponds to Sukhumi, then Moldova corresponds to...\n\nThe following are the specific experimental results, please see Figure1 at [this anonymous link](https://anonymous.4open.science/r/rebuttal-icml2025-r2) for additional figures.\n- Pre-trained model accuracies on these tasks were initially low: 46.74%, 27.60%, and 32.58% respectively. \n  - Comparison: $ \\Delta S_{edge} $ = 23.6%, $ \\Delta S_{node} $ = 9.6%\n  - Complex IOI: $ \\Delta S_{edge} $ = 17.3%, $ \\Delta S_{node} $ = 6.0%\n  - Capital-Country: $ \\Delta S_{edge} $ = 16.8%, $ \\Delta S_{node} $ = 7.3%\n\nThese results replicate the core conclusion of our main paper that edge dynamics dominate structural change during fine-tuning and confirm that our findings generalize beyond the original tasks.\n\n>Q4: If you only want to focus on mathematical problem solving for reasoning, I suggest extending the circuit analysis from LLM + SFT to LLM + RL to enhance the contribution.\n\n1, In this work, we focuse on SFT primarily due to SFT is more suitable for our task and model size. For models below 30B, the effect of reinforcement learning is very poor and it needs to be applied to more difficult tasks to be meaningful.\n\n2, In the previous question, we expanded our tasks beyond mathematical problem to include some natural language tasks. Futher, we also explore the circuits before and after reinforcement learning follow your suggestions. In the Add/Sub task, we used PPO for 10 epochs of training and compared the differences between the internal circuits of the model after reinforcement learning and before.\n\n3, The experimental results show $ \\Delta S_{edge} $  = 30.3% and $ \\Delta S_{node} $  = 14.7%. Besides, the added nodes are predominantly located in the middle and later layers of the circuit, and the nodes in the shallow layers rarely change. Please see Figure2 at the same anonymous link above for additional figure. **This is basically consistent with the conclusion in our original paper, and this result enhances the generalization of our research conclusions.**", "claims_and_evidence": "Claim1: Circuits can be identified in both pre-trained and fine-tuned models with high faithfulness and robustness, regardless of their significant performance differences.   -- Yes. The authors employ the pythia-1.4B model in section 4.1 to do the analysis.\n\nClaim2: Key Observation 1: Circuits can be identified in both pre-trained and fine-tuned models with high faithfulness and robustness, regardless of their significant performance differences. -- On different checkpoints the authors find that while node similarities remain high, there are significant edge changes that differentiate pre-trained and fine-tuned models. This indicates that circuit dynamics play a crucial role in the fine-tuning process. In Figure 3 (upper right), the authors show the plot. Actaully, I cannot agree with it. The total num of edges is much greater then total number of nodes. The ratio of changement can deliever more information.\n\nClaim3: The development of a circuit-aware LoRA method optimizes fine-tuning.\nEvidence: The paper describes a novel LoRA method that prioritizes circuit layers based on edge modifications. Experimental results validate this approach, indicating that circuit insights can lead to improvements in the fine-tuning effectivenes.", "methods_and_evaluation_criteria": "In general, the proposed methods and evaluations make sense for the problem.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Overall, the experimental design is sound. \n\nHowever, in Section 4 the authors use LoRA with the Pythia-1.4B model for fine-tuning, which I believe is not good. Typically, full-parameter fine-tuning is preferred. Especially, the authors use a small model 1.4B here. IIf a very large model were used, I could understand that, due to computational constraints, the authors might need to use PEFT directly. But for a 1.4B model, GPU memory constraints should not be a barrier to using full fine-tuning. \n\nThis choice raises concerns about whether the subsequent findings would hold true under normal full fine-tuning conditions.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Weaknesses:\n\n1. The study focuses on a specific set of mathematical tasks, and further research is needed to determine the generalizability of these findings.\n\n2. For the empirical observation, consider using full fine-tuning (FT) for small models and PEFT for large models.\n\n3. If you only want to focus on mathematical problem solving for reasoning, I suggest extending the circuit analysis from LLM + SFT to LLM + RL to enhance the contribution.", "other_comments_or_suggestions": "- Broaden the range of mathematical tasks to assess the generalizability of the findings.\n- For empirical observations, use full fine-tuning for small models and PEFT for large models.\n- If the focus is on mathematical problem solving and reasoning, consider extending the circuit analysis from LLM + SFT to LLM + RL to enhance the contribution.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L4ewBhSWmw", "reviewer": "Reviewer_9Uek", "summary": "* The paper investigates circuits in LLMs (subsets of the computational graph) that have been finetuned to complete various small mathematical tasks (e.g. add two numbers).\n* The paper computes circuits (using standard methods) at different stages in the finetuning process and on different data. After verifying that the circuits are faithful, the paper finds that there is less change in circuit structure as the finetuning process continues, that found circuits are generally robust to data perturbation, and that edges in a circuit change more than nodes in a circuit.\n* The paper then introduces a novel approach to performing parameter efficient finetuning distillation, given an already-finetuned model. The idea is to compute circuits for the task in the pre-finetuned and post-finetuned models, determine which layers contain the greatest differences in circuits (critical layers), and then perform parameter efficient finetuning with more parameters on the critical layers and fewer parmeters on other layers. The paper finds that this approach (called \"CircuitLoRA\") outperforms standard LoRA with the same parameter ratio, and even often outperforms LoRA with greater parameter count on many tasks.\n* Finally, the paper looks at a compositional task (which requires two subtasks to be solved) and finds more overlap between the union of the subtask circuits and the compositional task circuit than between circuits for unrelated tasks.\n\n## update after rebuttal\n\nI raised my recommendation to a weak-accept. This is largely because of new results provided by the authors that demonstrate that CircuitLoRA outperforms other layer-adaptive LoRA methods, and due to responses to other reviewers that explain that CircuitLoRA does not require the model to be fully finetuned before it is applicable (in fact, the authors stated that only one epoch out of five epochs is necessary). This suggests that CircuitLoRA may be valuable in allowing for greater parameter efficiency in finetuning. Because I now believe that CircuitLoRA is a worthwhile contribution, I raised my recommendation.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your helpful review! \n>Q1:The authors of this paper will provide a persuasive explanation…, then I would be happy to raise my score.\n\nWe apologize for the oversight and have added the missing reference. Both works use EAP-IG as a shared tool, not a core contribution. We clarify key differences from [1] in motivation and contributions.\n\nMotivational Distinction: \n- [1] focuses on analyzing how circuits and their components emerge and stabilize during pretraining.\n- We focus on why fine-tuning improves performance, including per-layer node/edge dynamics—a finer view not explored in [1]. Unlike prior MI work, we focus on low-performing tasks to better reflect practical fine-tuning scenarios.\n\nApplication Contributions Beyond [1]:\n- As noted in *Open Problems in Mechanistic Interpretability* (2024), MI research splits into understanding mechanisms and using MI for better predictions. Most prior work, including [1], contributes to the first category. Beyond understanding, we leverage MI insights to improve fine-tuning. CircuitLoRA shows the practical value of structural insights.\n\n- We also propose and empirically evaluate Compositional and Union Circuit. We show that merging two subtask circuits can approximate compositional circuit. These are hard to observe in [1], which focuses on pre-training.\n\n>Q2: But to me, the greater number of edge changes does not follow that these edge changes are more significant than node changes…because there are far more possible edges in a circuit than nodes…\n\nTo account for this, we use a **normalized change metric $ \\Delta S$**, measuring node/edge change rates relative to their initial values. The result shows edge change rates consistently exceed node change rates by **2–3x**.\n\nTo distinguish natural structural growth from fine-tuning effects, we added experiments: **the difference between the estimated edge changes caused by node changes and the actual observed edge changes.** In all tasks, the actual edge changes substantially exceeded the upper bound implied by node changes — often by a factor of **2.6x to 3.1x**. This supports our conclusion.\n>Q3: In Table 1, how do CircuitLoRA and RandomLoRA with $r_o$=32,$r_c$=64 have a lower parameter … Does the bolding represent \"best performance for a given parameter ratio\"?\n\nFirst, there is a labeling error in Table 1. CircuitLoRA ($r_o$=32, $r_c$=64) should be CircuitLoRA ($r_o$=16, $r_c$=64). \n\nSecond, Bold highlights the best method in similar parameter ranges. In the Table 1 caption, we have already specified the intended comparisons: CircuitLoRA ($r_o$=8, $r_c$=32) vs LoRA ($r_o$=16).\n\n>Q4: For Section 6, what is the faithfulness score of the union circuit on the compositional task? How does this change for different values of k?\n\n1, We evaluated the faithfulness of the Union Circuit and found it to be 89.18%, compared to 96.86% for the Compositional Circuit. \n\n2, We further report that the faithfulness of the union circuit across different percentage values of total edges. As Overlap is structural, we focus on the top 100–1000 scoring edges. Faithfulness evaluation requires more edges. See **table1** at [this anonymous link](https://anonymous.4open.science/r/rebuttal-icml2025-r3). The results show that a small subset of top-ranked edges is sufficient to achieve high faithfulness.\n>Q5: If a perturbed prompt already exists in the dataset, then is it resampled?...Would it not make more sense to simply partition the dataset into n disjoint splits...?\n\n1, We applied a duplicate avoidance mechanism during generation (up to 100 attempts per task).\n\n2, We also ran an experiment as suggested. The results show that circuits in **fine-tuned model score 0.73 vs. 0.84 (pre-trained model) and 0.55 (random model).** These results further confirm our original conclusion.\n>Q6: One possible \"nice-to-have\" would be to compare CircuitLoRA with another non-uniform-rank PEFT method.\n\nWe conducted experiments comparing CircuitLoRA with AdaLoRA. Below are results for the two methods with similar parameters.\n|Method|Param Ratio|Add/Sub(300)|Mul/Div|\n|-|-|-|-|\n|AdaLoRA|1.7481%|76.70|92.75|\n|CircuitLoRA ($r_o$=16, $r_c$=64)|1.4248%|83.10|97.00|\n\nPlease see **table2** at the above anonymous link for full results. This provides empirical support that MI insights can guide parameter-efficient fine-tuning effectively. \n>Q7: For CircuitLoRA, were the critical layers found for different tasks the same? How much overlap was there?\n\nIn our current experiments, we analyzed the top-5 critical layers identified for each task. Please see **table3** at the above anonymous link. Critical layers vary by task, with some overlap.\n>Q8: I would recommend that the paper explicitly include a graph that plots the number of edge/node modifications per layer.\n\nWe have added it. Please see Figure1 at the above anonymous link.\n>Q9: I assume that this this the same thing as a LoRALinear adapter but with a higher rank; is this true?\n\nCorrect—we've adjusted accordingly.", "claims_and_evidence": "* In Section 4.1, the paper states \"Key Observation 1: Circuits can be identified in both pre-trained and fine-tuned models with high faithfulness and robustness, regardless of their significant performance differences.\" This is supported by Figure 2, which shows faithfulness levels above 80% for obtained circuits (which increase with the number of finetuning checkpoints), and circuits with high robustness scores of over 0.9.\n* In Section 4.2, the paper claims that circuits stabilize over the course of finetuning; this is supported by Figure 3, which shows that the number of node changes and edge changes in circuits across tasks decreases as finetuning progresses.\n\n* At the end of Section 4.2, the paper mentions \"the pivotal role of edges as the primary drivers of structural adaptation during fine-tuning\". This is supported by the paper finding that when normalizing by the number of edges/nodes in a circuit before finetuning, a greater proportion of edges in a circuit change over the course of finetuning than nodes (Fig. 3). While this evidence itself may be true, it is unclear how this implies that edges **drive** structural adaptation during fine-tuning, or what it would mean for edges to do such a thing. Similarly, the paper states based on this evidence \"Key Observation 2: Fine-tuning performs more significant edge modifications than node modifications.\" But to me, the greater number of edge changes does not follow that these edge changes are **more significant** than node changes. In fact, because there are far more possible edges in a circuit than nodes, it seems reasonable to believe that a node change is more significant than an edge change. \n\n* In Section 4.3, the paper states that \"added nodes are predominantly located in the middle and later layers of the circuit, whereas added and deleted edges are concentrated in the middle layers\". The paper supports this with a diagram of edges and nodes that were added to/removed from the original addition-subtraction circuit over the course of finetuning (Fig. 3, left). Visually, looking at the figure, this seems true, but especially in the case of edge modifications (because there are so many edges), it feels difficult to be sure. I would recommend that the paper explicitly include a graph that plots the number of edge/node modifications per layer.\n\n* In Section 5, the paper states that \"Circuits can in turn improve fine-tuning with higher accuracy and parameter efficiency across various mathematical tasks.\" This is well-supported: according to Table 1, the paper's novel \"CircuitLoRA\" method, which makes use of circuit change information over finetuning, outperforms PEFT methods with even higher parameter ratios (although see Question 2 of mine regarding some confusion I have about these parameter ratios).\n\n* In Section 6.2, the paper states that \"the Union Circuit [the union of the circuits for the two subtasks in a compositional task] provides an approximate representation of the Compositional Circuit [the circuit for the compositional task]\". Similarly, it then states that \"The composition of the circuits can effectively represent the circuits of the compositional task\". However, the primary evidence for this comes from Table 2, which shows the overlap between the top $k$ edges in the union circuit and the top $k$ edges in the compositional circuit for different values of $k$, compared to the overlaps between different circuits (as baselines). While indeed the union and compositional circuits have greater overlaps (for $k=100$, their overlap is 69 edges versus 51 edges for the Add/Sub and Mul/Div circuits), no information is given about given about the performance/faithfulness of these circuits. It is thus hard for me to say that these claims are truly supported. In order for the claim to be supported, the paper should include this faithfulness information (see Question #3 later in this review).", "methods_and_evaluation_criteria": "* The paper uses EAP-IG for extracting faithful circuits, which is a standard, well-performing method. The faithfulness metric used in this paper is also standard and sensible.\n* I am a bit confused about the paper's newly-defined \"robustness\" metric. This metric is defined as the Jaccard similarity (intersection-over-union) of the edge set of the original circuit and the \"perturbed circuit\", where the \"perturbed circuit\" is computed by first adding noise to the dataset and then extracting the circuit for the same task on this \"perturbed dataset\". What doesn't make sense to me is why this notion is considered in terms of dataset noise, when it would be more principled to consider it in terms of extracting circuits from different disjoint *dataset splits*. All of the \"noising\" operations described in the paper actually seem to just be creating different dataset examples. Would it not make more sense to simply partition the dataset into $n$ disjoint splits, extract a circuit on each split, and then calculate the pairwise Jaccard similarities between these circuits?\n* The tasks that the paper investigates (addition/subtraction, multiplication/division, arithmetic/geometric sequences, least common multiples, linear function evaluation) all seem reasonable. The two-step compositional task introduced in Section 6.1 also makes sense.\n* The paper's newly-introduced CircuitLoRA algorithm is simple, sensible, and in keeping with the circuit-oriented focus of the paper. One possible \"nice-to-have\" would be to compare CircuitLoRA with another non-uniform-rank PEFT method (e.g. AdaLora, which the authors of this paper cited), to see if circuit-specific information outperforms more generalist algorithms. Even if not, it is still a \"sign of life\" for circuit analysis, suggesting that it is able to pick up on some real important properties of the model.", "theoretical_claims": "No theoretical claims were made.", "experimental_designs_or_analyses": "I already discussed the compositional overlap experiment from Section 6.2 in the \"Claims and Evidence\" section of this review.\n\nI also looked into whether learning rate was tuned separately for CircuitLoRA compared to vanilla LoRA, and happily found that it was not, thus putting the two methods on a more even playing field.\n\nBeyond this, I did not particularly investigate the validity of experiments in detail, although they all seem straightforward enough to me, and the authors mention that they split all datasets into a finetuning split and a separate circuit analysis split.", "relation_to_broader_scientific_literature": "This paper does a mostly good job in its related works section (Section 2.2 in particular) of contextualizing itself in terms of both the circuit analysis literature and also the literature on finetuning methods (such as PEFT methods). However, the related works section does not include [1], a paper from last year which addresses many of the same questions as this one on how circuits form throughout the training process of a large language model. I provide more specific discussion on [1] in the \"Essential References Not Discussed\" portion of this review; however, suffice it to say that I believe that in the context of [1], I fear that the paper under review lacks novelty in its approach and subject of analysis.\n\n[1] Tigges, C., Hanna, M., Yu, Q., and Biderman, S. LLM Circuit Analyses Are Consistent Across Training and Scale. arXiv preprint arXiv:2407.10827, 2024.", "essential_references_not_discussed": "Possibly the greatest lacuna from the references section of this paper is [1]. Just like this paper, [1] applies EAP-IG to find circuits in models at various stages of training; it then calculates the stability of these circuits throughout training using a Jaccard similarity-based score, and finds that circuits often stabilize throughout training. This anticipates many of the claims made in the paper under review. Furthemore, [1] goes beyond merely looking at graph-level properties of the circuits under consideration, and instead analyzes the specific functional roles of different components in specific well-studied circuits, along with how they evolve over time.\n\n[1] Tigges, C., Hanna, M., Yu, Q., and Biderman, S. LLM Circuit Analyses Are Consistent Across Training and Scale. arXiv preprint arXiv:2407.10827, 2024.", "other_strengths_and_weaknesses": "This paper is well-written, and most analyses are done in sensible way. The main reason why my recommendation for this paper is a weak-reject is that its analysis of circuit development and stability over finetuning largely is a retread of [1]'s more thorough analysis of circuit development and stability over the course of training. And while this current paper does introduce the CircuitLoRA PEFT method, I am somewhat skeptical of the utility of this method, given that it requires a model to be finetuned in the first place in order to compute circuits, and given the lack of comparison in this paper between this method and other adaptive PEFT methods.\n\nI hope that in their rebuttal, the authors of this paper will provide a persuasive explanation of how their paper differs from previous literature. If I am convinced by such an explanation, then I would be happy to raise my score.\n\n[1] Tigges, C., Hanna, M., Yu, Q., and Biderman, S. LLM Circuit Analyses Are Consistent Across Training and Scale. arXiv preprint arXiv:2407.10827, 2024.", "other_comments_or_suggestions": "**Minor questions**\n\n1. In Table 1, the entire line for each of the CircuitLoRA results is bolded, suggesting that each CircuitLoRA outperforms all other methods. However, this is not true across all tasks (e.g. $r_o = 32$ LoRA beats $r_o = 8, r_c = 32$ CircuitLoRA on the Sequence task). Does the bolding represent \"best performance for a given parameter ratio\"? Some clarification would be helpful, especially for readers simply skimming the tables and figures.\n2. For CircuitLoRA, were the critical layers found for different tasks the same? How much overlap was there? It might be the case that there are certain circuit-independent critical layers that benefit an outsized amount from LoRA finetuning. If this is true, then this would suggest that when performing LoRA finetuning in general, then those layers' adapters should have higher ranks.\n3. In Algorithm 1, what is EnhancedLoRALinear? I assume that this this the same thing as a LoRALinear adapter but with a higher rank; is this true? If so, then I would recommend replacing \"EnhancedLoRALinear\" with \"LoRALinear\".\n\n**Suggestions**\n\n* It would make the figures much easier to parse and cite if they were broken up into subfigures. \n\n**Typos**\n* In Table 2, \"Mul_Div\" is written instead of \"Mul/Div\".", "questions_for_authors": "1. When perturbing the dataset to calculate robustness, if a perturbed prompt already exists in the dataset, then is it resampled? If not, then this would suggest that the actual level of perturbation is less than stated.\n2. In Table 1, how do CircuitLoRA and RandomLoRA with $r_o = 32, r_c = 64$ have a lower parameter ratio (1.4248%) than the LoRA baseline with $r_o = 32$ (1.7479%)? Is this a typo, or am I not understanding something about how the CircuitLoRA algorithm works?\n3. For Section 6, what is the faithfulness score of the union circuit on the compositional task? How does this change for different values of $k$?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d80Qp2dZ8J", "reviewer": "Reviewer_Vy58", "summary": "This paper investigates the mechanisms of fine-tuning in LLMs through circuit analysis, focusing on mathematical tasks where pre-trained models perform poorly but improve significantly after fine-tuning. The authors identify that edge modifications in circuits (rather than node changes) drive performance gains and propose a circuit-aware LoRA method (CircuitLoRA) that dynamically allocates higher ranks to layers with greater edge changes. Experiments demonstrate improved accuracy and parameter efficiency over standard LoRA. Additionally, the paper explores compositional tasks, showing that union circuits of subtasks approximate compositional task circuits.\n\n## update after rebuttal", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and helpful suggestions! These are good points, which we answer below.\n>Q1: I'm curious if you combine the discovered circuit for the single task, whether the combined circuit can perform the compositional task like you measure the faithfulness.\n\nBased on your suggestion, we add the faithfulness obtained by the Union Circuit on the task. We observe that the **Union Circuit achieves a faithfulness of 89.18%**, which supports our claim that Union Circuit can perform compositional task like Compositional Circuit.\n\n>Q2: There are some other circuits that are related to the work: Some work about the reuse or combinations are not discussed...The work analyzes mainly based on the math tasks, but there are some other knowledge circuits proposed, and whether the compositional is also applicable should be mentioned.\n\nWe appreciate the opportunity to more clearly situate our contributions within these key works:\n\n[1] Circuit Component Reuse Across Tasks in Transformer Language Models (ICLR 2024)\nWhile this paper discusses the reuse of components, **we focus on the composability and reuse of circuits**. Our work complements this direction by studying not just reuse but also structural recomposition, showing how sub-circuits from simple arithmetic tasks can be merged to approximate more complex tasks.\n\n[2] Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization (NeurIPS 2024)\nThis paper mentions that this type of reasoning combined in implicit reasoning has something in common with our thinking. Our study builds on similar math reasoning tasks, but **focuses on how these circuits evolve during fine-tuning and how their compositionality can be leveraged to improve fine-tuning strategies like CircuitLoRA.** \n\n[3] Knowledge Circuits in Pretrained Transformers (NeurIPS 2024)\nWe only have some overlap in circuit discovery. Moreover, **Our research focuses more on the changes in the internal circuits during fine-tuning as the model accuracy increases, as well as the improvement of the fine-tuning mechanism from the perspective of Mechanistic Interpretability.**\n\nWe have explicitly cited these works in the Related Work and Discussion sections and expanded our discussion on how our findings support and extend these prior directions, especially regarding modular fine-tuning, compositionality, and general-purpose circuit reuse.\n\n>Q3: Experiments are limited to synthetic mathematical tasks. While these provide controlled settings, it is unclear if findings generalize to natural language tasks (e.g., text generation, reasoning).\n\n1, Motivation. We consider our five tasks because many recent works in Mechanistic Interpretability are based on tasks like IOI or greater-than where pre-trained models already achieve high accuracy on these tasks (e.g., 98% for GPT-2 on IOI), which is not practical for understanding fine-tuning. So we choose to explore scenarios where models start with low performance and improve significantly after fine-tuning — allowing us to observe meaningful structural changes in circuits.\n\n2, To follow your suggestion, we newly extended our experiments to both new mathmatical tasks and two natural language tasks. Compared to tasks of previous studies, our designed tasks are more challenging, involving more complex reasoning patterns. \n- Comparison Task: Is 121 > 112? Answer:\n- Complex IOI Task: Robert needed brush, Kelly wanted pan, Daniel handed brush to\n- Complex Capital-Country Task: If Abkhazia corresponds to Sukhumi, then Moldova corresponds to...\n\nThe following are the specific experimental results, please see Figure1 at [this anonymous link](https://anonymous.4open.science/r/rebuttal-icml2025-r4) for additional figures.\n- Pre-trained model accuracies on these tasks were initially low: 46.74%, 27.60%, and 32.58% respectively. \n  - Comparison: $ \\Delta S_{edge} $ = 23.6%, $ \\Delta S_{node} $ = 9.6%\n  - Complex IOI: $ \\Delta S_{edge} $ = 17.3%, $ \\Delta S_{node} $ = 6.0%\n  - Capital-Country: $ \\Delta S_{edge} $ = 16.8%, $ \\Delta S_{node} $ = 7.3%\n\nThese results replicate the core conclusion of our main paper that edge dynamics dominate structural change during fine-tuning and confirm that our findings generalize beyond the original tasks.\n\n>Q4: I have some questions about the CircuitLoRA. Since you need to substitute the critical layers for the CircutiLoRA, do you need to tune the model using the two ranks twice with different ranks?\n\nTo clarify, CircuitLoRA requires only **a single round** of fine-tuning. The training is performed once, with a unified LoRA configuration where Critical layers (identified via circuit analysis) are assigned a higher rank ($r_c$), Non-critical layers are assigned a lower rank ($r_o$).\n\nThis design is one of the key strengths of CircuitLoRA — it leverages mechanistic insights to redistribute parameter budget effectively, without introducing additional training complexity or computational overhead.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes\n- I'm curious if you combine the discovered circuit for the single task, whether the combined circuit can perform the compositional task like you measure the faithfulness.", "relation_to_broader_scientific_literature": "- I think the empirical support for modular fine-tuning strategies, relevant to efforts like task arithmetic and model merging.\n- Compositional reasoning work hypothesizes that models solve complex tasks by combining subtask circuits.", "essential_references_not_discussed": "There are some other circuits that are related to the work:\n\nSome work about the reuse or combinations are not discussed:\n- [1] Circuit Component Reuse Across Tasks in Transformer Language Models, ICLR 2024\n\nThe work analyzes mainly based on the math tasks, but there are some other knowledge circuits proposed, and whether the compositional is also applicable should be mentioned:\n\n- [2] Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization, Neurips 2024\n- [3] Knowledge Circuits in Pretrained Transformers, Neurips 2024", "other_strengths_and_weaknesses": "Strengths:\n- Insights into compositional tasks could inform strategies for complex task fine-tuning via subtask circuit unions.\n- Comprehensive experiments across multiple models (Pythia, GPT-Neo, OPT), tasks (arithmetic, sequences, LCM), and fine-tuning methods (LoRA variants, full fine-tuning).\n\nWeakness:\n- Experiments are limited to synthetic mathematical tasks. While these provide controlled settings, it is unclear if findings generalize to natural language tasks (e.g., text generation, reasoning).", "other_comments_or_suggestions": "NA", "questions_for_authors": "I have some questions about the CircuitLoRA.\n- Since you need to substitute the critical layers for the CircutiLoRA, do you need to tune the model using the two ranks twice with different ranks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: ## Summary

This paper investigates the mechanisms of fine-tuning in Large Language Models (LLMs) through circuit analysis, focusing on mathematical tasks where pre-trained models initially perform poorly but improve substantially after fine-tuning. The authors find that while circuit nodes remain largely stable during fine-tuning, the edges between nodes undergo significant modifications, contrasting with previous findings. Based on these observations, they develop a circuit-aware Low-Rank Adaptation (LoRA) method that assigns ranks to layers according to edge changes in the circuits, improving performance over standard LoRA with comparable parameter counts. Additionally, the paper explores how combining circuits from subtasks can enhance fine-tuning in compositional tasks, offering new insights into task design and circuit dynamics.

## Reasons to Accept

1. The paper addresses an important and timely question in mechanistic interpretability, closing gaps in our understanding of fine-tuning (42ZJ).
2. The proposed CircuitLoRA method is well-motivated, interesting, and delivers improved performance over standard methods, demonstrating practical value from mechanistic interpretability insights (42ZJ, 9Uek).
3. The work is well-written and provides comprehensive experiments across multiple models, tasks, and fine-tuning methods (42ZJ, Vy58).
4. The findings on compositional tasks could inform strategies for complex task fine-tuning via subtask circuit unions, offering valuable insights for modular fine-tuning (Vy58).
5. The paper provides novel observations on how circuit structure changes during fine-tuning, showing that edge modifications play a more significant role than node changes (4WDA).

## Weaknesses
The concerns mentioned by the reviewers are all addressed:  
1. The novelty of the approach and findings may be limited when compared to prior work, particularly Tigges et al. (2024), which also analyzed circuit stability throughout training (9Uek). The authors clarified in rebuttal that their work differs in motivation and contributions - focusing on why fine-tuning improves performance on low-performing tasks and leveraging MI insights for better fine-tuning, not just understanding mechanisms.
2. The comparison method using ratios to claim edge changes are more significant than node changes may not be an apples-to-apples comparison, as it doesn't account for different asymptotics of nodes vs edges in the graph model (42ZJ). The authors addressed this in rebuttal by normalizing change metrics and comparing actual observed edge changes with estimated changes from node modifications, showing edge changes exceeded expectations by factors of 2.6-3.1x.
3. The experiments were initially limited to synthetic mathematical tasks, raising questions about generalizability to natural language tasks (Vy58). In rebuttal, the authors extended experiments to both new mathematical tasks and natural language tasks, showing similar patterns of edge dominance in structural changes.
4. CircuitLoRA requires a model to be fine-tuned first to identify critical layers, potentially limiting its practical applicability (42ZJ, 9Uek). The authors clarified that identifying critical layers can be done with just one epoch using a low-rank LoRA of rank=2, making the approach more practical than initially suggested.
5. The robustness metric and associated experiments were not sufficiently clear, and it was unclear what value they added to the paper (42ZJ). The authors explained the motivation and details of the corruption process in their rebuttal.
Decision: Accept (poster)