Title: Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates
Venue: NeurIPS 2025
Accepted: yes
Keywords: mechanistic interpretability

language models

circuit discovery

logical gates
Abstract: Circuit discovery has gradually become one of the prominent methods for mechanistic interpretability, and research on circuit completeness has also garnered increasing attention. Methods of circuit discovery that do not guarantee completeness not only result in circuits that are not fixed across different runs but also cause key mechanisms to be omitted. The nature of incompleteness arises from the presence of OR gates within the circuit, which are often only partially detected in standard circuit discovery methods. To this end, we systematically introduce three types of logic gates: AND, OR, and ADDER gates, and decompose the circuit into combinations of these logical gates. Through the concept of these gates, we derive the minimum requirements necessary to achieve faithfulness and completeness. Furthermore, we propose a framework that combines noising-based and denoising-based interventions, which can be easily integrated into existing circuit discovery methods without significantly increasing computational complexity. This framework is capable of fully identifying the logic gates and distinguishing them within the circuit. In addition to the extensive experimental validation of the framework's ability to restore the faithfulness, completeness, and sparsity of circuits, using this framework, we uncover fundamental properties of the three logic gates, such as their proportions and contributions to the output, and explore how they behave among the functionalities of language models.

Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates

Abstract

Circuit discovery has gradually become one of the prominent methods for mechanistic interpretability, and research on circuit completeness has also garnered increasing attention. Methods of circuit discovery that do not guarantee completeness not only result in circuits that are not fixed across different runs but also cause key mechanisms to be omitted. The nature of incompleteness arises from the presence of OR gates within the circuit, which are often only partially detected in standard circuit discovery methods. To this end, we systematically introduce three types of logic gates: AND, OR, and ADDER gates, and decompose the circuit into combinations of these logical gates. Through the concept of these gates, we derive the minimum requirements necessary to achieve faithfulness and completeness. Furthermore, we propose a framework that combines noising-based and denoising-based interventions, which can be easily integrated into existing circuit discovery methods without significantly increasing computational complexity. This framework is capable of fully identifying the logic gates and distinguishing them within the circuit. In addition to the extensive experimental validation of the framework's ability to restore the faithfulness, completeness, and sparsity of circuits, using this framework, we uncover fundamental properties of the three logic gates, such as their proportions and contributions to the output, and explore how they behave among the functionalities of language models.

1 Introduction

As an intervention-based approach to mechanistic interpretability, circuit discovery allows for the extraction of subgraphs from the computational graph of a language model that play a significant role in task performance, referred to as circuits [1, 2, 3, 4]. Several key studies have supported its development [5, 6], such as those focusing on ensuring that circuits faithfully reflect the model's outputs [2, 7], enabling efficient circuit extraction [8], and addressing scalability challenges for models with extremely large parameters and corpora [9, 10, 11].

As the concept of circuits evolves, recent attention has increasingly focused on the completeness of circuits in addition to faithfulness . For example, completeness has been redefined such that

when a circuit is removed from the computational graph, the performance of the task should degrade significantly [12, 13]. Nevertheless, current circuit discovery methods have been found to lack completeness [9]. Moreover, theory analysis [14] indicates that incomplete circuits lead to two potential pitfalls: non-transitivity and preemption, which prevent the recovery of the key mechanisms underlying the circuit. Finally, incompleteness results in variability in circuit discovery outcomes, making the circuit appear more like an arithmetic solution to obtain the output rather than representing a closed-form solution with interpretability [15].

Incompleteness largely arises from the presence of OR gates [16, 2]. For instance, consider a model M that employs two identical and disjoint serial circuit paths, C 1 and C 2 , which operate in parallel and whose outputs are subsequently combined via an OR operation. In this case, identifying either path is sufficient to achieve faithfulness, and removing the other path is a preferable choice for promoting sparsity. However, restoring completeness by discovering OR gates remains a challenge. The simplest approach, which involves repeated interventions on combinations of components [14], could theoretically uncover the complete OR gate; however, this makes circuit discovery an NP problem (We explained it in Appendix E.1). Additionally, while denoising-based intervention methods can rapidly restore OR gates [17], they lead to a more severe loss of faithfulness. Furthermore, these methods fail to isolate the OR gates from the final circuit, resulting in a lack of logical interpretability.

To this end, we introduce the concept of logic gates, where any circuit can be decomposed into AND, OR, and ADDER gates, and propose a systematic framework to uncover and separate all the gates, and then to explain their correspondence to faithfulness or completeness with the sparsity constraint. Our specific contributions are as follows:

1. We systematically introduce three types of logic gates that compose a circuit: AND, OR, and ADDER gates. Through these gates, we are able to infer the minimum requirements for a circuit to achieve faithfulness and completeness, as well as assess the capability of noising-based and denoisingbased interventions in restoring these gates. Based on these corollaries, we analyze three types of prevailing circuit discovery methods, named greedy search [2, 18, 11], linear estimation [8, 19], and differentiable mask [9, 12, 10], by evaluating their ability to recover the three logic gates and their faithfulness and completeness. Moreover, we conduct experiments to provide empirical evidence supporting these theoretical conclusions.

2. We propose a framework capable of fully discovering the three logic gates , which can be easily extended to current circuit discovery methods with constant-time complexity. Our framework combines noising-based and denoising-based interventions, ensuring both the faithfulness and completeness of the circuit, and enabling the separation of AND, OR, and ADDER gates from the final circuit. Extensive experimental results demonstrate that our framework achieves promising faithfulness and completeness. Additionally, to ensure consistency in the granularity of noising-based and denoising-based interventions, we introduce a misalignment score for AND and OR gates to measure whether the scales of the two intervention strategies are aligned when combined.

3. We explore the characteristics of AND, OR, and ADDER gates in a circuit , including their proportions and contributions to the output, building upon our proposed logic gates and recovery framework. Furthermore, we examine the relationship between logic gates and the functionality of language models. Experimental results show that OR gates typically link multiple backup paths for the same function, while AND gates often connect paths for different necessary functions.

2 Preliminaries

2.1 Circuit Discovery

In Transformer decoder-based language models, the forward pass is typically conceptualized as a computational graph G , where the nodes represent components (such as attention heads, MLPs, or even more granular elements like the query, key, and value matrices) and an edge i → j denotes a connection where the output of component i serves as input to component j . Circuit discovery seeks to identify a subgraph (circuit) C ⊂ G that captures the task-relevant behavior of the model [3].

The process used to prune and obtain the circuit C is referred to as intervention (also known as knockout , ablation ) [17, 20, 21, 22]. For a given task T , each sample x is referred to as clean text , and the corresponding forward pass yields the clean activation x i at each component i . A perturbed version of the input, denoted ˜ x , is called corrupted text , producing a corresponding corrupted

activation [23, 17]. The corrupted activation ˜ x i depends on the specific ablation method used. For example, ZERO ABLATION sets ˜ x i = 0 , while NOISE ABLATION draws ˜ x i from a predefined noise distribution. A widely used method, INTERCHANGE ABLATION, defines ˜ x i as the activation resulting from an input text that has been minimally perturbed to produce a different task label [10].

The intervention is divided into two strategies: noising-based intervention (hereafter referred to as Ns ) and denoising-based intervention (hereafter referred to as Dn ) [24]. The Ns first runs the clean text in the computational graph. Then, corrupted activations replace each clean activation to observe the change in the final output y . If replaced (also known as removed or pruned) activations lead to a significant change in output, they are considered to make an important contribution to the task T and should be retained in the circuit C [17]. Let p G ( y | x ) denote the model's original output, p C ( y | x, ˜ x ) represent the circuit's output after intervention. Specifically, if an edge j → i is retained within C , the activation of component i keeps the clean one ( x i ). Conversely, it is replaced by the corrupted one ( ˜ x i ). Let s denote the requirement of sparsity, and D represent the distance used to quantify the difference between the two outputs. Ns has the following objective:

Equation 1 indicates that the circuit is a subgraph that most closely approximates the functionality of the computational graph, where the components and edges have the most significant effect on the output. Similarly, the Dn first performs the corrupted run in the computational graph, and then replaces the corrupted activations with the clean activations. Those activations that lead to significant changes in the output ( ˜ y ) consist of the circuits. Dn thus has the following objective:

Most of the related work on circuit discovery follows the Ns strategy. We categorize these works into three types: (1) Greedy search [2, 18, 11], which iteratively examines each edge (or node) through intervention to obtain a greedy solution for the circuit. (2) Linear estimation [8, 19], where the contribution of each edge is approximated by a gradient measure obtainable in a single backward pass. This approach ranks the importance of each edge to approximate the circuit. (3) Differentiable masks [9, 12, 10], where a learnable mask is assigned to each edge (or node), treating circuit discovery as an optimization problem to derive the optimal circuit.

2.2 Circuit Evaluation

Circuit evaluation is primarily defined by three aspects: faithfulness , completeness , and sparsity .

Faithfulness refers to the circuit's ability to perform task T in isolation, which is defined as the difference between the circuit's output and the model's original output [16, 9, 17]. This is represented in Equations 1 as E ( x, ˜ x ) ∈T [ D ( p G ( y | x ) || p C ( y | x, ˜ x ))] (simplified as D ( G||C ) ). Method ACDC [16] measures faithfulness by computing the average difference in the unnormalized output logits between the correct token and an incorrect option. Recently, work [2, 17, 25] proposes that KL divergence provides a better measure of the distribution over the vocabulary, while other work [9, 15] suggests that task accuracy can avoid the overemphasis on irrelevant vocabulary in the KL divergence. In this paper, we measure faithfulness using both KL divergence and task accuracy as metrics.

Completeness refers to whether the circuit includes all the important paths that have an effect on the output. The work [16] first introduces the concept of circuit completeness, stating that C and G should ensure similar outputs even under any knockout. Therefore, the incompleteness score is defined as the difference D ( C \ K||G \ K ) for any subcircuit K ⊂ C . Existing work [9, 15] proposes that insufficient sampling of K may lead to unreliable approximations (we show the practical results in Appendix A), and thus recommends evaluating completeness by assessing the performance after the circuit's removal from the computational graph on the task T , i.e., D ( G \ C||G ) [13, 12]. In this paper, we also adopt it to evaluate completeness.

Sparsity refers to that the circuit should be as small as possible. Currently, many works [10, 9, 15] recommend measuring sparsity using the ratio |C| / |G| , which represents the proportion of edges in the circuit relative to those in the computational graph. In fact, higher sparsity tends to result in lower faithfulness, meaning that the circuit always reflects some trade-off between sparsity and faithfulness.

3 Circuit Logic

3.1 Logical Gates

Recent studies [16, 2, 17] have increasingly observed that within a circuit, certain subcircuits influence the output according to logical relationships resembling AND or even OR operations. Building on these findings, we systematically introduce three fundamental circuit logic types: the AND gate, OR gate, and ADDER gate.

Definition 1. We assume a common paradigm in which a receiver node B , which is connected by more than 1 sender node A 1 , A 2 , ... . For any edge A i → B , we use binary values '0' and '1' to represent the activation state of a node. Specifically, A i = 0 indicates that node A i is removed, ablated, or deactivated, whereas A i = 1 indicates that node A i is retained and active. When the sender nodes are ablated, the effect of node B on the output exhibits three distinct patterns, which are as follows:

AND : All sender nodes satisfy an AND logical relationship with the receiver node, i.e., B = A 1 ∧ A 2 ∧ . . . . In this case, node B exerts a significant effect on the output only if all of its sender nodes are retained. If even a single sender node is ablated, the effect of B on the output is nearly eliminated.

OR gate: All sender nodes satisfy an OR logical relationship with the receiver node, i.e., B = A 1 ∨ A 2 ∨ . . . . In this case, node B always exerts a significant effect on the output if one or more of its sender nodes are retained. Only if all sender nodes are ablated, the effect of B on the output is nearly eliminated.

ADDER gate: all sender nodes satisfy an ADDER logical relationship with the receiver node, i.e., B = A 1 + A 2 + . . . . In this case, node B exhibits its maximal effect on the output only when all of its sender nodes are retained. If any single sender node is ablated, the effect of B on the output is substantially diminished; when all sender nodes are ablated, B 's effect on the output is reduced to zero. Accordingly, we define the state of B as taking values 0,1,2,. . . , where the total number of distinct states equals the number of sender nodes.

Theoretical analyses support the view that Ns is capable of recovering a complete AND gate but fails to recover a complete OR gate, whereas Dn demonstrates the opposite pattern [17]. This asymmetry is straightforward to interpret. The Ns procedure corresponds to the transition from a clean activation state (state = 1 ) to a corrupted activation state (state = 0 ). Since all gates can be regarded as being initialized with activation states equal to 1 , any transition to state = 0 induces a significant change in the effect of AND and ADDER gates on the output. Consequently, Ns can reliably identify AND and ADDER gates. Conversely, the Dn procedure corresponds to initialization with activation states equal to 0 . In this case, any transition to state = 1 produces a significant change in the effect of OR and ADDER gates on the output. Moreover, we design a toy model to explain ADD, OR, ADDER gates in Appendix D.

Therefore, we denote the circuit constructed under the Ns strategy as C Ns, and the one constructed under the Dn strategy as C Dn. Based on the above set-theoretic relationships between C Ns and C Dn, we extract subsets of edges corresponding to AND, OR, and ADDER gates as follows:

AND gate ( C AND): edges that are present in C Ns but absent from C Dn.

OR gate ( C OR): edges that are present in C Dn but absent from C Ns.

ADDER gate ( C ADDER): edges that are shared between C Ns and C Dn.

We conduct an ablation on these edges: for each gate, we randomly remove either one or two edges on the same receiver node and measure the resulting change in the KL divergence of the output. This procedure is repeated 30 times for each receiver node, and the distributions of ∆KL values are summarized via box plots, as shown in Figure 1 (Detailed results are shown in Appendix E.2). We selected the computational graph of GPT2-small as G , and Indirect Object Inference (IOI) [16] as the test task. For the baseline methods, we chose ACDC [2] to represent the greedy search method, EAP [8] to represent the linear estimation method, and EdgePruning [10] to represent the differentiable mask method. For details regarding the implementation of these strategies within each baseline, we refer the reader to Appendix E.

B1

B2

C

A1

A2

A3

A4

AND

OR

ADDER

(a) Complete Circuit

B1

B2

C

A1

A2

A3

A4

AND

OR

ADDER

(b) Max faithfulness and sparsity

B1

B2

C

A1

A2

A3

A4

AND

OR

ADDER

(c) Max completeness and sparsity

In Figure 1, for AND gates, the ∆KL values resulting from removing one versus two edges are similar, consistent with the conclusion that the disruption of any single edge in AND gates renders the gate ineffective. For OR gates, removing a single edge has little effect on ∆KL , supporting the idea that the OR gates remains functional as long as at least one edge in OR gates remains intact. In contrast, for ADDER gates, removing two edges leads to a significantly larger increase in ∆KL compared to removing one, indicating that the edges contribute independently to the gate's function.

ACDC

EAP

Edge Pruning

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

KL

ADDER-2edges

ADDER-1edge

AND-2edges

AND-1edge

OR-2edges

OR-1edge

3.2 How to Use Logical Gates to Interpret Faithfulness and Completeness

These logical gates reveal some interesting phenomena as shown in Figure 2. For optimal faithfulness and sparsity, the circuit only needs to include one edge from each OR gate. For optimal completeness and sparsity, the circuit only needs to include one edge from each AND gate. Based on the definitions in Section 2, we can draw the following corollary regarding these properties:

Corollary 1. The minimal edge subset that satisfies optimal faithfulness consists of all edges from the AND gates, all edges from the ADDER gates, and any one edge from each OR gate. The minimal edge subset that satisfies optimal completeness consists of all edges from the OR gates, all edges from the ADDER gates, and any one edge from each AND gate (The proofs are shown in Appendix C).

Corollary 1 provides an explanation for why existing circuit discovery algorithms predominantly adopt the Ns strategy. The reason is that Ns is able to recover complete AND and ADDER gates, while its recovery of OR gates remains incomplete. This tradeoff corresponds precisely to the optimal balance between faithfulness and sparsity. Conversely, the Dn strategy is not adopted because it fails to recover complete AND gates, thereby severely compromising the faithfulness of the resulting circuit.

Therefore, we further evaluate the performance of existing work in terms of faithfulness and completeness. Table 1 presents the specific results for the three types of circuit discovery methods mentioned in Section 2.

Among these, methods based on greedy search and differentiable masks can identify partial OR gates, whereas methods based on linear estimation are unable to detect any edges of OR gates. Similarly, methods from Dn exhibit a similar pattern. While they can completely identify OR and ADDER gates, methods based on greedy search and differentiable masks can detect partial AND gates, while methods based on linear estimation fail to identify any. In Appendix D, we explain why greedy search and differentiable mask methods are able to identify some edges, whereas linear estimation completely fails to do so. Moreover, inspired by [2], we design a simple one-layer transformer toy model to implement the basic AND, OR, and ADDER gates, and validate the performance of these circuit discovery methods corresponding to the conclusion from Table 1.

AND

OR

Noising

Denoising

Ground Truth

AND

OR

ADDER

Observed Results

ADDER

ADDER

(a) Aligned granularity

AND

OR

Noising

Denoising

Ground Truth

Observed Results

ADDER

ADDER

ADDER

OR

AND

(b) Misaligned granularity

3.3 Granularity Alignment between C Ns and C Dn

The intersection operation mentioned above raise concerns about the granularity alignment between C Ns and C Dn. As illustrated in Figure 3, if the number of edges in C Ns significantly exceeds that in C Dn, some edges identified as the AND gates could belong to the true type of the ADDER gates. This misalignment in granularity can also occur when C Dn is considerably larger. Therefore, we propose two metrics (refer to Appendix F) to assess the degree of misalignment between C Ns and C Dn when performing intersection.

We report the misalignment of C Ns and C Dn at different scales in Appendix F. The results indicate that when the number of edges in the Dn circuit is approximately equal to that in the Ns circuit, both the misalignment score and its standard deviation reach an acceptable level. Therefore, throughout this paper, we assume that the optimal alignment occurs when Ns and Dn contain an equal number of edges and conduct experiments based on this assumption by scaling the number of edges identified by Ns and Dn strategies in a similar range.

4 Discovering Logically Sound Circuit

4.1 Optimization for Logically Sound Circuit

Existing baseline methods are capable of recovering only complete AND and ADDER structures, as demonstrated in Table 1. Therefore, the recovery of complete OR gates remains a challenge. Several approaches can be considered to address this problem, such as introducing additional combinations of interventions or varying the order of the intervention to identify different surviving edges of the OR gate, or incorporating a completeness score, such as D ( G \ C ∥ G ) , into the circuit discovery process. However, these approaches come with significant drawbacks. Expanding the space of intervention combinations renders circuit discovery an NP problem. Meanwhile, the inclusion of completeness scores is incompatible with non-differentiable optimization strategies such as greedy search, and it also fails to effectively split the three logic gate types in the recovered circuits.

Therefore, we propose a combined Ns+Dn approach to recover logically complete gates. This method is compatible with a wide range of circuit discovery algorithms, introduces minimal additional computational overhead, and enables clear and effective separation of the three types of logic gates.

100

200

500

1000

2000

5000

#edges

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

KL

0

0.2

0.4

0.6

0.8

Ns of ACDC

Ns+Dn of ACDC

Ns of EAP

Ns+Dn of EAP

Ns of EdgePruning

Ns+Dn of EdgePruning

Ns of ACDC

Ns+Dn of ACDC

Ns of EAP

Ns+Dn of EAP

Ns of EdgePruning

Ns+Dn of EdgePruning

(a) IOI task

100

200

500

1000

2000

5000

#edges

0.5

1

1.5

2

KL

0

0.2

0.4

0.6

0.8

1

(b) GT task

100

200

500

1000

2000

5000

#edges

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

KL

0

0.2

0.4

0.6

0.8

1

(c) SA task

Ns+Dn has the following objective:

In brief, for each baseline, we modify its implementation to perform both the Ns and Dn strategies in parallel, whereas originally only the Ns strategy was applied.

4.2 Validation of Logically Sound Circuit

In this subsection, we focus on the faithfulness and completeness of the logically sound circuit (from our framework in Section 4.1, denoted by C Ns+Dn) and the circuit of existing work (since existing work generally adopts Ns as a basic intervention strategy, we denote it by C Ns). Similar to Section 3.1, we select GPT2-small as the computational graph, and ACDC, EAP, and EdgePruning as methods to represent greedy search, linear estimation, and differentiable mask, respectively. We examine the circuits obtained through Ns, Dn, and Ns+Dn. For instance, in ACDC, when intervening on each edge, we simultaneously compute the effect of substituting the clean activation with a corrupted one in the clean run, and the effect of substituting the corrupted activation with a clean one in the corrupted run. In EAP, we compute gradients under both clean and corrupted conditions. For EdgePruning, we replace Equation 1 with Equation 3 as the optimization objective. Detailed implementation can be found in Appendix E.3. These experiments are conducted on three mainstream tasks for circuit discovery, namely indirect object inference (IOI) [16], greater than (GT) [26], and syntactic agreement [9]. The details of these tasks are presented in Table 2.

4.2.1 Completeness

Following the definition of completeness in Section 2, we first compare the changes in KL divergence and accuracy for the corresponding tasks (IOI, GT, SA) when the circuit is removed from the computational graph. Specifically, we compare the differences between the original circuits (obtained through Ns) and the logically sound circuits (obtained through Ns+Dn) after removal, for three methods: ACDC, EAP, and EdgePruning. To account for the effects of sparsity, we constrain the number of edges in both circuits to remain consistent across six sparsity levels: 100, 200, 500, 1000, 2000, and 5000 edges. Figure 4 shows that, both in terms of KL divergence and accuracy, the performance of circuits removed through Ns+Dn is noticeably weaker compared to those removed through Ns. This corroborates Corollary 1, where we note that Ns, due to its inability to fully recover the OR gate, results in suboptimal completeness. Additionally, we observe that the gap between Ns and Ns+Dn is largest in both metrics in the EAP method (see the solid line in Figure 4), where Ns fails to recover any edges of the OR gate. Additionally, since ACDC and EdgePruning are generally able to identify one OR edge, the recovered OR edge exhibits some degree of randomness. In the case of ACDC, this randomness is influenced by the search order, while in the case of EdgePruning, it is influenced by the initial values of the mask. Moreover, we show the detailed results including Dn in Appendix G.

#edges

IOI

GT

SA

ACDC

EAP

EdgePruning

ACDC

EAP

EdgePruning

ACDC

EAP

EdgePruning

100

3.4±0.6

0.6±0.1

8.4±3.7

4.8±0.9

0.5±0.1

12.7±4.9

2.8±0.4

1.1±0.2

15.3±5.8

200

5.9±1.3

1.2±0.3

18.1±6.7

6.7±1.8

1.3±0.2

22.5±9.1

4.3±0.9

2.2±0.5

28.4±12.7

500

14.7±3.7

1.8±0.7

44.5±13.8

16.9±4.2

1.6±0.8

49.1±15.6

12.8±2.9

2.9±0.9

55.9±16.7

1000

21.8±5.3

4.7±1.8

89.6±27.9

23.6±6.4

4.4±1.6

97.5±29.4

19.7±4.3

5.7±2.8

108.2±31.4

2000

49.5±12.9

7.9±2.9

195.3±57.8

55.7±14.9

8.6±3.5

211.7±66.2

44.8±15.2

8.8±3.1

237.4±64.8

5000

127±28.5

14.5±6.9

509.5±164.7

136.5±33.4

15.9±6.1

564.8±181.1

113.7±5.8

15.4±5.8

688.9±144.5

(a) AND gate

(b) OR gate

(c) ADDER gate

<a0.9>

<a0.1>

<a5.9>

<a5.8>

<W>

<a0.5>

<a2.2>

OW>

<m6>

<m5>

<a7.3>

<a9.9>

<a5.5>

<m8>

<resid_post>

To further validate completeness, we test the overlap of randomly generated circuits by extracting 30 distinct circuits under different random seeds and calculating the Hamming distance between pairs of these circuits to assess the randomness of the discovered circuits. Table 3 shows that the randomness of ACDC and EdgePruning is significantly higher than that of EAP, and it increases with the sparsity scales (as more OR gates are discovered). Additionally, the randomness of the circuits obtained through Ns+Dn is consistently lower than that of the circuits obtained through Ns, further supporting the claim that the inclusion of all three logical gates ensures optimal completeness.

4.2.2 Faithfulness

In Appendix H, we compare the circuits obtained using three strategies-Ns, Dn, and Ns+Dn-under the same sparsity constraints (specifically, we select edge counts of 100, 200, 500, 1000, 2000, and 5000) in terms of KL divergence and accuracy. The results show that, in terms of faithfulness, we have the relationship: C Ns+Dn ≈ C Ns > C Dn. Figure 5 illustrates the average of the three methods on the IOI task to corroborate this conclusion. More results can be found in Figure 9 (a)-(c), which further supports the faithfulness requirements asserted in Corollary 1.

5 Exploration on Logical Gates

5.1 Graph Study

In Appendix I, we present the circuits of the AND, OR, and ADDER gates recovered by ACDC on the IOI task and map the functions of the components to those in the previous IOI circuit [16]. We show parts of these circuits as demonstrations in Figure 6; each color represents one function in IOI circuit and blocks represent components at different locations, such as 'a5.9' indicating the 9-th attention head in the 5-th layer, and 'm8' referring to the MLP in the 8-th layer. The complete gate circuit can be found in Figure 10 of Appendix I. The results are interesting, revealing that the AND gates typically receive edges from different functions , suggesting that these functions must work together to support the receiver's activation. In contrast, the OR gates almost exclusively receive edges from the same function , indicating that these edges are likely interchangeable due to their

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0

0.2

0.4

0.6

0.8

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

execution of the same function. The ADDER gates, on the other hand, tend to focus on combining two functions from different layers , with the activation generally considering the outputs of both shallow-layer and deep-layer functions.

5.2 Output Contribution

In Appendix J, we investigate the contribution of three types of logic gates to the output. The gate effect represents the contribution of the entire gate to the output and the edge effect represents the average contribution of each edge to the output. The results show that the contribution of the ADDER gates is significantly higher than that of the AND and OR gates. Furthermore, methods that focus on the edge effect, such as differentiable masks and linear estimation, lead to a higher average effect in the recovered circuit.

5.3 Proportion

In Appendix K, we present the number of AND, OR, and ADDER edges recovered by different methods. The results indicate that the proportion is closely related to the type of circuit discovery method used. For instance, greedy search selects all edges beyond the threshold, resulting in nearly equal numbers of AND, OR, and ADDER edges. In contrast, differentiable mask methods calculate the effect of each edge, which is disadvantageous for gates like AND and OR that contain multiple edges. As a result, the number of ADDER edges is significantly higher.

6 Conclusions

This paper systematically introduces three logic gates-AND, OR, and ADDER-to explain the essential requirements of circuit faithfulness and completeness. Furthermore, it provides an analysis of how existing circuit discovery methods perform with respect to these logic gates. Additionally, we propose an Ns&Dn-based method for separating the three logic gates, and for restoring a logically sound circuit. We empirically validate the differences in faithfulness and completeness between the logically sound circuit and existing circuits. Finally, we explore the relationships between the logic gates in terms of distribution, contribution, and functionality.

6.1 Limitations and Future Research

The three logical gates under discussion in this paper are, in principle, derived from the intersection of C Ns and C Dn . Nevertheless, it must be acknowledged that other types of logical gates may also exist-for instance, the XOR gate-whose underlying logic cannot be captured solely through the simple intersection of C Ns and C Dn . That said, we contend that the three gates obtained from this intersection-AND, OR, and ADDER-are already sufficient to cover all edges of the circuits constructed from C Ns and C Dn . For the purposes of the present scope of research, this coverage is adequate.

Moreover, with a complete understanding of the logical relationships between edges, the circuit becomes more useful for offering insights into model control. For instance, a logically sound circuit offers a novel approach for verifying the potential combination of tasks through boolean satisfiability, which is treated as our future study. We have demonstrated the potential contributions of completeness research with a toy task of model unlearning, as shown in Appendix L.

6.2 Societal and Ethical Impact

Our work aims to facilitate the process of understanding and explaining the logical connections in language models, which is crucial for their continued safe development and deployment. We do not foresee logically sound circuit and logical gates being used towards adverse societal or ethical ends.

Acknowledgments and Disclosure of Funding

This research/project is supported by the National Research Foundation, Singapore under its National Large Language Models Funding Initiative, (AISG Award No: AISG-NMLP-2024-005).

References

Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread , 2021. https://transformer-circuits.pub/2021/framework/index.html.

Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems , 36:16318-16352, 2023.

Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646 , 2024.

Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill , 5(3):e00024-001, 2020.

Aliyah R Hsu, Georgia Zhou, Yeshwanth Cherapanamjeri, Yaxuan Huang, Anobel Y Odisho, Peter R Carroll, and Bin Yu. Efficient automated circuit discovery in transformers using contextual decomposition. arXiv preprint arXiv:2407.00886 , 2024.

Tal Haklay, Hadas Orgad, David Bau, Aaron Mueller, and Yonatan Belinkov. Position-aware automatic circuit discovery. arXiv preprint arXiv:2502.04577 , 2025.

Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv preprint arXiv:2403.19647 , 2024.

Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages 407-416, 2024.

Lei Yu, Jingcheng Niu, Zining Zhu, and Gerald Penn. Functional faithfulness in the wild: Circuit discovery with differentiable computation graph pruning. arXiv preprint arXiv:2407.03779 , 2024.

Adithya Bhaskar, Alexander Wettig, Dan Friedman, and Danqi Chen. Finding transformer circuits with edge pruning. Advances in Neural Information Processing Systems , 37:1850618534, 2024.

Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla. arXiv preprint arXiv:2307.09458 , 2023.

Nicola De Cao, Leon Schmid, Dieuwke Hupkes, and Ivan Titov. Sparse interventions in language models with differentiable masking. In Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP , pages 16-27, 2022.

Deniz Bayazit, Negar Foroutan, Zeming Chen, Gail Weiss, and Antoine Bosselut. Discovering knowledge-critical subnetworks in pretrained language models. arXiv preprint arXiv:2310.03084 , 2023.

Aaron Mueller. Missed causes and ambiguous effects: Counterfactuals pose challenges for interpreting neural networks. In ICML 2024 Workshop on Mechanistic Interpretability .

Hang Chen, Jiaying Zhu, Xinyu Yang, and Wenya Wang. Unveiling language skills under circuits. arXiv preprint arXiv:2410.01334 , 2024.

Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. In The Eleventh International Conference on Learning Representations .

Stefan Heimersheim and Neel Nanda. How to use and interpret activation patching. arXiv preprint arXiv:2404.15255 , 2024.

Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. Knowledge circuits in pretrained transformers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems .

Neel Nanda. Attribution patching: Activation patching at industrial scale. URL: https://www. neelnanda. io/mechanistic-interpretability/attribution-patching , 2023.

Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. Advances in neural information processing systems , 33:12388-12401, 2020.

Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldwosky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing, a method for rigorously testing interpretability hypotheses. AI Alignment Forum , 2022. https://www.alignmentforum.org/posts/JvZhhzycHu2Yd57RN/ causal-scrubbing-a-method-for-rigorously-testing .

Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching. arXiv preprint arXiv:2304.05969 , 2023.

Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods. arXiv preprint arXiv:2309.16042 , 2023.

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems , 35:17359-17372, 2022.

Taehyeon Kim, Jaehoon Oh, NakYil Kim, Sangwook Cho, and Se-Young Yun. Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation. arXiv preprint arXiv:2105.08919 , 2021.

Michael Hanna, Ollie Liu, and Alexandre Variengien. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. Advances in Neural Information Processing Systems , 36:76033-76060, 2023.

Joy Crosbie and Ekaterina Shutova. Induction heads as an essential mechanism for pattern matching in in-context learning. arXiv preprint arXiv:2407.07011 , 2024.

Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Xipeng Qiu, and Dahua Lin. Identifying semantic induction heads to understand in-context learning. arXiv preprint arXiv:2402.13055 , 2024.

Benjamin L Edelman, Ezra Edelman, Surbhi Goel, Eran Malach, and Nikolaos Tsilivis. The evolution of statistical induction heads: In-context learning markov chains. arXiv preprint arXiv:2402.11004 , 2024.

Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems , 36:24678-24704, 2023.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper and a summary of the important work.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discussed the limitations in Section 6. Moreover, Section 3.1 shows the limitations of our corollary, including considering that the receiver node is directly connected to the output.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Section 3.1 provides full hypotheses and detailed examples to explain our theoretical definition.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide full settings of our experiments in Appendix E.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We will make it open-access with github link.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Section 4.1 and Section 3.3 provide full settings and details of our two experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Table 3, Figure 8 provide clear error bars with σ .

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: In Appendix E.3 we claimed that all experiments only need low computational cost as in the original setting of the existing work.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We checked the NeurIPS Code of Ethics and ensured that our paper satisfies it. Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discussed the broader impacts to ML society in Section 6.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: We listed the pretrained LMs in footnotes and described the scraped datasets in Appendix E.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We listed the URLs and references of existing assets that we used in the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We didn't release any pre-trained models and large-scale datasets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We didn't conduct any research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing and research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The paper does not involve any LLMs tools for core methodology, scientific rigorousness, or originality of the research.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/ LLM ) for what should or should not be described.

A The Preference between Two Types of Completeness Metrics

̸

The earlier completeness evaluation method (proposed by [16]) in fact measures incompleteness , defined as D ( C \ K ∥ G \ K ) , where a lower score indicates better performance. In contrast, the current method [9] measures completeness , defined as D ( G \ C ∥ G ) , where a higher score is preferable. Importantly, these two metrics are not directly interchangeable - that is, 1 -D ( C \ K ∥ G \ K ) = D ( G \ C ∥ G ) .

Nevertheless, a comparison can still be made in terms of the standard deviation . It has been theoretically demonstrated that the earlier completeness method suffers from high variance due to insufficient sampling, leading to instability in the final results. To illustrate this, we provide a simple comparison between the previous and current evaluation methods, as shown in the Table 4:

We conducted five different runs of circuit discovery using random seeds, with the random subset K chosen from subcircuits of sizes ranging from 2 to 5 nodes. For the previous evaluation metric, we performed 5, 10, 20, and 30 sampling iterations.

The experimental results clearly support two conclusions:

1. For algorithms that inherently lack completeness (such as ACDC, EAP, and Edge-Pruning), as well as for our own method, which ensures completeness, the previous evaluation metric exhibits large standard deviations across all trials. This indicates that the sampling process for the previous metric is insufficient, rendering the results highly unreliable.

2. The new evaluation metric significantly reduces variance for all types of circuits, as it does not suffer from sampling issues. The variance observed in the results arises from minor differences in the circuits generated by different random seeds.

B How Does Circuit Logic Model Intervention?

B.1 AND Gate

Consider a simple logical gate: A 1 ∧ A 2 = B (only if both x A 1 and x A 2 are present can B be activated), where B directly influences the output. For noising-based intervention ( Ns ), replacing x A 1 with ˜ x A 1 , or x A 2 with ˜ x A 2 , produces a significant effect on the output. Thus, Ns is capable of detecting the structure { ( A 1 , A 2 ) , B } .

However, for denoising-based intervention ( Dn ), substituting ˜ x A 1 with x A 1 alone does not yield a noticeable change in the output, as ˜ x A 2 remains present. Similarly, replacing ˜ x A 2 with x A 2 while ˜ x A 1 is still active also fails to significantly affect the output.

Under a greedy search strategy, if ˜ x A 1 is first replaced by x A 1 and the output remains unchanged, the algorithm concludes that A 1 is not relevant and removes it (i.e., replaces ˜ x A 1 with x A 1 in the scenario of Dn). Subsequently, replacing ˜ x A 2 with x A 2 causes a substantial shift in the output due to the presence of x A 1 , thereby restoring the structure { ( A 2 ) , B } , since A 1 has already been removed.

Analogously, a greedy search that begins with A 2 and then proceeds to A 1 would only recover the structure { ( A 1 ) , B } . Therefore, we conclude that Ns is capable of identifying the complete AND gate structure, whereas Dn either fails to detect the AND gates or only partially recovers it under greedy search conditions.

B.2 OR Gate

Consider a simple logical gate: A 1 ∨ A 2 = B (if either x A 1 or x A 2 are present can B be activated), where B directly influences the output. For Ns , replacing x A 1 with ˜ x A 1 alone does not yield a noticeable change in the output, as x A 2 remains present. Similarly, replacing x A 2 with ˜ x A 2 while x A 1 is still active also fails to significantly affect the output.

Under a greedy search strategy, if x A 1 is first replaced by ˜ x A 1 and the output remains unchanged, the algorithm concludes that A 1 is not relevant and removes it (i.e., retains ˜ x A 1 ). Subsequently, replacing x A 2 with ˜ x A 2 causes a substantial shift in the output due to the lack of support of x A 1 , thereby restoring the structure { ( A 2 ) , B } , since A 1 has already been removed.

Analogously, a greedy search that begins with A 2 and then proceeds to A 1 would only recover the structure { ( A 1 ) , B } .

However, for Dn , replacing ˜ x A 1 with x A 1 , or ˜ x A 2 with x A 2 , produces a significant effect on the output. Thus, Dn is capable of detecting the structure { ( A 1 , A 2 ) , B } .

Therefore, we conclude that Dn is capable of identifying the complete OR gate structure, whereas Ns either fails to detect the OR gates or only partially recovers it under greedy search conditions.

B.3 ADDER Gate

Consider a simple logical gate: A 1 + A 2 = B , where B directly influences the output. Since the influence of each edge in an ADDER gate is independent, removing an edge in either Ns or Dn directly impacts the output via its effect on B . For instance, in Ns, replacing x A 1 with ˜ x A 1 results in B ∗ = A 2 , which is significantly smaller than B = A 1 + A 2 . Similarly, in Dn, replacing ˜ x A 1 with x A 1 yields B ∗ = A 1 , which is substantially greater than B = 0 . Therefore, both Ns and Dn are capable of identifying the complete structure of the ADDER gate.

C How Does Circuit Logic Affect Faithfulness, Completeness, and Sparsity?

C.1 Faithfulness

As introduced in Section 2, faithfulness requires that D ( G || C ) be minimized. Let us consider the following scenarios:

For any gate { ( A 1 , A 2 ) , B } , if the circuit does not include all edges or nodes from this gate, it is always possible to find a circuit C ∗ = C ∪ A 1 , A 2 , B such that D ( G || C ) > D ( G || C ∗ ) .

For an AND gate { ( A 1 , A 2 ) , B } , if the circuit C only includes A 1 and B , the gate effect of this AND gate is not maximized (the influence of B is maximized when both A 1 and A 2 are present). Therefore, it is always possible to find a circuit C ∗ = C ∪ A 2 such that D ( G || C ) > D ( G || C ∗ ) .

For an ADDER gate { ( A 1 , A 2 ) , B } , if the circuit C only includes A 1 and B , the gate effect of this ADDER gate is not maximized (again, the influence of B is maximized when both A 1 and A 2 are present). Thus, there exists a circuit C ∗ = C ∪ A 2 such that D ( G || C ) > D ( G || C ∗ ) .

For an OR gate { ( A 1 , A 2 ) , B } , if the circuit C only includes A 1 and B , the gate effect of this OR gate is already maximized (the same applies if only A 2 and B are included). Therefore, for C ∗ = C ∪ A 2 , we have D ( G || C ) = D ( G || C ∗ ) . However, from the perspective of sparsity, | C ∗ | > | C | .

Thus, to achieve optimal faithfulness, the circuit must include all edges that result in the maximum gate effects, namely all edges from the AND, ADDER, and OR gates. However, considering sparsity, the gate effect sum remains maximal even if only one edge from each OR gate is retained.

C.2 Completeness

Similarly, completeness requires that D ( G \ C || G ) be maximized. Consider the following scenarios:

For any gate { ( A 1 , A 2 ) , B } , if the circuit does not include all edges or nodes from this gate, it is always possible to find a circuit C ∗ = C ∪ A 1 , A 2 , B such that D ( G \ C || G ) < D ( G \ C ∗ || G ) .

For an AND gate { ( A 1 , A 2 ) , B } , if the circuit C only includes A 1 and B , then G \ C will only contain A 2 or the edge A 2 → B (depending on whether pruning is applied to edges or nodes). Due to the AND operation, B will not produce a gate effect. Therefore, for the circuit C ∗ = C ∪ A 2 , we have D ( G \ C || G ) = D ( G \ C ∗ || G ) .

For an OR gate { ( A 1 , A 2 ) , B } , if the circuit C only includes A 1 and B , then G \ C will only contain A 2 or the edge A 2 → B . Due to the OR operation, B still produces a gate effect. Therefore, it is always possible to find a circuit C ∗ = C ∪ A 2 such that D ( G \ C || G ) < D ( G \ C ∗ || G ) .

For an ADDER gate { ( A 1 , A 2 ) , B } , if the circuit C only includes A 1 and B , then G \ C will only contain A 2 or the edge A 2 → B . Due to the ADDER operation, B still produces a gate effect. Therefore, it is always possible to find a circuit C ∗ = C ∪ A 2 such that D ( G \ C || G ) < D ( G \ C ∗ || G ) .

Thus, to achieve optimal completeness, the circuit must include all edges that result in the maximum gate effects, namely all edges from the AND, ADDER, and OR gates. However, considering sparsity, the total gate effect remains maximized even if only one edge is retained for each AND gate.

D Validation of Logical Gates

D.1 Toy Model

Motivated by [2], to study a toy transformer model with an AND, OR, and ADDER gates, we take a 1-Layer transformer model with two heads per layer, ReLU-based activations, and model dimension 1. Specifically, as shown in Figure 7, Let A 1 and A 2 be two attention heads with respective biases bias 1 and bias 2 , both set to 1. The activation function m is based on the ReLU nonlinearity. To ensure that the output of each attention head corresponds directly to its bias, we use a zero tensor as the input. For corrupted activations, we employ zero ablation-i.e., we directly remove the activations along the corresponding edges.

The activation function m is configured differently to simulate logical gates as follows:

AND gate: m ( x ) = ReLU ( x -1) . Under this setting, the output is 1 only when both A 1 and A 2 are active (i.e., not ablated); otherwise, the output is 0.

OR gate: m ( x ) = 1 -ReLU (1 -x ) . Here, the output is 1 as long as at least one of A 1 or A 2 is active; if both are ablated, the output is 0.

ADDER gate: The bias 2 is modified to 1.5, and m ( x ) = ReLU ( x ) . In this case, the output is 0 when both A 1 and A 2 are ablated; it is 1.5 when only A 1 is ablated, 1 when only A 2 is ablated, and 2.5 when both are active.

Under these configurations, we evaluate the performance of existing methods on the toy model, as summarized in Table 1. For example, under the default Ns, ACDC [2] (representing greedy search), EAP [8] (representing linear estimation), and EdgePruning [10] (representing differentiable mask) all successfully identify both A 1 and A 2 in the AND and ADDER gates. However, in the OR gate, ACDC and EdgePruning identify only one of A 1 or A 2 -the specific result depends on the search order in ACDC and the initialization of the mask in EdgePruning-while EAP fails to identify any high-effect edge.

Conversely, when these methods are executed under Dn, the outcomes are reversed. In the OR and ADDER gates, all three methods, ACDC, EAP and EdgePruning, can now identify both A 1 and A 2 . However, in the AND gate, only ACDC and EdgePruning are able to recover one of A 1 or A 2 , whereas EAP considers the effects of both to be insufficiently strong.

Ns can at least guarantee the full recovery of AND and ADDER gates. Greedy search, by retaining previous steps with removed results, can identify one OR

A1

A2

m

0

0

output

edge (a detailed analysis is provided in Appendix B). Differentiable mask, optimizing for faithfulness (as per Corollary 1), ensures that at least one OR gate is included (otherwise, optimal faithfulness cannot be achieved), while additional OR edges conflict with the sparsity constraint and are therefore removed. However, linear estimation, when computing the effect of each edge, keeps all other edges in their non-removed state, which results in a failure to detect the OR gate. For example, when computing the effect of the edge A → C in the OR gate A → C ← B , the edge C ← B is also in the clean activation state, leading to a very small effect for A → C . Similarly, when computing the effect of C ← B , the edge A → C remains in the clean activation state. In summary, linear estimation is unable to detect any edges of the OR gate.

The performance on Dn is the opposite of that on Ns, where, in addition to the full OR gates and ADDER gates, greedy search and differentiable mask can similarly recover one AND edge, just as they could with the OR gates in Ns. Linear estimation also fails to detect the AND gates due to the non-removed status of the other edges. Based on Corollary 1, we are also able to derive the performance of the three types of methods in terms of faithfulness and completeness.

E Experiment Details

E.1 Why Does Search Strategy by repeating interventions on combinations is NP Problem?

Due to the differences in circuit algorithms, this issue cannot be fully proven mathematically. However, we will provide some additional explanation: we treat each ablation as O (1) . Assuming the language model contains N nodes, if we need to use all possible intervention combinations to obtain a complete circuit, the time complexity would be O (2 N ) , which far exceeds polynomial time. However, different strategies can optimize the solution verification process to varying degrees. For example, EAP only requires O ( N ) time to verify whether the circuit is complete, while Edge-pruning involves gradient descent, making its time complexity difficult to estimate. Additionally, greedy search can only perform a greedy verification in O ( N ) time rather than a thorough verification. Therefore, we ultimately classify this problem as NP problem.

However, our proposed method, which employs the algorithm N s + D n , does not introduce additional computational complexity to the existing circuit discovery algorithm. Conceptually, it is equivalent to reapplying the N s algorithm once more under the strategy of D n . As a result, the overall time complexity increases by at most a factor of two relative to the baseline algorithm, without imposing any additional nonlinear burden.

E.2 Detailed Results of Figure 3

All three subcircuits exhibit a fundamental pattern: the effect on the output is maximized when both edges are retained, and it drops to zero when both edges are ablated (as there is no clean activation left to support the node in this case). However, ablation of a single edge yields divergent outcomes:

In AND gate, ablating either edge nearly eliminates the effect on the output. In OR gate, ablating either edge has almost no impact on the output. In ADDER gate, ablating either edge leads to a noticeable degradation in the output.

E.3 Baselines

In this work, we select ACDC [2] to represent greedy search methods, EAP [8] to represent linear estimation methods, and EdgePruning [10] to represent differentiable mask methods. In the following

sections, we provide a detailed exposition of the original design of each method under the Ns. strategy, the corresponding formulation under the Dn. strategy, and the final approach that integrates both-Ns.+Dn.-for recovering logically complete gates.

E.3.1 Greedy Search Example: ACDC

The ACDC method identifies important edges by iteratively removing each edge and observing the effect of this intervention on the model output. Edges whose removal causes an effect greater than a predefined threshold τ are retained, while those with an effect smaller than τ are pruned. The original algorithm (Ns. strategy), is outlined as follows:

Algorithm 1: The ACDC algorithm in Ns.

In the Ns. strategy, G denotes the clean run , and H\{ w → v } represents the replacement of the clean activation on the edge w → v with its corrupted activation. In contrast, under the Dn. strategy, G refers to the corrupted run , and H\{ w → v } indicates the substitution of the corrupted activation on edge w → v with the corresponding clean activation.

In the combined Ns.+Dn. approach, the effects from both strategies are jointly considered. Specifically, the original pruning condition D KL ( G ∥ H new ) -D KL ( G ∥ H ) < τ is replaced with the aggregated criterion: D KL ( G clean ∥ H new ) -D KL ( G clean ∥ H ) + D KL ( G corrupted ∥ H new ) -D KL ( G corrupted ∥ H ) < τ.

E.3.2 Linear Estimation Example: EAP

The EAP method approximates the effect of each edge using the first-order term of its Fourier expansion, enabling the estimation of all edge effects with a single forward pass. It is important to note that, during the computation of each edge's effect, all other edges remain in their unpruned (active) state.

Specifically, Ns. has approximation:

and Dn. has approximation:

Therefore, the approximation for Ns.+Dn. is (˜ x i -x i ) T ∂ ∂x i ] L ( x ) + (˜ x i -x i ) T ∂ ∂ ˜ x i ] L (˜ x ) .

E.3.3 Differentiable Mask Example: EdgePruning

EdgePruning assigns a learnable mask to each node or edge, where the mask is reparameterized using the hard concrete distribution. In the Ns. setting, the optimization objective corresponds to Equation 1. Consequently, the objectives for the Dn. and Ns.+Dn. settings are given by Equation 2 and Equation 3, respectively.

In the Ns.+Dn. setting, directly optimizing both objectives jointly can lead to gradient interference and convergence to Pareto-optimal solutions, rather than a unified optimum. To address this, we independently compute the final mask values for Ns. and Dn. using Equations 1 and 2, and then obtain the mask for Ns.+Dn. by averaging the two.

F Misalignment Score

Misalignment of AND : For any subcircuit K AND ⊂ C AND, C ∗ AND = C AND \ K AND. Let i, j ∈ C AND, i ∗ , j ∗ ∈ C ∗ AND be any two edges with the same receiver, respectively. The score of misalignment of AND reads:

Equation 6 indicates that the higher the score, the higher the misalignment. This is due to their properties: the effect caused by removing one and two edges from the AND gates is similar, while the effect for the ADDER gates differs significantly.

Misalighment of OR : Similarity, let any K OR ⊂ C OR , C ∗ OR = C OR \ K OR, i, j ∈ C OR, i ∗ , j ∗ ∈ C ∗ OR , respectively. The score of misalignment of OR reads:

Equation 7 utilizes the properties that the effect does not change by removing one edge from OR gates, while it significantly changes from ADDER gates. Additionally, to avoid the bias caused by both effects of ADDER and OR edges being marginally small in large-scale circuits, we replace the 'difference in one edge' with the 'difference in difference between one edge and two edges,' and introduce a constant m to ensure that the score > 0 (with m set to 1 . 5 in practice).

Therefore, for any pair of C Ns and C Dn, we can compute the misalignment using these two scores. We report the misalignment scores resulting from the intersection of Ns and Dn circuits at varying scales. Specifically, we select an Ns circuit consisting of 100 edges recovered from the IOI task and examine how the misalignment score changes as the number of edges in the Dn circuit varies from 60 to 140. Figure 8 illustrates that when Dn is significantly smaller than Ns, the misalignment score for the AND gates is high, as many ADDER edges are misclassified as AND edges. Conversely, when Dn is substantially larger than Ns, the misalignment score for the OR gates increases, due to many ADDER edges being misclassified as OR edges. When the number of edges in the Dn circuit is approximately equal to that in the Ns circuit, both the misalignment score and its standard deviation reach an acceptable level.

As shown in Figure 8, when the OR set significantly exceeds the AND set, a large number of misclassified ADDER gates appear only within the OR set, leading to a much higher misalignment score for OR compared to AND. The misalignment score is more of a "property," reflecting the optimal scale at which OR and AND sets should intersect. We determine the optimal ratio of AND to OR gates by minimizing the sum of the misalignment scores for AND and OR. The optimal results obtained with different methods and datasets are approximately as shown in Table 6:

The results show that the proportion of AND is always slightly higher than that of OR, but it can be approximated as 1:1. Therefore, throughout this paper, we assume that the optimal alignment occurs when Ns and Dn contain an equal number of edges .

G Experiments of Completeness

In this Appendix, we present the detailed results of the completeness validation, together with a comparison against the Dn strategy. The specific results are reported in Tables 7 and 8. In brief,

60

70

80

90

100

110

120

130

140

#edges with denoising

0

20

40

60

80

100

#edges of AND

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Score

ACDC #edges of AND

EAP #edges of AND

EdgeP #edges of AND

ACDC Score

EAP Score

EdgeP Score

(a) Misalignment of AND gate

60

70

80

90

100

110

120

130

140

#edges with denoising

0

20

40

60

80

100

120

140

160

#edges of OR

0.4

0.6

0.8

1

1.2

1.4

1.6

Score

ACDC #edges of OR

EAP #edges of OR

EdgeP #edges of OR

ACDC Score

EAP Score

EdgeP Score

(b) Misalignment of OR gate

our method generally outperforms the approach based on Dn. Although the Dn-based method is theoretically capable of identifying all OR gates, it suffers from certain shortcomings in practice. For instance, as shown in Table 1, EAP fails to identify one of the edges associated with an AND gate under the Dn framework. Additionally, the greedy strategy may miss some OR gates located beneath branches of AND gates due to its dependence on the search order. The differentiable mask approach, on the other hand, suffers from the drawback that, in gates involving multiple edges, the mask values assigned to each edge tend to be lower, reducing its effectiveness.

H Experiments of Faithfulness

In this section, we investigate the faithfulness of circuits obtained using three methods-ACDC, EAP, and EdgePruning-across three tasks: IOI, GT, and SA. Specifically, we examine the changes in KL divergence and accuracy between the original circuit (Ns.), the circuit with full OR and ADDER gates (Dn.) and the circuit with logically complete gates (Ns.+Dn.). For sparsity, we select edge counts of 100, 200, 500, 1000, 2000, and 5000.

Figure 9 illustrates that, under the same sparsity constraints, the circuits discovered using Dn. are significantly lower in both metrics compared to those discovered using Ns. and Ns.+Dn., which corroborates our assertion in Corollary 1: Dn. is incapable of fully recovering the AND gate, and thus cannot achieve optimal faithfulness.

Additionally, in the EAP method, Ns clearly performs much worse than Ns+Dn, whereas in the ACDC and EdgePruning methods, the performance of Ns and Ns+Dn is quite similar. This aligns with our reasoning in Table 1, where we note that only the linear estimation method completely fails to identify any OR edge, thus not satisfying the minimal requirement of faithfulness.

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

3.5

KL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(a) ACDC of IOI task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(b) EAP of IOI task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0

0.2

0.4

0.6

0.8

1

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(c) EdgePruning of IOI task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(d) ACDC of GT task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(e) EAP of GT task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(f) EdgePruning of GT task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

3

KL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(g) ACDC of SA task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

KL

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(h) EAP of SA task

100

200

500

1000

2000

5000

#edges

0

0.5

1

1.5

2

2.5

KL

0

0.2

0.4

0.6

0.8

1

Ns

Dn

Ns+Dn

Ns

Dn

Ns+Dn

(i) EdgePruning of SA task

I Graph Study

To investigate the relationship between the three logical gates and the functions of the language model, we extracted the circuits of AND, OR, and ADDER gates discovered through ACDC on the IOI task. We then reviewed the IOI circuit [16] to determine the function of each component.

Interestingly, each receiver in the AND circuit is almost always influenced by edges from different functions, indicating that the AND operation can be understood as combining different functions to jointly impact the subsequent layers. For example, in Figure 10(a), the Induction Head requires edges from both the Duplication Token Head and the Previous Token Head to function, which supports the mechanism behind the Induction skill [27, 28, 29]. Similarly, the Name Mover Head requires support from both the S-Inhibition Head and the Induction Head, which explains the functional mechanism of the AND operation.

In contrast, the OR circuit clearly shows that nearly every receiver node is influenced by edges from the same function, as shown in in Figure 10(b), suggesting that these edges from the same function are either backups or interchangeable. For instance, the S-Inhibition Head is influenced by multiple Induction Heads, and the Backup Name Mover Head is influenced by multiple S-Inhibition Heads.

Lastly, the ADDER circuit appears to focus more on the outputs of the MLP and often combines outputs from shallow-layer skills with those from deeper-layer skills, as shown in Figure 10(c). The Name Mover Head considers outputs from all functions between the Duplicate Token Head and the S-Inhibition Head, and the final output takes into account the combined results from all three Name Mover Heads.

Additionally, regarding the span of gates across layers, the OR gates typically operate over the shortest distances, usually occurring between two functions that are close in layer position. In contrast, the ADDER gates generally span the longest distances, typically combining shallow-layer functions with deeper-layer functions.

Duplicate Token Head

Previous Token Head

Induction Head

S-Inhibition Head

Negative Name Mover Head

Name Mover Head

Backup Name Mover Head

Output

Duplicate Token Head

Previous Token Head

Induction Head

S-Inhibition Head

Negative Name Mover Head

Name Mover Head

Backup Name Mover Head

Output

<a11.9>

<6W>

<a9.7>

<a9.0>

<a0.9>

<a0.1>

<resid_post>

<a3.0>

<m1>

<a4.11>

<a3.7>

<a2.2>

<m4>

<ow>

<a3.4>

<m2>

<a5.5>

<a0.10>

<m5>

<a5.9>

<a5.8>

<a8.10>

<a7.9>

<a7.3>

<m6>

<mw>

<a8.6>

<a9.9>

<m10>

<m1>

<m0>

<a5.9>

<a5.8>

<a3.10>

<a3.4>

<m6>

<m5>

<a8.6>

<6w>

<a3.0>

<a0.10>

<m7>

<a4.8>

<m4>

<m8>

<m11>

<a9.6>

<a9.9>

<a10.0>

<a8.10>

<a11.3>

<a11.8>

<a11.2>

<resid_post>

embed

<m6>

<ow>

<a0.1>

<a9.6>

<a10.0>

<a0.5>

<m5>

<a0.3>

<m4>

<m2>

<m1>

<m7>

<m8>

<mW>

<a11.3>

<a8.10>

<a7.9>

<a7.3>

<a11.11>

<o>

<a10.10>

<a10.6>

<a10.1>

<a9.9>

<resid_post>

<6W>

100

200

500

1000

2000

5000

#edges

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

KL

100

200

500

1000

2000

5000

#edges

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

KL

100

200

500

1000

2000

5000

#edges

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

KL

100

200

500

1000

2000

5000

#edges

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

KL

100

200

500

1000

2000

5000

#edges

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

KL

100

200

500

1000

2000

5000

#edges

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

KL

(d) Gate contribution in IOI task

(e) Gate contribution in IOI task

(f) Gate contribution in IOI task

J Output Contribution

In this section, we investigate the contribution of three types of logic gates to the output. Specifically, we calculate the change in the KL divergence of the output caused by replacing these gates. Considering the collection of gates, we analyze their contributions from two perspectives: the gate effect and the edge effect . The gate effect refers to the impact on the output caused by replacing an entire logic gate, while the edge effect corresponds to equally distributing the gate effect across each edge within the gate. For example, for an AND gate with two edges, if the activation of the receiver node contributes 0.8 to the output, the edge effect would be 0.4. Figure 11 illustrates the gate effect and edge effect of these logic gates across different tasks and baselines. Clearly, the ADDER gates exhibit the largest contribution, demonstrating its role as the primary framework of the circuit, while the contributions of the AND and OR gates are similar. Additionally, the average gate and edge effects in EdgePruning and EAP are significantly higher than those in ACDC. This is because the differentiable mask and linear estimation methods optimize (rank) based on the edge effect, ensuring that the effect within the circuit is maximized, in contrast to greedy search methods.

K Proportion of AND, OR, and ADDER Gates

Figure 12 illustrates the proportion of the three types of logical gates across different tasks for each method. Notably, in the ACDC baseline, the number of edges corresponding to each gate type is nearly equal. This is because ACDC employs a greedy search strategy without ranking edges by their effect on the output. In contrast, both EAP and EdgePruning yield significantly fewer OR edges, reflecting the fact that OR edges contribute the least to the output-a finding we detail in Section 5.2 and Appendix J. Furthermore, the results from EdgePruning indicate that the number of AND edges is similarly low, comparable to OR edges. This arises from the fact that EdgePruning optimizes based on individual edge effects rather than gate-level effects. For instance, in a gate comprising two AND edges, each edge contributes only half of the total gate effect. As a result, during optimization, the mask values for such edges may be suppressed, increasing the likelihood of pruning.

L Potential Contributions of Completeness: an Example of Model Unlearning

We have designed a toy task analogous to knowledge unlearning to demonstrate its actual effectiveness. Specifically, given a harmful-response datasets (PKU-SafeRLHF) [30], we first obtain its circuit C , and then remove this circuit from the computation graph G , observing whether the model

28

37

35

136

259

1480

157

312

1477

207

429

2043

(a) ACDC in IOI task

57

13

30

178

321

1498

191

433

2488

131

246

1054

(b) EAP in IOI task

22

12

66

110

222

1247

294

605

2833

96

173

920

(c) EdgePruning in IOI task

26

39

35

144

243

1359

242

356

1527

198

401

2114

(d) ACDC in GT task

44

21

35

149

306

1355

198

518

2459

153

176

1186

(e) EAP in GT task

23

26

51

104

214

1167

281

617

2922

115

169

911

(f) EdgePruning in GT task

27

41

32

139

249

1297

156

335

1536

205

416

2167

(g) ACDC in SA task

39

20

41

137

314

1349

216

527

2639

147

159

1012

(h) EAP in SA task

15

28

57

87

194

957

289

627

2996

124

179

1047

(i) EdgePruning in SA task

ADDER

AND

OR

AND

OR

ADDER

ADOER

AND

OR

AND

AND

AND

ADDER

OR

OR

ADDER

ADDER

AND

AND

AND

OR

ADDER

ADDER

OR

ADOER

OR

can still respond to the dataset. The dataset provides harmful responses for certain questions, which are used to evaluate the efficacy of preventing the model from generating these harmful responses.

In the Table 9, we present the performance of this toy task on the accuracy of G - C , which refers to the performance of the computational graph after the circuit has been removed (using the interchange ablation method), reflecting the completeness of the circuit. We test two circuit discovery methods: EAP and Edge-Pruning.

It is evident that, compared to existing circuit discovery methods, our method ensures that the computational graph loses almost all the necessary mechanisms for the task once the circuit is removed. Additionally, this toy task highlights the significance of studying circuit completeness, which holds great potential in unlearning harmful information: a complete circuit can help us pinpoint all neurons associated with harmful responses.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new way to interpret circuits in terms of three types of logical gate, which help shed light on and tease apart the important concepts of completeness and faithfulness. They explain how these gate types relate to the circuits found by existing methods, and propose a new method that compromises between existing methods to efficiently find better circuits by leveraging this new understanding. They apply these ideas to a number of well-studied tasks in mechanistic interpretability to validate key claims and show that these new concepts and methods can lead to new insights and improved interpretability in practice.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. I found section 2 in particular to be a well-written, clear, and concise summary of background on terms and methods for this paper.\n2. The work is well-motivated—I agree it is important to have a better understanding of completeness and faithfulness, and more generally interpretability, of discovered circuits. And furthermore this seems to be a novel contribution.\n3. The empirical demonstrations seem to be compelling, particularly Fig 3 and the conclusions drawn in 5.1, in showing that these classes of gates exist, can be identified, and can lead to meaningful insights that wouldn’t otherwise be achievable.\n\nWeaknesses:\n1. Legend in Fig 4 is unreadably small\n2. I don’t understand lines 144-148: what does it mean “B can influence the result”? And why should an OR gate remain functional as long as at least one edge remains intact? The function f(A_1, A_2) = A_1 ^ A_2 is not the same as the function g(A_1,A_2) = A_1, so I don’t see why it should be the case that an OR gate doesn’t care if one edge is dropped. Actually, it’s not even clear to me what function is being used when, say, A_2 is “dropped”—I guess it depends on the ablation method, but surely it’s not true that no matter what ablation method is used, ablating one edge in an OR gate doesn’t change it at all?\n3. There is a claim (repeated twice) that the simplest approach to uncovering OR gates would make circuit discovery an NP problem, but no reference or explanation is given.\n4. Line 153 states that any circuit CAN be represented via the proposed gates, but I don’t see how that means that every circuit IS represented via the proposed gates in terms of the nodes and edges it is actually defined by. Like, in order to represent a given circuit via only these gates, it seems you’d have to make a new graph that is computationally equivalent to the old graph but only uses these three gates; it’s not clear how to do this, nor that one would want to do this, since nodes and edges in the new graph will no longer have the same interpretation as those in the old graph.\n5. I think I have a high-level, very fundamental point of confusion about this paper. It seems to use the language of binary computation throughout (e.g., words like “logic” and even the names of the gates “and”, “or”), yet as far as I know this is not what the circuits people discover look like, i.e., the information being passed along edges is not binary, but instead is continuous, and in general multidimensional. I think this must be the case in their experiments as well, since they use GPT2-small, and Fig 6’s nodes are entire attention heads or MLPs. So then is the binary/logical language just sort of figurative/suggestive for classifying the components of these circuits and interpreting them? This disconnect between binary intuition/motivation and the reality of non-binary components in actual circuits seems to be unaddressed in the paper, which I found surprising given that it seems to otherwise be clearly and thoughtfully written—I feel I am missing something simple but major, which may also account for a number of weaknesses I listed above. So if the authors can clarify this, it could significantly impact my score.\n\nQUESTIONS:\nSee weaknesses above, particularly the last one.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an interesting abstraction of the behaviour that nodes and edges exhibit in circuits. In particular, it hypothesises that nodes are analogous to AND, OR, or ADDER logic gates in how they handle the input from incoming edges. The authors propose a framework that classifies nodes into one of the three gate classes based on their behaviour with respect to the intervention introduced its incoming edges. Using these identified behaviour, they were able to propose a more refined version of the completeness evaluation that could benefit the circuit discovery community.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strength**\n\n- This paper is very clearly written and easy to follow.\n- It proposes a refinement to the completeness aspect of the evaluation of circuit discovery methods, an essential component that is often poorly assessed.\n- The paper thoroughly enumerates its limitations, which is important for any interpretability research right now, as we are still in a very early stage of cracking the black box open. The paper does not overclaim anything and carefully states its findings. This should be much appreciated.\n\n### **Weakness**\n- I would like the paper to elaborate further on the limitations of existing completeness measures. Section 2.2 briefly mentions some critiques of sampling-based completeness scores, but the discussion is limited. It is more or less accepted that previous approaches to completeness evaluation are problematic, but, to the best of my knowledge, there has been no empirical or critical examination of this. I believe this work could have taken that step. It would also motivate their method strongly.\n\n### **Smaller points:**\n- The choice to abbreviate noising-based intervention as Ns and denoising-based intervention as Dn is somewhat unconventional.\n\nQUESTIONS:\n- Please refer to the Weaknesses section. I’d be interested to hear your thoughts on the issue of completeness.\n- Are there any previous convention to abbreviate noising-based intervention as Ns and denoising-based intervention as Dn?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1. This paper proposes a new framework for interpreting circuits in language models by decomposing them into three types of logic gates: AND, OR, and ADDER. The author argues that most existing circuit discovery methods fail to identify complete circuits due to the incomplete recovery of OR gates.\n2. This paper provides a detailed analysis showing the limitations of current noising-based and denoising-based strategies, and proposes a combined approach to recover logically complete circuits. \n3. This paper provides extensive experimental validation using three different circuit discovery methods (ACDC, EAP, EdgePruning) on three tasks (IOI, GT, SA). The results show that the proposed Ns+Dn framework discovers circuits that are more complete and less random than circuits found using Ns alone, while maintaining comparable faithfulness.\n4. This paper explores the functional roles these different gates play, suggesting AND gates integrate diverse functions while OR gates provide redundant pathways.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The idea of decomposing a circuit into AND, OR, and ADDER gates to understand circuit functionality is a strong and meaningful contribution. It provides an intuitive and straightforward way to understand the relationships between components in a neural network.\n2. The paper analyzes how the two primary intervention strategies (noising-based and denoising-based) are naturally suited to recovering different types of logic gates. The proposed framework (combine *Ns* and *Dn*) is modular and compatible with existing methods, making it practical for adoption.\n3. Extensive experiments are conducted to support the paper's claims. The framework is tested across three different classes of circuit discovery algorithms on three distinct benchmark tasks.\n4. The paper is clear and easy to follow.\n\nWeaknesses:\n1. One of the key parts of the method for separating the logic gates is based on the assumption that \"*optimal alignment occurs when Ns and Dn contain an equal number of edges*\" (L-231). This is a strong assumption that is not deeply justified. Is it possible that the true underlying circuit has an inherent imbalance between necessary (AND-like) and redundant (OR-like) pathways? How will the author handle this case? More discussions on this are suggested.\n2. The \"logically complete\" circuits discovered by the proposed Ns+Dn method are not significantly more faithful than those found by standard Ns methods alone (as acknowledged by the authors in Sec 6.1). Given that faithfulness is often a key goal, this may limit the method’s practical appeal for users who prioritize that metric.\n3. Some of the figures (e.g., Fig. 4 and 9) are difficult to understand.\n(1) For example, the multi-line graphs in these 2 figures show two metrics across multiple methods on dual y-axes. It is visually dense and makes it challenging for reviewers to compare results effectively. \n(2) Also, it seems to me (from Fig. 5) that KL divergence and task accuracy of Ns+Dn are very close to Ns-only. This raises my concern: why should we adopt this more complex method if existing approaches already perform nearly identically on core metrics? Is it possible to show the improvement through specific quantitative numbers rather than a line chart?\n4. The paper refers to the \"*gate effect*\" as the contribution of an entire gate, which is then used to calculate the \"*edge effect*\" (e.g., Sec. H). However, the method for isolating and calculating the total effect of a multi-edge gate (e.g., for AND and OR gates where inputs are not independent) is not fully detailed. It is unclear to me how these contributions were measured, and more explanations on this are suggested.\n5. All experiments are done on GPT2-small. There is no discussion showing whether the proposed Ns+Dn approach scales to other models (e.g., GPT-3, Llama).\n\nQUESTIONS:\nPlease see \"Weaknesses\".\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a circuit discovery method using AND, OR, and ADDER gates. Categorizing existing intervention methods for circuit discovery as noising-based (Ns) and denoising-based (Dn), the authors argue that Ns recovers AND and ADDER gates while Dn recovers OR and ADDER gates. Furthermore, for optimal faithfulness, the circuit needs only one edge from each OR gate, while for optimal completeness it needs only one edge from each AND gate. The authors then propose to combine noising- and denoising-based interventions to discover all three types of logic gates. The proposed framework is evaluated through experiments on GPT2-small considering three tasks (indirect object inference, greater-than, and syntactic agreement). In particular, they show that the proposed Ns+Dn achieves higher completeness (measured by KL divergence and accuracy) than Ns only.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed approach to circuit discovery using logical gates is novel and interesting. Circuits with multiple, clearly-defined types of gates can lead to richer interpretations of language models.\n\nThe proposed method for discovering these nodes, by combining noising- and denoising-based strategies, is fairly straightforward and can be adopted by tweaking existing Ns methods. Thus, there are immediately a few variants of Ns+Ds for logical circuit discovery based on which baseline was chosen. This adds to usability and potential impact.\n\nMy main concern of the paper is that some definitions or results are described not quite rigorously, but rather using high-level arguments using intuition. This made it difficult to check technical correctness and significance: e.g., proofs for Corollary 1 and 2 and how the Ns+Dn objective in Eq 3 can be optimized using different baseline strategies. Also, faithfulness and completeness are defined in Section 2.2 such that there are multiple ways to quantify them, and I wasn’t sure if the proof for Corollary 2 still applies if faithfulness is measured by accuracy for example.\n\nThe authors compare their method mainly to Ns methods, with the reasoning that most existing baselines are noising-based. However, Table 1 shows Dn methods as well (although I am confused that the methods for Ns and Dn are identical), and it would be appropriate to compare against them wrt completeness.\n\nQUESTIONS:\n1. Can there exist gates other than AND, OR, and ADDER, and would that affect Corollaries 1 and 2? I would also suggest adding a short proof sketch for them in the main text even though the full proofs are in the appendix.\n\n2. Regarding how optimizing for faithfulness allows removing an edge from OR gates, while only one edge being present will activate the OR gate, it doesn’t necessarily mean that each input activates all edges into the OR gate. In the extreme case, if the inputs that activate $A_1$ are disjoint from those that activate $A_2$, then removing either from gate $B = A_1 \\vee A_2$ will hurt faithfulness. Could the authors clarify why removing an edge in this case is still ok?\n\n3. When running Ns and Dn in parallel to identify the three types of gates, are there any assumptions such as them having to use the same strategy (greedy, linear estimation, ..)? Does misalignment vary depending on which method was used?\n\n4. In Section 4.3.2, the comparison should also include Dn methods especially because they aim to optimize for completeness.\n\nOther questions/comments:\n- Is the term “logically complete circuits” used with a precise meaning? I think this might not be the best name, potentially confused with completeness.\n- I don’t quite follow how the paragraph starting in line 171 is implied by Corollary 2.\n- Line 241: “Expanding the space of intervention combinations renders circuit discovery an NP problem”. Is this referring to it being an NP-hard problem? There should be a proof for the statement.\n- The marks and legends in Figure 4 were too small and very hard to read.\n- Figure 5: is there a statistically significant difference between Ns+Dn and Ns? I would hope to see error bars.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWhen finding which subgraph (circuit) of the NN is responsible for a task, we want to have three properties: faithfulness (the circuit by itself does the task), minimality (the circuit is as small as possible) and completeness (the circuit contains all parts of the NN that are responsible by the task, even redundant ones).\n\nCircuit discovery algorithms until now focused mainly on faithfulness and minimality and the tradeoff between them. The circuits discovered by them often lack completeness. The \"automatic circuit discovery\" paper [ref 2 in paper, appendix M] also talked about a logical OR gate as something that its proposed method had trouble with, and methods since has not improved.\n\nThis paper argues that traditional circuit discovery algorithms miss OR gates because they use a noising (Ns) approach. Ns algorithms start with NN activations on 'clean' inputs, and replace them with 'corrupted' activations; which leads them to see that replacing one if the inputs is OK (so it has no effect). \n\nThe paper introduces denoising (Dn) as an alternative circuit discovery approach: modify any current algorithm to start with the corrupted activations, and replace parts of it with clean activations. The same way that Ns algorithms easily discover AND gates, and AND is the dual of OR (DeMorgan's law style, see lines 199-201), Dn algorithms easily discover OR gates.\n\nTo make an algorithm which discovers both kinds of gates (as well as linear ADDER gates), we can combine Dn and Ns approaches in the same loss.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\nThis paper engages reasonably well with the existing literature. Its main idea, to introduce Denoising as an approach, is excellent: it makes it obvious everyone was making an assumption that restricted the design space. The resulting design plausibly addresses the largest remaining within-paradigm weakness of circuit discovery (OR gates).\n\nI really like the conceptual clarity of Table 1. In contrast with the attempted formal Definition 1 of the article, Table 1 very clearly defines what we expect of \"redundant + voting\" circuits (OR) or \"all outputs must work\" circuits (AND), or linear (ADDER) circuits.\n\nThe end result is a brilliant idea that, if I were writing a circuit discovery paper, I would definitely try out and give credit to.\n\n# Weaknesses\n\nUnfortunately, the paper suffers from two issues: superficial empirical validation, and conceptual problems.\n\n## Empirical problems.\n\nThere is not enough grounds for readers to believe that the presented method is actually better than the existing ones, despite the clearly motivated idea. A\n\n## Conceptual problems\n\n### Correctness of OR/AND/ADDER definitions\n\nDefinition 1 (L133-143) introduces OR, AND and ADDER gates with seemingly formal definitions, but these are very unclear. what is the input space to which the As and the Bs belong to? If it is booleans, what does the sum (+) operation mean? If they are numbers, what do OR (v) and AND (^) mean?\n\nWhat reason do we have to believe that NN circuits can be decomposed into these three gates? Neural networks do other operations as well, for example negation (if we stick to the boolean framing, which I also think is too restrictive).\n\nIt seems that the real definition of OR, AND and ADDER is whether they can be discovered by linear approximation methods or existing methods in Ns or Dn versions (L196-206 in Section 4). I think you should stick to that.\n\n### It's not true that you can just remove edges from OR and AND gates, or that OR gates \n\nIn the caption of figure 1: \"when optimizing for faithfulness and sparsity alone, it is possible to remove one edge from the OR gate\". Why? In the subset of inputs in which the node corresponding to the removed edge is activated, and the edge that remains is not, this changes the output of the circuit. So removing one edge *does* decrease faithfulness, which would make existing algorithms find the OR gates sometimes.\n\n### It's not true that all circuits can be decomposed in OR, AND and ADDER (as defined)\n\nLine 8-9: \"decompose the circuit into combinations of [OR, AND and ADDER]\" -- which circuit do you decompose in this way? You mean *any* circuit? How do we know that any circuit can be expressed with these three gates? (This claim is made again in L46 and L153-155).\n\nHow to fix?\n\nIf you change the definition to \"AND can be discovered by Ns, OR by Dn, and ADDER by both\" then you would just have to show that the OTHER gates (which can be discovered by neither Dn or Ns) don't exist. Then, every circuit is composed of these gates (note, I'm not sure that's true).\n\nYou could also argue that all circuits are OR and AND and NOT gates if they're made of booleans, but that seems like a huge stretch given that neural networks inputs and outputs are continuous.\n\n## Missing literature\n\nIn line 103, these two papers are missing for linear estimation: EAP-IG (https://arxiv.org/abs/2403.17806) and AtP* (https://arxiv.org/abs/2403.00745). Both of these are reasonably cited elsewhere, especially EAP-IG. They address some of the issues of EAP/AtP for circuit attribution.\n\nQUESTIONS:\n# Upgrading my score\nWhen the authors address all the weaknesses I wrote about, I will recommend this paper for acceptance.\n\n# Questions about experiments\nThis section contains some clarity problems that should be addressed\n\n## Section 4.1\n- In section 4.1, how do you choose the C_AND, C_OR and C_ADDER circuit parts? Using *which* Dn and Ns algorithm specifically? On how many random runs?\n- How do you choose the 1 and 2 edges to remove for Figure 3? Randomly, you average over all sets of 1 edges and sets of 2 edges, ... ?\n- I think the \"granularity alignment\" concept from Figure 2 and Lines 222-233 needs more explanation. Why is Figure 2 a good picture of what's really happening in the circuit? The misalignment scores from Appendix E, they're completely separate for OR and AND; from figure 2 I thought misalignment was a property of the ADDER intersection? Very confused here.\n\n## Completeness validation 1: removing the circuit\n\nThis corresponds to Section 4.3.1 (in page 7) and Figure 4\n\nFigure 4 is really hard to read, the dots are small and everything is the same color, so I don't know what to make of the results at all. In any case, the difference between circuit discovery methods doesn't seem that large? But it's consistent at least.\n\nFrom the text in lines 265-279, it seems that you're measuring completeness by removing the discovered circuit from the model and checking how much the KL divergence changes. Is that true?\n\nIf so, that doesn't match the definition of completeness from Wang et al. 2023: that's about whether removing a *subcircuit* K from both the model and the circuit, has the same effect on both.(as you well specify in lines 118-124). Why is this method a valid way to measure completeness?\n\n## completeness verification 2: average Hamming distance\nWhat is the Hamming distance on: which edges are present/absent? Is this the average of every pair of circuits from the 30 runs (900 pairs total), or from a few randomly selected pairs?\n\nIt would be much easier to read Table 3 if you would just compute the Hamming distance for C_Ns and for C_{ns + dn} and state both separately,  instead of subtracting them. Then we can compare which distance is greater.\n\n## Figure 6 + section 5.1\n\nHow are you supporting the claim that \"the AND gates typically receive edges from different functions\" vs \"the OR gates receive edges from the same function\"? What is a function here? Do you literally mean whether incoming edges are from MLPs (\"same function\") or attentions and MLPs (\"different function\")? That's a very strange thing to check, different MLPs and different attention heads will be computing different function, so I don't see why they're the same \n\n# Limitations that aren't actually limitations\nIn Line 336-338, you say that \"since there is no difference in faithfulness between the logically complete circuit and the existing circuit, it is difficult to directly demonstrate its 'superiority' over current work\". This seems false: you can just demonstrate superiority by demonstrating that the discovered circuit is more complete. (which the Hamming distance experiment kind of showed). Why did you say this?\n\n\n\n# Minor awkward wording\nL46: \"we introduce the concept of logic gates\" - no, you do not, it's at least a century old. You talk about logic gates in circuits.\n\nLIMITATIONS:\nThe one limitation they acknowledge, in lines 335-340, that this is only theoretical research and cannot be shown empirically to be better, is straight up not a limitation.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-18"}