Light Differentiable Logic Gate Networks
Authors: Lukas Rüttgers
Till Aczel
Andreas Plesner
Roger Wattenhofer
Venue: ICLR 2026 Poster (2026)
Keywords: reparameterization
logic gate networks
vanishing gradients
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: This paper proposes an efficient reparametrization of Differentiable Logic Gate Networks that mitigates vanishing gradients, discretization errors and training cost.

Abstract: Differentiable logic gate networks (DLGNs) exhibit extraordinary efficiency at inference while sustaining competitive accuracy.
But vanishing gradients, discretization errors, and high training cost impede scaling these networks.
Even with dedicated parameter initialization schemes from subsequent works, increasing depth still harms accuracy.
We show that the root cause of these issues lies in the underlying parametrization of logic gate neurons themselves.
To overcome this issue, we propose a reparametrization that also shrinks the parameter size logarithmically in the number of inputs per gate.
For binary inputs, this already reduces the model size by 4x, speeds up the backward pass by up to 1.86x, and converges in 8.5x fewer training steps.
On top of that, we show that the accuracy on CIFAR-100 remains stable and sometimes superior to the original parametrization.

Published as a conference paper at ICLR 2026
LIGHT DIFFERENTIABLE LOGIC GATE NETWORKS
Lukas Rüttgers, Till Aczely Andreas Plesner* & Roger Wattenhofer
ETH Zürich
Zürich, Switzerland
Iruettgers, taczel, aplesner, taczel,
wattenhoferGethz.cd atteeea.eepeeeeeeettteeeeeeette.
ABSTRACT
Differentiable logic gate networks (DI.GNs) exhibit extraordinary efficiency at inference while sustaining competitive accuracy. But vanishing gradi- ents, discretization errors, and high training cost impede scaling these net- works. Even with dedicated parameter initialization schemes from subse- quent works, increasing depth still harms accuracy. We show that the root cause of these issues lies in the underlying parametrization of logic gate neurons themselves. To overcome this issue, we propose a reparametriza- tion that also shrinks the parameter size logarithmically in the number of inputs per gate. For binary inputs, this already reduces the model size by 4x, speeds up the backward pass by up 1.86x, and converges in 8.5x fewer training steps. On top of that, we show that the accuracy on CIFAR- 100 remains stable and sometimes superior the original parametrization. "Equal contribution.
INTRODUCTION
Contemporary large, overparametrized neu- ral networks have demonstrated remark- able expressivity (Allen-Zhu al. 2019)) but their computational cost necessitates efficiency improvements while sustaining their approximation accuracy (Gusak et al. ]022]. With that goal, several approaches directly draw from the physical structure of the underlying hardware to parametrise
Original
Reparametrized
2:3112
3%%
198410
Relative Scale
0.4
model classes /Wang et al. 2020 Benamnra ltt al. 2024
1562
Bacellar et al.) 2024 Hubara
et al.
2016 Among them, differentiable
logic gate networks maintain an unparal- leled perfornance-eflliinny trade-off (Pe- tersen et al. 2022). Subsequent works have since advanced this model to convolutional or recurrent architectures (Petersen et al., 2024 Bührer al. 2025). Yet, several is- sues like vanishing gradients, discretization errors, and high training cost impede scal- ing these models in depth.
0.0
Forword pass Backward pass Training steps
Moriel site
Test aOC.
Figure For a CIFAR DLGN Petersen et al, 2022), reparametrized DLGNs require 4x less memory, converge in 8.5x fewer training steps, and perform the forward and backward passes in up to 8% and 45% less time, respec- tively. Details Scction and Section B.5
So far, prior works have mainly patched these problems with alternative parameter initial- ization schemes (Petersen al. 2024 Yousefi et al. 2025). But these remedies do not fully resolve the issues, as they neglect that the primary root cause lies in the parametrization of logic gate neurons themsclvos. For that reason, scaling the convolutional DLGN from Petersen et al. (2024) in depth still grossly degrades its discretized accuracy (cf. Figure 5b).
In this work, tackle the DLGN parameterization, study how it gives rise to the prob- lems mentioned above, and propose a reparametrization that overcomes the problems; the reparametrization is illustrated in Figure 2 Over and above, explicate the impact that

---

Published as a conference paper at ICLR 2026
initializations have gradient stability and optimization dynamics in deep logic gate net- works. In particular, we identify RIs as proposed by Petersen et al. (2024) as one of the simplest instances of larger class of aagaaiion-symmmezeee heavy-tail inifializations, and elucidate why such initialization schemes are particularly beneficial for the information flow in both the forward and backward pass during training. Combining such initializations with the reparametrization, we overcome the issues mentioned a above and obtain logic gate networks that are more expressive, more scalable, and more efficient train (cf. Figure I
Petersen et al. 2022) showed that DLGNs can process one million MNIST or CIFAR-10 im- ages per second on a single CPU core, Petersen et al. (2024) later showed that convolutional DLGNs take less than 10 nanoseconds per CIFAR-IU image on an FPGA, and Bührer et al. (2025) showed that recurrent DL.GNs require 20'000 times fewer logic operations deliver performance comparable to RNN, GRU, and Transformer-baser models in the WMT14 German to English translation task (Bojar et al., 2014b). These values show that DI- GNs are very suitable for real-world deployment once the accuracy matches state-of- of-the-art models. To facilitate the research needed to close this accuracy gap, our reparametriza iion makes training more efficient without altering the inference dynamics that make DLGNs attractive. We find that models require 4x less VRAM to train, process backward passes up to 1.86x faster, and 8.5x fewer training steps.
BACKGROUND ON LOGIC GATE NETWORKS & RELATED WORK
2.1 LOGIC GATE NETWORKS
In essence, differentiable logic gate nelworks differ from feed-forward neural networks in the parametrization of each neuron. In standard architectures, each neuron is composi- tion of vector vector-algebrann algebraic operations with non-linear activation functions (Fukushima 1980) Schmidhuber 2015) LeCun et al. 2015 Goodfellow et al., 2016). By contrast, differen- tiable logic gate networks (DLGNs) associate each neuron with a binary Boolean function 2022). That way, each neuron connected to only two neurons in the previous layer. Combined with bit-level operations, this extreme sparsity renders DLGNs particularly suitable for high-performance inference on devices with low computational resources.
G: 10 (0, 1} (Petersen et al.
Adhering to the canonical ordering of Boolean functions (cf. Table 5 we denote the 16 binary Boolean functions by G 1 & & 16. We categorize these functions based on the number non-zero outputs into four ANDs, four ORs, two constants, two XORs, and four pass-throughs, which merely forward one of the inputs, negated or non-nogated. A layer of such ncurons is referred as the logic layer.
Naturally, the space of Boolean functions and the functions themselves are discrete, and thus do not immediately give rise to differentiable neurons. To apply gradient-based optimization methods, the original paper proposed to cont relax each neuron to the probability simplex over all 16 functions (Petersen et al. 2022
16
g(p. 2) = Σ (..(p..), p.g.E 1J, wi w. 0,
(1)
where each function gi is a probabilistic surrogate of the deterministic G₁, defined as g..pg))=
E [G],A [.. B-Ber(g)
(2)
A-Ber(p).
Such a surrogate is necessary to deal with real-valued inputs p,g during training, for which the underlying G are not defined. Accordingly, we will refer w; as the weight of gi-
Moving back to the parameters of each neuron, the authors decided to initialize the weights in Equation [[] via softmax of ii.d. random variables
exp(1,) w: E,Exx(())))
Ω i.d.d N10,0²).
(3)
2

---

Published as a conference paper at ICLR 2026
Likewise, softmax operation is used to eventually obtain differentiable class scores from this network for classification tasks. In particular, for C classes, a layer coined GroupSum partitions the gate outputs of the final logic layer into C contiguous bins and accumulates them to obtain the corresponding logits.
At inference, all these softmax operations replaced by argmax operations. This rounds cach ncuron to the binary gate with the highest weight wi, which yiclds a logic gate circuit that can be directly embedded in hardware such as FPGAs or ASICs (Zia al,, 2012). Naturally, this rounding operation entails a discretization error that might lurther reduce performance deployment. We hence refer both versions of the network as the continuous and discretized DLGN.
Contending with both this discretization error and vanishing gradients, Petersen et al. (2024) observed superior performance when they replaced the Gaussian initialization in Equa- tion (3) by an RI, which deterministically assigns a high initial weight to the pass-through gate lunction G4(A, A,
i= 10, i 4
5,
Ω =
1, 16.
(4)
Similar to the original idea of residual connections (He et al., 2016). this pass-throus through bias stabilized training. On top of that, it notably reduces the number of non-trivial logic gates that remain after the discretized DLGN undergoes a logic simplification. That way, they obtained logic gate circuit that achieves test accuracy of 85% CIFAR-10 with less than 29 million gates, which is far less than what competitive networks required (Petersen ct al. 2024 Sec. 5.1).
Albeit effective, this initialization is still subject to limitations that arise from the underlying parametrization, which will pinpoint in Section 3 But first, present other related work and explain how they differ from DLGNs in both their reparametrized and original form.
2.2 OTHER RELATED WORK
Several works have exploited that learning circuits of logic gates with more than two inputs allows for embedding more functional expressivity on the same hardware (Umuroglu et al. 2020 Bacellar al.) 2024). On the contrary, DLGNs were practically limited learn logic gales with very few inputs, as processing 22 parameters per logic gate with a inputs quickly becomes intractable With our reparametrizzation that reduces the number of parameters to 2", advancing DLGNs to process more than two inputs per gate becomes viable option.
In contrast to our reparametrization, these works do not directly estimate the outputs of the logic gates. Instead, they use a different representation class and quantize this class logic gates after training. However, these indirect represcntations either fall short of exploiting the expressivity of logic gates (Umuroglu et al., 2020) or are costlier to paramctrize (Andronic
& Constantinides
2023 2025 We provide a detailed comparison in Section H
REPARAMETRIZING LOGIC GATE NEURONS
3.1 WRAKNESSES OF THE CURRENT PARAMETRIZATION
We demonstrate that redundancies in the parametrization are the primary cause of vanishing gradients and large discretization errors.
3.1.1 GRADIENT STABILITY
Each Boolean function G; has a negated counterpart. Adhering to the canonical ordering of Boolean functions (cf. Table 5), denote this counterpart by G-, := GI7- 1 G, = GG, The same holds for the probabilistic surrogates 9,. Under this condition, choosing independent weights for each yi and its negated counterpart, g- is fatal, as it provokes self-
3

---

Published as a conference paper at ICLR 2026
cancellations in the partial derivatives, progressively diminishing the gradient norm during backpropagation.
To see this, equally denote w-t W17-1 to expose the symmetry in Equation
as
g(p.)) wigi (p, 9) Σ -wig-i(p.
(5)
wige(p. + w ((1 (1
(6)
Having i.dd. Wa this translates to a weighted sum of sign-symmetric random variables in the partial derivatives
Og(p.g) ②) 0p
Σ (w) wii) (p. ōp
(7)
Initializing S2 with the default variance 1.0 will concentrate the gradient norm around O (cf. Figure 23ah and entail vanishing gradients with high probability, as Petersen et al. (2024) have already encountered. Notably, even with a variance as large as o² 16.0, many partial derivatives remain concentrated at zero (ef. Figure 23b
RIs as proposed by Petersen et al.
2024
successfully break this sign-symmetry to 0: a.. (g. 4)
Ōg(p, 4) Ōp
(p:
04 c + 15
Σ
(8)
Op
Σ, e4 0p
Once more, symmetric overparametrization traps RIs in tension between maintaining gradient stability and stalling optimization for other gate functions (cf. Section F.1
While sign-symmetries interfere with the gradient signal in destructive way, there are also other redundancios the parametrization that contribute to the discretization error.
3.1.2 DISCRETIZATION ERROR
When converting the continuous relaxation a logic gate circuit, the softmax- to-ggaa-------axxx rounding principle (cf. Section 2.1 discretizes each neuron to the logic gate function with the highest weight W/. But with the redundancies in this parametrization, the logic gate that is rounded to is not necessarily the one that the neuron is closest to. For example, assume neuron with weight 0.4 for the one pass-through gate G4 B) A, weight 0.3 for the other pass-through gate Go(A, B) B, and weight 0.3 for the OR function Gs(A, B) Av B. For the four binary inputs (0, 0), (0, 1), (1, 0), (1, 1), the neuron will output 0, 0.6, 0.7, Clearly, this output behaviour is closest to the OR function Gş, although the argmax is the pass- through gate G4. Argmax effective only when applied to inputs that are exclusive and independent.
Redundancies in the parametrization are the leading cause of vanishing gradients. In the following, we present exact, redundancy-free paramctrization.
3.2 INPUT- -ISE PARAMETRIZATION
In fact, each binary function G {0. 1)2 - (0.1) has a unique decomposition
G(k, E) Eoo E) auz Fo1 (k: E) + E10 () En
(9)
where ai {0, 1}, and Eg is the indicator function (k, 1(kk, f) (i.j)). This exact representability also transfers to the probabilistic surrogates
g Cpp + 001 + 010610 + @11611
(10)
where ()) E[E, (p. g)] as in Equation 2 Relaxing the binary coefficients ay to
the contimuous interval Wiz [0, 1] and rounding back via Wij > 0.5, we obtain the exact parametrization
4

---

Published as a conference paper at ICLR 2026
0,
0,
SOFT MAX
1, 0
X
Σ
1, 1
Ω
(a) Original parametrization
Input-wise parametrization
Figure 2: Illustrating the reparametrization for logic gates with one input. It requires only 2n learnable parameters for inputs, opposed 22 for the original paramctrization.
(p. = (1 - p) q) W00
(11)
+(1 p) W01 +
① W10
+
Wi1-
Similar to Petersen et al. (2022] one could learn such a bounded coefficient wa E [0, I] by mapping a real parameter 12,, E to an activation function p R - [0, 1]. We will defer
the specific function choice to Section C.1.1 and stick with the standard sigmoid function
for now, i.e. p(x) I+exp(-a)
Since the basis of the class of Boolean functions with n inputs has cardinality 2", this equally expressive parametrization requires logarithmically fewer parameters than the soft- max parametrization used by Petersen et al. (022)) which assigns an individual parameter to each of the 22" Boolean functions. For the class of binary functions used here, this already shrinks the model size by factor of 4, and renders lcarning higher-dimenssonall Boolcan functions computationally more viable. We hence also refer to this reparametriza- tion as inse paramelrization (IWP), and use the abbreviation OP for the original parametrization.
3.3 No GRADIENT STADILITY WITHOUT APPROPRIATE INITIALIZATIONS
We now show that the IWP climinates the pathways causing gradient cancellations and discretization errors. Any remaining gradient instability arises from other architectural factors, particularly parameter initialization.
To begin with, rounding the outputs of to their closest binary numbers clearly attains minimal errors with respect to any Minkowski norm and any other norm that based on uniform distance metric between outputs of the function. Proof in Section F.3
Moving on with gradient stability, the partial derivative now becomes
(g. (1 (w10 - woo) + g(w11 wo1)
(12)
Op
E [WiB woBl.
(13)
B-Ber(g)
An i.id. parameter initialization with sufficiently low variance would still entail cancella- tions, but, opposed to the OP, the IWP itself does not compound this problem further. Heavy-tail initializations that concentrate most weights wij close to 0, would already re- solve these cancellations inside neuron to a sufficient extent, as we explain exhaustively in Section F.4 But a heavy tail alone is not enough in general. As long as the parame- ter initialization treats each function and its negated counterpart independently, gradients will distribute iignsyymmttrically between different neurons during backpropagation. The more subsequent gates neuron passes its output to, the more likely it is that the sum of partial derivatives that it receives during backpropagation will concentrate at 0. Therefore, appropriate initialization schemes should be negation-asymmelric well.

---

Published as a conference paper at ICLR 2026
A residual initialization (RI) as pro- posed by Petersen et al. (2024) that assigns high imitial bias to the pass-
0.5
through Ga(A, B) A a sim- ple instance satisfying both require- ments. More complex instances, like an AND-OR initialization that concen- trates each gate cither to the AND or OR function, are also feasible princi- ple. However, it turns out that RIs en- tail a gate output distribution (cf. Fig- ure 3) that organizes the optimization of logic gate networks consecutively from earlier to later layers, which is ad- vantageous for training deep networks. We substantiate this argument in Sec- tion F.4.1 where we study the class of heavy-tail, negation-asymmest tialization schemes in more detail.
Activation Value
0.4
Density
0.3
0.5
0.2
0.1
0
0.0
5
10
15
20
25
Logic Layer
Figure 3: Distribution of gate outputs for an IWP DLGN right after residual initialization (RT), aver- aged over 100 images of CIFAR-100. That way, RI postpones gate learning later layers until carlier layers more refined. This incremental refine- ment allows to learn complex deep networks.
ini-
To conclude, we pair our IWP with RIs and show that the result is more scal- able in depth and expressive complex- ity.
EXPERIMENTS
To verify our claims of better gradient stability, discrotization accuracy, and training effi- ciency of our IWP, we adopt the original DLGN models and the experimental training setup from Petersen et al. (2022). We also cover the models and experimental setup from Petersen et al. [2044 to show the benefits apply to CDLGNs well.
4.1 BENCHMARKS
In both works, the networks were evaluated on several image classification benchmarks, with CIFAR-10 (Krizhevsky] 22099) as the most challenging dataset. However, the shallow models used there already pertectly fit the training dataset after few iterations, which restricts the measurability of further expressive benefils when scaling the networks in depth. Thus, we decided lift the complexity of the task in two ways: Firstly, we transition to CIFAR- 100, 100-class extension of CIFAR-10 (Krizhevsky 2009). Secondly, employ random resized crops and horizontal flips as standard data augmentations Py'Torch Core Team 2023
We need to account for the 10-fold class increase in the final prediction head of the model. Apart from that, no further adjustments to the original experimental setup for CIFAR- 10 are required. The class increase can be encountered in two different ways. Following recommendations of [Petersen el. al. (2024 Appendix A.2), we explore both options; see Section D.2.1 for details.
As a trade-off between computational feasibility and expressiveness, we finally decided to consider the medium-sized M models for both papers. When refer to the DLGN and CDLGN in the experiments, we hence always mean the specific CIFAR-10 M architecture from Petersen al. (2022) Appendix A.1) and Petersen al. (2024 Appendix A.1.1), respectively. To estimate uncertainty, we train each model on three different seeds.
4.2 IMPLEMENTATION OF REPARAMETRIZATION
To implement our IWP and the adjusted initialization schemes in the given Python and CUDA implementation, we merely have to override the weight initialization and the forward and backward functionality according to Equations 11 and 12

---

Published as a conference paper at ICLR 2026
While we assumed the sigmoid function as the binary gate output estimator p in Equa- tion (11) for the sake exposition, we observed slightly superior expressivity with a rescaled sinusoidal estimator sinor (z) 0.5 + 0.5 sin and adopted that one for subsequent exper-
iments. See Section C.1.1 for details.
4.3 SCALING MODELS IN DEPTH
Eventually, we want to reliably assess how increasing model depth affects performance for both parametrizations. To scale both DLGNs and CDLGNs in comparable, architecture- agnostic way, we introduce a depth-scale parameter D e N, and obtain depth-scaled networks by placing D (convolutional) logic layers instead, where only one was placed in the original architocture. Section D.I presents implementation details of this depth scaling.
5
RESULTS
5.1 REPARAMETRIZATTON REDUCES VANISHING GRADIENTS
To begin with, vanishing gradients as the major hindrance for scaling DLGNs in depth, the input-wise parametrization drastically reduccs the shrinkage of gradient norm as we backpropagate over layers. As Figure [8a] showcases, the gradient norm undercuts machine precision after 16 logic layers already, and vanishes to 10-34 over 40 layers, when the OP is used. But as already discussed in Section [.3, an TWP alone without appropriate negation- asymmetric, heavy-tail initializations can not reduce the gradient norm shrinkage sufficiently and also ends up with an average gradient norm of 10-16 16 after 40 layers.
5.2 RESIDUAL INTTIALIZATIONS SCALE BEST WITH DEPTH
The residual initialization (RI), although biasing towards a single gate function only, proves most effective for training deep DLGNSs On the one side, all other single-gate biases quickly concentrate the gate outputs at one value (cf. Figure (27] where their gradients bocome 0 and stifle gradient flow (cf. Figure 24b). On the other side, some multi-gate biases appeared competitive alternatives to RI, such as the AND-OR initialization, which exerts theoretically more appealing anticoncentration that retains inputs close to 0 and over the layers (cf. Figure 27c). Nonetheless, these methods remain slightly inferior to RI in terms of both gradient stability (cf. Figure 24ah and accuracy (cf. Figure 26a While the former drawback is rather obvious because the pass-through gate G4 is unparalleled in retaining a uniformly high gradient of the latter relates to the more intricate discrepancy in the optimization dynamics that each of the two initialization schemes gives rise to. As discussed in Section F.4.2 RIs order optimization of neurons from earlier to later layers. On the contrary, AND-OR OR initializations allow for non-uniform updates of the four gate outputs for neurons at later layers right from the beginning. This additional freedom, however, scems not only detrimental to the accuracy of the continuous relaxation. Surprisingly, despite its anti-concentration, this alternative initialization grossly compounds to the discretization error as we further increase depth (cf. Figure 26b
5.3 REPARAMETRIZATION DRASTICALLY STRENGHTENS HYPERPARAMETER ROBUSTNESS
Hyperparameters needed to be carefully chosen for the original DLGN. Besides the choice of optimizer and learning rate, the softmax temperature of the GroupSun layer was already identified a highly sensitive parameter in Petersen et al. (2022). By contrast, we find that IWP DLGNS are far more robust. than OP DI.GNs against variations of these hyperparametors. When swapping the Adam optimizer for established alternatives such as Stochastic Gradient Descent (SGD), Nesterov Accelerated Gradient (NAG), or Adadelta, the test accuracy of the OP DLGN deteriorates drastically while the IWP DLGN maintains a stable discretized test accuracy of above 30% for all of them (cf. Figure 4aa. On top of that, the large discretization errors of the OP DLGNs become particularly apparent when shrinking the GroupSum temperature т (cf. Figure [b]. Conversely, raising the temperature already reduces the test accuracy of the continuous OP DI.GN. A complete
7

---

Published as a conference paper at ICLR 2026
Original
Reparametrized
Original
Reparametrized
31.020
20
30.540
30
34
20
20
10
Adam
NAG
SGD
Adadelta
(a) Choice of optimizer
(b) GroupSum temperature
Figure 4: Discrotized test accuracy of the CIFAR DLGN with 3-fold depth when varying hyperparameters, averaged over two sceds. Details in Section B.1
overview of continuous and discrete accuracies is provided in Table 2 We continue these
stability experiments in Section [.22 where we show that IWP DLGNs remain consistently
superior to the OP for different Tearning rates and binary input encodings aswell.
5.4
ORIGINAL PARAMETRIZATION SCALES WORSE WITH DEPTH
30.0
Accuracy (%)
Accuracy (%)
35.0
28.0
27.0
Reparametrized
30.0
Reparametrized
Original
Original
26.0
2
6
8
10
12
16
18
20
Depth Scale Factor
Depth Scale Factor
(a) DLGN architecture
(b) CDLGN architecture
Figure 5: Discretized test accuracy, averaged over three seeds, when scaling the CIFAR M DLGN Petersen et al. 2022) and CDLGN (Petersen et al.) 2024) in depth.
Reparametrized
300
Original
Original
140
200
40
u45
100
20
10
10
10
10
10
10a
Batch size
Batch size
(a) Forward pass
(b) Backward pass
Figure 6: Training times for the DLGN with 20-fold depth. Mcan and standard deviation were computed over 20 batches of CIFAR-100.
RIs also suppress the undesirable propertics of the OP but cannot fully level them out, demonstrating that inappropriate underlying parametrization can irreversibly condition shortcomings for optimization. The IWP addresses these weaknesses, and pairing it with RI achieves superior performance to the OP with RI. For one, the TWP with RI

---

Published as a conference paper at ICLR 2026
still retains higher gradient norm than the OP with RI (ef. Figure 8b For another, we observe a clear gap in the predictive performance as well. For the DLGN, this gap is already apparent with baseline depth scale D 1. Moreover, the IWP consistently maintains this gap, which the OP cannot close even with 20-fold depth (ef. Figure 5a and eventually plateaus roughly 28% test. accuracy. For the CDLGN, the shallow baseline network performs almost equivalently. But increasing depth now begins to expose a drastic performance gap that culminates in more than 1.3 times better test accuracy of the IWP for D = 5 (cf. Figure 5b).
We observe that this gap mainly attributable to large discretization error for the OP. Figure 9b shows that the accuracy of the continuous OP CDLGN trails the IWP only by a few percent. Unfortunately, the increasing depth ceases benefit performance for the IWP as well at some point, at least when not increasing the number of optimization steps. Henceforth, we suppose that this caused by shared underlying characteristics of the overall DLGN architecture, and discuss potential reasons later in Section 6
5.5 TRAINING EFFICIENCY
By reducing the number of real parameters per neuron from 16 4, we shrink the model size by a factor of (cf. Figure D. This reduction also reduces the working set size during the forward and backward passes the CUDA kernel. This advantage becomes particularly apparent for small batch sizes, where the parameter tensors dominate the memory footprint. For an laylr DLGN trained with batch size 1, we observe 1.86x speedup for the backward pass and 1.11x speedup for the forward pass (cf. Figure 6
However, for large batch sizes, as they are typically used during training of such large models, the parametrization plays increasingly negligible role in the overall memory and operation usage, and the relative speedup over the OP fades. We discuss further potential efficiency improvements in Section B.5
Besides the models being lighter, the significant benefit our IWP lies in the better gradient signal. In Figure 11 (Section B.5.1), , that IWP converges in 8.5x fewer training steps than the OP, and as the steps are slightly faster, this means we can converge more than 8.5x faster in terms of wall clock time.
5.6 CONSISTENT IMPROVEMENTS ACROSS VISION AND LANGUAGE BENCHMARKS
The superior performance of IWP DLGNs not particular CTFAR-100, but holds across several vision datasets (cf. Table D. We evaluate DL.NNs on five vision datasets ranging from the simple MNIST to the more challenging vision dataset ImageNet332 (Krizhessky
2009 LeCun et al. 2010 kovsky et al.
2015 Xiao et al. 2017], and on the NLP English to German translation task (Bojar et al.) 2014a For contextualization, we compare the
achieved vision accuracies to vanilla convolutional neural network (CNN) (Krizhevsky let al. 2012) with 2 convolutional layers that get as input the same quantized low-resolution
inpuls the DLGNs.
Model
ImageNet32
C1FAR-100
CIFAR-10
Fashion-MNISII
MNIST
WMT'14
DLGN OP
4.84 0.02
27.7 + 0.05
55.33 1 0.23
81.39
0.07
92.43 0.17
15.11 1 0.64
DLGN IWP
4.93 1 0.02
29.5 - 0.02
57.47
1 0.20
82.34 - 0.15
94.02 0.08
17.38 1 0.04
CNN
5.19 1 0.37
39.2 - 0.07
64.01 1 0.16
77.66 + 0.85
92.91 1.61
Table 1: Discretized test accuracy (%.1) across vision datascts for the DLGN, compared to
a vanilla 2-layer CNN (Krizhevsky et al.) 2012), and corpus BLEU scores (t) for the NLP 1MT'14 English to German task. For ImageNet32, we consider DLGNs with 3-fold depth. Details in Section D44
9

---

Published as a conference paper at ICLR 2026
6 DISCUSSION
IWP DLGNs are not prone to performance degradation for increasing depth, as they mitigate the discretization error and improve gradient stability. On top of that, they are more robust against variations in optimization and input distribution hyperparameters. However, scaling these networks in depth did not yield large expressivity benefits. And DLGNs still have considerable generalization gap despite data augientatiios. We want to discuss how to overcome both problems in the following, and present further avenues for future research.
6.1 REMAINING EXPRESSIVITY BOTTLENECKS IN DLGNS
Although scaling DLGNs in depth provides slight benefits, the expressive advantage of deep DLGNNs fades beyond a certain depth despite the IWP. This is expected, as the CDLGG baseline with depth D already contains learnable gate layers. Reducing initialization variance does not alleviate this expressivity bottleneck (ef. Figure [Oap. We hypothesize that this limitation does not arise from the expressivity of DLGNs. Instead, we identify possible bottlenecks in the randomized, fixed connection topology and input preprocessing. We expose the former bottleneck in Section B.7 where we show that DLGNs fail to leverage higher imput resolution in contrast to CNNs. The latter bottlencck can be seen from Table where the vanilla CNN performs comparably to DLGNs when provided with the same low- resolution quantized inpul data. An encoding-aware connection heuristie or even learned connections, as in Bacellar et al. (2024), might overcome these limitations.
6.2 CLOSING THE CENERALIZATION GAP OF DLGNS
Even before discret ization, TWP DLGNs only slightly outperform the OP on test accuracy, despite a substantial increase in the training set (cf. Figure 9b. Dataset augmentations alone do not close this gap, and standard techniques like dropout (Srivastava et al. (2014), random interventions, or residual connections (He al.) 2016) fail improve test performance (cf. Section G). Designing constraints that promote generalizable functionality in DLGNs remains an open problem.
6.3 LEARNING GATES WITH MORE INPUTS
As discussed in Section 2.2 advancing DLGNs to learn logic gates with more than two inputs finally becomes a viable option. We showcasc several benefits for DLGNs with six- input gates, namely stronger expressivity (cf. Figure 14) and another 8.4x convergence
speedup in the number of training steps (cf. Figure 15 Moreover, arity 6 DLGNs could
also result more efficient hardware embeddings on modern FPGAs that lypically admit
six inputs to their lookup tables (Bacellar et al. 2024 Zia et al. 2012
We leave this avenue
to be explored in future research.
7 CONCLUSION
We proposed an input-wise parametrization (IWP) of logic gate networks with tailored initializations that allow scaling DLGNs in depth without degrading performance, while reducing parameter count logarithmically in the number of inputs per gate. Moreover, the IWP is significantly more robust against variations in hyperparameters and reduces the training time. We demonstrate the benefits of IWP across vision classífications (MNIST to Tm)g and language translation task (WMT'14 English to German). That way, it facilitates research for learning logic gate circuits that are not only far deeper, but also far more complex per logic gate by increasing the number of gate inputs.
Finally, closing the generalization gap in DLGNs has become pressing problem because the IWP and the higher gate arity notably increase the expressivity of DLGNs. But in view of the appealing perfommanc--ofictenccy trade-off, DLGNs continue to lend themselves for deployment on computationally restricted hardware like real-time systems or edge devices, and advancing their potential remains a promising avenue for future research.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY
The source code of IWP DLGNs and the associated experiments is made available soon GitHut There, step-by-step guide explains how set up the runtimo environment in which we conducted our experiments, and how to reproduce any particular experiment in this environment. That way, we hope to make our experiment infrastructure as conveniently accessible as possible. To guaranlee the reproducibility of our experiments, we restrict PyTorch to deterministic algorithms and fix the seeds of the random number generators that are used for the randomized initialization of weights and connections and for data loading.
REFERENCES
Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overparameterized neural networks, going beyond two layers. In H. Wallach, H. Larochelle, A. Beygekzime, F. d' d'Alché-Buc, E. Fox, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https:/ neurips httppscccppppp.ppppppp htcc/apeerfflls/ppppeppeee /tt.e///ppppe//0///111ee 22aabe7766733402277222776666
pdf
Marta Andronic and George A. Constantinides. Polylut: Learning piecewise polynomials for ultra-low latency FPGA lut-based inference. In International Conference on Field Programmable Technology. ICFPT 2028, Yokohama, Japan, December 12-14, 2023, pp. 60-68. IEEE, 2023. doi: ELL109/ICFP59550555 S0T.PP/55.052222..0000 2023.00012. 1109/ICFPT59808 2023 2023.00012
URL https://doi. http:////oo.rrg//... org/10.
Marta Andronic and George A. Constantinides. NeuraLUT-AAsemble Hardwarr-Await Assembling of Sub-Neural Networks for Efficient LUT Inference. In 2025 IEEE 33rd Anroual International Symposium on Field- Field-Programmnnn Programmable Custom Computing Machines (FCCM), pp. 208 216.. TEEE, 2025. doi: 10. FF100//CCCM62333 202022273..22200077..
Marta Andronic, Jiawen Li, and George A. Constantinides. Polylut: Ultra-low latency poly- nomial inference with hardwarc-aware structured pruning. IEEE Trans. Computers, 74
(9):3181-3194, 2025. doi: N.1100//CC.222.333366 URL http::///doo.rg///... TC.
Alan Tendler Leibel Bacellar, Zachary Susskind, Mauricio Breternitz Jr, Eugene John, Lizy Kurian John, Priscila Machado Vieira Lima, and Felipe M.G. França. Differ- entiable weightless neural networks. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 2277 2295. PMLR, 21 27 Jul 2024. URL https://prooceednng: mlr. ppress/vv35/bacellar2222 html
Adrien Benamira, Thomas Peyrin, Trevor Yap, Tristan Guérand, and Bryan Hooi. Truth table net: Scalable, compact and verifiable neural networks with a dual convolutional small boolean circuit networks form. In Kate Larson (ed.), Proceedings of the Thirty- Third International Joint Conference on Artificial Intelligence, IJCAI-24, pp. 13-21. International Joint Conferences Artificial Intelligence Organization, 8 2024. doi: 144963/jjaai.222//
URL https:///oi. org/10 24963/1fcai. 2024/2
Main Track.
Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Kochn, Jo- hannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Ale s Tamchyna. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the Ninth Workshop on Slatistical Machine Trans- lation, pp. 12 58, Baltimore, Maryland, USA, June 2014a. Association for Computational Linguistics. URL htRl://ppp/www..ccooooo/////a/go
http:///ttthhhoomm///sasmeeeettt

---

Published as a conference paper at ICLR 2026
Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Jo- hannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Amand, Radu Soricut, Lucia Specia, and Ales Tamchyna. Findings of the 2014 workshop on statistical machine translation. In Ondřej Bojar, Christian Buck, Christian Federmann, Barry Had- dow, Philipp Koehn, Christof Monz, Matt Post, and Lucia Specia (eds.), Proceedings of the Ninth Workshop on Statistical Machine Translation, pp. 12 58, Baltimore, Maryland, USA, June 2014b. Association for Computational Linguistics. doi: 10.3115/v1/W14-3300 URL
wrg/W14-3302/
Simon Bührer, Andreas Plesner, Till Aczel, and Roger Wattenhofer. Recurrent deep differ- entiable logic gate networks, 2025. URL httts://aaxii: www.aawwww.....aaaaaooooaaeoooee httpg//bs/2508.0000 06097
Hugo C.C. Carneiro, Felipe M.G. França, and Priscila M.V. Lima. Multilingual part- of-speech tagging with weightless neural networks. Neural Networks, 60:11-21, 2015.
ISSN 0893-6080. doi: https://doiie
www.:e................... 2015.02.012 URL https: /ettp::/www...
Kunihiko Fukushima. Neocognitron: A self-organizing neural network model for a mech- anism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36:
193-202, 1980.
Tan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016.
httt:://ww..deepeermmnngoooo org
Julia Gusak, Daria Cherniuk, Alena Shilova, Alexandr Katrutsa, Daniel Bershatsky, Xunyi Zhao, Lionel Eyraud-Dubois, Oleh Shliazhko, Denis Dimitrov, Tvan Oseledets, and Olivier Beaumont. Survey efficient training of large neural networks. In Lud De Raedt (ed.), Proceedings of the Thirty- First International Joint Conference on Artificial Intelligence, IJCAI-22, pp. 5494-5501. International Joint Conferences Artificial Intelligence Or- ganization, 2022. doi: 10.24963 124966//jcci..22 (ijcai.2022//111 ijcai 2022/769 Survey Track.
URL https://dol: org/10. /4963/
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Im- age Recognition. In 2016 IEEE Conference on Compuder Vision and Pattern Recognition (CVPR), pp. 770 778, Las Vegas, NV, USA, June 2016. IEEE. ISBN 978-1-4673-8851-1. doi: (0.110//CVPR.20
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran Eaniv,, and Yoshua Bengio. Binarized neural networks. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Newral Information Processing Systems, volume 29, Curran Associates, Inc., 2016. URL htttps://proceeednngg// www.eediiiinnnnnuuuuuee neur cc/paper f BB5300555T77773222770006666666-3 .pdf]
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technicol Report, University of Toronto, pp. 32 32-33, 33, 2009. URL https://www. cs toronto..oo... edu/~kriz.
Seaniing-eeturess 2009- pdf
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), Advances in Newral Information Processing Systems, volume 25. Curran Associates, Inc., 2012. URL G3962289667666666662444442-22222
ptpppppeeecpppppprrrrr..... httpe/cappppeeeeeeeeaaeeeeeeeeee filess/ppper/2022/1110
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann,.eee hww.///wnnnnnnnnn.eemm www.aa.coooee..aeeeeeee cow/ezdb/ hte////db//iis, 2, 2010.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553): 436-444, 2015.
Ryan O'Donnell. Analysis of Booleon Functions. Cambridge University Press, 2014. ISBN 9781107033325. First edition published June 2014; corrected via arXiv version 2021 (10301
12

---

http://tooo.og//..1.1//. 11t///0001117/1110/22 Published as a conference paper at ICLR 2026
Felix Petersen, Christian Borgelt, Hilde Kuchne, and Oliver Deussen. Deep dif- ferentiable logic gate networks. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Informa- lion Processing Systems 35: Annual Conference on Neural Information Process- ing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 Decem- wwweee/pppee/00///
ber 9, 2022, 2022. URL http://papers: nips. oatpe//eeeeaa990900000000000eeee
Felix Petersen, Hilde Kuehne, Christian Borgelt, Julian Welzel, and Stefano Ermon. Convo- lutional differentiable logic gate networks. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Ad- vances in Neural Information Processing Systems 38: Annual Conference on Neural In- formation Processing Systems 2024, NeurIPS 2024, Vancowver, BC, Canada, December 10 15, 2024, 2024. URL http://papers. mipppprr..nipss..ommmm cctpaper_ppiiaeeeeceeee Filesspaper/200/0 hashh/ html
PyTorch Core Team. PyTorch Documentation. PyTorch, 2023. https://pytorch.org docs/
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Compuler Vision (IJCV), 115(3):211 252, 2015. doi: 10.1007/s112663-015-00006
Jürgen Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61: 85-117, 2015.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhut- dinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1999-1958, 2014. URL http://jmlr. (eljjillrrrr../..em..m..eeoo /tg/papers/ v15/srivastaval4a.1 html
Yaman Umuroglu, Yash Akhauri, Nicholas Fraser, and Michaela Blott. Logicnets: Co- designed neural networks and circuits for extreme-throoghpuue applications. In Proceedings of the International Conference on Sield-Programmaabe Logic and Applications, pp. 291- 297, Los Alamilos, CA, USA, sep 2020. IEEE Computer Society.
V. N. Vapnik and A. Ya. Chervonenkis. On the uniform convergence of relative frequencies of events their probabilitics. Theory of Probability & Its Applications, 16(2):264-280, 1971. doi: 10.1137/1116025, URL
Erwci Wang, James J. Davis, Peter Y. K. Cheung, and George A. Constantinides. Lut- net: Learning fpga configurations for highly efficient neural network inference. IEEE Trunsactions Compulers, 19(12):1795 1808, 2020. doi:
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms, 2017. URL https://arxiv.. wwwaariii..ogaaaaa. wrg./bs////aa...//e///s// 07747.
Shakir Yousefi, Andreas Plesner, Till Aczel, and Roger Wattenhofer. Mind the gap: Re-
moving the discretization gap in differentiable logic gate networks, 2025. URL https:
//arxiv. wtwpi//orrgabs//255006 07500
Razia Zia, Muzaffar Rao, Arshad Aziz, and Pervez Akhtar. Efficient Utilization of FPGA Using LUT-6 Architecture. Applied Mechanics and Materials, 241-244:2548 2554, 12 2012. htwp///wwttifficc..m.../ Stt.iittffi.tteAAMMMMM1-11-2222. AM4./MMM.2111244225555
doi: 10.4028/ 10.4028/www.scimmt///
13

---

Published as a conference paper at ICLR 2026
T
IWP DLGN (cont.)
IWP DLGN (disc.)
OP DLGN (cont.)
OP DLGN (disc.)
3
26.53 + 0.05
18.64 + 0.01
26.21 + 0.20
12.48 + 0.05
10
29.13 + 0.12
27.05 + 0.12
28.02 + 0.04
25.49 t 0.11
30
32.33 + 0.25
32.46 + 0.08
30.99 + 0.04
31.02 + 0.09
100
24.75 0.01
24.81 + 0.02
21.52 + 0.12
21.52 + 0.13
300
18.25 + 0.12
18.25 + 0.14
14.85 + 0.04
14.84 + 0.03
Table Test accuracies (continuous and discretized) for CIFAR DLGNs with 3-fold depths when varying the GroupSum temperature While the continuous test accuracy of the OP DLGNs closely tracks the IWP for smaller temperatures, the large discretization error exposes clear gap in the discretized accuracy compard to the IWP. For higher temperatures, the OP DLGNs are already worse in their native continuous version. Mean and standard deviation are evaluated on the CIFAR DLGN with 3-fold depth trained on two seeds.
A USAGE OF LLMs
We have used LLMs to polish the writing of this paper and for code generation through chats, Cursor, and Claude code. ChatGPT, Claude, Gemini, and Grammarly were employed for spellchecking, refining and condensing text, and reviewing to improve clarity and readability, Furthermore, ChatGPT, Claude, and Cursor were used to assist with code completion and generate visualizations. These tools served as auxiliary aids for writing and implementation, while all core research ideas, experimental design, and interpretation of results are our own.
B FURTHER EXPERIMENT RESULTS
B.1 SUPERIORITY ACROSS HYPERPARAMETERS AND INPUT DISTRIBUTIONS
B.1.1 OPTIMIZER
All new optimizers, i.c. SGD, NAG and Adadelta, use the same learning rate of 100. Lower learning rales slowed down convergence for both the OP and IWP. Figure 7b substantiates that the IWP does not only previal on this particular learning rale, but remains superior across this range of learning rates.
B.1.2 GROUPSUM TEMPERATURE
To convert the real-valued inputs x E [0, 1] to binary encodings, Petersen et al. 2022) adopt the thermometer encoding Ith. (z fi,t t2 .......2 > tx), where = i/k evenly spaced thresholds in [0, 1] (Carneiro et al.] 2015). We extend our experiments beyond this particular choice and evaluate DI.GNs
diverse range of encodings, namely
1. binary number system encoding, where is express 2-moduli at various granularity,
2. a graycode encoding, where is express how centered the value at various subin- tervals, and
3. sparse interval indicator encoding, where is placed only for the interval which the value is contained in.
For any of these encodings, the IWP attains superior performance (cf. Figure 7a The same holds for deviating the learning rate from the default 0.01 (cl. Figure 7b
B.2 VANISHNG GRADIENTS DEEP DLGNS
While the IWP eliminates cancellations inside neuron, cancellations between partial deriva- tives of different neurons are out the control of the parametrization. For that reason, IWP alone does not reduce the gradient norm shrinkage sufficiently, and also ends up with
14

---

Published as a conference paper at ICLR 2026
Original
Reparametrized
Original
Reparametrized
35
32, 16.
2LAg01
NARZOLI
30
28 940.
25
211
15
11
Thermo
Graycode
Binary
Interval
(a) Choice of binary encoding
(b) Choice of learning rate
Figure 7: TWP DLGNs achieve consistently superior lest. accuracy across variations of the experimental setup. Here, we compare different input encodings and deviate the learning rate from the default value 10- that was used in Petersen al. (2022).
10->
Gradient Norm [log scale)
10-1
Gradient Norm [log scale)
IWP RI
OP RI
10-12
10-1
10-1
10-7
IWP
10
OP
10
OP
10-1
12
18
24
30
36
0
30
36
Layer
Layer
(a) IWP with inappropriate initialization
(b) IWP with RIs
Figure 8: Gradient norm decrease of an IWP DLGN with 40 layers.
average gradient norm of 10-16 after 40 layers. Avoiding this requires appropriate negation- asymmetric, heavy-tail initializations as already discussed in Section 3.3
B.3 DISCRETIZATION ERROR OF OP
The discretization error is major reason for the performance docrease of the OP for deeper models. For five-fold depth, the discrelization gap is already substantial for both the DI.G and CDLGN architecture (cf. Figure Dh
B.4 DEEP NETWORKS DO NOT REQUIRE LOWER INITIALIZATION VARIANCE
Deeper IWP CDLGNs with RIs do neither converge faster nor improve test accuracies (cf. Figure 10al when lowering the initialization variance and concentrating the weights wiy closer to U, 1, as illustrated in Figure [0b] We believe that this is also attributable to the implicit organization of neuron optimization for RIs (cf. Section F.4.2..
15

---

Published as a conference paper at ICLR 2026
Original
Reparametrized
Original
Reparametrized
150.9
68-7-148
70.0
40.0
36.42000 69
60.0
30, 30,360226
2621022
52 2:0-0.19
30.0
290 200000.11
50.0
40.0
36,5=0 5FF
20.0
29 29.060.66
30.0
20.0
10.0
10.0
0.0
0.0
Train. (cont.)
Test (cont.)
Test (disc.)
Train. (cont.)
Test (cont.)
Test (disc.)
(a) DLGN architecture
(b) CDLGN architecture
Figure 9: Accuracies of the DLGN and CDLGN with five-fold depth on CIFAR-100
Std Dev
+: 1,2, 0.25
F.1.S.
0.025
2000
0.01/0.99
Density
15.0%
10.0%
J02
5.0%
20.0
0.0
(cont.)
(cont.)
(disc.)
0.0%
0.2
0.4
(a) Accuracy of CDLGN with 2-fold depth CIFAR-100
(b) More concentrated distribution for
1.5,
0.125
Figure 10: Reducing the initialization variance by concentrating the weights wiy even further at the tails 0, for deeper models does not improve performance.
B.5 TRAINING EFFICIENCY
B.5.1 FASTER CONVERGENCE OF IWP
We show in Figure 11 a roofline plot (running maximum) of the test accuracy for 20-fold depth models under IWP and the OP. We show in red the maximum accuracy of the OP, which is achieved after 222000 steps. Meanwhile, our IWP achieves this after only 26000 steps. Thus, we can converge 8.5x faster in the number of training steps. As shown in Figure D the steps under IWP are as fast or faster than the OP. Thus, we can also train more than 8.5x faster in terms of wall-clock time.
B.5.2 MINIMAL EFFICIENCY IMPACT OF GATE OUTPUT ESTIMATOR
We observe that the choice of the gate output estimator p(x:) has noticeable impact on both the runtime of the forward computation, but only minimal effect on the backward pass. We compare the sinusoidal gate output estimator p(x) = 0.5 0.5-sin(()) with a custom double-capped linear p(z) max(0, min(1,x)), whose gradient is set to I throughout. This not only avoids arithmetic operations during the forward and backward pass, but also alleviates memory requirements because the constant gradient does not require saving the particular input tensor for the backward pass. Although the forward pass speeds up by 22%, the computationally dominant runtime of the backward pass reduces only by 4% (cf.
16

---

Published as a conference paper at ICLR 2026
0.30
0.25
0.20
Accuracy
0.15
0.10
0.05
Reparametrized
Original
0.00
Best accuracy
50000
100000
200000
250000
Training Step
Figure 11: For the DLGN with 20-fold depth, we juxtapose the best discretized accuracy that has been achieved far during training for both parametrizations. The OP reaches its best accuracy after 222000 steps, which is indicated by the red roofline. The IWP already surpasses this accuracy after only 26000 steps. It hence achieves more than 8.5x faster convergence.
SIN
SIG-ST
LIN-ST
11.6114:21
75.20= 2.14+ 2.2..22...9
Relative Scale (per metric)
27.611887
25 13
0.8
0.6
0.4
0.0
Forward pass [us]
Backward pass [us]
Figure 12: Runtime of the forward and backward pass for IWP DLGN with different gate output estimators. 'Tee default sinusoidal estimator (SIN) is compared to straight- through sigmoid (SIG-ST) and the linear straight- through estimator (LIN-ST) as introduced in Section B.5.2 Measurements are taken for DLGN of 20-fold depth and are averaged over 20 batches with 25 CIFAR-100 instances.
Figure After all, we have not mcasured whether linear straight- estimator can meel the performance of the sinusoidal estimator.
B.6 LEARNING LOGIC GATES WITH HIGHER ARITY
The exact, redundancy- free reparametrization of logic gates finally allows to optimize circuits with gates of higher arities 2. This further leverages the expressive potential of modern FPGAs which typically admit n = inputs to their lookup tables (Bacellar et al. 2024 Zia et al.) 2012). To showease the computational feasibility, gradient stability and aptimization efficiency of training DLGNs with such an arity, we have implemented and profiled CUDA kernels for
4 and 6.
17

---

Published as a conference paper at ICLR 2026
Input probabilities
Output estimators
(1- (1-P))(L
(1))11122)1 (1-p:)(1-Pa))) (1p)
W000
Partial sums
Wpo
(1 (1-P1)(1 - -p2) ps
Wp00
Partial sumss
W/
x
(1 (1-p1) pi) p2 ps)
Wo10
W01
x
(1-P) pp) P2
P3
W011
Total
x
(1 - -1 - -p3)
W100
110
P1
(1 - p2) p3
W101
W11
P1
P2 (1 Pss)
0110
W11
X
P1
P2
Ps
W11
Figure 13: Computation graph of the forward kernel of a differentiable logic gate with arity 3. The tree-like accumulation order gathers terms with similar probability weights and thus mitigates numerical instability.
B.6.1 COMPUTATIONAL FEASIBILITY
Even for doublo-procision floating point computations and fully unrolled single scalar as- signment instructions, the register usage of the forward and backward kernels for gate arity remains below the maximum 255 registers per thread on modern NVIDIA GPUs. By contrast, OP DLGNs exceed this boundary for gate arily 3 already, where the number of parameters alone becomes 256.
Still, the number of registers is evidently larger than for arity 2 kernels, and do need to decrease the number threads per thread block by a factor of up to 4 for the backward kernels to meet the register constraints of an NVIDIA streaming multiprocessor. Moreover, the computation graph of cach logic gate deepens by a factor of 3. The overall runtime per training step thus increased by factor of 8.
B.6.2 ANALYTICAL AND NUMERICAL GRADIENT STABILITY
Computing the forward and backward pass boils down to a computing a weighted sum of the output estimators w. Recall Equation 11 for the case n 2. At low-level implementation in the forward and backward kernels, this accumulation is performed in tree-like accumulation order which we visualize in Figure 13 for n 3. This tree structure is not only a natural choice to minimize instruction dependencies and latency. It also ensures that terms have a similar magnitude when added together. For example, no matter how extreme pı and P2 concentrate towards 0 and 1, the tree accumulation consecutively accumulates "ncarest- neighbour" weights whose probability term only differs in Pa. (1 -P)). That way, the risk of precision loss due to numerical instability increases only moderately with higher arity.
18

---

Published as a conference paper at ICLR 2026
IWP, arity
IWP, arity
Relative Scale (per metric)
1.0
2.35=0.11
73511.1
32.5100
Gradient Norm (log scale)
10-1
IWP RI. arity
IWP RI, arity
0.8
0.220011
30"4
AV, arity
46.12110
0.6
10-1
0.4
0.0
10-1
Train Loss
Train (cont.)
Test (disc.)
0
24
30
36
Layer
(a) Oplimization metrics after 250,000 training steps.
(b) Initial gradient norms the first batch.
Figure 14: Juxtaposition of the gradient norm and prediction accuracies of IWP DLGNs with arity n and n 6.
Consequently, the average gradient norm remains similarly stable as for arity = 2, both for the residual initializations and the AND-OR initialization (cf. Figure 14b).
B.6.3 OPTIMIZATION EFFICIENCY
Increasing the gate arity from 2 to 6 drastically raises the model's ability to fit the training dataset in a small number of steps. However, the test accuracy improves only slightly (cf. Figure 14a We emphasize that this is not due to high discretization error but to high generalization error.
However, increasing the arity to 6 unleashes another 8.4 convergence speedup. Figure 15 showcases that the arity IWP DLGN reaches its best test accuracy after 210000 steps, while the arity 6 IWP DLGN already surpasses this accuracy after 25000 training steps only. However, this benefit in convergence speed remains on par with the eightfold longer runtime of each training step that was reported in Section B.6.1 However, the current CUDA kernels are just naive implementations without common optimizations such as pipelining or improved memory access patterns. Such optimizations could further improve runtime performance and eventually render arity 6 DLGNs more efficient.
Another caveat is that the arity model is larger, because each neuron in the arity 2 model only corresponds to a lookup-table (LUT) with 2 inputs, of which multiple can be merged into single LUT-6 during hardware embedding.
B.7 REMAINING ARCHITECTURAL WEAKNESSES OF DLGNS
B.7.1 RANDOMIZED CONNECTION TOPOLOGY FAILS TO EXPLOIT STRUCTURE IN ENCODING
To convert the real-valued inputs E J0, to binary encodings, Petersen et al. 2022 adopt the thermometer encoding Fzh := (z > f..t .2....,.'' tx), where t, i/k k are evenly spaced thresholds in [0, (Carneiro et al. 2015). The number of thresholds directly determines the discretization resolution, and an increase should hence further decrease the approximation error. For standard CNN architecture (cf. Figure 17 there is indeed noticeable improvement. But for the convolutional DLGN, such an improvement fails to appear (cf. Figure 16
Since the DLGN architecture does not lag behind the CNN architecture in expressivity (cf. Figure 16a), we hence locate the bottleneck in the random, fixed initialization of connections. In the early layers, an encoding-aware connection heuristic or even learned connections as in Bacellar et al. (2024) might overcome this limitation.
19

---

Published as a conference paper at ICLR 2026
0.3
Accuracy
0.1
IWP RI, anty IWP RI. arity
Best accuracy
0.0
0
50000
100000
150000
200000
250000
Training Step
Figure 15: Increasing the gate arity from 2 to 6 yields another 8.4x convergence speedup. We still consider the CIFAR DLGN with 3-fold depth. We juxtapose the best discretized accuracy that has been achieved so far during training for both arities. The best test accuracy of the arity DLGN is indicated by the red roofline.
95.0
CLGN
90.0
42.0
Accuracy (%)
Accuracy (%)
85.0
41.0
80.0
40.0
75.0
39.0
70.0
CLGN
38.0
20
20
24
Number of thresholds
Number of thresholds
(a) Training accuracy (cont.)
(b) Test accuracy (disc.)
Figure 16: Approximation improvement for increased resolution in the thermometer encod- ing for the CDLGN and standard CNN architecture.
B.7.2 EXTRACTING CHROMATIC INFORMATION FROM COLOR SPACE ENCODING
The randomized connection topology not only hinders from exploiting the structure in the binary encoding of each real-valued channel, but also mix information from different chan- nels. This might explain why switching from the standard RGB color space encoding used Petersen al. (2022 2024) a HSV color space improves performance on ImageNet32 (cl. Table 3). While the hue channel in HSV solely encodes the chromatic information, all channels in RGB need to combined to obtain a proper estimate of the chromatic value.
C IMPLEMENTATION DETAILS FOR REPARAMETIIIZATIO
C.1 ESTIMATION FUNCTION OF LOCIC GATE OUTPUTS
While the OP slightly benefited from weight decay (Petersen et al. 2024h, we have to disable it for our IWP. The reason is that for both the sigmoid and simusoidal estimators, weight w close to 0 corresponds to an undecisive gate output w 0.5. Weight decay hence actively encourages high discretization error and entails weaker performance at inference.
20

---

Published as a conference paper at ICLR 2026
Layer #
Description
Conv2D (in channels=..., out channels=256, kernel=3x3, padding-1)
ReLU
Conv2D (in_chamels=2556, out.chammels-61, kernel=3x3, padding=1)
ReLU
MaxPool2D (kernel=2x2, stride-2)
AdaptiveArgPPol2D (output.sizo=
Flatten
Linear (in features=512, out fealures=256) ReLU Linear (in_features=256,, out_features=100)
Figure 17: CNN Architecture built via torch. nn for CIFAR-100 classification in Figure
Color space
Test accuracy (cont.)
Test accuracy (disc.)
RGB
5.25 = 0.05
5.11 i 0.03
HSV
6.25 + 0.05
6.15 0.06
Table 3: Test accuracies (continuous and discretized) for CIFAR DLGNs with 3-fold depths on ImageNet32 for different color space encodings. Mean and standard deviation are com- puted from two seeds. We juxtapose the default RGB to the HSV color space, which isolates chromatic information in single channel.
C.1.1 SINUSOIDAL ESTIMATOR
Heavy-tail initializations as motivated in Section 3.3 can be adjusted by adjusting shift and 0.25, which results in
variance of the normal initialization. We choose / 1.2 and
distribution like Figure 18b
3,0%
8.0%
Mean = 0.05/0.95
2.5%
Density
2.0%
Density
Mean 0.50
1.5%
4.0%
1.0%
2.0%
0.5%
0.0%
0.0%
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
1.0
(a)
0.0,
1.0
(b) 1.2, 0.25
Figure 18: Initial distribution of coefficients wij when initialized with and without a RT for the sinusoidal output estimator, i.e. wiy 0.5 + 0.5 sin(Dy,) ~ (N),
C.1.2
SIGMOID ESTIMATOR
For the sigmoid estimator that is more commonly used in logistic regression, we can similarly adopt heavy-tail initializations by shifting the weights Pig by 3.0 (cf. Figure 19h.
C.1.3 PERFORMANCE AND GRADIENT STABILITY
Although the sigmoid function has been widely adopted for its theoretically desirable prop- erties, its gradients vanish faster for large input values. At the same time, the periodicity of the sinusoidal estimator avoids such dead end. But for heavy-tail initialization in
21

---

Published as a conference paper at ICLR 2026
6.0%
Std Dev
Sid Dev
8.0%
Mean 0.05/0.95
5.0%
Mean 0.07/0.93
6.0%
4.0%
Density
Density
3.0%
4.0%
2.0%
1.0%
0.0%
0.0%
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
(a)
3.0, a = 0.5
(b) 3.0, a
1.0
Figure 19: Initial distribution of coefficients wij when initialized with and without a RI for the sigmoid output estimator, i.c. wij = (1 + exp(S2,)) P45 ~ oN,,
Figure 19a that does not shift the weights too strongly into this flat region, the gradient is still sufficiently high allow deviation from the initialization region. Although the gradi- ent norm is initially smaller across layers compared to the sinusoidal (cf. Figure 20b we observe that the gradient norm recovers quickly after a few batches only and approaches the curve of the sinusoidal. However, logic gale networks with a sinusoidal estimator still achieve slightly superior accuracies (ef. Figure 20a), which is why eventually stuck with them.
10
30.0
Gradient Norm (log scale)
sigmoid
sigmoidal
IWP RI
29.5
Accuracy (%)
29.0
28.5
10
27.5
3 Depth Scale Factor
0
24
30
36
Layer
(a) Performance comparison.
(b) Average gradient norm over layers.
Figure 20: Performance and gradient stability comparison for the sigmoid and the sinusoidal gate output estimator.
D EXPERIMENTAL SETUP
D.1 SCALING DLGNs DEPTH
As the original DIGN uses a uniform width for all logic layers, we simply scale the DI.G in depth by placing D logic layers everywhere a logic layer was placed in the original architecture.
For the CDLGN architecture, we place block of D convolutional logic layers instead of one, but apply the max pooling layer only once at the end. Because the kernel size, padding, and stride in the original architecture (Petersen et al. 2024 Sec. 3.4) preserve the spatial dimensions of the data tensors, further adjustments DLGN, channel increases and decreases are only performed once at the initial and final convolutional logic layer of the block. Finally, we do not restrict. the CDLGN architecture to
needed. As for the original
22

---

Published as a conference paper at ICLR 2026
partition the range of channels into separate, independent streams motivated by Petersen et al. (2024 Sec. 3.4) for more efficient hardware embeddings and data movement during training, but allow connections to be formed between any combination of channels.
D.2 DEVIATIONS FROM ORIGINAL EXPERIMENTAL SETUP
Scaling DLGNSs in depth increases the overall computational cost for training. To ensure that gradient. descent. converges iterations from 200,000 to 250,000. Furthermore, when training sufficientyy deep CDLGNs, GPU memory limitations hinder us from loading batches of original size 100. 'o ensure comparable optimization conditions for these models, we hence employ batch accumulation for depths D > In particular, we accumulate four batches of size 25 in one backward pass for depths D 4, 5, and tested that it behaves identically to training on the original batch size 100.
for deep models, increase the number of training
D.2.1 10-FOLD CLASS INCREASE FOR C-FAR-100
The 10-fold class increase be encountered in two different ways: On the one hand, one could keep the final logic layer unchanged and accumulate 10 times fewer gate outputs per class in the GroupSum layer. Petersen et al. (2024) proposed the heuristic to shrink the softmax temperature by the square root of the class increase /10 in such case for optimal performance. On the other hand, one could increase the final logic layer to 10-fold width, which does not change the number of gate outputs per class and hence does not require any temperature adjustment.
For both the DLGN and CDLGN, increasing the width 10-fold further improves performance (cf. Figure 21 J. At the same time, decrcasing the temperature as proposed by Petersen et al. (2024) indoed maintainod optimal performance, with only minor changes when decrcasing
the temperature further by /IO (cf. Figure 21b
10W, 1.0t
W, 0.1
50.0
10W. 1.0g
10W, 0.15
W. 0.1
40.0
86.1
80.0
30/1
31.2
60.0
20.0
41.9
41.1
35.6
10.0
20.0
Train. (cont.)
Test (cont.)
Test (disc.)
0.0
(cont.)
Test (cont.)
Test (disc.)
(a) DLGN architecture
(b) CDLGN architecture
Figure 21: Performance comparison for differont final logic layer widths and temperatures
class score accumulation.
But for our experiments, we do not consider the choice between keeping the width and decreasing the temperature or increasing the width to keep the absolute temperature crucial one. The reason is that we merely focus on different parametrizations each neuron that leave their functional characteristies unchanged. We hence do not expect the trends that we observe when scaling these networks in depth to alter across these slightly varying widths of the final logic layer. To cover both options, choose to keep the width and decrease the temperature for the DLGN, and keep the temperature and increase the width for the CDI.GN.
23

---

Published as a conference paper at ICLR 2026
D.3 RUNTIME MEASUREMENTS
Our objective is to assess the runtime performance of both parametrizations in comparable way. To rule out possible discrepancies that are unrelated to the IWP, we build a Python subclass of the original classes for logic layers that execute both our TWP and the OP. At runtime, Boolean variable determines which parametrization is chosen. Apart from the different weight initialization and the invocation of the custom autograd function, the footprint of this algorithm on the machine is hence identical. We mcasure the past nanoscconds for an entire forward and backward pass each, and enforce synchronization at both time points to ensure that the total computation of all streams on the GPU is captured the time measurements.
To quantify uncertainty, we take measurements for 20 different, randomly sampled batches.
D.4 DLGN ARCHITECTURES FOR OTHER DATASETS
D.4.1 IMAGENET32
For ImageNet, we use the same DLGN architecture and the same data augmentations as for CIFAR-100. To accomodate the 10-fold increase classes, we downscale the temperature by 10 as proposed in Petersen et al. (002)). For both the CNN and DI.GNs, the RGB channel values are quantized with 7-threshold thermometer encoding. Note that we only used thresholds for CIFAR-100, However, Figure [6 demonstrates that increasing the number of thresholds benefits the CNN more than the DLGN architecture.
D.4.2 CIFAR-10
For CIFAR-10, use the baseline CIFAR-10 M. LGN architecture from Petersen et al. 2022 with layers of 128, 000 neurons each, just as we have done for CIFAR-100.
D.4.3
CIFAR-100
For CIFAR-100, we presented the results for the DLGNs with 3-fold depth.
D.4.4 MNIST AND FASHION-MNIST
For the simpler gray-scale datasots, we use 1-threshold thermometer encoding and smaller width of 32, 000 neurons, was done in Petersen et al. (2022) The resulting model is shown Figure 22
Layer # Description
Encoding (type=thermometer) resolution=1) LogicLayerIWP (in 784, out 32000)))====0000)
3
LogieLayerIWP (in fnnffaauures-333
out out_featture==22000
LogicLayerIWP (in features=32000, out aat_featture==2200))
LogicLayerIWP (in fel.eauuurs=20000 o2t.f
GroupSum (k=10, 7 = 100.0)
Figure 22: DL.GN architecture for MNIST and Fashion MNIST.
D.4.5 14
For the 14 DLGNs, the architectures used by Bührer al. 2025). The only changes we make are: 1) We make the model use our IWP rather than the original OP. and 3) We vary the vocabulary and context sizes. See complete results in Section
24

---

Published as a conference paper at ICLR 2026
2.5%
Densty
2.0%
1 Sid Dev
Dematy
1.5%
1 Std Dev
1.5%
Mean 0,002
Mean 0.004
1.0%
0.5%
0.5%
(a) Logit initialization variance o² 1.0 (b) Logit initialization variance d² 16.0
Figure 23: Self-cancellations the sign-symmetric sum Σn (w) w..:) concentrates the gradients around zero (cf. 23a as long the initialization variance a of the logits is not overly high (cf. 23b). Empirical distribution for N 10 gradient samples da(p.g) with op
q = 0.5.
E NLP TASK: ENGLISH TO GERMAN WMT'14
We show in Table the corpus BLEU scores for discretized and relaxed DLGNs under OP and IWP for the English German translation task from WMT114 (Boarr et al. 2014a). See the experimental setup in Section D.4 We see that for almost all the parameters, IWP significantly outperforms OP. It is only for vocabulary and context sizes of 16,000 and 32, respectively, that the difference is not significant. Notably, see that IWP decreases less when going from the relaxed to the discretized setup.
BLEU
Discretized
Relaxed
Vocabulary size
Context size Parameterization
OP
15.11 0.64
17.91 t 0.27
16
IWP
17.38 + 0.04
19.09 + 0.53
8,000
OP
13.61 + 3.21
16.71 + 0.54
32
IWP
17.36 + 0.52
17.77 + 0.06
OP
10.65 + 0.20
15.17 + 0.12
IWP
14.40 0.60
15.63 + 0.11
16,000
OP
10.22
0.28
14.61 + 0.82
32
IWP
13.50 + 0.41
14.91 + 0.21
Table 4: Performance (corpus BLEU 1) of DLGNs with OP or IWP on the WMT'14 English to German translation task. We use the LGNs from Bührer al. (2025). We mcasure the BLEU score using the sacrebleu Python library. We measure the metrics across two different seeds and report the mean and standard deviation.
F THEORETICAL ANALYSIS OF PARAMETRIZATION
F.1 VANISHIG GRADIENTS IN OP
Although RIs successfully suppress vanishing gradients, the symmetric parametrization still traps them in dichotomy between gradient stability and stalling optimization towards other gate functions. On the one hand, Petersen al. (2024) 's choice of 5 will sufficiently preserve the gradient norm, as will decrcase by at most e*+15 0.9. On the other hand, already slightly decreasing layers, e² -1
would again elicit vanishing gradients after only few
2=415 0.55.
25

---

Published as a conference paper at ICLR 2026
F.2 ALGEBRAIC INTERPRETATION OF THE IWP
To understand the redundancies from an algebraic viewpoint, we can regard the space of binary functions G2 == {G 1] {0, 1}] as a vector space over the field Zz. Firstly, seven of the eight aforementioned negation symmetries correspond to linear dependencies G, + G + between elements G2. Secondly, the redundancy that led to the suboptimal rounding in the example on the discretization error can be captured in the linear dependency G3 + G6 G8 +
F.3 MINIMAL ROUNDING ERROR OF THE IWP
When rounding the gate estimator Iw to a logic gate Ja: we round each output estimator Wiy to its closest binary number Oaj == arg min Why - b/- be(0.1)
This achieves a minimal discretization error ll Ja in terms of any Minkowski norm Ilf - مllp = " Ere [f()) - g(z) because for any binary input x = (i,j), the term (x)(x) - 9o (x) = Wij - Osyl min way - bj by definition. (0.1))
F.4 REMAINING CAUSES OF VANISHING GRADIENTS IN IWP
Even with heavy-tail initializations that concentrate the wiy close to 0, 1, destructive inter- ference between gradient signals can still arise for precisely three reasons. Still, all of them are out of the control of the parametrization.
The first reason is destructive interferences that arise from the probabilistic relaxation of the Boolean functions. For example, for binary inputs (1, the gradient of the OR function (s(p.g) = p q-pq will be 0 for both inputs. We obtain a symmetric case with input (0, 0) and the AND function g2(p,g) pq-
Opposed to that, the remaining two reasons both relate to the parameter initialization of the DLGN architecture. We divide them into cancellations inside neuron and between neurons.
Inside a neuron, cancellations arise if the two terms in 12 have different signs. This happens precisely if w11 > wio and wol < woo: or vice versa, which holds only the relaxation is close the XOR function gr(p.g) NXOR. Similar to the first reason, this behaviour is not problematic and even intended as long as the inputs carry information about the desired output. If the gate outputs wiy are close to 0, 1, the gradient norm will remain close to 1. But in the case of low information, where both inputs p. 9 = 0.5 are highly uncertain, the gradients of the probabilistic surrogate of XOR and NXOR will both collapse to 0 and annihilate the gradient signal. Depending on the logic gale distribution, this undesirable scenario will, however, inevitably occur as we scale logic gate networks in depth (cf. Figure 27.. A heavy-tailed tailed initialization of the logic gate distribution alone does not suffice to prevent this. In particular, we will observe later that RIs suffer from this information collapse. But in theory, this is only problematic XOR functions are present in the network, which is not the case for RIs.
p + q- 2pq or its negated counterpart
Finally, even if we avoid cancellations inside a neuron, gradients from different neurons might still cancel when they pass the same neuron. Because of the negation symmetry in Boolean functions, a parameter initialization that treats each function and its negated counterpart independently will result in sign-symmetric gradients across different neurons during backpropagation. If the gate output of neuron is used the input of multiple subsequent neurons, this gate will receive sum of sign-symmetric partial derivatives. The more gates this neuron connected to, the more this sum will concentrate at 0.
F.4.1 HEAVY-TAIL, NEGATION- COCAATION-ASMMMTRRE INITIALIZATIONS
We maintain that an ideal initialization scheme should satisfy three properties to scale logic gate networks in depth: heavy tail, information preservation, and negation asymmetry.
26

---

Published as a conference paper at ICLR 2026
10"
10-2
Gradient Norm (log scale)
Gradient Norm (log scale)
10-9
10
10-14
10-7
10-34
IWP RI
IWP
10-1
IWP AV
10-20
IWP XOR
IWP AV-RI
10-34
IWP OR
10-9
IWP Uniform
IWP AND
0
12
18
24
30
36
0
12
18
24
30
36
Layer
Layer
(a) RI vs. multi-gate biases
(b) Single-gate biases
Figure 24: Gradient norm decrease for different heavy-tail initializations of IWP DLGN with 40 layers. While Ris stand out as the only stable single-gate bias, other multi-gate biases also relain stable gradients.
The normal initialization Gy 1id. ((O, violates all of these properties. The resulting cocfficients wij will concentrate symmetrically around 0.5 and evoke vanishing gradients, Figure 24a illustrates.
First of all, one could ensure a heavy-tail distribution of the coefficients wij at 0 and by shifting the normal distribution in a negative or positive direction. The overall sign combination in Say f.dd ((tus, 1) hence attributes high initial bias towards one of the sixteen logic gate functions. Choosing the pass-through gate G1(A, = A for all neurons recovers the idea of Rls.
Indeed, if we restrict ourselves to choosing only a single function for all neurons, RIs are the only viable approach. While the constant functions have no gradient anyway, the AND, OR, and XOR functions alone rapidly concentrate the intermediate feature distribution to 1, O, and 0.5, as Figure 27 exemplifies. At that point, their gradients collapse to and stifle any information in the imput. In terms of our three necessary properties, these initializations fall short information preservation.
On the other hand, the pass-through gate G1 does not change the input value p, and maintains a gradient of with respect to that input p. independent of what value takes. However, we increase the model depth, the intermediate feature distribution will also collapse to 0.5 with RIs (cf. Figure 3). This is because even small initial uncertaintics in the coefficients, i.e. laaj - wigl > 0.05, will accumulate over the layers. But because no gate is initially close to the XOR functions when employing RIs, this high uncertainty in later layers is harmless. On the contrary, we will discuss in the following subsection F.4.2 why this increasing uncertainty can even benefit the optimization of deep logic gate circuits.
For heavy-tail initializations that bias towards single function in all neurons, RIs are hence indeed the unique scalable choice. But we might also combine multiple logic gate functions into a heavy-tail initialization. In the extreme case, cach logic gate could bias towards one of all sixteen functions with uniform probability 1/16. But this brings us back to the third and last property, namely, negation asymmetry.
Allowing both Boolean function and its negated counterpart will provoke cancellations sign-symmetrie partial derivatives merge during backpropagation, Fortunately, this con- dition only holds for architectures with drastically increasing width between layers. For the architecture of Petersen et al. (2022) with uniform width, even negation-symmetric initializations such as the uniform initialization will retain sufficiently stable gradients (cf. Figure 24ah.
But this might not hold in general. Formally speaking, any subset of the binary functions G C G2 that does not contain a function and its negated counterpart is a feasible negation- asymmetric subset. In particular, such a subset be obtained by fixing one output to 0 or
27

---

Published as a conference paper at ICLR 2026
0.5
0.5
Activation Value
0.4
Activation Value
0.4
0.3
Densio
0.3
Density
0.5
0.5
0.2
0.2
0.1
0.1
0.0
10
15
20
25
30
35
40
5
10
15
20
25
30
35
40
Logic Layer
Logic Layer
(a) Initial training batch
(b) After 100 batches
0.5
0.5
Activation Value
0.4
Activation Value
0.4
0.3
Density
0.3
Dense
0.5
0.5
0.2
0.1
0.0
0.0
10
15
20
25
30
35
40
5
10
15
20
25
30
35
40
Logic Layer
Logic Layer
(c) After 200 batches
(d) After 500 batches
Figure 25: Distribution of intermodiate gate outputs an IWP DLGN with RIs. Mea- surements were taken al different timesteps over the course of training, where each batch comprises 100 CIFAR-100 images.
and taking half of the binary functions that coincide with this mapping. For example, by enforcing 00 O, we admit the constant 0, the two pass-through gates, three AND functions, one OR function, and the XOR function. Therefore, an alternative to the RI is to combine the AND and OR functions into an AND-OR initialization. Indeed, the complementary concentration behaviour of the AND and OR functions avoids the information collapse at 0.5 that RIs inevitably entail. Instead, Figure [77e depicts how the feature distribution balances at values close to 0 and 1, and hence reduces the uncertainty in the signal in later layers. However, this alone does not render the AND-OR initialization more desirable than Rls. Conversely, while collapse at 0.5 might be harmful in general, we explain in the next section why it actually benefits the optimization process in the case Rls.
F.4.2 RESIDUAL INITIALIZATIONS DELAY FEATURE LEARNING AT LATER LAYERS
When initializing all neurons with a pass pass-through gate, Figure 3 displays how the features eventually concentrate at 0.5 at later layers. At those layers, holds that 1 p p dc 2c du.. is roughly equal for all i,j. Because q q, hence the gradient update owyy &a.. Awiy of that, the neurons in the later layers will maintain their tass-through function until the uncertainty reduces sufficiently. This pass-thre tass-hhrough enforcement at the later layers allows the network to begin with optimizing the earlier layers first. The more the earlier neurons approach specific gates, the more declines the uncertainty of their outputs, allowing the later layers to refine their functionality. Practically, the model first optimizes a shallow logic gate circuit and increasingly advances this circuit in depth over time. Figure 25 showcases this consecutive gate collapse at earlier layers and uncertainty decrease at later layers over the course of training. This implicit organization of feature learning not only tames the overall discretization error but will also lead to faster convergence. On the contrary, the initial feature distribution of the AND-OR initialization will allow neurons at all layers to update
28

---

Published as a conference paper at ICLR 2026
AND-OR-EL
45.0
+2.5
00.43
40.0
35.0
21.0
30.0
27.5
17
(cont.)
Train. (diec)
[cont)
Vaid. (diec)
[cont)
Train (dac
]
(cons.)
Valil (dic)
(a) 5-fold depth
(b) 20-fold depth.
Figure 26: In contrast to the structured layer optimization of DLGNs with RIs that steadily maintains low discretization error, the simultaneous layer optimization for AND-OR ini- tialization drastically increases the discretization error and harms overall performance.
their coefficients in a non-umine non-umiform fashion at the same time. The drawbacks of such more chaotic optimization process become noticeable as we scale those networks in depth.
While the discretized accuracy of both initializations remains similar for shallower logic gate networks with gap for the AND-OR initialization. At the same time, the RI maintains low rounding error over the course of training and exhibits slightly superior predictive performance (cf. Figure 26 Similar drawbacks also hold for a uniform initialization or an initialization that combines AND, OR, and pass-through gates (cf. Figure 27..
layers, scaling these networks 80 layers exposes a clear discretization
To conclude, pairing exact IWP with RIs results in logic gate networks that scalable in depth and can harness the associated expressive benefits.
G REGULARIZING LOGIC GATE NETWORKS
To mitigate the generalization error, try impose several constraints on the DLGN architecture that have benefited standard neural network architectures. Unfortunately, the methods that we have tried did not raise the test accuracies further, leaving the generaliza- tion gap an open problem. In the following, we present the measures we have taken, how we implemented them for logic gate networks, and how they impacted performance.
G.1 DROPOUT
When applied in standard feed-forward neural networks, dropout (Srivastava et al., 2014] typically randomly zeroes neurons. For the logic gate network, the zeroing operation is, however, only a neutral operation in the algebraic sense when we apply it in the summation in the GroupSum layer. For logic gates, the zero is not neutral element, but on equal terms with its binary complement 1. We hence decide to realise dropout by randomly masking logic gate outputs at the final logic layer. To determine which outputs are affected, randomly select. channels of the input tensor and mask the outputs of all gates that are path-connected inputs from at least one these channels. For all affected gates, we ensure that they receive gradient update. Each channel feature dimension is selected independently with a probability Pdropout > This selection is repeated for every single batch in training. For Peropout 0.02, roughly 30,000 of the 120,000 logic gates in the final layer are masked. For Pdropout = 0.05, this number increases to 70,000, and culminates in 100.000 for Paropout 0.1.
However, Figure [8a shows that the test accuracies degrade with increasing dropout prob- ability. This regularization strategy does not, hence, seem beneficial.
29

---

Published as a conference paper at ICLR 2026
0.6
0.6
Activation Value
Activation Value
0.4
0.4
Density
Density
0.5
0.5
0.0
3
12
15
18
0
3
12
15
18
Logic Layer
Logic Layer
(a) heavy-tail towards AND
(b) heavy-tail towards OR
0.5
0.5
0.4
Activation Value
0.4
Activation Value
0.3
Densi
0.3
Density
0.5
0.5
0.2
0.1
0.1
0.0
0.0
0
3
12
15
18
0
3
12
15
18
Logic Layer
Logic Layer
heavy-tail towards XOR
(d) heavy-tail towards pass-through (RI)
0.5
0.5
0.4
0.4
Activation Value
Activation Value
0.3
0.3
Density
Densio
0.5
0.5
0.2
0.2
0.1
0.1
0.0
0.0
3
15
18
12
15
18
Logic Layer
Logic Layer
(e) heavy-tail towards AND and OR
(f) heavy-tail towards all gate functions
Figure 27: Initial distribution of intermediate gate outputs, averaged over 100 CIFAR-100 images, when initialized with different heavy-tail initializations.
G.2 RANDOMIZED GATE INTERVENTIONS
Similarly, we try to randomly intervene in the output of cach gates in the network with probability Pinterwene > 0. We explore several strategies to replace the actual gate output: from simple replacement by constant value to replacement by random uniform b U[O0 11) or a symmetric Bernoulli b B(0.5). We explore the impact of magnitude for the intervention probability Pintervene and find Pintervene 0.05 to yield the best results in the end. Indeed, the generalization gap narrows substantially, but the test accuracies still trail the unregularized DL.G for all intervention strategies (cf. Figure 28b
30

---

Published as a conference paper at ICLR 2026
40.0
40.0
35.5
35.0
35.0
17-4
30.0
22-4
26.3 11.7.26 26.5
25.0
25.8
25.0
2401
203
20.0
18.7
15.8
20.0
15.0
15.0
Nome
None
10.0
0.02
10.0
0.05
Uniform
0.
Zero
0.0
Train. (cont.)
Test (cont.)
Test (disc.)
Train. (cont.)
Test (cont.)
Test (disc.)
(a) Dropout
(b) Random gate interventions
Figure 28: Accuracics of the DLGN with dropout and random gate interventions.
Res. Initializations
Res. Connections
Gradient Norm (log scale)
Initializations
40.0
10-3
Res. Connections
35.0
both
30.0
33.5
26-5
25.0
20.0
10
0
18
24
30
36
Train. (cont.)
Test (cont.)
Test (disc.)
Layer
(a) Accuracy
(b) Gradient stability
Figure 29: Accuracies and gradient norms of the DLGN with residual connections compared
to Rls.
G.3 RESIDUAL CONNECTIONS
Finally, we explore if the network benefits from enforcing explicit residual connections (He el. al. 2016) between layers instead of Rīs. From the first to the last layer, linearly increasing fraction of gates are fixed to directly pass their output to unique neuron in the subsequent layer. We ensure that incoming residual streams from earlier layers are continued until the last layer. That way, each layer is guaranteed to receive a fraction of unreduced gradient signals, even when the remaining weights are not initialized with a heavy tail, but a standard Gaussian. Unsurprisingly, the gradient norms of DLGNs with residual connections are even more stable than for Rls, which still include some uncertainly in the weights wij (ef. Figure 29b). However, Figure 29a indicates that both the training and test. accuracies suffer slightly from this functional constraint. Although they half the mumber of learnable parameters and allow retain gradients norms without heavy-tail initializations, residual connections do not seem to play beneficial role for generalization.
H RELATED WORK
H.1 LEARNING SINGLE LOGIC GATES
Several works have exploited that learning circuits of logic gates with more than two inputs allows to embed more functional expressivity on the same hardware (Umuroglu et al. 2020 Bacellar et al. h
31

---

Published as a conference paper at ICLR 2026
The reason is that single logic gate with n inputs has VC dimension of 2n (Vapnik & Chervonenkis 1971). On the contrary, a circuit of binary logic gates with n inputs has strictly smaller discriminative power, as the VC dimension of subcircuits merely accumulates additively and not multiplicatively (Andronic & Constantinides 2025).
On the contrary, DLGNs were practically limited to learn logic gates with very few inputs, as processing 2d parameters per logic gate with n inputs quickly bocomes intractable. With our IWP that reduces the number of parameters 2", advancing DLGNs to process more than two inputs per gate becomes a viable option.
In contrast to our IWP, these works do not directly estimate the outputs of the logic gates. Instead, they use a different representation class and quantize this class to logic gates after training. However, these indirect representations either fall short of exploiting the expres- sivity of logic gates (Umuroglu et al.) 2020) are costlier to parametrize (Andronic & Constantinides 2023 2025). To begin with, [accelar et al. (2024) do not relax the logic gate at all and approximate gradients via a finite difference method that accumulates all 2" function values in a weighted sum. Most other works relax each logic gate to a continuous function class during training and quantize it back afterwards (Umuroglu et al. 2020 An- dronic & Constantinides 2023 2025p. Our IWP also falls within this category. However, these works differ from IWP in that these function classes either do not completely exploit the ex expressivity of logic gates or require more parameters to train. On the one hand, Umuroglu et al. (2020) merely regress affine transformation w x: b that is fed through an activation function after batch normalization. Here, x is the input vector, and w, b are learnable weights and bias. Although the parameter size of each neuron grows only linearly in the number of logic gate inputs, this relaxation can also express only small subset of Boolean functions. [Andronic & Constantinides (2023) hence extends this relaxation to kernelized regression w Ø(x) + b with polynomaal kernel Φ that maps x to all monomials of degree at most D, where is a configurable parameter. The size of w hence scales to nD, where n dim(z) is the number of inputs. To completely cover the class of Boolean functions, one needed to scale D to n in order to incorporate the conjunction all n inputs. The resulting weights would then have dimension n", which is larger than our 2", Finally, Andronic & Constantinides (2025) Iearn even larger neural networks within cach logic gate relaxation.
H.2 UNRELATED ADVANCEMENTS
Finally, these works contributed several advancements that do not relate to the parametriza- tion, such as learning and simplifying the connection topology or regularization.
H.2.1 LEARNING CONNECTIONS
Petersen et al. (2024) maintained that randomly initializing the conncctions between logic gale lunctions ab initio and leaving them fixed during training does not degrade perfor- mance. Instead, |aacllar el al. (2024) learn these connections via softmax relaxation. This degree of freedom however comes at the cost of learnable weight matrixes whose di- mensions correspond to the widths of contiguous layers.
H.2.2 REGULARIZATION
While Andronic et al. (2025) employ pruning strategies that incorporate the connection topology of the hardware, Bacellar et al. (2024) exert regularization on the Fourier transform
of cach logic gate (O'Donnel] 2014).
H.2.3 CLASSIPICATION HEAD
To convert the logic gate outputs into a classification, DLGNs counts the bits for each class and outputs the class index with the highest sum. To avoid the additional overhead of embedding these operations in FPGA hardware, Bacellar et. al. (2024) replace them by learnable lookup tables.
32

---

Published as a conference paper at ICLR 2026
Table 5: All binary logic functions with real-valued relaxations and gradients
id
G:
G)(0,0)
1)
G,(1,0)
G,(1,1)
9:
ôg
Gg.
1
0
0
0
2
A
B
0
0
0
1
AB
B
A
3
-A B)
0
0
A(1 B)
1
B
-A
4
A
0
D
1
1
A
1
5
-B - A)
0
O
0
B(1 A)
B
1-
D
6
B
0
1
B
0
7
AΦ
0
A+ 2AB
1- 2B
1-24
8
B
1
B-
1- B
1-A
9
(Av
0
0
1- A-
-1+ B
-1+ A
10
(AOB)
1
0
1
A-B++2R
-1 2B
-1 2A
11
-B
0
1-B
12
B-A
1
O
1
1 1-B AB
B
-1 A
13 -
0
14
A B
1
1- ABAB
-1+B B
15
(A B)
1
O
1- AB
-B
-A
16
1
1
1
1
0
33


Reviews:
[{"review_id": "o93seFzVXM", "reviewer": "Reviewer_fw6a", "summary": "The paper tackles the problem of vanishing gradients, discretization errors, and high training cost of Differentiable logic gate networks (DLGNs). The authors claimed to root cause the issues and proposed a reparametrization solution to resolve it. Redundant parameters of input grates are replaced while maintaining the representability. With binary inputs, it can achieve 4x smaller model size, 1.86x backward pass speedup and 8.5x fewer training steps to converge.", "questions": "It makes sense that light weight gate parameters help resolving the vanishing gradient issue. How is this  solution working on a larger data set like imagenet?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Less weight parameters compared w/ the original DLGN paper.", "weaknesses": "DLGN seems to have a low test accuracy, which makes it less appealing as a practically useful solution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lqnsJaAQd0", "reviewer": "Reviewer_H3Lt", "summary": "This paper proposes an input-wise parameterization (IWP) of logic gate networks with tailored initializations that mitigate vanishing gradient issues of  DLGNs when increasing depth. Specifically, the paper points out that in previous DLGN, since there are negation pairs of logic gate in each layer, initializing the weight of each gate independently will cause gradient cancellation during back propagation. To address this issue, the paper reparameterize the logic function with fewer independent components. Experimental results show that the proposed method achieve better performance than existing DLGNs when increasing network depth.", "questions": "1. It would be better if the paper add comparison experiment on more complex dataset like ImageNet 32", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written and easy to follow\n2. The paper is well motivated, as addressing the vanishing gradient issue of deep DLGNs will improve their applicability for more challenging tasks\n3. The analysis of the root cause of vanishing gradient issue in DLGN , and the solution proposed by the paper make intuitive sense and are all backed by solid proof.\n4. Experimental results and ablation study are comprehensive and positively support the proposed design.", "weaknesses": "1. The paper conducts experiment only on CIFAR 100.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LKngKXO7XM", "reviewer": "Reviewer_Nt7w", "summary": "This paper introduces an input-wise parametrization (IWP) for differentiable logic gate networks that removes redundancy, stabilizes gradients, and reduces discretization error. It cuts parameters by 4×, speeds up backward pass up to 1.86×, and converges 8.5× faster while preserving or improving accuracy on CIFAR benchmarks.", "questions": "1) Missing ablation studies on hyperparameters such as learning rate and optimizer choices. Since logic-gate networks can be highly sensitive to gradient flow and gating stability, how do different learning rates and optimizers affect the gate sharpening process and convergence behavior in IWP-based DLGNs? Additionally, how robust is the model to hyperparameter variation, and does IWP mitigate instability compared to the original parametrization (**OP**)?\n\n2) The experiments are conducted only on CIFAR-10/100, which are datasets from the same image family and share similar structure. To test generalization of the proposed IWP method to diverse problem domains, would including simpler symbolic datasets (e.g., **MNIST**) or non-vision benchmarks provide stronger evidence that the approach generalizes beyond natural images and is not over-specialized to CIFAR distributions?\n\n3) Results show that IWP scales effectively with depth under residual initialization (**RI**). However, the study appears limited to thermometer encoding and nearest-rounding. Could the authors kindly clarify whether the stability and discretization improvements of IWP are expected to extend to alternative binary encodings or rounding strategies, and whether similar behavior holds without residual biasing?\n\n4) Since IWP's parameter count scales as $2^n$ for $n$-input gates, did the authors examine performance and memory implications for $n>2$? Additionally, are there any observed or expected stability issues or computational constraints as the logic arity increases, particularly in terms of gradient behavior, training efficiency, or numerical sensitivity?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a highly efficient reparametrization that eliminates redundancy in logic gate networks and significantly stabilizes gradients. It is grounded in a strong theoretical analysis of vanishing gradients and discretization error. The method delivers substantial improvements in training speed and resource efficiency. It also enables deeper DLGNs to train reliably without performance collapse. Finally, the approach maintains hardware compatibility, making it practical for edge and FPGA deployment.", "weaknesses": "See Question Below", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AsVzW17kl6", "reviewer": "Reviewer_Pghb", "summary": "The paper introduces a new parameterization called input-wise parameterization (IWP) for Differentiable Logic Gate Networks (DLGNs), which converts the original parameterization (OP), where mixture-over-Boolean-functions parameterization is learned. IWP directly learns the numerical outcome of four input cases, i.e., (0,0), (0,1), (1,1), (1,0), leading to a 4x parameters reduction per gate. The authors claim that the new proposed parameterization can alleviate gradient vanishing and less computational costs. Experiments on CIFAR-100 show superior performance and faster computation of IWP compared to the original parameterization.", "questions": "1. In line 196-197, should the argmax is the pass-through gate G4 rather than G3?\n2. The expression of Equation 9 is a bit confusing to me. The equation would be more self-contained with more details like $G(k,l) = \\alpha_{0,0} E_{k,l,0,0} + ... $ and $E_{k,l,i,j} = \\textbf{1}_{\\\\{(k,l)=(i,j)\\\\}}$.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The analysis in discretization error, redundancy, and gradient stability of OP in Section 3 is insightful.\n2. The manuscript is well-organized and easy to follow.", "weaknesses": "1. My major concern of the paper is the experiment. The authors only evaluate IWP on single CIFAR-100 datasets. I would expect the authors to include more diverse datasets, for example, MNIST, CIFAR-10, WMT’14, and possibly TinyImageNet. Experiments on more diverse datasets are needed to evaluate the proposed parameterization robustly. \n2. My second concern is that the improvement over OP is marginal. For example, in CIFAR-100 the authors show 2% improvement with 8x faster convergence. However, there is no breakthrough in representation power, i.e., the depth scaling behaviors are unchanged compared to the OP as seen in Figure 4. However, note that the overall test accuracy is 29% while very small CNNs like resnet-20 can have 70%+ accuracy on CIFAR-100. Hence, results would be more compelling if deeper models with IWP closed the gap to standard architectures, not just DLGNs with OP.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)