Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Authors: Zhaoxian Wu
Quan Xiao
Tayfun Gokmen
Omobayode Fagbohungbe
Tianyi Chen
Venue: NeurIPS 2025 oral ()
Keywords: Analog AI; in-memory computing; stochastic gradient descent; stochastic optimization
Primary area: optimization
TL;DR: Leveraging a residual learning framework to support the model training on non-ideal analog in-memory computing hardware

Abstract: As the economic and environmental costs of training and deploying large vision or language models increase dramatically, analog in-memory computing (AIMC) emerges as a promising energy-efficient solution. However, the training perspective, especially its training dynamic, is underexplored. In AIMC hardware, the trainable weights are represented by the conductance of resistive elements and updated using consecutive electrical pulses.  While the conductance changes by a constant in response to each pulse, in reality, the change is scaled by asymmetric and non-linear response functions, leading to a non-ideal training dynamic. This paper provides a theoretical foundation for gradient-based training on AIMC hardware with non-ideal response functions.  We demonstrate that asymmetric response functions negatively impact Analog SGD by imposing an implicit penalty on the objective. To overcome the issue, we propose residual learning algorithm, which provably converges exactly to a critical point by solving a bilevel optimization problem. We show that the proposed method can be extended to deal with other hardware imperfections like limited response granularity. As far as we know, it is the first paper to investigate the impact of a class of generic non-ideal response functions. The conclusion is supported by simulations validating our theoretical insights.

Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu", Quan Xiao", Tayfun Gokmen', Omobayode Faghohungbe Tianyi Chen
"Cornell University, New York, NY
IBB T. J. Watson Research Center, Yorktown Heights, NY
Renssellar Polytechnic Institute, Troy, NY
{zw868, qx232)8cornel1. edu,
{tgokmen, Omobayode Fagbohungbe -imm. com,
tianyi. www.ecconeccnnnccccomo edu
Abstract
As the economic and environmental costs of training and deploying large vision language models increase dramatically, analog in-memory computing (AIMC) emerges as a promising energy-efficient solution. However, the training perspective, especially its training dynamics, is underexplored. In AIMC hardware, the trainable weights are represented by the conductance of resistive elements and updated using consecutive electrical pulses. While the conductance changes by constant response to each pulse, in reality, the change is scaled by asymmetric and non-linear response functions, leading to a non-ideal training dynamics. This paper provides a theoretical foundation for gradient-base based training on AIMC hardware with non- ideal response functions. We demonstrate that asymmetric response functions negatively impact Analog SGD by imposing an implicit penalty on the objective. To address the issue, we propose residual learning algorithm, which provably converges exactly to critical point by solving bilevel optimization problem. We demonstrate that the proposed method can be extended to address other hardware imperfections, such limited response granularity. As we know, it is the first paper to investigate the impact of a class of generic non-ideal response functions. The conclusion supported by simulations validating theoretical insights.
Introduction
The remarkable success of large vision and language models is underpinned by advances in modern hardware accelerators, such as GPU, TPU [1], NPU [2), and NorthPole chip [3]. However, the computational demands of training these models are staggering. For instance, training LLaMA [4] cost $2.4 million, while training GPT-3 [5] required $4.6 million, highlighting the urgent need for more efficient computing hardware. Current mainstream hardware relies the Von Neumann architecture, in which the physical separation of memory and processing units creates a bottleneck due to frequent, costly data movement between them.
In this context, the industry has turned its attention analog in-memory computing (AIMC) accelera- tors based on resistive crossbar arrays [6-10]. which excel at accelerating ubiquitous, computationally intensive matrix-vector multiplications (MVMs) operations. AIMC hardware, the weights (ma- trices) are represented by the conductance states of the resistive elements in analog crossbar arrays [11, 12], while the input and output of MVM are analog signals like voltage and current. Leveraging
"The work was done when the authors were Rensselacr Polytechnic Institute. The work supported by IBM through the -BM-Rensselarr Future Computing Research Collaboration, the National Science Foundation Projects 2401297, 2532349, and 2532653, and by the Cisco Research Award.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

weight (conductance)
weight (conductance)
Atiiin q+ (w)
Atiiii (+(1x)
10
- Minin g- (w)
min (w)
45
00
20
# of pulse cycles
# of pulse cycles
At
Ataii
AtFii
Figure 1: The weight's ss response curve. Positive and negative pulses are fired continuously on the left and right halves, respectively. One pulse is fired per cycle. Given w, the weight becomes w" or w after one positive and negative pulse, respectively. The response factors q4 (w) and q- (w) are approximately the slope of the curve at w; and Ammin is the response granularity. (Left) Ideal response functions q+ (w) q- (w) = Every point is symmetric point. (Right) Asymmetric response functions (w) ) + q 9- (w) almost everywhere expect for the symmetric point wº.
Kirchhoff's s and Ohm's 's laws, AIMC hardware achieves 10x-10,000x energy efficiency than GPU [13-15] in model inference.
Despite its high efficiency, analog training is considerably more challenging than inference since involves frequent weight updates. Unlike digital hardware, where the weight increment can be applied to the original weight in the memory cell, the weights in AIMC hardware are changed by the so-called pulse update.
Pulse update. When receiving electrical pulses from its peripheral circuits, the resistive elements change their conductance in response the pulse polarity [16]. Receiving pulse each pulse cycle, the conductance is updated by Awmin q+ (w) or A Aiiin q- (w). depending on the pulse polarity, where A AWmin is response granularity, and q+ (w) and (ww are response functions. Geometrically, 2+ (w) and q- (w) are the slopes of response curves, see Figure 1. All AWmin 4+ (w), and q- q are element-specific parameters or functions that are set before training and hence remain fixed during training. Typically, AWmin is known while g (w) and (w) are not.
maadient-based training implemented by analog update. Supported by pulse update, the gradient- based training algorithms are used to optimize the weights. Consider a standard training problem with objective f(-): RD + IR and model parameterized by W E RD
W = arg min s(W) = IEg (((:EE)
WERD
where & is random data sample. Similar to stochastic gradient descent (SGD) in digital training (Digital SGD), the gradient-based training algorithm on AIMC hardware, Analog SGD, updates the weights by stochastic gradients Ex). Digital SGD updates the weight by W = Wa - VV f(We; (We:&e) Ex) with learning rate a. Given desired update AW -aVf(Wes Ex), AIMC hardware implements Analog SGD by sending [[AW]a] Jal/ Awiin pulses to the d-th element. Idcally, (w) = (w) = for every conductance states. If so, with each pulse updating [Wa]a by A Wmiin [W&la is ultimately updated by about [AW]a-
Challenges of analog training. Despite its ultra-efficiency, gradient-based training on AIMC hardware is challenging. First, the generic response functions are asymmetric (i.e. 4+ (w) 전 (w)),, and non-linear [17-19]. Due to the variation of response functions and conductance states, gradients are scaled by different magnitudes across different coordinates, leading to biased gradients. Furthermore, the response granularity Awmin is a constant. When the gradients or the learning rate decay below Awmin. pulse update no longer provides sufficient precision to perform gradient descent [20]. Other imperfections include, but are not limited to, noisy input/output (IO) of MVM operations and analog-digital conversion error [18]. This paper aims to investigate the impact of non-ideal response functions and develop method to mitigate their negative effects. We also discuss extending the proposed method to deal with other hardware imperfections.
1.1 Main results
Complementing existing empirical studies in analog in-memory computing, this paper aims to build a rigorous theoretical foundation of analog training. By introducing bias to the gradient, the asymmetric

---

response function plays a central role in differentiating digital and analog training. In contrast, the other non-idealities hinder the training process by causing precision-relates issues. Therefore, we approach the problem progressively, beginning with simplified case that involves only the asymmetric response functions, and extending the proposed methods to more general scenarios.
As warm-up, building upon the pulse update mechanism, we propose the following discrete-time mathematical model to characterize the trajectory of Analog SGD
Analog SGD
WK+1 W a V f(W Ex) @ F(W+) VWW:: Ea] G(W))) (2)
where 0 is the learning rate and &A is the data sample of iteration k; and O represent the element-wiss wise absolute value and multiplication, respectively; and F(-) and G(-) are hardware- specifie matrix which are defined by g+ ) and q- (-). In Section 2, we will explain the underlying rationale of (2). Compared with the standard Digital SGD, the gradients in (2) are scaled by F(-)
and extra bias term is introduced. Typically, hardware imperfections lead to non-ideal response functions, i.c., F(.) # 1 and G(-) ) # O. Thus, we ask a natural question that
Q1) What is the impact of non-ideal response functions and how to alleviate it?
Recently [21] partially answers the question by showing that Analog SGD suffers from a convergence issue due to the asymmetric update, and a heuristic algorithm, Tiki- [22-24], converges exactly by reducing the weight drift. However, their work is limited to a special case of linear response functions, which are in the form of 9+ (w) = 1 - w/T,qe (-w) ) = 1 + w/T with hardware-speciee parameter 7 > O. Given more general q+ (w) and q- (w), the convergence of Tiki -Tka does not trivially hold, even though the response functions are still linear.
Gap between theory for special linear and generic response functions. Consider a more generic linear response q+ (w) = (1+ (1
10a
w/ (w) = (1 (1 + w/T) with parameter Cii. which reduces to the setting in [21] when CLin = O. Figure 2 shows the damage from a non-zero CLin to Tiki-Take Consistent with the conclusion in [21]. Tiki-Take signifi- cantly outperforms Analog SGD when CLin = However, when CLin is perturbed from 0.1 to 0.3. Tiki-Taka degrades dramatically and even be- comes worse than Analog SGD does. The mod- ification is slight, but the convergence guarantee in [21] fails, and the convergence of Tiki-Taake is harmed significantly. This counter-example indicates a gap between the theory for special linear and generic response functions, and necessitates the study of the analog training with generie response functions and the exploration of exact convergence conditions.
10
Qa 0.0
10
0.1
0.2
0.3
10-1
Analag SGD
200
400
600
800
1000
Numberol Gradient Computation (k)
Figure 2:
Comparison of Analog SGD and Tiki-Taka under different parameter CLin. The error plateau in the order 10-5 comes from the limited response granularity Awmin = 10-4.
Ignoring other imperfections temporarily, this paper first analyzes the impact of response functions. We show that Analog SGD suffers from asymptotic error due to the mismatch between the algorithmic stationary point and physical symmetric point. Inspired by that, we propose novel algorithm framework that aligns two points, overcoming the asymmetric issues. Building on that, we endeavor to extend the proposed algorithm to more practical scenarios that involve other imperfections like limited granularity and noisy readings, prompting a second critical question:
Q2) How to extend the framework to address the limited response granularity and noisy 10 issues?
To answer this question, propose two mechanisms further overcome these two issues.
Our contributions. This paper makes the following contributions:
C1) Building on the pulse update equation, we propose an approximate discrete-time dynamics for analog update. Enabled by this, study the impact of response functions directly, without being limited to specific element candidates.
C2) Based on that, we show that instead of optimizing ʃ(.), Analog SGD optimizes another penalized objective implicitly. An implicit penalty is introduced by the asymmetric response functions, which attract the weights towards symmetric points. Consequently, Analog SGD can only converge to the optimal point inexactly.
3

---

C3) We propose a novel Residual Learning theoretical framework to alleviate the asymmet- ric update and implicit penalty issues. Residual Learning explicitly introduces another residual array, which has a stationary point O. This framework leads to Tiki- - Taka heuristi- cally proposed in [22] while it offers an understanding of how Tiki-Taka deals with the challenge from generic response functions. By properly zero-shifting so that the stationary and symmetric points overlap, Residual Learning provably converges to a critical point. C4) Building on C3), we propose a variant, Residual Learning v2, tailored for more practical training scenarios. We propose introducing a digital buffer to filter out reading errors caused by I0 noise. Furthermore, we propose a threshold based transfer rule to alleviate instability caused by limited granularity.
1.2 Prior art
AIMC training, Analog training has shown promising early successes with tremendous energy advantage [25, 26]. Among them, on-chip training, which performs forward. backward, and update directly on analog chips [22-24, 27, 28] is considered to be the most efficient paradigm, but it more sensitive to hardware imperfections. Sacrificing energy efficiency for robustness, hybrid digital-analog off-chip training is proposed [29-32], which offloads some computation burden to digital components. This paper focuses on the more challenging on-chip training setting.
energy-baseee model and equilibrium propagation. AIMC training leverages peck-propagation compute the gradient signals. Recently, class of energy-based models has been studied, which performs equilibrium propagation to compute gradient signals [33-37]. Focusing on the training dynamics instead of concrete gradient computing, our work orthogonal to them and is expected to provide insight for algorithm designs of energy-based model training.
2 Analog Training with Generic Response Functions
This section examines the discrete- time dynamics of analog training and introduces the challenges posed by generic response functions. After that, we introduce family of response functions that reflect crucial physical properties that interest us.
Compact formulations of analog update. We first investigate the dynamics of one element w in W R.. This paper adopts to represent the element of the weight We without specifying its index. As we discuss in Section 1, the response granularity Awmin is scaled by the response functions 4+ (w) or q (w). Since desired update Aw requires series of pulses with each scaled by approximately 4+ (w) or (wo) it is sensible that the Awe is approximately scaled by 4+ (w) or q- (w) as well, Accordingly, we propose that an approximate dynamics of analog update given by w' z Ug(w, Aw), where (w, Aw) is defined by
(w Aw) == w + Aw q+ (w), Aw 2 0, w + Au q- (w), Aw < O.
(3)
The update (3) holds at each resistive element. At the k-th iteration, We stack all the weights wk and expected increment Aw together into vectors WR, A W. E R.. Similarly, the response functions q+ (.) and (-( are stacked into (-) and Q-(-), respectively. Let the notation UW,,, AW) on matrices WA and AW denote the glement-wise operation on We and AW, i.e. [W, AW)]a U[Q)a ((w (W&&) kl,, [Am]aa) Vid [D] with [D] == {1, D) denoting the index set. The element-wise update (3) can be expressed WR+1 UO AWx). Leveraging the symmetric decomposition [21, 22], we decompose Q (W) and Q+ (W) into symmetric component
F(.) and asymmetric component G(.)
F(W) (Q- (2)) and G(W) = (Q- (W) (W))2/2
(4)
which leads compact form of the Analog Update
Analog Update
Wh+1 = Wi + AWA O F(W2) AW4 e G(W.).
(5)
Gradient-based training algorithms on AIMC hardware. In (5), the desired update AW, varies based on different algorithms. Replacing AWa with the stochastic gradient V f(W;; Ex), we obtain

---

Response function
(-(u)
W
W
1
Figure Examples response functions from Definition I; w the symmetric point.
the dynamics of Analog SGD shown in (2). This update reduced to the mathematical form for linear response functions in [21] special case; Appendix B for details.
Response function class. Before proceeding to the study of response functions, we first define the response function class that interests us. Since the behavior of resistive elements is always governed by physical laws, function class should reflect certain crucial physical properties.
The most crucial property of the response functions is the asymmetric update, i.e., q- (.(gg-(w) o (w) for most of w. Specifically, if a point u satisfies q- point. Stacking all w² into a vector W E IR.. Observe that the function G(W) is large if q-(w) and q+ (w) are significantly different, while is almost zero around W At the same time, F(W) the average of the response functions in two directions. As we will see in Sections 3.2 and 4, the ratio G(W) plays critical role in the convergence behaviors.
q+ (w²). we say wº is symmetric
VF(W)
In addition the asymmetric update, the function class should possess other properties. First, the conductance increases upon receipt of a positive pulse, and vice versa, resulting in positive response functions. In addition, assume that the response functions differentiable (and hence continuous) for mathematical tractability. Taking all factors into account, we define the following class of response functions.
Definition (Response function class). (+(-) and q-(-) satisfy
(Positive-definitennss) There exist positive constants Quin O and Amax O such that Quin S (w) S 9max and Quin S (-(w)= I 9maxx Vw; and,
(Differentiable) The response functions q+( and (-) are differentiable.
Definition 1 covers wide range of response functions, including but limited PCM, ReRAM, ECRAM, and others mentioned in Section A. Figure showcases three examples from the response functions class, including linear, non-linear but monotonic, and non-monotonic functions.
Implicit Penalty and Inexact Convergence of Analog SGD
This section introduces critical impact of the response functions, implicit penalized objective. Affected by this, Analog SGD can only converge inexactly with non-diminishing asymptotic error.
3.1 Implicit penalty
We first give intuition through situation where We is already critical point, i.e., E& [V[V((W>:)) O. Recall that stochastic gradient descent on digital hardware (Digital SGD) is stable in expectation, i.e. Es, [W++1] = Wt Bg, [WW:(W()))) Wt. However, this does not work for Analog SGD
Eqe [W++1] W6 - Eç, (VV O F(WA) &
(6)
= Wi - a || V (() O G(W))) + Wt.
Consider a simplified version that the weight is scalar (D = 1) and the function G(W) is strictly monotonically decreasing" to help gain intuition on the impact of the drift in (6). Recall G(Wo)
0 at the symmetric point W G(W) > O when W > W and G(W) < 0 otherwise. Consequently. (6) indicates that E2, [W < We when W, W and Eg, [W W++1] > Wt otherwise. It implies that Wi suffers from a drift tendency towards W In addition, the penalty coeflicient proportional
It happens when both (+(+) and q- g-(-) are strictly monotonic.

---

to the noise level since the drift is proportional to B (WW: which is the first moment of noise s,, f Ex) E& (WW E)lll in essence.
The following theorem formalizes the implicit penalty effect. Before that, we define an accumulated
asymmetric function Re(-): IR_ RD, whose derivative is R(W) :- F(W), i.e.
G(W) d[R((W]
d[Wl2
[R(W)]a- Fa If RRW) is strictly monotonic, R&) reaches its minimum at the symmetric point W where R(W") = 0, so that it penalizes the weight away from the symmetrie point.
Theorem 1 (Implicit penalty, short version). Suppose W* the unique minimizer of problem (1). Let = E VVW"; E D. Analog SGD implicitly optimizes the following penalized objective
min fx: == f(W) + Rc()). W
(7)
The full version of Theorem and its proof are deferred to Appendix G. In Theorem 1, (R) plays the role of penalty to force the weight towards a symmetric point. As shown in Appendix G, (W) has a simple expression on linear response functions when CLin 0, leading (7) to minw (=:WW= == s(W)= ス I| which is an & regularized objective. In addition, the implicit penalty has coefficient proportional to the noise level Σ and inversely proportional the dynamic range It implies that the implicit penalty becomes active only when gradients noisy, and the noise amplifies the effect.
With noisy gradients, an implicit penalty attracts Analog SGD towards symmetric points.
3.2 Inexact Convergence of Analog SGD under generic devices
Due to the implicit penalty, Analog SGD only converges to critical point inexactly. Before showing that, We introduce a series of assumptions on the objective, as well as noise.
Assumption (Objective). The objective f(W) is L-smooth and is lower bounded by f*. Assumption 2 (Unbiasness and bounded variance). The stochastic gradient is unbiased and has bounded variance o². i.e. E& (VWW; <)] = /(W) and [[WVV/(W; - -/S(W) S o².
Assumption 1-2 are standard in non-convex optimization [38]. This paper considers the average
squared norm of the gradient as the convergence metric, given by E = : Σk-1 (W+] Now, we establish the convergence of Analog SGD.
Theorem 2 (Inexact convergence of Analog SGD). Under Assumption 1-2, if the learning rate is set as a = O(1/VK), it holds that
E4S00 0o <0 + 0 )
(8)
where SteED denotes the amplification factor given by K
R k
K-1 VF(W2)
G(W.)
The proof of Theorem 2 is deferred to Appendix H. Theorem 2 suggests that the convergence metric FAAED is upper bounded by two terms: the first term vanishes at a rate of O(vo2/K) K). which matches the Digital SGD's convergence rate [38] up to a constant; the second term contributes to the asymptotic error of Analog SGD, which does not vanish with the number of iterations K.
Impact of saturatiionasymmetrrii update. The exact expression of SABGD depends on the specific noise distribution and thus is difficult to reach. However, SAScD reflects the saturation degree near the critical point W* when W6 converges point W S becomes large, leading to large E and a large asymptotic error. In contrast, if W* remains close to the symmetric point W the asymptotic error is small.
neighborhood of W If W* is far from the symmetric
4 Mitigating Implicit Penalty by Residual Learning
The asymptotic error in Analog SCD is a fundamental issue that arises from the mismatch between the symmetric point and the critical point. An idealistic remedy for the inexact convergence carefully shifting the weights to ensure the stationary point is close to symmetric point. However, determining

---

the appropriate shifting is challenging, as the critical point is unknown before training. Therefore, an ideal solution address this issue is to jointly construct sequence with a proper stationary point and proper shift of the symmetric point.
Residual learning. Our solution overlaps the algorithmic stationary point and physical symmetric point on the special point O. Besides the main analog array, W4. we maintain another array, Pk. whose stationary point should be 0. A natural choice the residual of the weight, P P*(W), defined by the P that minimizes the objective f(W + 7PP) with non-zero Notice that P ("(We)) - 0 as WE + W* Additionally, the goal of the main array is minimize the residual so that the model We approaches optimality. This process can be formulated as the following bilevel problem, whose optimal points can be proved to be those of f(W)
(9) Now we propose gradient-based method solve (9). The stochastic gradient of f(W + 7P) with respect to P. given by V VrJ(W + &)) yV f( + PP;E), is accessible with fair expense, enabling us to introduce a sequence P& to track the residual of Wi by optimizing f(Wx + +PP)
Residual Learning
W WERD IP"(") s.t. P"(W) E arg min f(W + 7P).
min
PER
Pk+1 = Pa a (WW:: @ F(P)) - a / ((W:: &x) o G(Ps).
(10)
where W1 == Wi + 7P: is the mixed weight. We then derive the hyper-gradient of the upper-level objective. Notice V|| 12 P* W) (W). Assuming W is the unique minimum of f(.), we know PM P"(W) satisfies PP satisfies
W = =W* W Taking gradient with respective to W on both
sides, we have (W)) = 1 and hence VW]P?(W)]|= = (W). Approximating P (W)
by Pk and absorbing 2 into the learning rate 3, we reach the update of the main array
?
W++1 We + BP&+1 @ F(WA) P @ G(W)).
(11)
Featuring moving the residual Ps to Wa. (11) is referred to transfer process. The updates (10) and (11) are performed alternatively until convergence. Tiki-Tale mentioned [21] is the special case with linear response functions and 7 O.
On the response functions side, it is naturally required to let zero be symmetric point, i... G(()) which can be implemented by the zero-shifting technique [39] by subtracting a reference array.
Convergence properties of Residual Learning. We begin by analyzing the convergence of Residual Learningwithout considering the zero-shift first, which enables us to understand how zero-shifted response functions affect convergence.
the optimal point W* exists and is unique, the solution the lower-level objective has closed form P*(W)= == -W, At that time, the upper-level objective equals | 21 However, the solutions of f(()) are generally non-unique, especially for non-convex objectives with multiple local minima. To ensure the existence and uniqueness of W*, we assume the objective is strongly convex.
Assumption (u-strong convexity). The objective f(W) is p-strongly convex.
Under the strongly convex assumption, the optimal point W* is unique and hence the optimal solution of the lower-level problem in (9) unique. Since the requirement of strong convexity is non-essential in the development of bilevel optimization [40-43], we believe the proof can be extended to more general cases and will extend it for future work.
Involving two sequences W4 and Ps. Residual Learning converges in different senses, including: (a) the residual array P& converges the optimal point P (W&); (b) W₁ converges the critical point of f(-) or the optimal point W*; (c) the sum Wt- = W₁ + Pe converges to a critical point where ((W,) = 0. Taking all these into account, we define the convergence metric as
K-1
(We) + w113))
E
E
+
(12)
K
k=0
For simplicity, the constants in front of some terms FRt are hidden. Now, we provide the convergence of Residuul Learning with generic responses.
7

---

100
Teot ACCURa
97
Text Accurar
60
T-0
40
96
0.5
0.7
99
T-
r=0.8
Tik-Bka
Tik-Taaa
20
95
30
Digiall S2D
Digial SGD
0
5
10
15
20
25
30
5
10
15
20
25
30
Epoch
Epoch
Figure 4: Test accuracy of training on MNIST dataset under different (Left) FCN. (Right) CNN.
Theorem 3 (Convergence of Residual Learning). Under Assumptions 1-3, with the learning rale a = (=1(/1//22) = 0(a73322), it holds for Residual Learning that
F o OV02/KK+ +
(13)
where SĀt denotes the amplification factor of Pk given by Skk :- 1 E
G(R VF(P2)
The proof of Theorem 3 is deferred to Appendix Theorem 3 claims that Residual Learning converges at the rate O (Vo0 (V to neighbor of critical point with radius Oto which share almost the same expression with the convergence Analog SGD. The difference lies in the amplification factor K and SAB0D where the former depends on Ps while the latter depends on Wa.
Impact of response functions. Response functiin affects the Analog SGD and Residual Learning similarly. However, attributed to the residual array, constructing response functions enable exact convergence of Residus iusidual Learning is viable.
As we have discussed, P& tends to Pa (W&) which tends to 0 given We tends to W Therefore, response functions with G(P) when P are required for the exact convergence.
Assumption 4. (Zero-shifted symmetric point) P is a symmetric point, i.e. G((0) = 0.
Under it and the Lipschitz continuity of the response functions, it holds directly that G(Px) /F(Ps)
S
Pslloe for a constant Lg 2 O. Consequently, when Pş + PW&))) - 0 as W. - W the
asymptotic error disappears. Formally, the following corollary holds true.
Corollary (Exact convergence of Residual Learning). Under Assumption 4 and the conditions in Theorem 3, y 2 l(g it holds that E < (VOLL/K)
The proof of Corollary is deferred to Appendix I.5. Corollary demonstrates the failure of Tiki Tiki-Taka in Figure The symmetric point is w = CLinT in this example, which violates Assumption when CLin * 0 and hence introduces asymptotic error into Residual Learning.
5 Extension of Residual Learning: limited granularity and noisy IO
This section extends Residual Learning to practical scenarios with additional hardware imperfec- tions. To be specific, consider the noisy 10 and limited granularity examples. We highlight that we are not trying to diminish the importance of imperfection, but rather focus on two the primary ones known to be erucial.
10 of resistive crossbar arrays introduces noise during the reading of Pe41 the transfer process (11), given by WK+1 W₃ ((P +ex FF F(W)) - implicit penalty issues again, leading to penalized upper-level objective (P)())]l' + (Ee, (W),, claimed by Theorem I, where Σ. B| is assumed to be constant. To filter out the noise, propose to use digital buffer H₂ to take a moving average of noisy Pk+1 signals by
P +ex/ CateiiteeGGGWWaa with noise It incurs the
HK+1 (1 B) + ((P++ +x411).
(14)

---

CIFAR10
DSGD
ASGD
TT/RL
TTv2
RLv2
ResNet18
95.43 0.113
84.47
+3.40
94.81
+:009
95.31 4-000
95.12 10.14
ResNet34
96.48 0.02
95.43 +0.12
96.29
0.112
96.60+10.0
96.42 10.13
ResNe150
96.57 0.10
1..6
96.34 0.04
96.63 #0.09
96.56=00.00
CIFAR100
DSGD
ASGD
TT/RL
TTv2
RLv2
ResNet18
01.1
68.98 1.001
76. 0.23
78.26
79.83
00.13
ResNet34
83,86
78.98
10.55
80.58 0011
81.81 10.15
82.85 10.19
ResNet50
83.98 10.11
79.88 1.26
80.80
$2.8210.33
83.90
Table 1: Fine-tuning ResNet models with the power response on CIFARI0/100, Test accuracy is reported. DSGD, ASGD, TT/RL, TTv2, and RLv2 represent Digital SGD, Analog SGD, Residual Learning/Tiki-Taka, and Residual Learning v2, respectively.
Intuitively, with fixed PR-11 H₃ will converge neighborhood of Pk-1 with radius 0 Therefore, a sufficiently small 3 renders Hy a fair approximation of noiseless Ps. cnabling optimizing the upper-level objective with clearer signals. After that, Hk+1 is transferred to Wi as follows
W8+1 = W + 33H4+ @ F(W.) 3 || O G(W&).
(15)
Furthermore, the transfer process suffers from constant error of O(Awmin) due to the discrete pulse firing, each of which changes the weight by O(Awmin). To overcome these issues, we propose introducing threshold mechanism that does not transfer the entire H to W' at each iteration, as in (15). Instead, we compute an intermediate value by Ha+ (1 3) + 3(Pk+1 + Ek+1) first. At each coordinate d, if the value |[A +lal AWmin. one pulse will be fired to [W&]a and AWmin [Ha+1]a [H yla + Atmmn where
update the digital buffer by [Ha+1]a the sign of increment is determined by the sign of [H++gld fld Otherwise, no transfer triggered if the intermediate value falls below the threshold, i.e., [Hx+lld = [ The proposed algorithms are referred to as Residual Learning v2.
[H +
6 Numerical Simulations
In this section, we verify the main theoretical results by simulations on both synthetic datasets and real datasets. We use the open source toolkit IBM Analog Hardware Acceleration Kit (A AAHHWAII KIT) [44] to simulate the behaviors Analog SGD, Residual Learning (which reduces to Taki-Taka). Each simulation is repeated three times, and the mean and standard deviation reported. We consider two types of response functions in our simulations: power and exponential response functions with dynamic ranges [-7, - and the symmetric point being O, as required by Corollary 1. More details, simulations, and ablation studies can be found in Appendix K. The code of our simulations available gi thub. com (hhaxiian--o Ww/ analanaaaaaaaaaaa wraiggiggriinnnnn....i
FCN/CNN @ MNIST. We train fully- cully-connected network (FCN) and a convolutional neural network (CNN) the MNIST dataset and compare the performance of Analog SGD and Tiki-Taka under various dynamic range т on power responses; see the results in Figure 4. By tracking residual, Residual Learning outperforms Analog SGD and reaches comparable accuracy with Digital SGD. For both architectures, the accuracy of Residual Learning drops by < 1%. In contrast, Analog SGD takes few epochs to achieve noticeable increase in accuracy in FCN training, rendering a slower convergence rate than Residus duaal Learning. In CNN training, Analog SGD s accuracy increases more slowly than Residual Learning, eventually settling at about 80%. It is consistent with the theoretical claims.
ResNet @ CIFARIO/CIEARIOO, We fine-tune three ResNet models with different scales on CRRARRO/IIFRRIIO 100 datasets. The power response functions are used, whose results are shown in Table 1. The results show that the Tiki-Taka outperforms Analog SCD by about 1.0% in most of the cases in ResNer34/50, and the gap even reaches about 7.0% for ResNet18 training on the CIFARI00 dataset. On top of that, also compare the proposed Residual Learning v2 and Tiki-Ti -Taaa v2.

---

82
96
80
Accuracy
Accuracy
Resnet18
78
Resnet:3 et34
Resnet50
Resnet18
94
Resnet34
76
Resnet50
93
Figure The test accuracy ResNet family models after 100 epochs trained by Residual Learning under different in (10); (Left) CIFARIO. (Right) CIFARI00.
Both of them outperform Residual Learning since they introduce digital buffer filter out the reading noise. However, Residual Learning v2 outperforms Tiki-Taka v2 on the CIFARI00 dataset, demonstrating the benefit from the bilevel formulation.
Ablation study on 7, We conduct simulations study the impact of mixing coefficient y in (10) on the CIFARIO 10 or CIFAR 100 dataset in the ResNet training tasks. The results are presented in Figure 5, which shows that Residual Learning achieves great accuracy gain from increasing 7 from to 0.1, 1, while the gain saturates from 0.1 to 0.4. Therefore, we conclude that Residual Learning benefits from a non-zero y, and the performance robust the y selection.
7 Conclusions and Limitations
This paper studies the impact of a generic class of asymmetric and non-linear response functions on gradient-based based training in analog in in-memory computing hardware. We first formulate the dynamics of Analog Update based on the pulse update rule. Based on it, we show that Analog SCD implicitly optimizes penalized objective and hence can only converge inexactly. To overcome this issue, propose a Residual Learning framework which solves a bilevel optimization problem. Explicitly aligning the algorithmic stationary point and physical symmetric point, Residual Learning prov- ably converges to the optimal point exactly. Furthermore, we demonstrate how to extend Residual Learning to overcome the noisy reading and limited update granularity issues. The efficiency of the proposed method is verified through simulations. One limitation of this work is that the current analysis considers only the three hardware imperfections. While they are known be crucial for analog training, it is also important to extend our convergence analysis and methods to more practical scenarios involving more imperfections future work.
References
[1] Norm Jouppi, George Kurian, Sheng Li. Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, al. TPU v4: An optically reconfig- urable supercomputer for machine learning with hardware support for embeddings. In Annual International Symposium on Computer Architecture, pages 1-14, 2023.
[2] Hadi Esmaeilzadeh, Adrian Sampson, Luis Ceze, and Doug Burger. Neural acceleration for general-purpose approximate programs. JEEE/ACM international symposium on microarchi- tecture, pages 449-460. IEEE, 2012.
[3] Dharmendra S Modha, Filipp Akopyan, Alexander Andreopoulos, Rathinakumar Appuswamy, John V Arthur, Andrew S Cassidy, Pallab Datta, Michael V DeBole, Steven K Esser, Car- los Ortega Otero, et al. Neural inference at the frontier of energy, space, and time. Science, 382(6668)::22 335, 2023.
[4] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 2023.
10

---

[5] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877-1901, 2020.
[6] An Chen. comprehensive crossbar array model with solutions for line resistance and nonlinear device characteristies. IEEE Transactions on Electron Devices, 6084):1388-1326, 2013.
[7] Wilfried Haensch, Tayfun Gokmen, and Ruchir Puri. The next generation of deep learning hardware: Analog computing. Proceedings of the IEEE, 107(())108-122, 2019.
[8] Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE, 105(12):295--3329 2017.
[9] Abu Sebastian, Manuel Le Gallo, Riduan Khaddam- Aljameh,, and Evangelos Eleftheriou. Memory devices and applications for in in-memory computing. Nature Nanotechnology, 15:529- 544, 2020.
[10] Manuel Le Gallo, Riduan Chaddam-Alljaehh Milos Stanisavijevic, Athanasios Vasilopoulos, Benedikt Kersting, Martino Dazzi, Geethan Karunaratne, Matthias Brändli, Abhairaj Singh, Silvia M Mueller, memory for deep neural network inference. Nature Electronics, 6(9):680 693, 2023.
A 64-core mixed-signal in-memory compute chip based on phase-change
[11] Geoffrey W Burr, Robert M Shelby, Abu Sebastian, Sangbum Kim, Seyoung Kim, Severin Sidler, Kumar Virwani, Masatoshi Ishii, Pritish Narayanan, Alessandro Fumarola, et al. Neuromorphic computing using non-volatile memory. Advances in Physics: X, 2(1):89-124, 2017.
[12] J Joshua Yang, Dmitri B Strukov, and Duncan Stewart. Memristive devices for computing. Nature nanotechnology, 8(1):13, 2013.
[13] Shubham Jain et al. Neural network accelerator design with resistive crossbars: Opportunities and challenges. IBM Journal of Reseurch and Development, (3(6):10-1,2013
[14] Stefan Cosemans, Bram-Ernst Verhoef, Jonas Doevenspeck, Ioannis A. Papistas, Francky Catthoor, Peter Debacker, Arindam Mallik, and Diederik Verkest. Towards 10000TOPS/W DNN inference with analog in-memory computing a circuit blueprint, device options and requirements. In IEEE International Electron Devices Meeting, pages
2019.
[15] Ioannis A Papistas, Stefan Cosemans, Bram Rooseleer, Jonas Doevenspeck, M-H Na. Arindam Mallik, Peter Debacker, and Diederik Verkest. A 22 nm, 1540 TOP/s/W, 12.1 TOP/s/mm 2 in-memory analog suurix-veetor-multiile for DNN acceleration. In IEEE Custom Integrated Circuits Conference, pages 1-2. IEEE, 2021.
[16] Tayfun Gokmen and Yurii Vlasov. Acceleration of deep neural network training with resistive cross-point devices: Design considerations. Frontiers neuroscience, 10:333, 2016.
[17] Geoffrey W Burr, Robert M Shelby, Severin Sidler, Carmelo Di Nolfo, Junwoo Jang, Irem Boybat, Rohit S Shenoy, Pritish Narayanan, Kumar Virwani, Emanuele U Giacometti, al. Experimental demonstration and tolerancing of large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element. IEEE Transactions on Electron Devices,
-307, 2015.
[18] Sapan Agarwal, Steven J Plimpton, David R Hughart, Alexander H Hsia, Isaac Richter, Jonathan A Cox, Conrad D James, and Matthew Marinella. Resistive memory device require- ments for a neural algorithm accelerator. In International Joint Conference on Neural Networks, pages 929 938. IEEE, 2016.
[19] Paiyu Chen, Binbin Lin, I-Ting Wang, Tuohung Hou, Jieping Ye, Sarma Vrudhula, Jac-sun Sco, Yu Cao, and Shimeng Yu. Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. In IEEE/CCM International Conference on Computer-Aided Design, pages 194-199. IEEE, 2015.
[20] Vinay Joshi, Manuel Le Gallo, Simon Haefeli, Irem Boybat, Sasidharan Rajalekshmi Nan- dakumar, Christophe Piveteau, Martino Dazzi, Bipin Rajendran, Abu Sebastian, and Evangelos Fleftheriou. Accurate deep neural network inference using computational phase-change memory. Nature communications, 11(1):2473, 2020.

---

[21] Zhaoxian Wu, Tayfun Gokmen, Malte J Rasch, and Tianyi Chen. Towards exact gradient-based training on analog m-memory computing. In Advances Neural Information Processing Systems, 2024.
[22] Tayfun Gokmen and Wilfried Haensch. Algorithm for training neural networks on resistive device arrays. Frontiers in Neuroscience, 14, 2020.
[23] Tayfum Gokmen. Enabling training of neural networks on noisy hardware. Frontiers in Artificial Intelligence, 4:1-14, 2021.
[24] Malte Rasch, Fabio Carta, Omobayode Fagbohungbe, and Tayfun Gokmen. Fast and robust analog in-memory deep neural network training. Nature Communications, 15(1):7133-7147, 2024.
[25] Peng Yao, Huaqiang Wu, Bin Gao, Sukru Bure Eryilmaz, Xueyao Huang, Wenqiang Zhang, Qingtian Zhang, Ning Deng, Luping Shi, H-S Philip Wong, et al. Face classification using electronic synapses. Nature communications, 8(1):15199, 2017.
[26] Zhongrui Wang, Can Li, Peng Lin, Mingyi Rao, Yongyang Nie, Wenhao Song, Qinru Qiu, Yunning Li, Peng Yan, John Paul Strachan, et al. In situ training of feed-forward and recurrent convolutional memristor networks. Nature Machine Intelligence, 1(9):434-442, 2019.
[27] Yaoyuan Wang, Shuang Wu, Lei Tian, and Luping Shi. SSM: pigo- scheme for in situ training of imprecise memristor neural networks. Neurocomputing, 407:270-280, 2020.
[28] Shanshi Huang, Xiaoyu Sun, Xiaochen Peng, Hongwu Jiang, and Shimeng Yu. Overcoming challenges for achieving high in-situ training accuracy with emerging memories. In Design, Automation & Test in Europe Conference Exhibition, pages 1025-1030. IEEE, 2020.
[29] Weier Wan, Rajkumar Kubendran, Clemens Schaefer, Sukru Bure Eryilmaz, Wenqiang Zhang, Dabin Wu, Stephen Deiss, Priyanka Raina, He Qian, Bin Gao, et al. A compute-in iemoory chip based on resistive random-acee accesss memory. Nature, 608(7923):504-511 2022.
[30] Peng Yao, Huaqiang Wu, Bin Gao, Jianshi Tang, Qingtian Zhang, Wenqiang Zhang. J Joshua Yang, and He Qian. Fully hardware-implemees Nature, 577(7792):6411 1-66, 2020.
memristor convolutional neural network.
[31] S. R. Nandakumar, Manuel Le Gallo, Irem Boybat, Bipin Rajendran, Abu Sebastian, and Evangelos Eleftheriou. Mixed- preedis-eeisioon architecture based on computational memory for training deep neural networks. In IEEE International Symposium on Circuits and Systems, pages 1-5, 2018.
[32] S. R. Nandakumar, Manuel Le Gallo, Christophe Piveteau, Vinay Joshi, Giovanni Mariani, Irem Boybat, Geethan Karunaratne, Riduan Khadddam-Aljameh Urs Egger, Anastasios Petropoulos, Theodore Antonakopoulos, Bipin Rajendran, Abu Sebastian, and Evangelos Eleftheriou. Mixed- precision deep learning based on computational memory. Frontiers in Neuroscience, 14, 2020.
[33] Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
[34] Mohamed Watfa, Alberto Garcia-Orie Ortiz, and Gilles Sassatelli. Energy-based analog neural network framework. Frontiers in Computational Neuroscience, 17:1 14651, 2023.
[35] Benjamin Scellier, Maxence Ernoult, Jack Kendall, and Suhas Kumar. Energy-based learning algorithms for analog computing: a comparative study. Advances in Neural Information Processing Systems, 36, 2024.
[36] Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio, and Benjamin Scellier. Training end-to-end analog neural networks with equilibrium propagation. arXiv preprint arXiv:2006.019// 2020.
[37] Maxence Ernoult, Julie Grollier, Damien Querlioz, Yoshua Bengio, and Benjamin Scellier. Equilibrium propagation with continual weight updates. arXiv preprint arXiv:2005. 0404 168, 2020.
12

---

[38] Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60(2):223-3) 2018.
[39] Hyungjun Kim, Malte Rasch, Tayfun Gokmen, Takashi Ando, Hiroyuki Miyazoe, Jae-Joon Kim, John Rozen, and Seyoung Kim. Zero-shifting technique for deep neural network training on resistive cross-point arrays. arXiv preprint arXiv:1907 10228, 2019.
[40] Quan Xiao, Songtao Lu, and Tianyi Chen. A generalized alternating method for bilevel learning under the polyak-tojasiewiee condition. In Proc. Advances in Neural Info. Process. Syst., 2023.
[41] Michael Arbel and Julien Mairal. Non-convex bilevel games with critical point selection maps. In Advances in Neural Information Processing Systems, 2022.
[42] Han Shen, Quan Xiao, and Tianyi Chen. On penalty-based bilevel gradient descent method. In Proc. of International Conference on Machine Learning, 2023.
[43] Jeongyeol Kwon, Dohyun Kwon, Steve Wright, and Robert Nowak. On penalty methods for non- convex bilevel optimization and first-order stochastic approximation. In Proc. of International Conference Learning Representations, 2024.
[44] Malte J Rasch, Diego Moreda, Tayfun Gokmen, Manuel Gallo, Fabio Carta, Cindy Goldberg, Kaoutar EI Maghraoui, Abu Sehastian, and Vijay Narayanan. A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays. IEEE International Conference on Artificial Intelligence Circuits and Systems, pages 1-4, 2021.
[45] Geoffrey W Burr, Matthew BrightSky, Abu Sehastian, Huai-Yu Cheng, Jau- Yi Wu, Sangbum Kim, Norma Sosa, Nikolaos Papandreou, Hsiang-L Lung, Haralampos Pozidis, Evangelos Eleftheriou, and Chung H Lam. Recent Progress Thase-Change Memory Technology. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 6(2):146-162, 2016.
[46] Manuel Le Gallo and Abu Sebastian. An overview of phase-change memory device physics. Journal of Physics D: Applied Physics, 53(21))2 2020.
[47] Jun- Woo Jang, Sangsu Park, Yoon-Ha Jeong, and Hyunsang Hwang. ReRAM-based synaptic device for neuromorphic computing. In IEEE International Symposium on Circuits and Systems, pages 1054-1057, 2014.
[48] Jun-Woo Jang, Sangsu Park, Geoffrey W Burr, Hyunsang Hwang, and Yoon-Ha Jeong. Opti- mization of conductance change in Pr Mn0,,nnOaaattt synaptic devices for neuromorphic systems. IEFE Electron Device Letters, 36(5):457-459, 2015.
[49] Tommaso Stecconi, Valeria Bragaglia. Malte J Rasch, Fabio Carta, Folkert Horst, Donato Falcone, Soficke C Ten Kate, Nanbo Gong, Takashi Ando, Antonis Olziersky, et al. Analog resistive switching devices for training deep neural networks with the novel Tiki-Taka algorithm. Nano Letters, 24(3):866-87) 2024.
[50] Scokjae Lim, Myounghoon Kwak, and Hyunsang Hwang. Improved synaptic behavior of CBRAM using internal voltage divider for neuromorphic systems. IEEE Transactions on Electron Devices, 6579):3976-3981, 2018.
[51] Elliot J Fuller, Scott T Keene, Armantas Melianas, Zhongrui Wang, Sapan Agarwal, Yiyang Li, Yaakov Tuchman, Conrad D James, Matthew J Marinella, J Joshua Yang, Alberto Salleo, and A Alec Talin. Parallel programming of an ionic Hoating-gate memory array for scalable neuromorphic computing. Science, 364(6440):570-577 2019.
[52] Jianshi Tang, Douglas Bishop, Seyoung Kim, Matt Copel, Tayfun Gokmen, Teodor Todorov, SongHoon Shin, Ko-Tao Lee, Paul Solomon, Kevin Chan, et al. ECRAM as scalable synaptic cell for high-speed, low-power neuromorphic computing. In IEEE International Electron Devices Meeting, pages 13-1. IEFF, 2018
[53] Murat Onen, Nicolas Emond, Baoming Wang, Difei Zhang, Frances M Ross, Ju Li, Bilge Yildiz, and Jesús Del Alamo. Nanosecond protonic programmable resistors for analog deep learning. Science, 377(6605) 2022.
13

---

[54] Seungchul Jung, Hyungwoo Lee, Sungmeen Myung, Hyunsoo Kim, Seung Keun Yoon, Soon- Wan Kwon, Yongmin Ju, Minje Kim, Wooseok Yi. Shinhee Han, et al. A crossbar array of magnetoresistive memory devices for in-memory computing. Nature, 601(7892)211-216, 2022.
[55] Zhihua Xiao, Vinayak Bharat Naik, Jia Hao Lim, Yaoru Hou, Zhongrui Wang, and Qiming Shao. Adapting magnetoresistive memory devices for accurate and ti--chi-training-free in-memory computing. Science Advances, 10(38)eeddp3700, 2024.
[56] Rui Guo, Weinan Lin, Xiaobing Yan. T Venkatesan, and Jingsheng Chen. Ferroic tunnel junctions and their application in neuromorphic networks. Applied physics reviews, 7(1), 2020.
[57] Panni Wang, Feng Xu, Bo Wang, Bin Gao, Huaqiang Wu, He Qian, and Shimeng Yu. Three- dimensional NAND flash for vector-matrix multiplication. IEEE Transactions on Very Large Scale Integration Systems, 27(4):988-99) 2018.
[58] Yachen Xiang, Peng Huang, Runze Han, Chu Li, Kunliang Wang, Xiaoyan Liu, and Jinfeng Kang. Efficient and robust spike-driven deep convolutional neural networks based on NOR flash computing array. IEEE Transactions on Electron Devices, 67(6):2329-233) 2020.
[59] Farnood Merrikh-Bayer Xinjie Guo, Michael Klachko, Mirko Prezioso, Konstantin K Likharev, and Dmitri B Strukov. nigh-performance mixed-signal neurccoputtng with nanoscale floating- gate memory cell arrays. IEEE Transactions on Nneural Networks and Learning Systems, 29( 29)444477824 2017.
[60] Bonan Zhang, Peter Deaville, and Naveen Verma. Statistical computing framework and demonstration for in-memory computing systems. ACM//EEE Design Automation Conference, pages 979-984, 2022.
[61] Peter Deaville, Bonan Zhang, Lung- Chen, and Naveen Verma. A maximally row- row-paralle MRAM tomemory-computiee macro addressing readout circuit sensitivity and area. In IEEE European Solid State Circuits Conference, pages 75-78. IEEE, 2021.
[62] lung-Hoon Lee, Mong-Hyeoo, Lim, Hongsik Jeong, Huimin Ma, and Luping Shi. Exploring cycle-to-cycle and sorrice-to-deviie variation tolerance in mlc storage-based baaed neural network training. IEEE Transactions on Electron Devices, (6(5):2172-2178 2019.
[63] Jintao Zhang, Zhuo Wang, and Naveen Verma. In-memory computation of a machine-learning classifier in a standard 6t SRAM array. IEEE Journal of Solid-State Circuits, 52(():915- 2017.
[64] Tayfun Gokmen, Malte J Rasch, and Wilfried Haensch. The marriage of training and inference for scaled deep learning analog hardware. In IEEE International Electron Devices Meeting, pages 22 3. IEEE, 2019.
[65] Corey Lammie, Athanasios Vasilopoulos, Julian Büchel, Giacomo Camposampiero, Manuel Le Gallo, Malte Rasch, and Abu Sebastian. Improving the accuracy of analog- in-memory computing accelerators post-training In IEEE International Symposium on Circuits and Systems, pages 1-5. IEEE, 2024.
[66] Qing Jin, Zhiyu Chen, Jian Ren, Yanyu Li, Yanzhi Wang, and Kaiyuan Yang. PIM-QAT: Neural network quantization for processing-in-memory (PIM) systems. arXiv preprint arXiv:2209.08617 2022.
[67] Malte J Rasch, Charles Mackin, Manuel Le Gallo, An Chen, Andrea Fasoli, Frédéric Odermatt, Ning Li, S. R. Nandakumar, Pritish Narayanan, Hsinyu Tsai, et al. Hardware-awan training for large-scale and diverse deep learning inference workloads using in-memory computing-basee accelerators. Nature Communications, 14(1):5282, 2023.
[68] Bonan Zhang, Chia- Yu Chen, and Naveen Verma. Reshape and adapt for output quantization (RAOQ): Quantization-awar training for in-memory computing systems. In International Conference Machine Learning, 2024.
14

---

[69] Beiye Liu, Hai Li, Yiran Chen, Xin Li, Qing Wu, and Tingwen Huang. Vortex: awaar aware training for memristor x-bar. Proceedings of the 52nd Annual Design Automation Conference. pages 1-6, 2015.
[70] Abhiroop Bhattacharjee, Lakshya Bhatnagar, Youngeun Kim, and Priyadarshini Panda. NEAT: Non-line Non-linearity aware training for accurate and energy-efficicen implementation of neural networks on It-Ir memristive crossbars. arXiv preprint arXiv:20/2.0026. 2002000 2020.
[71] Tayfun Gokmen, Murat Onen, and Wilfried Haensch. Training deep convolutional neural networks with resistive cross-point devices. Frontiers in neuroscience, 1 11:538, 2017.
[72] Zhongrui Wang, Saumil Joshi, Sergey Savel'Ey, Ev, Wenhao Song. Rivu Midya, Yunning Li, Mingyi Rao, Peng Yan, Shiva Asapu, Ye Zhuo, et al. Fully memristive neural networks for pattern classification with unsupervised learning. Nature Electronics, 1(2):137-145, 2018.
[73] Nanbo Gong, Malte Rasch, Soon-Cheon Seo, Arthur Gasasira, Paul Solomon, Valeria Bragaglia, Steven Consiglio, Hisashi Higuchi, Chanro Park, Kevin Brew, et al. Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. In IEEE International Electron Devices Meeting, 2022.
[74] Zhaoxian Wu, Quan Xiao, Tayfun Gokmen, Hsinyu Tsai, Kaoutar El Maghraoui, and Tianyi Chen. Pipeline based model training on analog in-memory accelerators. arXiv preprint arXiv:2410. 15155, 2024.
[75] Logan G Wright, Tatsuhiro Onodera, Martin M Stein, Tianyu Wang, Darren T Schachter, Zoey Hu, and Peter I. McMahon. Deep physical neural networks trained with backpropagation. Nature, 601(789)):549 555555 2022.
[76] Ali Momeni, Babak Rahmani, Benjamin Scellier, Logan G Wright, Clara C McMahon, Peter andWanjura, Yuhang Li, Anas Skalli, Natalia G. Berloff, Tatsuhiro Onodera, Ilker Oguz, Francesco Morichetti, Philiip del Hougne, Manuel Le Gallo, Abu Sebastian, Azalia Mirhoseini, Cheng Zhang, Danijela Marković, Daniel Brunner, Christophe Moser, Sylvain Gigan, Florian Marquardr, Aydogan Ozcan, Julie Grollier, Andrea J Liu, Demetri Psaltis, Andrea Alú, and Romain Fleury. Training of physical neural networks. arXiv preprint arXivv2406.03332, 2024.
[77] Demetri Psaltis, David Brady, Xiang-Guang Gu, and Steven Lin. Holography in artificial neural networks. Nature, 343(6256):325-33,, 1990.
[78] Tyler W Hughes, Ian AD Williamson, Momchil Minkov, and Shanhui Fan. Wave physics as an analog recurrent neural network. Science advances, 5(12)xeaa66946, 2019.
[79] Alexander N Tait, Thomas Ferreira De Lima, Ellen Zhou, Allie X Wu, Mitchell A Nahmias, Bhavin J Shastri, and Paul R Prucnal. Neuromorphic photonic networks using silicon photonic weight banks. Scientific reports, 7(1):7430, 2017.
[80] Nanbo Gong. T Idé, S Kim, Irem Boybat, Abu Sebastian, V Narayanan, and Takashi Ando. Signal and noise extraction from analog memory elements for neuromorphic computing. Nature communications, 9():2110, 2018
[81] Mingyi Rao, Hao Tang, Jiangbin Wu, Wenhao Song, Max Zhang, Wenbo Yin, Ye Zhuo, Fatemeh Kiani, Benjamin Chen, Xiangqi Jiang, et al. Thousands conductance levels memristors integrated on CMOS. Nature, 615(7954):823-829 2023.
[82] Deepak Sharma, Santi Prasad Rath, Bidyabhusan Kundu, Anil Korkmaz, Damien Thompson, Navakanta Bhat, Sreebrata Goswami, Stanley Williams, and Sreetosh Goswami. Linear symmetric self-selecting 14-bit kinetic molecular memristors. Nature, 633(8030):560-566, 2024.
[83] Wenhao Song, Mingyi Rao, Yunning Li, Can Li, Ye Zhuo, Fuxi Cai, Mingche Wu, Wenbo Yin, Zongze Li, Qiang Wei, et al. Programming memristor arrays with arbitrarily high precision for analog computing. Science, 383(6685):903-110 2024.
15

---

[84] Shubham Jain, Hsinyu Tsai, Ching-Tzu -zu Chen, Ramachandran Muralidhar, Irem Boybat, Mar- tin M Frank, Stanislaw Wožniak. Milos Stanisavljevic. Praneet Adusumilli, Pritish Narayanan. et al. A heterogeneous and programmable compute-in-memory accelerator architecture for analog-ai using dense 2-d mesh. IEEE Transactions on Very Large Scale Integration Systems, 31(1):114-127, 2022.
[85] Yurii Nesterov. Introductory Lectures Convex Optimization: A Basic Course. Springer, 2013.
16

---

Appendix for "Analog In-memory Training on Non-ideal Resistive Elements: Understanding the Impact of Response Functions"
Table of Contents
Literature Review
17
B Relation with the result in [21]
19
C Dynamics of Non-ideal Analog Update
21
Comparison of Residual Learning v2 and Tiki-Taka v2
22
E Estimation of time consumption
23
F
Useful Lemmas and Proofs
23
F.1
Lemma 1: Properties of weighted norm
23
F2 Lemma 2: Properties of weighted norm
23
F3 Lemma 3: Lipschitz continuity of analog update
23
Lemma 4: Flement-wise product error
................................
24
G Proof of Theorem 1: Implicit Bias of Analog Training
24
H Proof of Theorem 2: Convergence of Analog SGD
27
Proof of Theorem 3: Convergence of Residual Learning
30
L1 Main proof
30
1.2
Proof Lemma 5: Descent sequence W
34
1.3 Proof of Lemma 6: Descent of sequence Wi
37
1.4 Proof of Lemma 7: Descent of sequence
38
1.5 Proof of Corollary 1: Exact convergence of Residual Learning
41
J Proof of Theorem 6: Convergence of Analog GD
42
Simulation Details and Additional Results
43
Power and Exponential Response Functions
K.2
Least squares problem
44
K.3
Classification problem
45
K.4
Additional performance on real datasets
45
K.5
Ablation study on cycle variation
46
K.6
Ablation study on various response functions
46
L Broader Impact
A Literature Review
This section briefly reviews literature that is related to this paper, as complementary to Section 1.
Training on AIMC hardware. Analog training has shown promising early successes in tasks such as face classification [25] and digit classification [26], achieving 0000x lower energy consumption than digital implementations. Researchers are also exploring approaches to mitigate the impact of hardware non-idealities. For example, [27, 28] proposes leveraging the momentum technique to stabilize training by reducing noise. To address other potential non-idealities, hybrid training paradigm is also being explored.
[29] leverages the chip-in-the-loog technique to train models layyre,,-layer, while [30] proposes to train the backbone in the digital domain and train the last
17

---

layer in the analog domain. In general, these works have provided valuable insights into analog training, shedding light on many critical technical challenges. However, their focus has largely been on experimental and simulation aspects, with limited systematic and theoretical analysis of how specific imperfections affect the training process. our paper, we present an alternative viewpoint and novel tools to explore the effects of non-idealities.
Resistive element. A series of works seeks various resistive elements that have near-constant or at least symmetric responses. The leading candidates currently include PCM [45, 46], ReRAM [47-49], CBRAM [50, 51], ECRAM [52, 53), MRAM [54, 55], FTJ [56] or flash memory [57-59].
However, a resistive element with symmetric updates may not be the best option for manufacturing. For example, although ECRAM provides almost symmetrie updates, it remains less competitive than ReRAM, which offers faster response speed and lower pulse voltage [49]. The suitability of the resistive elements is evaluated using metrics across multiple dimensions, including the number of conductance states, retention, material endurance, switching energy, response speed, manufacturing cost, and cell size. Among them, this paper is only interested in the impact of response functions in the training.
Imperfection of AIMC hardware. Besides the response functions, analog training suffers from all kinds of hardware imperfection, especially when the task's ' scale increases, like asymmetric update [17, 191, reading/writing noise [18, 60, 61], device/cycle variations [62], non-linear current response due to IR drop [18, 6, 63]. This paper mainly focuses on asymmetric response functions. However, this paper is not trying to diminish the importance of other hardware imperfections but rather foeuses on one of the primary ones known to be very important (19, 16].
Hardware-aware training. For inference on AIMC hardware purposes, models pretrained on digital hardware will be programmed on analog hardware. Due to hardware imperfections, the pretrained models suffer performance drops. Hardware-aware training (HWA) is technique designed to bridge the gap between ideal pretrained models and non-ideal programmed models. In contrast to standard training methods, hardware-an aardeere-aware training explicitly incorporates device-specific imperfections, such as weight drift [20], device fail [64], bounded dynamic range [65], quantization error from ADC [66-68], device variation [69], and non-linear current output [70], into the training loop. By modeling these constraints during training, the learned parameters become inherently more robust to real- world deployment conditions. It is worth highlighting that HWA is still performed on digital hardware, and the trained model will be programmed onto AIMC hardware. On the contrary, this paper considers a different, more challenging setting in which training is performed directly on analog hardware.
Gradient-based training on AIMC hardware. A series of works focuses on implementing back- propagation (BP) and gradient-based training on AIMC hardware. The seminal work [16. 71] leverages the rank-one structure of the gradient and implements Analog SGD by a stochastic pulse update scheme, rank-update. Rank-update significantly accelerates the gradient descent step by avoiding the ((N2)) computation of gradients and instead using two vectors with O(N) elements for the update, where N is the number of matrix rows and columns. To alleviate the asymmetric update issue, researchers also design various of Analog SCD variants, Tiki-Taka algorithm family [22-24]. The key components of Tiki-Taka are the introduction of a residual array to stabilize training. Apart from the rank-update, a hybrid scheme that performs forward and backward passes in the analog domain but computes gradients in the digital domain has been proposed in [31, 32]. Their solution, referred to as mixed-precision update, provides a more accurate gradient signal but requires 5x-10x higher overhead compared the rank-update scheme [24).
Attributed to these efforts, analog training has empirically shown great promise, achieving accuracy comparable to that of digital training chip prototypes while reducing energy consumption and training time [72, 73]. Simultancously, the parallel acceleration solution with AIMC hardware is under exploration [74]. Despite its good performance, it remains mysterious when and why the analog training works.
Theoretical foundation of gradient-based training. The closely related result comes from the convergence study of Tiki-Taka [21]. Similar to our work, they attempt to model the dynamics and provide the convergence properties of Analog SGD and Tiki-Taka. However, their work is limited to a special linear response function. Furthermore, their paper considers a simplified version of Tiki Taka, with
y 0 (see Section 4). As we will show empirically and
18

---

theoretically, benefits from non-zero 7. Consequently, We compare the results briefly in Table and comprehensively in Appendix B.
7
Generic response
Linear response
Tiki-Taka [21]
0
X
o (V K 1 1-33P man!
Tiki-Taka [Corollary 1]
* 0 O
(V
(V P Pmxx/7
Table 2: Comparison between our paper and [21]. Mixing-coefficiee coefficient ry is a Tiki-Taka. "Generic response" and "Linear response" columns the convergence rates in the corresponding settings. represents the number of iterations. M min and P / p saturation while the former reduces the latter linear response functions.
of
measure the
Energy- models and equilibrium propagation. Apart from achieving explicit gradient signals by the BP, there are also attempts train models based on equilibrium propagation (EP, [33]), which provides biologically plausible alternative to traditional BP. EP is applicable to a series of energy-basee
models. where the forward pass performed by minimizing an energy function [34, 35]. The update signal in EP is computed by measuring the output difference between a free phase and an active phase. EP eliminates the need for BP non-local weight transport mechanism, making it more compatible with neuromorphic and energy-efficient hardware [36, 37J. We highlight here that the approach to attain update signals (BP or EP) is orthogonal to the update mechanism (pulse update). Their difference lies in the objective f (W&)). which is hidden in this paper. Therefore, building upon the pulse update, our work is applicable both BP and EP.
Physical neural network. The model executing on AIMC hardware, which leverages resistive crossbar array to accelerate MVM operation, is concrete implementation of physical neural networks (PNNs, [75, 76]). PNN is generic concept of implementing neural networks via physical system which a set of tunable parameters, such as holographic grating [77], wave-based systems [78], and photonic networks [79]. Our work particularly focuses training with AIMC hardware, but the methodology developed in this paper can be transferred to the study of other PNNs.
B Relation with the result in [21]
Similar to this paper, [21] also attempts to model the dynamics of analog training. They show that Analog SGD converges to a critical point of problem (1) inexactly with an asymptotic error, and Tiki-Taka converges to critical point exactly. In this section, we compare our results with our results and theirs.
As discussed in Section I, [21] studies the analog training on special linear response functions
w
q+ (w)
q- (-w
(16)
It can be checked that the symmetric point is 0 while the dynamic range of it [-T, The symmetric and asymmetric components are defined by F(W)= and G(W) w respectively. It indicates Fraxx 1. Furthermore, they assume the bounded weight saturation by assuming bounded weights, i.e. W& W Wmax, vk E [K] with a constant Wmax т. Under this assumption, the lower
bounds of response functions are given by
W Wmax
W max
4max = 1+
Amin = 1
(17)
W 1loo
2
min))(WW)) = min min(Q+ (Ww) @ (AAI)@QQ(C Q-(w)) =1
(18)
W max
2
M ASGD min(M((W.))
(19)
Challenge of analyzing the convergence of Tiki-Taka with generic response functions. For linear response functions (16), the recursion of residual array Ps has special structure, where the
19

---

first and the biased term can be combined
Pk+1 = Pk a (WW((W) a [<W: Pk
(20)
= (1 a [V[(W::))) O Ps aa
which a weighted average of Ps and V/(Wk: Ex). Consequently, Ps can be interpreted as an approximation of the average gradient. From this perspective, the transfer operation can be interpreted as biased gradient descent. However, given a generic G(-) ) the combination is no longer viable, bringing difficulties to the analysis.
Convergence of Analog SGD. As we will show in Remark at the end of Appendix H, inequality (8) can be improved when the saturation never happens
K-
((WWl(
(21)
k=0
2 G(WA)
4F2 max M min
(f(W)) ff)
2F F
MW
K
VF(WA)
0o (V (Vurww)
- -
K
[ 1
+ 2
K
W max
W 1
k=0
which is exactly the result in [21].
Convergence of Tiki-Take It is shown empirically that non-zero 7 in (10) improves the training while this paper considers a non-zero 7, Pxll S Punxx
accuracy [22]. However, [21] only considers
With the linear response, if also assume the bounded saturation of Pa by letting
the minimal average response function is given by M 1 - [raax The upper bound in Corollary becomes
K-1 K
(f(W)
f
[V 0o 1
(22)
P2 max/
/22
K
k=0
As comparison, without a non-zero y, [21] shows that convergence rate of Tiki is only
1 K-1
(f(W)) - )
Wf 00 0
(23)
K
1-33P2
33P2 max
K
Even though it is not completely fair comparison, since the two papers rely different assump- tions, is still worth comparing their analyses. [E([I
[21] assumes the noise should be non-zero, i.e. O vd E [D] holds for a non-zero constant Czoise- Instead, this paper does not make this assumption but assumes that the objective is strongly convex. As mentioned in Section 4. the strong convexity is introduced only ensure the existence of (*WW).. Therefore,
believe it can be relaxed and that the convergence rate can remain unchanged, which is left for future work. Taking that into account, we believe the comparison provide insight into how the non-zero improves the convergence rate of Tiki-Taka.
Why does non-zero y improve the convergence rate of Tiki-Taka? As discussed in Section 4, Pk is interpreted as residual array that optimizes f(W + +PP). the ideal setting that F(W) = and G(W) 0, it can be shown that Ps converges to P* (W&) ) if W.. is fixed and Px is kept updated, even though the W₁ * W* (hence (V( #0).
Instead, without a non-zero 7, [21] interprets Ps as an approximation of clear gradient by showing
Eeg P|| CV (WW)(W))
(24)
-1- - |P CVS(W)) + remainder
where C, C' are constants depending on the resistive element and model dimension, and the "remain- der" is the non-essential terms. Consider the case that W; is fixed and (10) is kept iterating, in which
20

---

case the increment on Ps: is constant since 7 = 0. Telescoping (24), we find that the upper bound above only guarantees that
lim - ((CC)///WW
(25)
k-+0c
which means that Ps tracks the gradient accurately only when V (WWx reaches zero asymptotically. The less accurate approximation results in a slower rate than the one reported in this paper.
C Dynamics of Non-ideal Analog Update
This section presents details on how to obtain the dynamics of the analog update (3) appearing in Section 2, along with its error analysis. The primary distinction between digital and analog training is the method of updating the weight. As discussed in Section I, the weight update in AIMC hardware implemented by Analog Update, which sends a series of pulses to the resistive elements.
Pulse update. Consider the response of one resistive element in one cycle, which involves only one pulse. Given the initial weight w, the updated weight increases or decreases by about Awmin depending on the pulse polarity, where AWmin > 0 is the response granularity determined by elements. The granularity is further scaled by factor, which varies by the update direction due to the asymmetric update property of resistive elements. The notations q+ (-) and q-(-) are used to denote the response functions on positive or negative sides, respectively, to describe the dominating part of the factor. In practice, the analog noise also causes deviation of the effective factor from the response functions, referred to as cycle variation. random variable & with expectation 0 and variance 1. Taking all of them into account, with {+, }} being the update direction, the updated weight after receiving one pulse is Ug s) where -.,: IR [+.-} IR is the element-dependent update that implements the resistive element, which can be expressed as
represented by the magnitude oa times
(g( s) == + Awmia (q:(w) + ozE)
(26)
w
8 8= + Awmin (q+(w) + orLa),
w Awmin (q- (w) + ocLe), =
The typical signal and noise ratio T./ 0e/qx(w) is roughly 5%-100% [80, 49], varied by the type of resistive elements. Furthermore, the response functions also vary by elements due to the imperfection in fabrication, called element variation (also referred as device variation in literature [16]).
Equation (26) is a resistive element level equation. Existing work exploring the candidates of resistive elements usually reports the response curves similar to Figure 1, [73, 52, 49]. Taking the difference between weights in two consecutive pulse cycles and adopting statistical approaches [80], all the element-dependent quantities, including AWmin: 4+( ), (-(-) and oe, can be estimated from the response curves of the resistive elements.
Analog update implemented by pulse updates. Even though the update scheme has evolved over the years [16, 71], we discuss simplified version, called Analog Update, retain the essential properties. To update the weight w by Aw, series of pulses are sent, whose bit length (BL) is computed by BL :=
After received BL pulses, the updated weight w' can be expressed as the function composition of (26) by BI. times
Awl
w/ = Üg o Úg o ..... (w, 8) =: ÜBI. (w, s).
(27)
x BL
Roughly speaking, given an ideal response q+ (w) g- (w) = 1 and o. = 0, BL pulses, with Awmin increment for each individual pulse, incur the weight update Aw. Since the response granularity A A'min scaled by the response function 9u (w), the expected increment is approximately scaled by (ww as well. Accordingly, we propose approximate dynamics of Analog Update is given by w/ Rs (w, Awr) where (w, Aur is defined in (3). The following theorem provides an estimation of the approximation error. It has been shown empirically that the response granularity can be made sufficiently small for updating [81, 82], implying A Awmin & Aw. Therefore, we establish the error estimate for the approximation under small-responee resooss-ggggg condition.
21

---

Theorem 4 (Error from discrete pulse update), Suppose the response granularity is sufficiently small such that Awiin I o(Aw). With the update direction signl Aw), the error between the true update BL Ug,((, s) and the approximated U,( Aw) bounded by
[ s) - Ug( Aw)] = 0.
lim Aw-5
(28)
JUBL (w,
In Theorem 4, |ÜBL(W, (w, s) - (w, Aw) is the error between the true update and the proposed dynamics, while 17 (w, s) w| is the difference between original weight and the updated one. Theorem 4 shows that the proposed dynamics dominate the update, and the approximation error is negligible when Aw is small, which holds Aw always includes small learning rate in gradient- based training.
Takeaway. Theorem 4 enables to discuss the impact of response functions directly without dealing with element-specific details like response granularity AWmin and cycle variation de- Response functions are the bridge between the resistive element level equation (pulse update (26)) and the algorithm level equation (dynamics of Analog Update (3)).
Proof of Theorem 4. Recall the definition of the bit length is
Aw
Aww
BL
(29)
Wimin
Wanin
leading to
BT. Atmmin
|Aw/|
<
Wanin
or
|s BL. AL
Awl < Ammin-
(30)
Notice that the update responding each pulse ((Awmin) term. Directly manipulating (BL (w, s) and expanding it in Taylor series to the first-order term yields
BL
BL
UBL (w. s) = + AWmin
go (ww O(tA wmin) )) + AWmia
Σ JeE
(31)
p
BL -1
BL-1
BL-1
= w + 8 Atmii
(1(w)) + Σ
((t(AWiin))) + A Winin
Σ
aehi
1=
1=
Awmia BL as (w) + ( ( (Awmin) ) A Awmin vBL. 0c5
= w + Aw (w) + (s BL Awmin - Aw) + (((Am)(A))))))))) + Awmin
Aw abf
Ug(( (w, Aw) + O(Awmin) (((Aw))) + θ Awmin VAw -e)
where & : BL DBL-L & is the accumulated noise with variance 1. The proof is completed.
D Comparison of Residual Learning v2 and Tiki-Taka v2
Introducing digital buffer, the proposed Residual Learning v2 has similar form of Tiki-Taka v2 [23]. However, there are slight differences. Tiki-Taka v2 updates the digital buffer by
H Hr 3(Px+1 +Ex)
(32)
which do not include a decay coefficient front of HE. Furthermore, Tiki-Taka v2 uses gradient ([(W,; that are solely computed on the main array W,. Instead, Residual Learning v2 computes gradient on a mixed weight Wt Wt + Pt. suggested by the ablation simulations in 6, the training benefits from non-zero 7
22

---

E Estimation of time consumption
Residual Learning introduces extra resistive element array, which increases overhead. How- ever, the extra overhead is affordable in practice. Compared to Analog SGD, the analog memory requirement doubles, but the latency remains almost unchanged since Residual Learning does not explicitly compute the mixed weights during the forward and backward passes. As [83] suggests, Wa and Pa can share the same maalog-digital convertor (ADC), which implements the weight mixing without introducing extra latency. On the other hand, as suggested by [22], the forward, backward, and update steps for Wx and Ps are performed parallel. thereby avoiding a significant increase in latency. Consequently, introducing an extra residual array does not incur substantial extra latency.
Following the evaluation in Table in [24], compared the lateney of Analog SCD and Residual Learning in Table 3. We consider that each gradient update step requires 32 pulse cycles, each consuming nanoseconds (ns). Following the estimation in [24], preprocessing the input vectors for each MVM operator takes 5.9ns. Compared with Analog SGD, Residual Learning adds an extra MVM step read from Ps. The results suggest that the overhead is only about 2x that of Analog SGD. As the update is typically not the bottleneck of the whole training process, the extra overhead is affordable.
Analog SGD
Residual Learning
Forward/backward [84]
40.0
40.0
Update
165.9
371.8
Table 3: Comparison of time (nanosecond) consumption in each layer
F Useful Lemmas and Proofs
F1 Lemma 1: Properties of weighted norm
Lemma 1. [[]]ss has the following properties: (a) |[|]s = W @ V5I|: (b)
[W]ls S
V (c) |[ /Wllmmmi((S])
Proof of Lemma 1. The lemma can be proven easily by definition.
F.2
Lemma 2: Properties of weighted norm
direct property from Definition 1 is that all g+(.). g- 9- and F(-) are bounded, as guaranteed by the following lemma.
Lemma 2. The following statements are valid for all W E R. (a) F(.) is element-wise upper bounded by constant Fuax > i.e., [[ < Fmaxi (b) Q-(-) and VQ-(-) are element- wise bounded by LQ, i.e., [ (W)ll S LQ. VVQ (W) loo I LQ-
F.3 Lemma 3: Lipschitz continuity of analog update
Lemma 3. The increment defined in (5) is Lipschitz continuous with respect to AW under any weighted norm || [lss, for any W, AW, AW' RDD and S E R2, it holds
AA @ F(W) |AW| @ G(W) (AW" @ F(W) AW' @ G(W))
(33)
< F max AW - A.
Proof of Lemma 3. proven similarly. Notice that the absolute value and vector norm || ||. scalar multiplication x and element-wise multiplication O are equivalent at that situation. We adopt both notations just for readability.
We prove for the case where
and S and the general case can be
AW @ F(W)- AW| 6 G(W) - (AW' @ F(W)- Aw"l O G(W))]
(34)
23

---

-AW AW") O F(W) - - 3 G(W)II-
Since |AW - AW 2 |||W AWI and [G(W] - F(W)) ) we have
[(W AW") @ F(W) - (IAW] |AW")) @ G(W)
(35)
(AW AW") @ (F(W)-(G(W))
AW AW"I [F(W) - [G(W)]
Fax AW AWWI
which completes the proof.
F44 Lemma 4: Element-wise product error
Lemma 4, Let V. RDD be vectors indexed by [D]. Then the following inequality holds
(U.V Q) C (U, C_ (11),1V)>
(36)
where the constant C+ and C_ defined by
C+ = (max(Q) + min( C_ = (max - min
(37)
Proof of Lemma 4. For any vectors U, V. R², it is always valid that
(U,V Q) = E
(38)
dEID)
Σ
Uld[V
E
[V
min
Σ
IulaVaaa
max x
Σ
[allVa
C+
Σ
[[a[V]a - -C-
Σ
[[]][V]al
+ +C+
Σ
UllaVVaa -C-
Σ
[VaaVVaall
C4
Σ
[V] C_ Σ
[[U]|[V]al
dE(D) = V) C_ (IU),
dE(D)
where (a) uses the following equality
min min (Q)U]alV C+1V)aVV -C max max maxx()]Uja[V]J C+IU1alVia C_IVolaVial ir[u][[V] 0.
ir 2 0,
(39)
(40)
This completes the proof.
G Proof of Theorem 1: Implicit Bias of Analog Training
In this section, we provide a full version of Theorem 1. Before that, we introduce the asymmetry ratio and its element-wise anti-derivative. Since F(-) and G(-) act element-wise, define R(-)
R + P and Re R + R element-wise by
G(W) F(W)
[W]
R(W)
and [R)(W)) /
d d (D),
(41)

---

where the division is taken element-wiss It holds that [[W]. (W) = [R(W]a for each coordi-
d
nate E [D], and V (C, Rc(W)) = C' o R(W) for any vector C' I P_ Consequently, if R(W) is strictly monotonic, (W) 2 and it reaches its minimum value the symmetrie point W where R(W")= = 0.
Define the scaled effective update of Analog SGD at W as
G(W)
T(W) = VVW -4 ([W
(42)
F(W)
direct verification shows that if (W:, 0, then Analog SGD (2) (equivalently, (5) with AW; -aVf(WWss Eu)) is stable at W₂ in conditional expectation, i.e., B&g [Wk+1] = Wk.
Rather than characterizing the exact limit of Analog SGD, establish local statement showing the existence of a point that is simultaneously (i) nearly critical for fy; and, (ii) nearly stationary for the mean Analog SGD dynamics.
Theorem 5 (Implicit Penalty, full version of Theorem 1). Suppose W is the unique minimizer of problem (1). Define
W V F(W") + Diag(2) V
("" (V"F(W"*) W Diag(2)) V)( (43)
where (M) E DxD denotes the diagonal matrix whose diagonal entries are given by the vector M E IR. Let Σ := [ E IR be the element-wise first moment of stochastic gradients W*. Analog SGD implicitly optimizes
min fe(W) == f(W) + (Z, WER
(44)
the sense that as | WI|l + 0,
[T(Ww)
V (Wo)]]] +0 0 and
min [[ + 0. (45)
II,
Proof of Theorem 5. By the definition of W it holds that
I W+| [ Diag ()) VR(W9))-'V> (W Wo)l @[[[ - W*ll)
(46)
|| - = [[V2/(W>) + Diag Σ) V Diag Σ) (W( - Wo)l
-(I - I)]
(47)
We separately show the two parts of Theorem 5. We first show that [T(W') < O(I 11)
and ||T(W))]| OlIlWo WIIl) and [[T(W*)] O(Il W'Il). It reaches the first limit in
(45). Similarly, we show the second limit by showing that [W )| S O(I II and [|V]z(W>)]| = O(]lW> - (lI and [V (W*)]l @[|l - W*
(Step 1a) Proof of (W= s O - ) The gradient of fs(W) is given by v (W) (/W)) + V R(W).
(48)
Leveraging the fact that Vf(W*) = 0,
c(w F[Wo)
0, as well as Taylor expansion given by
Vf(W") V2 f(W') (I w O( W=)3),
(49)
G(W")
VR(W (W w + O( - -W9)3),
(50)
F(W-)
where "(WW~~W**? and O((W~WW9)) 2 are vectors with norms - and || W respectively. We bound the gradient of the penalized objective as follows
G(W*)
[V== = Vf(W")) -o F(W*)
(51)
25

---

[<f(">> - - to ( + (1933]
= - - 0 = O(1
=
- 1
where the last inequality holds by (46) and (47).
(Step 1b) Proof of [T(W)] = @(I - - and [T(W>)] < W'll).
V (W [V/(())) + Eo -e R(W") Σo R(W"] O(IIW* W)
(52)
V = [[V/(W)+ Eo R(W)) = V/ = [ -W
(53)
(Step 2a) Proof of [T(WW>]| O(| 13) By the definition of T(W*), we have
Eg - G(W*)
[TW =
(54)
F(W*)
G(W7)
<
VS(W>) - E. 901
F(W-)
G(W*)
(E [Ec[IV((W'<<)) 01 -<e///((WWWW 0 G(W")
$ VS(W") - E4 / O1)
F(W-)
F(W*)
G(W
VV (W + (E")) E VV(W O F(W-)
The first term in the right-hand side (RHS) of (54) is bounded by (51). By applying ||x|-|ull S -x- for any 2, IR at all components, the second term in the RHS of (54) is bounded by
G(W*)
[Be[V ; E3 :
(VS
OID)
(55)
F(W-)
G(W")
G(W9)
B<(IV/(W": :g) <S("=
F(W.)
F(W')
G(W")
G(W")
EeIIV (E) VS(W")))) :00
F(W*)
F(W))
G(W*) G(Wo)
Es Vf(W vs(W=: =)
F(W-) F(WO)
< <O(I - W / - - = O - -W913).
Plugging back (51) and (55) into (54) shows T(W*) - 0( w2)
(Step 2b) Proof [[W*)]| = @ w-|l and [[(W*)]] = @ W+]l)
G(W9)
[T(W>)| Ee V ; B& IVV :) F(W")
= |V/(())]
(56)
O Wol),
Eg [W : (eI G(W*)
[T(W*)|I
I R(W")]
(57)
F(W*)
@[[ - Iol).
Now we complete the proof.
Special case with D 1 Before proceeding the next section, also present special
where the response functions are linear, i.e., Q+ (W) = W.
Q (W) = W. F(W) = 1 and G(W)= W based on definition (4): and hence R(W) G(W) W Accordingly, the accumulated F(W)
r asymmetric function is given by
[W], مiiر
M [W]a a(W) Auiia
[W)]a
[R(W)]a d(W),
(58)
26

---

2r ([W]a)² (T:"iuy²
2r
Therefore, the last term in the objective (7) becomes
1 (T+ (rrmin)
(2, (cW)
Σ (w)
22
(WWa))2 2r
(59)
2r
W + const.
2r
which is weighted l2 norm regularization term. Furthermore, if W is scalar, i.c., D 1, (44) reduces to minw fx(W)= = f(W) + 27 [WI, which a lz-regularized problem with an approximated solution
W ("(W") W RE(WO)E f"(W*) R'
(60)
H Proof of Theorem 2: Convergence of Analog SGD
Theorem 2 (Inexact convergence of Analog SGD). Under Assumption 1-2, if the learning rate is set as a O(1/VK), it holds that
EK So <O(Vo//K
+ oz
(8)
where S45ED denotes the umplification factor given by SK
S450D
1 K-1
G(W,) VF(W&)
K 2==0
Proof of Theorem 2. The L-smooth assumption (Assumption implies that
L Be,
[f. s f(W) + Eq. (VV((W)) W6+1 W&))
[|
1
(61)
2
(b)
Next, we will handle the second and the third terms in the RHS of (61) separately.
Bound of the second term (a). To bound term (a) in the RHS of (61), we leverage the assumption that noise has expectation 0 (Assumption
E&a (VF(W)) W W&)
(62)
allee
VS(WW) VF(W)).
W - We
+ - o VF(W.)
av F(W)
a
(.((W)) @ 2(W))l
1
Ber
W4+1 - WR VF(WW)
(V(V((W))))) - vS(Wa) R VF(Wa)
2a
We+1 We VF(W))
+
Eg.
+aVf(WE:E)) O VF(W))
2a
The second term of the RHS of (62) is bounded by
WK+1 Wk
Ee
+ a(VS(WWEE)) v.(W2)) @ 1 FF(W)
(63)
2x
F(W+)
Wk - Wr a (V[(Wi:E)) v.(W+)) F(WA)
Eg
2x
VF(Ww)
27

---

2 22.Fmax Be, [[ WA + aVV - Vf(WA) o F(W))1
The third term in the RHS of (62) can be bounded by variance decomposition and bounded variance assumption (Assumption 2)
WK+1 We VF(W))
Eşe
(WV / F
(64)
2a
G(WA)
2
VF(WW)
a 5 VS(Wa)) O (F(WL)
G(WA)
ao²
G(W&)
+ 2
2
VF(WL)
Define the saturation vector M(W)) E R by
M(W == (2 G(W,) (F(W)) + G(W))) @ (F(W+) G(W)))
(65)
= (w+ 3 QWw).
Note that the first term in the RHS of (62) and the second term in the RHS of (64) can be bounded by
G(W2)
[f(W&)l
(66)
VF(Wi)
(G(WW))
Σ
[VS()))
[F(WW))
[F(Wa)la)
dE(P)
[F(W&)] (G(W
)) [F(W
Σ
(V](W))
dE(D)
([VS(W&)) (IF(W))) (G(W))1))
2F 2F
dE(D)
(V(
(Br S.
2F max
Plugging (63) (66) into (62), we bound the term (a) by
Eg (VF(W), WK+1 - W
(67)
a02
G(We)
(V +
2F max
2
VF((Ws
[[ W + ((WisEE)) v((W)) O F(WW))]
Ee,
20 max
Bound of the third term (b). The third term (b) in the RHS of (61) is bounded by
BEall WK+1 11
(68)
<. LLEE,, [ k+1 W4 + aVV V((W))) @ F(W
+ +a [WW V)(W))) O F(W
$ LIE [|] k+1 W4 + aVV (x) Vf(Wi)) O F(W+) LF2 max 0
where the last inequality leverages the bounded variance of noise (Assumption 2) and the fact that F(Wz) is bounded by Frax element-wise.
Substituting (67) and (68) back into (61), we have
Ex, [[(+)+))
(69)
28

---

ax²
G(WA)
< s(We)
V
+ a² LF max
2ama
(F(WW)
1
LF max
W& + +V(V)(Wa:5s) vf(W))) F(W:)
Fmax
20
The third term in the RHS of (69) can be bounded by
WglllWa++1 Wi + ((V)(WA: v((W))) & FI
(70)
= = [I) V ((W)) @ ( @G(W)]
1
2 a EE (l (FW 7
-
LE (WW))+ / GG(W))
- - (V - v.((w:)))) C G(WW)
1 0 EG (WW)) O F(W.) о G(WL)) (W)))))
G(WE) F(W+)
a² F
where the first incquality holds because || || - y112 - |1//1 for any x, E RP, the second inequality comes from
||>|| Ivll s | yl for any v, TR, and the last inequality holds because
Egg (VW,))) (W(W::))) @ G(W))
(71)
(V](W)) vS(W&:E)) @ G(W+)
Es,,
(F(W))
VF(WL)
- o G(WA)
$ FeaxEEe
VF(W&)
G(W+)
& Eçg
V [[(W:
VF(W,)
G(WA)
S 2
FF(W))
The learning rate a S 4LF max
implies that
1 LF LF S 4a 2n
1 in (69), which leads (61) to
G(W+)
Eçg (f( S f(Wa))
( a LF2 max
o² ta02
(72)
2F 2F maax
VF(W,)
Vf(W)) O F(W.) VW( G(WA)l
SF max
Reorganizing (72), taking expectation over all fx,EK=1 Eo. and averaging them for & from 0 to K deduce that
K
ASGD
BllV)(WA) O F(W&) - [V/(w) G(W))) + (((wa)
(73)
K
K=0
8F 3Fmax(((1)) -
K
G(W))
S&LF3 02 + F2
x
Σ
aK
K
FF(W))
k=
2
8F 33aaax (l'ma(((WW) - SoLF3 o 0² + 8F 22
G(Wa)
aK
/ F((
29

---

16F max / ( -
+ 8F o 5K
[(W))-f' The proof is completed.
where the last equality chooses the learning rate as a
Fiaxx
aLLK
Remark (Tighter bound without saturation). Assuming the saturation never happens during the training, i.e. M(W)) 2 M > 0 for all k E [K], we get a lighter bound in (72) by leveraging
(VW)(W) 2 (Wx)) (IWW))
M [WlW))l
G(WA)
(f f(WA)
[VS(W))) a² LF o² a002
(74)
Fmmax
VF(W.)
which leads to
K
(75)
K
K=0
4F2 min
- (f(Wo)
1
K-1
G(W+)
+ 2F Fmaxx
x
min (W
K
K
VF(W)
k=0
exactly reduces to the result for the convergence of Analog SGD in [21] on special linear repsonse functions, discussed in Appendix B.
I Proof of Theorem 3: Convergence of Residual Learning
This section provides the convergence guarantee of the Residual Learning under the strongly convex assumption.
Theorem 3 (Convergence of Residual Learning). Under Assumptions 1-3, with the learning rate
a= 0 (V1/o=k) 3 = 0/a)3/2), it holds for Residual Learning that
E < (VVo2/K+ o
(13)
where SAL denotes the amplification factor Pu given by SAL + Σk
G(P))
VF(P%)
L1
Main proof
Proof of Theorem 3. The proof relies on the following two lemmas, which provide the sufficient descent of We and Wk. respectively.
Lemma 5 (Descent Lemma of Wt). Suppose Assumptions 1-2 hold. It holds for Residual Learning that
Ee. ([(W+++))) 1)] S(WW) 4F max
a
(V/((s)) (M(()) 2002
G(Pp)
+ 2nx LF max
[F((P)
(76)
(V((W) o F(PA) ( O G(PA)l2
F max
F max ES,
[ Wellir( Eg. [ Wx.l
a
Lemma 6 (Descent Lemma of Wt). It holds for Resi dual Learning that
W 3 < W - 2 3
WR - I(
(77)
2 Fmax
30

---

By
(WP) 2 F(WA) [P*( G(WA)]
2F max
2 max
1 PPH1-PP(WWWIMMM
P
The proof of Lemma 5 and 6 are deferred to Section I.2 and I.3, respectively.
For sufficiently large y, Pd (W)) ensured be located in the dynamic range of the analog array Pk- Therefore, we may assume both q+ (Px) and (P() are non-zero, equivalently, there exists non-zero constant M such that ( 2 M for all k. Under this condition, we have the following inequalities
aMRL
a 4/max
IV
2
min 4Fmax
[VS(W>)113
(78)
F Fmax at
Fiax
[[k+11 Wellim(PM)1
[| W
(79)
arM min
Similarly, we bound the term || PM (Wi) (l in (76) by
2
23F Fax
|| - P* PWWW) (M(we)) S
|P - P*(W)))
(80)
r min (W
Notice it only required have min (M(()) > for the inequality hold. By inequality (79), the last two terms in the RHS of (76) is bounded by
F Fmax
[I
Bec
WallBa(P Bg. [1] k+1 Wxll [[|Wa+1 Wx
(81)
Fmax
Eq, [Il 1
a.
(a)
2. 2F max
[[[Wa+1 - Ww|l 2.3 Fmax
EEN
|| O F(W,) i G(W))
Or o&MEL min
a M&L
min
4,3 Fmax
P** @ F(W+) - PM (W&) O G(W)))
O.MAL min
4,8 FgfFFma Fmax
| @ F(W)) PPetil o G(W)) - (P*(W)) O F(WA) (P*(W&) o G(W
a.MRL
min
(6) 4,32 1.2Fmme max
4 Fmax
(W*) o F(W)) (WW) G(WW)
|| P'WW)
a.MRL min
AL min
where (a) holds if learning rate a is sufficiently small such that
Emasy
2 1; (b) comes from the
M min
Lipschitz continuity of the analog update (c.f. Lemma 3).
With all the inequalities and lemmas above, we are ready to prove the main conclusion Theorem now. Define Lyapunov function by
VK = f(Wx) - f" + C - 3
(82)
By Lemmas and 6, we show that VA has sufficient descent in expectation
EELIVA+) [V++1]
(83)
= Eee [f(W - + - -
G(P))
< ((WW) - f
4Fmax [VVF + 2a02
a
+ 2a. LF2 02
VF(Pk)
a
+ G(P))]
8F 8F max
48 F Faax
4 F Fax
(P((W) F(W)) (W.) G(W)))
Eeg [1] [||1111 P*(W)]1
0.1/ALL min
RL min
31

---

3
33F max
+C
| [ W01l
1 W (M
Be SelllPt+11 p (WW))
2-Fmmxx
y y miin(M (M (W+]
By
| PP*(W)) 2 -(W2) (P- G(WW)l?
2F max
V
a
MRL
min
[]((W)]]| 2002
G(Pp)
+ 2a LF LF
4F max
(P(PP))
Cx
(Vf(WA) @ F(P+) O G(PA)]I
8F max
By
4/3 Fiax
C
[P'(W&) O F(W)) - P'(W))) @ G(W+)]l
2F Fmax
a.MaL. min
33F
4 Fmax
B
max
C+
- -
C
(WW)
aM RL
2yFmaxx
10812 max M
Let C
which leads to the positive coefficient in front of || (_WW))),,
Ee
(84)
2
a.RL min
G(PP)
I V
[V((W))11 200²
+ 2a 2 LF 2
4F 4Fmax
(F((Ps)
(Vf(W)) @ F(P)) [V(())) O G(PA)l?
8F
B2 Funx
P**(Wk) O F(W)) (P G(W+)
aMaL
min
30,32 Fs
4,22 Fmax
5,82 Fmax
Ege [1 - (WW)))
I - W (2
O7 ( (WW)).M min
a.MAL min
aM
min
Notice that the || |P P* (W+) appears in the RHS above, we also need the following lemma to bound it in terms of | Ps: - P* (W))
Lemma 7 (Descent Lemma of Pt). Suppose Assumptions 1-2 and hold. It holds for Tiki- Taka that
Ex. [||t++ - PW
(85)
s 1 any
4(m+ +) |Pp 1 PPW))l" + 20( + L)F o
G(Pp)
+ FFF2 F
TuL
VF(Ps)
The proof of Lemma is deferred Section 1.4. By Lemma 7, we bound the P terms of Pr P*
P (1 in
11 as
30,32 max
4,3² Fmax R.MRL
Eg. [|l|PPt41 PP
(86)
ary min (M (W+)) J Bi
WN
3252 25
EEG [|| (WW))]
ary min M (WW))MR min
12. max
(1-9) α
uL
- P
ary min (M(W)) MRL
4 7+1++++1 + T.))
32,5 FS
2a(μ + F)x
G(P))
ary min max
+ a 122 12
MRL min
Ju
VF((P)
323² ps max
G(P.)
- PWWW)l +0
1802
+ a F2 02
ary min M (W&
VF((Pt
32

---

32,32
G(Pp) VF(Pk)
max
|| P*(WW]l + t002
+ a LP
ary min (W)) Mat
where (a) assumes 4.3 Fmas
2,9 Pm
an min (W&) aaa with lost of generality to keep the formulations
min simple since n min (W&)) is typically small; (b) holds given a and / is sufficiently small. In
addition, the strong convexity of the objective (c.f. Assumption 3) implies that
aMai min
a min
ap² M
|V/(WW)]]
I|
w-1l2²
|| 7Pk 1=12
(87)
max
max
max
ap²
M
min
W
Wx
2 M
Pk
P P (W)))l??
8F FFmax
7
8a
Substituting (86) and (87) back into (84) yields
Egg [V++1l
(88)
MML
G(PP)
V 8F max
min
[V](W))] 300
+ +- LF2 max
VF(P))
VVf(W)) F(P)) [V((W)) G(P))
8 max
8 F max
P*(WW) ( F(W.) PP (Wx) G(W)))
RRL
min
aap²n² MAL
32,3² ps max
5,3 F max
PP P* PWW(I))
I l1
8F Fax
Q7 min M (W)))MM
a.MRL AL min
min
aM min
G(PP)
R
11 3002
+ 3a +3a2LF2 LF2 LF 02
8F max
(F(P))
VS(WW) F(P)) + [V((W)) G(PA)]1
SF max
ap 2 3 min (W+)
PP*)W>) o F(W)) - (P (Ww)) G(WW)l
512F3
MRL
"1p22"" min
2
P P*(WW)l
W ll
16 16Fmax max
512F5 M
max min
where the last step chooses the transfer learning rate by
apry 8
min (M (W&))
(89)
16 max
Rearranging inequality (83) above, we have
VS(WE) O F(P&)+ + @ G(PA)l2 + 8Fmax MAL
|V/(W>))
(90)
8F max
min
2 (Ww))
PP*(W)) O F(W.) - (P*(WW) G(W+))l
512FS MRL
2 min (MWW))
2 min
W. 1 W1
+
|| - PPeeill
MRL 512F5 max min
16F
G(Pk)
V Ee, EV.VV (V+1l + 3a 2FFFF2 m + 3002
/F(Pk)
20
Define the convergence metric FBL
K-1 Σ
ERL
O
+
[VF((W))1 (91)
K
MRL
K=
min
33

---

y² min (Min(MM() [WW]
PP* o F(W.) - |P (P- @WW))
64F4
MAL min
max
7
2
I W (l( +
min
Px
P'(W)))
64F max
M min
2
Taking expectation over all EK 5k-1---
Eu: averaging (90) over is from to K and choosing
the parameter a as a O
deduce that
F
mmnx
FLK
Vo - K+1
1
G(Ps)
Ex S 8F max
FaLFF FanLFmxx
mixx
+42FFim 2
(92)
aK
(F(P))
< 8F
max
Vo
33LFF? BaLFmaxO?) )
+ 24F 2
G(Px)
ak
VF(P))
=0 =o
Voo2
F2
max
+ 24F
o
K
The strong convexity of the objective (Assumption 3) implies that
Vo = f(W)) f" C w711 < 11
2C
(S(W)) S*)
(93)
Plugging it back the above inequality, we have
Ex O FM V (s(W)) 12)
+ 24F 24Fmaxx0 SBL
(94)
K
The proof is completed.
1,2 Proof of Lemma 5: Descent of sequence Wi
Lemma 5 (Descent Lemma of Wa). Suppose Assumptions 1-2 hold. It holds for Residual Learning that
Eq. (1)
a 4F 4Fmax
IVF(W)))MRR) 2ao2
G(PA)
+ 2a LF2 o²
/F(Pk)
max
(76)
ary
O (VS(We) @ F(PA) + ( G(Px))?
8 F max
F max
Ee,
|| W Wi Will
+ Eg. [|| Wxll3).
Proof of Lemma 5. The L-smooth assumption (Assumption 1) implies that
L Ee, [I|W Wx
L/W ([(W+++)) )] < ((Wa) + Ee, (Vee.V/// (V)(WW), WR+1 WR)| + 2
(95)
= f(WW) + Be. (V)(W)), Pa+1 - - Ee, ( ((W),, WA+1 Wall +
Esc [|| - Wa
(a)
(e)
Next, we will handle the each term in the RHS of (95) separately.
Bound of the second term (a). To bound term (a) in the RHS of (95), we leverage the assumption that noise has expectation 0 (Assumption
Ege (VF(W)))...... Pa+1 - Px)]
(96)
aaE ( (T((W) o (F(P)) P Pe + (Vf(WW:))) - -V/(a)) O
F(F(P))
34

---

((WW) /F(P)]?
1
PA+1 Ps (VV vs(WW)) @ F(Pt)
2a
VF(P))
1
PR+1 - Pa VF(Pk)
EEn
(WW o (F(P))
2a
The second term in the RHS of (96) can be bounded by
1
Pa+1 F((k)
Be
+ (VV[(WE:).)) - V/(Ww)) @ 1OOOFFFFF ((P))
(97)
20
Pk+1 Pk a
& F(P.)
2a Egg,
(F(P))
Eg, (
@ F[P.]]]
20.F max
The third term in the RHS of (96) can be bounded by variance decomposition and bounded variance assumption (Assumption 2)
1
PA+1 - Pk
+ (WVf(W O F(P))
(98)
2a
(F(P))
a Ex
v[(WwiEH] G(PA)
2
VF(Ps)
G(PP)
ao?
G(Px)
V((W))
+
2
VF(P))
2
VF(Pk)
Notice that the first term in the RHS of (96) and the second term in the RHS of (98) can be bounded together
G(PA)
2 (VV @ /F(P)) +
/f(Wk)l
(99)
VF(P))
(G(P&)
Σ
(V)(W)))
[F(PE))
[F(Pa)]a
dElp
[F(Pa) [G(P:)]3
Σ [V)(WW))
[F(Pk)]a
dE(D)
< 2F Fmax
a
([V
(
dElD)
2F (V $ 0.
Plugging (97) to (99) into (96), we bound the term (a) by
Exe (VVf(W)) Pa+1 Pa)1
(100)
ao²
G(Pp)
[V/(WW)|
2F max
(F(P.)
Es, [|
Pa + VV v)(W)) @ F(P.)
2a
35

---

Bound of the third term (b). By Young's 's inequality, we have
E.. (V WK+1 W& - a
(V Fmax
Be. [+] Well M
4F
(101)
Bound of the third term (c). Repeatedly applying inequality |IU + vII² - 2||111 + 2|| for any U, R., we have
L
c/ [||
Wxll3]
(102)
2
< LEg,
[|| Wi + LEg, Pk+1 - Pı.1
- LEg, [||
W&ll2] + 2. [||k+t1 - Px + a(V[(((WW))) - vS(W)) @ [(PA))]")
+ +2a² LIEEN [I(V Ex) - vs(Wa)) @ F(PW)]
S E5g [llll - W + 2LE [|| - Ps + aVVf(W(VWW)) v)(WA)) @ F(PA))
+ +2a² LF OF
where the last inequality comes from the bounded variance assumption (Assumption 2)
20x LEEn
[I(VS((W:EE)) -V o F(P)]
(103)
< 20 LF2 max Een
([) - v
< 20 L
max
Combination of the upper bound (a), (b), and (c). Plugging (100), (101), (102) into (95), we derive
+ ao² G(Ps)
Eg, [WW+)l) & f(W))
a 4Fmax
[VF(WW)(MM())
(104)
2
VF((#)
21.
Een ||| - Pk + ar (VV/(WW: - V/((W)) @ [(
2a Fiax
Fraxx
Egg.
[[Wa+1
R.. ||| k+1 W2ll + 2. x
0²
We bound the fourth term in the RHS of (104) using the similar technique as in (70)
[| Ps (W o F(P.)]
Eg,
(105)
G(PP)
- om
VF((P)
Inequality (105) as well as the learning rate rule a S ILFmax leads to the conclusion
2
G(Px)
Eg. [ ((W))
a WV// 2002
+ 2o LF2 F
4F
/F(Pk)
(106)
OF
( (V[(w))) GGPA))
max
F max
Ese
W Willirc
+ Eg. [1 Wz
a
The proof is completed.
36

---

1.3 Proof of Lemma 6: Descent of sequence W
Lemma (Descent Lemma of Wt). It holds for Residual Learning that
3
Wk+1 - W < We- - 1'11 27 max
W - (20
(77)
37
|| 3 F(W)) (WW)) G(W
2 max
23F max
MMWw) + 2 PW
7
Proof of Lemma 6. The proof begins from manipulating the norm || - W'
[ - 2 1
I - 1 + (W₂ - W W - Wx) + | Willa (107)
Revisit that we interpret Ps as the residual of Wa, namely P*(W)= = W W Therefore, we bound the second term in the RHS of (107) by
(W₂ W., WK+1 W&)
(108)
=2( - -W*33P&+1 @ F(W)) 3) P G(WA))
23 (Wx- (W,, @ F(We) P (WW)) G(W.))
+ 2 (Wa - Px-1 @ F(W&) PP @ G(W)) - (P.(W)) @ F(WA) (P*(W&) o G(W+)))
The first term in the RHS of (108) is bounded by
28 ( W* (W(,) F(W+) - (W @ G(W+))
(109)
(W& W*) ④ (F(WW). (*(Wa) F(W() (W&)) G(WA)
28
VF(W+)
23
(Wh W @ (F(W)) (W - W') O F((W))
?
23
(W&. w²) o /F(W) W 1 G(W+)
+
7
VF(WE)
(0)
1( w*) F(WW)))
W lo G(W+)
VF(Ws)
8
(Wi- - w*) 3 F(W)) + W 1 G(WA)
7
VF(WA)
3
G(W))
W - (lll)
(Wr. - W*) 3 F(WA) + W W 0 VF(WA)
Y Finx
3
By
W Wlllaa))
IP* 3 F(WA) P'(WW)) G(W
F Fmaxx
Fmax
where (a) leverages the equality 2 (U, V)
|| || - UU VII for any U,V V E ERD, (b)
achieved by similar technique (66), and (c) comes from
G(W,)
(Wh W O / F(WW) + W
w O
(110)
VF(WA)
1
WR W*
WE W
B7
F(W))
G(W+)
VF(WA)
7
BY
< F Fmax
(W+) O F(WA) - PP' (W+)) 2
37

---

The second term in the RHS of (108) is bounded by the Lipschitz continuity of analog update (c.f. Lemma 3)
28
(W4- W PR+1 F(W))
|PR+1 @ (P
(CW.))-- G(WW))
7
1 WV + 2.Fmax
(111)
27 Fmax
x PR++ @ F(W:)) PA++1 1 G(W&) (P*(WA) o F(WE) (W&)) @ G(Wa))l
I| W M((w) + 233
|| - P"
27 PyFxax
Substituting (109) and (111) into (108), we bound the second term in the RHS of (107) by
<W W
Wk+1 Wx)
(112)
B
BY Fmax
IW
W W'lla
IP* (FWA) - |P* (W&)) o G(W))l
7 F max
2 max
PA++ - P" (WW))lla
The third term in the RHS of (107) is bounded by the Lipschitz continuity of analog update (c.f. Lemma 3)
[W&+1 11 3 F(W&) o
(113)
$ (3 @ F(W&) [P*(W)) G(W))I
+ + 1 @ F(Wx) - (P*()) @ F(W&)- - (P* @ G(W)))
< 2 @ F(W,) - -PP"(W)) G(W+)I 28 |P (W)))
Plugging (112) and (113) into (107) yields
3
W 1
W 1111
W W
(114)
27Fmaxx
By
2/32
[IP)(Ww) + F(W.) - (P=( OGWW)I
F Fmax
2
max
|Pat1- PP - P
7
Notice the learning rate is chosen 35
have
28
B
[ 1 < W
2
(115)
2-Fmmss max
B7
[P*(W&)) 3 F(W+) |P' e G(W))]
2F
2
[Px41 (WW))) + 2 P
which completes the proof.
1.4 Proof of Lemma 7: Descent of sequence Pk.
Lemma 7 (Descent Lemma of Pt). Suppose Assumptions 1-2 and hold. It holds for T that
Eeg P|| - P PWWW)]
(85)
< (1 arpi
M(++L) 1 |Pp PP'(W)) 20( L)
G(P))
+ F22 2
NuL
VF(Pk)
38

---

Proof of Lemma 7. The proof begins from manipulating the norm |P P P0 (Wa)ll
||P441 PP -P +
P*
+ (Px - (Ka), Pk) +
P ||
Px
(116)
To bound the second term, we need the following equality.
2EE, - (W)), Pa+1 - Pa))
(117)
20xBei 1 P'WW,), VI(WA o o
- 2ex P"(W,), VI(W& O F((P))
+2aBei((P' -P*(WE), - [VS(W:: G(Pt))]
2 ( Pk V/(W)) @ F(P)) +20 (1))
+2aBs,
2a - (WW),, V/(W)) - GGPP)
(T)
+ 2a By. IPp - (" = v C(PA))
(72)
Upper bound of the first term (TI). With Lemma 4, the second term in the RHS of (116) can be bounded by
- -2a - PW(W), Vf(WW) - (W)( o G(PP))
(118)
= -22 (Pr - (WW,,, V/(W)) (p.)
S -2aCk,+ (Pr- - P (WWA), V((WW) + 2aa (1M (F(WW))
where Ck,+ and Ck,- are defined by
1 Ck
max {qo ([Pk)a)) min f9. ((Pxla)) dE(D)
(119)
dE|Di
1
Ck. =
max (q. (PPa)a) min (g. (P dE((D
(120)
dEiD]
In the inequality above, the first term be bounded by the strong convexity of f. Let (p(
f(W +PP) which is 72 L-smooth and "²u-strongly convex. It can be verified that (P() has gradient
VP((%) P.. f(W 7P%) f(W)) and optimal point (WW). Leveraging Theorem 2.1.9
in [85], we have
(Vf(W)), Ph (WW)) 1 (Vi((P)) Ps P P*(W)))
(121)
722
|| - PPWWW)11
7u + 2 [
72 + L
TuL T, ||Pa P (#W))
1
+) V (W(W))
The second term in the RHS (118) can bounded by Young's inequality 2 (x., v) S +| with any u > 0 and x, y E R
2aC (|Px - P* P"W(W
(122)
aC a L)
aCk+ [VS(()))
P P*(W2)1
Ch.+
7(p + L)
where u chosen to align the coefficient in front of (W Therefore, (T1) in (118) becomes
2a (Ps - P vf(W)) F(P.) Vf((W)] G(P))
(123)
39

---

2ampLCCki 1+L
aCk a
aCk,
1122 - P-
(((W)))
Ch,+
7(1 + L)
Upper bound of the second term (T2). Leveraging the Young's inequality (z, v) S u||x||2 Illll with any u >> O and x, y E IR D. we have
P" (W&), (VV)(W) - ( 2.B (
(124)
(
(Ps- (IPs- (Ps- (P&-)'(WW)) o
o G(Pt)
=2aB3.
/F(Pk)
I1(Pk - P*(W)) O (((P))
(n + Fmxx
alp L)Fmax Es
[ IVS(W VF(Ps)
G(P.)
JULCO,+
acppLCk.t 11(Pk - PP((W)) @ /F(PA)|
(1 + Fmmaax
a(μ L)) Rag
(I VS(WE:E) G(PP)
J/LCCk++
VF(Pk)
( OCiHLCCktt |1(Pk PP((k)) @ /F(P))) + alu + F
G(PA)
(1+ Fmax
ypI.Ck.+
VF(PP)
anHLCk,
alμ + Fmaxx
G(PA)
PP(WW))
++
IpI.Ck++
(F((k)
where (a) choose > 0 to align the coefficient in front of || P (l in the RHS of (123), (b) applies ||||| - bl| < | y for any x. E R, (c) uses the bounded variance assumption (c.f. Assumption 2), and (d) leverages the fact that ((PA) is bounded by Fmax element-wiss
Combining the upper bound of (T1) and (72), we bound (117) by
2Eg, 1(P (*(W),, Pkt1 - PE)]
(125)
aYpLCk,t x 11 PPPPa)
1+L
Chi+
7(1+ L) aCk,t ||V](W)] + alu + Fmaxo22
G(Px)
WLLOKt
F(P))
2
S a -PPP--P((WWa) ('(W) 2(u+ -
aCk,+
alμ + Fm
G(Px)
//(
7(++ L)
JpLCktt
VF(PA)
where the last inequality holds when y is sufficiently large, Ps well as Ck are sufficiently closed to 0, and the following inequality holds
C2 C
UL
(n L)
(126)
2(p + L)
Furthermore, the last term the RHS of (116) can be bounded by the Lipschitz continuity of analog update (c.f. Lemma 3) and the bounded variance assumption (c.f. Assumption 2)
Eg. PM - 1 = Ee, a V (W Ex) O F(Px) - -al))aaV//(MM (W..)( @ G(P))l]]
(127)
< a² F2 Ee, ||| ((W::)))
a² F2 max Vf(WA)]l + a F2 F
40

---

aCk,t 7/1 + L)
[W + a F2 F
where the last inequality holds a is sufficienlly small. Plugging inequality (125) and (127) above into (116) yields
Eg [k| - (WW)1
(128)
11 a.HLCCk.t) 2(1++1 L.)
|P& - P*(W))) a(u+ F Tpt.Cktt
G(PA) VF(Pk)
a² F
max o².
By definition of Ck.+, when the saturation degree of Pk is properly limited, we have Ck,+ 2 Therefore, have
1 2"
Eg. -PPP)-)11
(129)
$ (1 1 4(m+ L)
ayHL ) ||Ps - PM (W,)) 20(p + L)F
G(PP)
+ a² p2
max
o²
Mp
VF(P))
which completes the proof.
1,5 Proof of Corollary 1: Exact convergence of Residual Learning
Corollary (Exact convergence of Residual Learning). Under Assumption 4 and the conditions
in Theorem 3, i 2 00 it holds that EK S (
Proof of Corollary 1. From Theorem 3, we have
|VS(Wa)] O
HL S So
F2
(U(W)) - 2
+ 24 Fmaxx CAL
(130)
max
K
Under the zero-shift assumption (Assumption 4) and the Lipschitz continuity of the response functions, it holds directly that
G(P)) VF(PP)
G(PA)
G(PP)
G(O)
< PAIl
(131)
VF((P)
VF(Ps)
VF(0))
where Lg 2 0 is constant, Using |U + 2 < 2|1112 + VI2 for any U, V E R,, we have
| < Pr P* (*WW)) + (W))l 2 11PP-PP((AA) P"(())) 2 W w (132)
where the last inequality comes from the definition of P" ('Wa), as well as the definition of P* (W). Recall that convergence metric EAL defined (91) in the order of
RLL 2
Ω 7" WR - 1
W (12r 72 Pr P'(WW))i²) w0 + 2² 11p - (WW)I))
(133)
2 min (Min((
WR - w 12 7 - PP (waa)l))).
2
Therefore, have
SAL 1 K
G(Pt)
2
K K
2| - P*(Wa))1 + 2 | w-1l2) < (
Σ K
S
VF(PA)
K=
(134)
where the last inequality holds 7 sufficiently large. Considering that, FRL SAL 2 (((F 2 and the conclusion is reached directly from Thcorem 3.
41

---

J Proof of Theorem 6: Convergence of Analog GD
In Section 3.2, we showed that Analog SGD converges a critical point inexactly with asymptotic error proportional to the noise variance o². Intuitively, without the effect of noise, Analog GD converges the critical point. Define the convergence metric by
K-1 Σ K=
A =
1
(IV o F(W,) - (V(( o G(W))l++
(V((W)))).)))) (135)
K
The convergence is guaranteed by the following theorem.
Theorem (Convergence of Analog GD). Under Assumption 1-2, it holds that
SL(f(W)) F Faax
E
S
(136)
Further, M [[k] min min Q-(w))) > O, it holds that
K-1
2T(S(W)) ) KM A8AD min
[VS(W&)
(137)
K
Proof of Theorem 6. The L-smooth assumption (Assumption implies that
L
f(WK+1) f(Wa) (VS(W))) W Wk)
2
Wiel
(138)
-[(Wk) _ [VVf(W の VF(WW)112 1
1
LF max
[1
Will
Fmax
2a
2
1
W - W4 + aVS(W)) o VF(WW)
+
20
VF(W))
where the second inequality comes from
WK+1 We
(VJ(W)) We) = a VS(WW) o /F(W)),
(139)
oV F(W
- (VV o /F(W)) 1
a
Wh+1 W4
2a
VF(W&)
WK+1- WR
+ 2o
+ +aVf(WW) 3 F(WA)
VF(WN)
well as the inequality
2
W W VF(Wk)
W W
(140)
Fmax
The third term in the RHS of (138) can be bounded by
1 WK+1 Wt
G(W)).
+ +VVf(WW) F(WW)
Vf(Wa)l
(141)
2a
VF(W))
VF(W))
Define the saturation vector M(W1) RD by
M(W) = F(W, ( O (F(W)) - G(Ww) = 4+ (W+) (W+))))) (Wx).
(142)
Notice the following inequality valid
2
G(W))
Vf(WW) V
2 [V((We))
(143)
(F(W))
42

---

G(W))l
Σ
(VS(Wa)) 11
(P(WW))a
2 dE(P)
(((a)]a
[F(W&)] (G(W )) [F(Wa)]a
Σ
[V](W>)l
dE[D)
([VS(W&))) (IF(W))) G(W))13)
2F max
DE(D)
a
IV((W))ls. S.
2F max
Substituting (141) and (143) back into (138) yields
LF Faaax
|W - Will S f(W&) - f(Wk+1).
(144)
F
Faax
2(x
Noticing that ||
Woel2 a² (VS(W)))))))) @ F(W.)
VS(WL)) G(W and averaging
for k from 0 to K 1, we have
K
ACD
Σ
(VV((W)) o F(W)) -V/())))) G(W))l 1 (145)
K
k=0
2(f(Wo)
f(WK+1]))))) max
8L(f(Wo) F
a(1 F...Faa K
K
where the last inequality choose a 2LFomxx
Further, given the response functions bounded, (138)-(143) implies that there is lower bound M such that
a
ACD min
[f(W2))
[ S f(Wx) f(W++1).
(146)
2
Averaging (146) for & from 0 to K deduce that
1 K-1
[Vf(W)) 2(f(W))
11) 2L(f(Wo) - F
(147)
a. AMAD min
K K.M. AGD min
where the second inequality holds because the learning rate is selected as a LF
K Simulation Details and Additional Results
This section provides details about the experiments in Section 6. All simulation is performed under the PYTORCH framework https : gi thub. com/ conn/ytoorch/pytoree algorithms, including Analog SGD and
The analog training are provided by the open-source simulation toolkit AIHWKIT [44], which has MIT license; see github Caom/IBM/aala
Optimizer.
The baseline Digital SGD optimizer is implemented by FloatingPointRPUCoaii ig in AIHWKIT, which is equivalent the SGD implemented in PYTORCH. The Analog SGD is implemented by selecting SingleRPUConfig as configuration, and Tiki-Taka optimizers are implemented by UnitCel1RPUConfig with TransferCompound devices AIHWKIT.
As suggested by [22], in the implementation of Residual Learning, only a few columns of Ps are transferred per time to W₁ in the recursion (11) to balance the communication and computation. In our simulations, we transfer column every time.
RPU Configuration. AITWKIT offers fine- iin-ggrinedd simulations of the hardware imperfections, such as the noise, analog-digital conversion, and so on. They are specified by the resistive processing unit (RPU) configurations. Without other specifications, we use the configuration list in Table 4. The experimental setup uses a specific I/O configuration, as detailed in the relevant table. The system's input and output signal bounds explicitly defined. Regarding signal quality, the setup employs
43

---

input noise but introduces additive Gaussian noise to the output signal, the statistical properties of which are precisely specified. Finally, the resolution of the digital conversion process is determined by distinct bit values for both the input (DAC) and the output (ADC).
In addition, noise, bound, and update management techniques used [71]. A learnable scaling factor is applied after each analog layer and updated using SGD. For each gradient update step, more than BL 32 pulses desired, only BL pulses fired.
Table 4: Hardware imperfection setting
configuration
value
input bound
1.0
input noise
None
input resolution (DAC)
7 bits
output bound
12.0
output noise
additive Gaussian noise (V000.066
output resolution (ADC)
9 bits
Update granularity Awmia Bit length BL
1 x 10-3
32
Simulation hardware. We conduct our experiments on one NVIDIA RTX 3090 GPU, which has 24GB of memory and a maximum power of 350W. The simulations take from 30 minutes to 5 hours, depending on model sizes and datasets.
Statistical Significance. The simulation data reported in all tables is repeated three times. The randomness originates from the data shuffling, random initialization, and random noise in the analog hardware. The mean and standard deviation are calculated using statistics library.
K.1 Power and Exponential Response Functions
We consider two types of response functions in our simulations: power and exponential response functions with dynamic ranges [-r,7], The power response is a power function, given by
g+ (w) = -1 Tam
w
(-ww) = -11 W
(148)
which can be changed by adjusting the dynamic radius and shape parameter Tres- We also consider the exponential response, whose response is an exponential function, defined by
4+ (w) exp (7ree (1 w/T)) -
(w)w exp (Tree (1 + w/T)) -1 exp (Trea)
(149)
exp (Trea)
could be checked that the boundary of their dynamic ranges are max = T and min = -r, while the symmetric point is 0, required by Corollary Figure 6 illustrates how the response functions change with different Trea-
K.2 Least squares problem
In Figure (see Section 1.1), we consider the least squares problem on synthetic dataset and ground truth W E RD_ The problem can be formulated by
1 f(W): |AW b|2 - 1112=11100 1,IAAW W WERD
min
(150)
The elements of W are sampled from a Gaussian distribution with mean O and variance oi... Consider matrix A DDuD size = 50 and Dans 100 whose elements are sampled from a Gaussian distribution with variance 03 The label E R D.. is generated by b = AW* where W are sampled from a standard Gaussian distribution with ...... The response granularity Awmin=ee- while 7 = 3.5. The maximum bit length is 8. The variance are set as of 1.002. ow- 0.52.
44

---

Power Step: Response Functions
Exponential Step: Response Functions
a (w)
q. q.(w)
Yis 0.2
Yos 1.0
Yiss 2.0
Power Step: Sym/Asym Components
Exponential Step: Sym/Asym Components
G(w)
Yess 0.2
Yes 1.0
Yiss 2.0
Figure 6: Examples of response functions. The dependence of the response function the weight w can grow at various rates, including but not limited to power (Left) or exponential rate (Right). the radius of the dynamic range, and Tres is parameter that needs to be determined by physical measurements.
K.3 Classification problem
We conduct training simulations of image classification tasks on series of real datasets.
3-FC @ MNIST. Following the setting in [16), train model with 3 fully connected layers. The hidden sizes are 256 and 128. The activation functions are Sigmoid. The learning rates are a 0.1 for Digital SGD, 0.05, B = 0.01 for Analog SGD and Tiki-Taka. The batch size is 10 for all algorithms. In Figure 4, the power response functions with Yres = 0.5 are used, and various are used indicated in the legend.
CNN @ MNIST. We train convolutional neural network, which contains 2 convolutional layers, 2 max-pooling layers, and 2 fully connected layers. The activation functions Tanh. The first two convolutional layers use 5x5 kernels with 16 and 32 kernels, respectively. Each convolutional layer is followed by subsampling layer implemented by the overlapping pooling windows of size x 2. The output of the second pooling layer, consisting of 512 neuron activations, feeds into fully connected layer consisting of 128 tanh neurons, which is then connected to a 10-way softmax output layer. The learning rates are set as a = 0.1 for Digital SGD, 0.05, 5 0.01 for Analog SGD are Residual Learning/Tiki -aka.. The batch size is 8 for all algorithms. In Figure 4, the power response functions with Tres = 0.5 are used, and various are used indicated in the legend.
pooling function over
ResNet/MobileNet CIFARIO/CIFARIOO, We train different models from the ResNet family, including ResNet18, 34, and 50. The base model is pre-trained the ImageNet dataset. The last fully connected layer is replaced by an analog layer. The learning rates are set as a = 0.075 for Digital SGD, a 0.075, 3 0.01 for Analog SGD, Residual Learning/Tkii-Ten Tiki-Taka v2, and Residual Learning v2. Tiki adopts = 0.4 unless stated otherwise. The batch size is 128 for all algorithms.
K.4 Additional performance on real datasets
We train different models from the MobileNet family, including MobileNet2, MobileNetV3L. MobileNetV3S. The base model is pre-trained on ImageNet dataset. The last fully connected layer is replaced by an analog layer. The learning rates are set as a = 0.075 for Digital SGD, 0.075, 0.01 for Analog SGD Aiki-Taka. Tiki-Taka adopts y 0.4 unless stated otherwise. The batch size is 128 for all algorithms. Power response function with Yrea 4.0 and 0.05 is used in the simulations.
ResNet CIFAR1O/CIFAR R100. We fine-tune three models from the ResNet family with different scales CIFARIO/CIFAR 100 datasets. The power response functions with Tres 3.0 and 0.1.
45

---

and the exponential response functions with = 4.0 and T = 0.1 are used, whose results are shown in Table 1 and 5, respectively. The results show that the Tiki Taka outperforms Analog SGD by about 1.0% in most of the cases in ResNet34/50, and the gap even reaches about 10.0% for ResNer18 training on the CIFAR100 dataset.
CIFARIO
DSGD
ASGD
TT/RL
TTv2
RLv2
ResNet18
95,43 10.13
84.47
94.81 10.09
10.00
95.12 10.14
ResNet34
96.48 0.00
90.13
96.29 10.12
96.60
96.42 10.13
ResNet50
96.57 0.100
94.36+1.16
96.34 +.004
96.63 +0.09
96.56 +0008
CIFAR100
DSGD
ASGD
TT/RL
TTv2
RLv2
ResNet18 8
81.12
68.98 1.01
76.17 -.233
18.56-1.22
79.83
10.13
ResNet34
$3.86=0.12
78.98 +0.55
80.58 0.11
81.81 +0.15
82.85
10
ResNet50
83.98 10.11
79.88 1.26
80.2
82.82 +0.33
$3.90+0.22
Table 5: Fine-tuning ResNet models with the exponential response on CIFAR10/100 datasets. Test accuracy is reported. DSGD, ASGD, and TT represent Digital SCD, Analog SGD, Tiki-Take respectively.
MobileNet @ CIFARIO/CIFARION We fine-tune three MobileNet models with different scales on CIFARIO/CIFARIO datasets. The response function is set as the power response with the parameter Tres 4.0 and т 0.05, whose results are shown in Table 6. In the simulations, the accuracy of Analog SGD drops significantly by about 10% in most cases, while Tiki- -aaka remains comparable the Digital SGD with only slight drop.
K.S Ablation study on cycle variation
To verify the conclusion of Theorem 4 that the error introduced by cycle variation is higher-order term, we conduct numerical simulation training on an image classification task on the MNIST dataset using Fully-comnected network (FCN) or convolution neural network (CNN) network. In the pulse update (26), the parameter de is varied from 10% to 120%, where the noise signal is already larger than the response function signal itself. The results are shown Table 7. The results show that the test accuracy of both Analog SGD and Tiki-Taka is not significantly affected by the cycle variation, which complies with the theoretical analysis.
K.6 Ablation study various response functions
We also train a FCN model on the MNIST dataset under various response functions. As shown in the figure, larger Yres leads to steeper response function. The results are shown in Table 8. The
CIFAR10
DSGD
ASGD
TT/RL
TTv2
RLv2
MobileNetV2
95.28 10,20
94.34 10.27
95.05 10.11
95.20
95.26 1.000
MobileNetV33
94.45 10
80.66
93.65 +0224
93.04
93.79-10.00
MobileNetV3L
95.95 10008
80.99
95.39
95.27
95.33
CIFAR100
DSGD
ASGD
TT/RL
TTv2
RLv2
MohileNetV22
80.60++0.18
63.41 +1.20
73.33 +0.94
78.41 +0.15
79.60 99.60+-0.10
MobileNetV33
78.94 10.05
51.79
71.1410.93
75.39 79.16-10.10
74.51 +0.37
Mo
10.26
6.400
78.81 10.52
80.18 1-0.07
Table 6: Fine-tuning MobileNet models with power response on CIFAR 10/100 datasets. Test accuracy is reported. DSCD, ASGD, and TT represent Digital SGD, Analog SCD, Tiki Maki,, respectively.
46

---

FCN
CNN
DSGD
ASGD
TT
DSGD
ASGD
TT
10%
97.22
97.66 10.04
92.68 10..4
10.07
De 30%
96.97
97.07 10.12
93.55.10..5
1.0.55
98.89 -0.05
0. 60%
18.07
96.33 0.221
97.70=00.09
99.09
93.07
98 28.68 =0.09
0. 90%
95.99 0.15
97.44
91.87 +0.48
98 00.02
= 120%
96.19 00.20
96.97 +0.20
91.57 +0.58
98.85 00.04
Table 7: Test accuracy comparison under different cycle variation levels de on MNIST dataset. DSGD, ASGD, and TT represent Digital SGD, Analog SGD, Tiki-Take respectively
accuracy < 15.00 in the table implies that Analog SCD fails completely all trials, which is close to random guess. The results show that Analog SGD works well only when the asymmetric is mild, i.e.
Tree is small and 5 is large, while Tiki- aaka outperforms Analog SGD and achieves comparable accuracy with Digital SGD.
Power response
Exponential response
DSGD
ASGD
TT/RL
ASGD
TT/RL
T = 0.6
96.01 0.26
96.92 +0.19
<15.00
97.27 +0.07
Yres
0.5
0.7
97.40=0.1
97.05
+0.05
<15.00
10.15 10.15
T = 0.8
97.38 0010
96.82 1.0.77
94.00
97.16 10.16
T 0.6
<15.00
97.39 10.05
<15,00
97.46 LO.08 10.08
98.17
Yres 1.0
T 0.7
<15.00
97.33
<15.00
97.49 L0.04
T = 0.8
<15.00
97.34.0.00
<15.00
97.25 10.16
0.6
<15.00
96.93 +0.15
<15.00
10.19
Yrea = 2.0
T 0.7
<15.00
97.27 +0.02
<15.00
97.72 +0.07
0.8
<15.00
97.18 #0.04
<15.00
97.06
Table 8: Test accuracy comparison under different response function parameters 7 and Tres for FCN training MNIST dataset with power exponential response functions. DSGD, ASGD, and TT represent Digital SGD, Analog SGD, Tiki -akaa, respectively.
L Broader Impact
This paper focuses on developing theoretical analysis for gradient-based training algorithms class of generic AIMC hardware, which can leveraged boost both energy and computational efficiency of training. While such efficiency gains could, in principle, enable broader and potentially unintended uses of machine learning models, we do not identify any specific societal risks that need to be highlighted in this context.
47

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper and important assumptions and limitations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed in the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
48

---

Justification:
See Section 3 and Section 4.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 6 and Section K.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
49

---

Answer: [Yes]
Justification: All of the data used in this paper is public accessible and we include the link. The full details of algorithms have been provided in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips. ec/
wweeiiisssooooooio oolloooooo plycy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/ mublic/guidess Caleee/CooSSubbieee bebbbmisssiins ssisnnoliccyyyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section 6 and Section K.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
See Section 6 and Section K.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
50

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section K.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httppp///eeerpppp.ppppclee/e/// de ines?
csGuidel
Answer: [Yes]
Justification:
This paper strictly follows the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section L.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
51

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We have not included any generation tasks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See Section 6 and Section K.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
52

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We did provide any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
53

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method we developed is original and completely without LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https /ttpa/////eeup.ppeeeee / for what should or should not be described.
54


Reviews:
[{"review_id": "rKjVuWgy1u", "reviewer": "Reviewer_GpyD", "summary": "This paper investigates the theoretical foundations of gradient-based training on analog in-memory computing (AIMC) hardware, specifically focusing on how non-ideal response functions of resistive elements affect training dynamics. The authors develop a mathematical framework to analyze Analog SGD and propose a \"Residual Learning\" algorithm to address the convergence issues caused by asymmetric response functions. The key contributions include a discrete-time model for analog weight updates, theoretical proof that Analog SGD implicitly optimizes a penalized objective leading to inexact convergence, a Residual Learning algorithm that provably converges to critical points through bilevel optimization, and extensions to handle additional hardware imperfections like limited response granularity and noisy I/O.", "questions": "Please see the weaknesses above. Additionally, I am curious about a few questions:\n\n1. Can the authors validate if the idealized response function model can capture most hardware complexities encountered in practice?\n\n2. The analysis focuses primarily on resistive crossbar arrays. I am curious if it can generalize to other emerging analog computing paradigms.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper provides a strong theoretical foundation with rigorous mathematical analysis of a practically important problem. The bilevel optimization formulation for Residual Learning is theoretically elegant and well-motivated, supported by comprehensive proofs with detailed assumptions and convergence guarantees.\n\n2. The proposed response function model captures actual hardware behavior observed in various resistive memory technologies including PCM, ReRAM, and ECRAM.\n\n3. The identification of implicit penalty effects in Analog SGD provides novel and valuable insights, particularly the connection between hardware asymmetries and optimization bias.\n\n4. The authors demonstrate methodological rigor through careful treatment of different hardware imperfections, appropriate use of assumptions with clear justifications, and good experimental validation of theoretical predictions using standard datasets.\n\nWeaknesses:\n\n1. The experimental scope is somewhat limited, as simulations are conducted using AIHWKIT rather than actual hardware, which constrains real-world validation. \n\n2. The dataset scale remains relatively modest with MNIST and CIFAR experiments, raising questions about scalability to larger, modern datasets (however, most papers on analog training adopt these small-scale datasets as benchmarks)\n\n3. The strong convexity assumption for main convergence results is quite restrictive for neural networks, while the bounded saturation assumption may not hold as hardware degrades over time. The idealized response function model may not capture all hardware complexities encountered in practice.\n\n4. The Residual Learning approach introduces computational overhead by requiring maintenance of an additional array and computing gradients on mixed weights.", "was_revised": true, "final_justification": "The authors resolved most of my concerns, and hence, I increase my rating to 5.", "rebuttal": "Thanks for acknowledging that **this paper provides a strong theoretical foundation with rigorous mathematical analysis**, and the support. We address your comments as follows.\n\n\n---\n> Q1: The experimental scope is somewhat limited, as simulations are conducted using AIHWKIT rather than actual hardware, which constrains real-world validation.\n\n\nThanks for your valuable suggestion! Analog in-memory computing is an emerging technology that attracts researchers from material science, device design, chip design, computer systems, and AI algorithm design. Our focus was mainly on the algorithm design for analog in-memory computing, where evaluation using off-the-shelf hardware simulators is very common. Since the reliable devices for analog in-memory training are not commercially available, we plan to leave the hardware evaluation for future work. Nonetheless, we still perform additional simulations to validate the approaches in real resistive element configurations, including ReRAM, EcRAM, and EcRAM-MO. The simulation results, presented in the response to **Q1 of reviewer Kwur**, demonstrate the potential in real-world applications.\n\n---\n> Q2. The dataset scale remains relatively modest with MNIST and CIFAR experiments, raising questions about scalability to larger, modern datasets (however, most papers on analog training adopt these small-scale datasets as benchmarks)\n\nThanks for your valuable suggestion. We evaluate the scalability by investigating their performance on more training tasks, where the models are deeper or have more weights. The preliminary results below indicate that analog training is promising for scaling to larger and more modern datasets and tasks.\n\n**Task 1: Training language models from scratch.**\nWe train a Transformer model from scratch on the *Coriolanus* dataset, a play by Shakespeare. The model consists of 6 layers with ~10 million parameters. The weights for all query, key, value, and output projections in attention layers, as well as linear layers in feed-forward networks, are updated in the analog domain. The other parameters, such as token/position embeddings, scale/bias in layer normalization layers, are put and updated in the digital domain. For a fair comparison, we do not use an adaptive learning rate for either digital or analog training. In Table T1.1, we reach the same conclusion that Residual Learning outperforms Analog SGD significantly and is comparable with Digital SGD.\n\nTable T1.1: Loss (lower is better)\n| Digital SGD | Analog SGD | Residual Learning v1 | Residual Learning v2 |\n|---|---|---|---|\n| 2.46 | 3.10 | 2.67 | 2.61 |\n\n**Task 2: Fine-tuning language models.** \nWe finetune a pre-training tiny MobilieBERT model with 17M parameters on the MRPC dataset via LoRA training. A total of 462 LoRA layers and 0.99M trainable parameters are introduced. We implemented different portions of the LoRA layers on analog arrays, while the others were left on digital hardware and trained using AdamW. The algorithms reduce to digital Adam if the percentage is 0%, while they reduce to a pure analog algorithm (Analog SGD or Residual Learning) when the percentage is 100%. \n\nTable T1.2: Accuracy (higher is better)\n| Analog percentage | Analog SGD | Residual Learning |\n|---|---|---|\n| 0% (Digital training) | 87.21 | |\n| 20% | 87.00 | **88.72** |\n| 40% | 86.02 | **87.00** |\n| 60% | 84.06 | **85.06** |\n| 80% | 78.01 | **80.01** |\n| 100% | 76.63 | **77.69** |\n\nTable T1.3: F1 score (higher is better)\n| Analog percent | Analog SGD | Residual Learning |\n|---|---|---|\n| 0% (Digital training) | 91.03 | |\n| 20% | 90.94 | **91.01** |\n| 40% | 90.35 | **90.87** |\n| 60% | 88.61 | **89.51** |\n| 80% | 81.61 | **84.35** |\n| 100% | 77.67 | **80.95** |\n\nAn interesting observation is that Residual Learning achieves slightly higher accuracy than its digital counterpart when the analog percentage is 20%. We speculate that it happens since the analog imperfection implicitly applies a regularization on the training, leading to a better generalization performance. It is worth investigating in future work.\n\n---\n> Q3. The strong convexity assumption for the main convergence results is quite restrictive for neural networks.\n\nThank you for pointing it out. Following nonconvex bilevel optimization work [R2.1-R2.2], the strong convexity assumed in the current paper can be relaxed to the Polyak-Lojasiewicz (PL) condition with similar conclusions, which accommodates nonconvex landscapes such as those in overparameterized neural networks [R2.3]. We will discuss it in the revision.\n\n---\n> Q4. The bounded saturation assumption may not hold as hardware degrades over time.\n\nThank you for pointing it out. We did not consider hardware degradation over time since typical training processes for moderate-scale models or fine-tuning large models only take hours to days, during which hardware degradation is not a dominant imperfection. It is worth considering in large-scale model training tasks if the training takes a longer time, like weeks to months. We plan to relax this assumption in our future work.\n\n---\n> Q5. Can the authors validate if the idealized response function model can capture most hardware complexities encountered in practice?\n\nThanks for your valuable suggestion. As we discuss in Appendix A, a series of empirical works measured the response curves on various resistive elements [45-59], which demonstrate that the non-linear and asymmetric response is one of the main challenges of analog training. On top of that, AIHWKIT [44] provides a fine-grained simulator of the analog hardware, which includes most of the known hardware imperfections like the IO noise, limited granularity, device variation, and quantization error in analog-digital conversion. **Building upon their work, the validation in this paper can capture most hardware complexities.**\n\nTo further validate the proposed idealized response function model, we compare two settings: \n- **S1)** training models following the proposed dynamics (5); \n- **S2)** training models following the standard settings in the AIHWKIT. See Appendix K and *Table 4.1 in the response to the Reviewer Uooc* for the concrete configurations.\n \nTable T1.4 presents the simulation results. The results show that the accuracy of S1 and S2 is similar for both Analog SGD and Residual Learning. It demonstrates that the proposed response function model captures the training complexities in practice well. In response to your feedback, we will discuss it in the revision.\n\nTable T1.4: Comparison of S1 and S2 in the CIFAR10 training task\n|  | ASGD-S1 | ASGD-S2 | RL-S1 | RL-S2 |\n|---|---|---|---|---|\n| Resnet18 | 87.21 | 84.87 | 95.04 | 94.81 |\n| Resnet34 | 95.56 | 95.43 | 96.31 | 96.29 |\n| Resnet50 | 95.50 | 94.36 | 96.50 | 96.34 |\n\n---\n> Q6. The Residual Learning approach introduces computational overhead by requiring maintenance of an additional array and computing gradients on mixed weights.\n\nWhile Residual Learning introduces extra computational overhead, it is affordable in practice. Compared to Analog SGD, the analog memory requirement doubles, but the latency remains almost unchanged since Residual Learning does not explicitly compute the mixed weights during the forward and backward passes. As [R1.4] suggests, $W_k$ and $P_k$ can share a same analog-digital convertor (ADC), which implements the weight mixing without introducing extra latency. On the other hand, as suggested by [22] in our paper, the forward, backward, and update on $W_k$ and $P_k$ are performed in parallel, which avoids significant latency increase. Consequently, *introducing an extra residual array does not incur substantial extra latency*. Following the evaluation in Table 1 in [24], we compared the latency of Analog SGD and Residual Learning in Table T1.5. The results suggest that the extra overhead is mild. We will discuss it in the revision.\n\nTable T1.5: Comparison of time (nano-second) consumption in each layer\n|  | Analog SGD | Residual Learning |\n|---|---|---|\n| Forward/backward [R1.5] | 40.0 | 40.0 |\n| update                  | 30.9 | 50.8 |\n\n---\nWe hope our response resolves your concerns. Thanks!\n\n---\n\n\n[R1.1] Shen, et al. On penalty-based bilevel gradient descent method. ICML 2023. \n\n[R1.2] Kwon, et al. On penalty methods for nonconvex bilevel optimization and first-order stochastic approximation. ICLR 2024. \n\n[R1.3] Liu, et al. Loss landscapes and optimization in over-parameterized non-linear systems and neural networks. 2022.\n\n[R1.4] Song, et al. Programming memristor arrays with arbitrarily high precision for analog computing. Science 2024.\n\n[R1.5] Jain, et al. A heterogeneous and programmable compute-in-memory accelerator architecture for analog-ai using dense 2-D mesh. VLSI 2022."}, {"review_id": "09gViiNQHM", "reviewer": "Reviewer_Kwur", "summary": "This paper addresses the question of how to overcome the fact that parameter updates in analog in-memory computing have non-linear responses to update pulses. They find that asymmetric response functions negatively impact analog SGD and propose residual learning as a solution. They evaluate their approach with numerical experiments and generally reach a good match to digital SGD.\n\nThe main contributions are: \n- To propose an approximate discrete-time dynamic to the analog pulse update and to study the impact of response functions based on it\n- They show that means that analog SGD implicitly optimizes for a penalized objective. This means that it cannot converge to the optimum of the non-penalized loss\n- To address this they propose a \"Residual Learning\" algorithm, which they claim provably converges to a critical point under assumptions of strong convexity and expand this to include additional concerns that could occur in analog in memory computing hardware.", "questions": "- How would you validate your approach on either a hardware model or actual hardware?\n- Can you illustrate the difference in the algorithms in a case where the assumptions of the theoretical results are met?", "limitations": "Sufficiently discussed.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The paper addresses an important problem: How to deal with the fact that analog in memory computing can only approximately implement SGD updates. \n\nStrength:\n- The identified problem in analog learning is crucial to overcome for AIMC to succeed.\n- Thorough theoretical analysis of Analog SGD.\n- Well motivated and executed analyses of problem and solution. \n- \"Residual Learning\" and its extension appears to be a novel solution. \n- Numerical evaluation on multiple datasets and models (MNIST, CIFAR10, CIFAR100) and (FC/Conv, Resnet), among others demonstrates the ability of the method to achieve Digital SGD level of performance.\n\nWeaknesses:\n- No evaluation on any actual analog hardware\n- Limited evaluation on more larger machine learning datasets like Imagenet\n- No evaluation on a specific simulated analog hardware\n- The paper would benefit from an illustration of the proposed alternative algorithm on toy datasets that meet the assumptions of the theoretical results.", "was_revised": true, "final_justification": "This paper would be a good fit for neurips. I recommend its acceptance in particular in light of the updates to the results.", "rebuttal": "Thanks for acknowledging that **the identified problem in this paper is crucial to overcome for successful training on AIMC hardware**. We address other comments as follows.\n\n---\n> W1 & W3 & Q1. No evaluation on any actual analog hardware. No evaluation on a specific simulated analog hardware. How would you validate your approach on either a hardware model or actual hardware?\n\nThanks for your constructive suggestion! As mentioned in the response to **Q5 of Reviewer GpyD**, our work builds upon a wide range of empirical study [45-59] and all our simulations are done on AIHWKIT simulator [44]. Since the AIHWKIT already captures most of the known hardware imperfections, the conclusion from our paper should be able to generalize to actual hardware.\n\nAlthough we do not perform it in this work, we will discuss how to validate the proposed approach on actual hardware. We will follow the methods presented by [R2.1], which proposes a device-algorithm co-optimization framework that validates the Tiki-Taka [22-23] on HfO$_2$-based ReRAM with 14nm CMOS technology. Similar to them, we will first need to measure the response curve of a given resistive element (e.g., ReRAM) and simulate its behaviour in the analog hardware simulator. We then validate that the proposed approach can achieve comparable accuracy with digital training benchmarks on different machine learning tasks. Based on the simulation results, we can optimize the hardware to reproduce the simulation results on actual hardware. \n\nAs the first step, we validate the proposed approaches on two real resistive element configurations. The results presented in Table T2.1 suggest that the proposed algorithm works well on various real resistive elements. The rest of the hardware validation involves chip design, fabrication, and extensive testing. We will do so in future work.\n\nTable T2.1: Test accuracy of training FCN models on MNIST dataset \n| Digital SGD | ReRAM [R2.1] | EcRAM [52] | EcRAM-MO [R2.2] |\n|---|---|---|---|\n| 98.17 | 96.10 | 97.55 | 96.76 |\n\n\n---\n> W2. Limited evaluation on more larger machine learning datasets like Imagenet.\n\nThanks for your valuable suggestion! We include additional simulations in the revision to make our paper more convincing. In the response to **Q2 of the reviewer GpyD**, we present additional simulation results in the language model training during the limited rebuttal period, including training from scratch and finetuning tasks. Since the simulations on ImageNet training take a longer time, we will complete them after the rebuttal and add them in the revision.\n\n---\n> W4 & Q2. The paper would benefit from an illustration of the proposed alternative algorithm on toy datasets that meet the assumptions of the theoretical results. \n\n\nThanks for your constructive suggestions! We will add a new subsection to discuss and illustrate the algorithms at the end of Section 4. We consider a toy regression problem $\\min_{W\\in{R}^2}||W-W^*||^2$, whose objective is smooth and strongly convex. \n\nWe choose a symmetric point $W^\\diamond\\ne W^*$. We add a Gaussian noise with mean 0 and variance $\\sigma^2\\ne 0$ to the gradients. We plot the convergence trajectories of Analog SGD (equation (2)) and Residual Learning (equation (10)-(11)). \n\nWe show that the weights generated by Analog SGD are attracted toward the symmetric point $W^\\diamond$, as suggested by Theorem 1. Instead, Residual Learning converges to the optimal point and the auxiliary array $P_k$ is a good approximation of the residual $\\frac{W^*-W_k}{\\gamma}$, as suggested by the theory. We will discuss this example in the revision to provide more insight.\n\n---\nWe hope our response resolves the concerns. Thanks!\n\n---\n[R2.1] Gong, et al. Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. IEDM 2022.\n\n[R2.2] Kim, et al. Metal-oxide based, CMOS-compatible ECRAM for deep learning accelerator. IEDM 2019."}, {"review_id": "yEqNwailVS", "reviewer": "Reviewer_TGon", "summary": "The authors provide a detailed analysis of the shortcomings of Analog SGD for training on AIMC hardware, and offer a learning algorithm that can converge to an exact critical point. They provide very detailed theoretical proofs, and very good motivation for the problem.", "questions": "I wonder if it would be possible to incorporate more simulations with more practical scenarios and other realistic imperfections in the rebuttal period. If the authors were able to achieve more significant and convincing improvements, as the theory seems to entail, it would make the paper extremely strong. As of now , although very good, it feels \"top-heavy\" although the challenges of simulating realistic hardware imperfections and noise are well-noted.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "This is an extremely strong paper theoretically and is very well written, well presented and well-motivated. In light of the foundational nature of the problems they highlight (for e.g. analog SGD, and for AIMC training in general) I would have expected more significant improvements on the simulations. As the authors note, this is perhaps due to the artificial nature of simulating hardware imperfections, and realistic noise (a famously hard problem!)", "was_revised": true, "final_justification": "The authors have added simulations which was in my opinion the major missing component from this work.", "rebuttal": "Thanks for acknowledging the theoretical contribution of our work and the support. We address some other comments as follows.\n\n> Q1: I wonder if it would be possible to incorporate more simulations with more practical scenarios and other realistic imperfections in the rebuttal period.\n\nThanks for your constructive suggestion! Following your suggestion, we incorporated more simulations to reflect more practical setting challenges in analog training:\n\n**1) More challenging machine learning tasks.** We conducted a series of additional simulations on various tasks, including training language models from scratch and their finetuning. Compared with the existing simulation results, the additional ones involve deeper models and more parameters in the analog domain. The simulation results, presented in the response to **Q2 in reviewer GpyD**, show that the proposed approaches generalize well on more challenging tasks.\n\n**2) Validation on real resistive element configuration.** To validate the approaches in real resistive elements, we performed simulations on various resistive elements, including ReRAM, EcRAM, and EcRAM-MO. The training on them is more challenging since they suffer from the asymmetric update issue, and they have limited update granularity and dynamic ranges. The simulation results, presented in the response **Q1 in reviewer Kwur**, show that it is promising to apply the proposed methods on real resistive elements.\n\nWe hope our response resolves your concerns. We are keen to engage in further discussion to ensure the clarity and rigor of our work. Thanks!"}, {"review_id": "vX1n5aJQTN", "reviewer": "Reviewer_Uooc", "summary": "The aim of this paper is to address nonlinearities involved in gradient-based training for Analog In-Memory Computing (AIMC) systems by proposing new methods for robust training. The authors have developed rigorous proofs to analyze the properties of the proposed residual learning methods and have demonstrated their working using simulation studies.", "questions": "While the proposed methods demonstrate performance improvements over the vanilla analog SGD, no discussion has been provided regarding its computational overhead. Is there any significant increase of computational complexity? How does it scale with the model size?", "limitations": "N/a", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "*** Strengths:\n\nThis paper primarily addresses two important types of nonliearities occurring in AIMC training: asymmetric weight update response, and other nonlinearities such as I/O reading errors. For the first problem, which is a key issue, the authors develop theoretical work to show that asymmetry influences training negatively by adding a penalty on the objective function, which provides a motivation for the development of the proposed methods. \n\nThe key contribution of this work is its  residual learning algorithm, which has a provable convergence to the critical point. This residual learning algorithm is further extended to deal with I/O reading errors. \nThe work provides an in-depth understanding of negative impacts of several nonliearities of the AIMC weight update process, and develops theoretically rigorous methods for reducing these negative impacts. \n\n**** Weaknesses:\n\nThe experimental aspects of the work can be improved to make the proposed work more convincing. \nSpecifically, the application of the proposed residual learning algorithm and its variants have been only demonstrated on fairly small datasets for which, some modest performance gains have been observed compared to other recent methods. \n\nMore details of physical modeling of AMIC systems and weight updating process can be provided. For example, it is not clear what kind of I/O reading errors have been considered in the experimental studies.", "was_revised": true, "final_justification": "I have read the rebuttal and other reviewers' further comments. I maintain my support of this paper.", "rebuttal": "Thanks for acknowledging the theoretical contribution of our work and the support. We address other comments as follows.\n\n> W1: The experimental aspects of the work can be improved to make the proposed work more convincing. Specifically, the application of the proposed residual learning algorithm and its variants have been only demonstrated on fairly small datasets for which, some modest performance gains have been observed compared to other recent methods.\n\nThanks for your valuable suggestion! We incorporate more simulations to make the paper more convincing, including training language models from scratch and their finetuning. Compared with the existing simulations, the additional ones involve deeper models and have more weights. The simulation results, presented in the response to **Q2 of reviewer GpyD**, show that the proposed approaches generalize well on more challenging tasks.\n\n---\n> W2: More details of physical modeling of AMIC systems and weight updating process can be provided. For example, it is not clear what kind of I/O reading errors have been considered in the experimental studies.\n\nThanks for your valuable suggestion. In all simulations, we adopt the following I/O settings. We will attach the details in the revision.\n\nTable T4.1: I/O setting in the paper\n| configuration | value |\n|---|---|\n| input bound | 1.0 |\n| input noise | None |\n| input resolution (DAC) | 7 bits |\n| output bound | 12.0 |\n| output noise | additive Gaussian noise $\\mathcal{N}(0, 0.06^2)$ | \n| output resolution (ADC) | 9 bits |\n\n\n---\n> Q1: While the proposed methods demonstrate performance improvements over the vanilla analog SGD, no discussion has been provided regarding its computational overhead. Is there any significant increase of computational complexity? How does it scale with the model size?\n\nThanks for your construction question! We compare the computational overhead in the response to **Q4 in reviewer GpyD**. Although an extra residual array $P_k$ is introduced, the latency does not significantly increase. Furthermore, as existing work presents, the resistive crossbar array implements the MVM operator with $O(1)$ time complexity as the matrix size increases, ensuring its strong scalability [R1.5]. \n\n---\n\nWe hope our response resolves your concerns. We are keen to engage in further discussion to ensure the clarity and rigor of our work. Thanks!"}]

Author rebuttal:

Meta-review: The paper considers training on analog in-memory hardware, providing a theoretical framework for gradient-based training on such hardware in the presence of non-idealities in response functions. This framework is used to show that these non-idealities negatively affect SGD, and as a mitigation, the paper proposes a residual learning algorithm that is shown to converge provably and through simulations.

Reviewers are unanimous in their positive opinion of the paper, with all four giving it a score of 5. AC concurs and enthusiastically recommends acceptance.
Decision: Accept (oral)