Title: Towards Exact Gradient-based Training on Analog In-memory Computing
Venue: NeurIPS 2024
Accepted: yes
Keywords: Analog AI; in-memory computing; stochastic gradient descent; stochastic optimization
Abstract: Given the high economic and environmental costs of using large vision or language models, analog in-memory accelerators present a promising solution for energy-efficient AI. While inference on analog accelerators has been studied recently, the training perspective is underexplored. Recent studies have shown that the "workhorse" of digital AI training - stochastic gradient descent (SGD) algorithm converges inexactly when applied to model training on non-ideal devices. This paper puts forth a theoretical foundation for gradient-based training on analog devices. We begin by characterizing the non-convergent issue of SGD, which is caused by the asymmetric updates on the analog devices. We then provide a lower bound of the asymptotic error to show that there is a fundamental performance limit of SGD-based analog training rather than an artifact of our analysis. 
To address this issue, we study a heuristic analog algorithm called Tiki-Taka that has recently exhibited superior empirical performance compared to SGD. We rigorously show its ability to converge to a critical point exactly and hence eliminate the asymptotic error. The simulations verify the correctness of the analyses.

Towards Exact Gradient-based Training on Analog In-memory Computing

Abstract

Given the high economic and environmental costs of using large vision or language models, analog in-memory accelerators present a promising solution for energyefficient AI. While inference on analog accelerators has been studied recently, the training perspective is underexplored. Recent studies have shown that the "workhorse" of digital AI training - stochastic gradient descent ( SGD ) algorithm converges inexactly when applied to model training on non-ideal devices. This paper puts forth a theoretical foundation for gradient-based training on analog devices. We begin by characterizing the non-convergent issue of SGD , which is caused by the asymmetric updates on the analog devices. We then provide a lower bound of the asymptotic error to show that there is a fundamental performance limit of SGD -based analog training rather than an artifact of our analysis. To address this issue, we study a heuristic analog algorithm called Tiki-Taka that has recently exhibited superior empirical performance compared to SGD . We rigorously show its ability to converge to a critical point exactly and hence eliminate the asymptotic error. The simulations verify the correctness of the analyses.

1 Introduction

Large vision or language models have recently achieved great success in various applications. However, training large models from scratch requires prolonged durations and substantial energy consumption, which is very costly. For example, it took $2.4 million to train LLaMA [1] and $4.6 million to train GPT-3 [2]. To overcome this issue, a promising technique is application-specific hardware accelerators for neural networks, such as TPU [3], NPU [4], and NorthPole chip [5], just to name a few. Within these accelerators, the memory and processing units are physically split, which requires constant data movements between the memory and processing units. This slows down computation and limits efficiency. In this context, we focus on analog in-memory computing (AIMC) accelerators with resistive crossbar arrays [6, 7, 8, 9] to accelerate the ubiquitous matrix-vector multiplications (MVMs), which contribute to a significant portion of digital computation in model training and inference, e.g., about 80% in VGG16 model [10]. Very recently, the first analog AI chip has been fabricated in IBM's Albany Complex [11], and has achieved an accuracy of 92.81% on the

CIFAR10 dataset while enjoying 280 × and 100 × efficiency and throughput value compared with the most-recent GPUs, demonstrating the transformative power of analog computing for AI [12].

In AIMC accelerators, the trainable matrix weights are represented by the conductance of the resistive elements in an analog crossbar array [13, 14]. Unlike standard digital devices such as GPU or TPU, trainable matrices, input, and output of MVMs in resistive crossbars are all analog signals , which means that they are effectively continuous physical quantities and are not quantized. In resistive crossbars, the fundamental physics (mainly Kirchhoff's and Ohm's laws) enable the devices to accelerate MVMs in both forward and backward computations. However, the analog representation of model weights requires updating weights in their unique way. To this end, an in-memory pulse update has been developed in [15], which changes the weights by sending consecutive pulses to implement gradient-based training algorithms like stochastic gradient descent (SGD). Pulse update reduces energy consumption and execution time significantly.

While the AIMC architecture has potential advantages, the analog signal in resistive crossbar devices is susceptible to noise and other non-ideal device characteristics [16], leading to training performance that is often suboptimal when compared to digital counterparts. Despite the increasing number of empirical studies on AIMC that aim to overcome this accuracy drop [17, 18, 19, 20], there is still a lack of theoretical studies on the performance and performance limits of SGD on AIMC accelerators.

1.1 Main results

The focus of this paper is fundamentally different from the vast majority of work in analog computing.

We aim to build a rigorous theoretical foundation of analog training, which can uniquely characterize the performance limit of using a hardwareagnostic digital SGD algorithm for analog training and establish the convergence rate of gradient-based analog training. Consider the following standard model training problem

0

500

1000

1500

2000

2500

Iteration

k

10

9

10

7

10

5

10

3

10

1

f

(

W k

)

f

*

=2e-01

=1e-01

=5e-02

=3e-02

Digital SGD

Analog SGD

Figure 1: Digital/Analog SGD under different learning rates. where f ( · ) : R D → R is the objective function, W is all the trainable weight stored in an analog way and D is the model size.

In digital training, the workhorse algorithm for solving problem (1) is SGD, which iteratively updates weight W via the following recursion

where k is the iteration index, α is a positive learning rate and ε k is the gradient noise with zero mean at iteration k . In digital training, the noise usually comes from the mini-batch sampling. In analog training, the noise also arises from the non-ideality of devices, including weight read noise, input/output noise, quantization noise, digital/analog conversion noise, and even thermal noise [21].

In Figure 1, we first numerically show that the asymptotic performance of SGD running on the analog devices that we term Analog SGD does not follow that of (2), and thus ask a natural question:

Q1) How to better characterize the training trajectory of SGD on analog devices?

Building upon the pulse update in [15, 22], in this paper, we propose the following discrete-time mathematical model to characterize the trajectory of Analog SGD on analog computing devices

where | · | and ⊙ represent the coordinate-wise absolute value and multiplication, respectively and τ is a device-specific parameter. We will explain the underlying rationale of this model (3) in Section 2.2. But before that, compared to (2), the extra term of (3) comes from the asymmetric update of analog devices. Following this dynamic, we prove that the Analog SGD only converges inexactly to a critical point, with the asymptotic error depending on the non-ideality of the analog device. Since the inexact convergence guarantee usually leads to an unfavorable result, it raises another question:

Q2) How to mitigate the asymptotic error induced by the asymmetric analog update?

To answer this, we revisit a heuristic algorithm Tiki-Taka that has been used among experimentalists [22], and establish the first exact convergence result of Tiki-Taka on a class of AIMC accelerators.

Our contributions. This paper makes the following contributions (see the comparison in Table 1):

C1) We demonstrate that Analog SGD does not follow the dynamic of SGD in (2). Leveraging the underlying physics, we propose a discrete-time dynamic of the gradient-based algorithm on analog devices, and show that it better characterizes the trajectory of Analog SGD .

C2) Based on the proposed dynamic, we establish the convergence of Analog SGD and argue that the performance limit of Analog SGD is a combined effect of data noise and device asymmetry. We prove the tightness of our result by showing a matching lower bound.

C3) To improve the performance limit of analog training, we study a heuristic algorithm Tiki-Taka that serves as an alternative to Analog SGD . We show that Tiki-Taka exactly converges to the critical point by reducing the effect of asymmetric bias and noise.

C4) To verify the validity of our discrete-time dynamic for analog training and the tightness of our analysis, we provide simulations on both synthetic and real datasets to show that the asymptotic error of Analog SGD does exist, and Tiki-Taka outperforms Analog SGD .

1.2 Prior art

Since the seminal work on pulse update-based Analog SGD [15], a series of gradient-based training algorithms have been proposed to enable training on analog devices. Despite its potential energy and speed advantage, Analog SGD suffers from asymmetric update and noise issues, leading to large errors in training. To overcome the asymmetric issue , a new algorithm so-termed Tiki-Taka (TT-v1) introduces an auxiliary array to estimate the moving average of gradients, whose weight is then transferred to the main array periodically [22] . However, the weight transfer process between arrays still suffers from noise. To deal with this issue, TT-v2 [25] introduces an extra digital array to filter out the high-frequency noise. Enabled by these approaches, researchers successfully trained a model on the realistic prototype analog devices and reached comparable performance on a real dataset [20].

Another major hurdle comes from noisy analog signals , which can perturb the gradient computed by analog devices. As an alternative, a hybrid scheme that accelerates the forward and backward pass in-memory and computes gradient in the digital domain has been proposed in [17, 18], which provides a more accurate but less efficient update. In addition, the successful training by TT-v1 or TT-v2 relies on the zero-shifting technique [26], which corrects the symmetric point of devices as zero. However, the correction is inaccurate because it also involves analog signals. To deal with this issue, Chopped-TTv2 (c-TTv2) and analog gradient accumulation with dynamic reference have been proposed in [19]. Because of these efforts, analog training has empirically shown great promise in achieving a similar level of accuracy as digital training, with reduced energy consumption and training time. Despite its good performance, it is still mysterious about when and why they work.

2 The Physics of Analog Training

Unlike digital devices, analog devices represent the trainable weights by the conductance of the base materials, which have undergone offset and scaling due to physical laws . This difference leads to entirely different training dynamics on analog devices, which will be discussed in this section.

2.1 Revisit SGD theory and its failure in modeling analog training

This section shows that the convergence theory developed for digital SGD fails to characterize the analog training. Before that, we introduce standard assumptions for analyzing digital training.

Assumption 1 ( L -smoothness) . The objective f ( W ) is L -smooth, i.e., for any W,W ′ ∈ R D , it holds

Assumption 2 (Lower bounded) . f ( W ) is lower bounded by f ∗ , i.e. f ( W ) ≥ f ∗ , ∀ W ∈ R D .

Assumption 3 (Noise mean and variance) . The noise ε k is independently identically distributed (i.i.d.) for k =0 , 1 , . . . , K , and has zero mean and bounded variance, i.e., E [ ε ] = 0 , E [ ∥ ε ∥ 2 ] ≤ σ 2 .

In non-convex optimization, instead of finding a global minimum, it is usually more common to find a critical point , which refers to W ∗ satisfying ∇ f ( W ∗ ) = 0 . Under Assumptions 1-3, if W k follows the digital SGD dynamic (2), the convergence of SGD is well-understood [23, Theorem 4.8]

This result implies that the impact of noise can be controlled by the learning rate α , i.e., after K ≥ Ω(1 /α 2 ) iterations, the error is O ( α ) . By forcing α → 0 , the error will reduce to zero.

Analog SGD violates the SGD theory. The dynamic of digital SGD exactly follows (2) up to the machine's precision. To verify whether analog training adheres to the same dynamic, we conduct a numerical simulation on a least-squares problem. We compare Analog SGD implemented by an analog devices simulator, IBM Analog Hardware Acceleration Kit (AIHWKIT) [27], and digital SGD implemented by PYTORCH. The same level of noise obeying Gaussian noise is injected into each algorithm. Beginning from a large learning rate α = 0 . 2 , we reduce the learning rate by half each time and observe the convergences. As Figure 1 illustrates, digital SGD behaves as what (5) predicts: it converges with a smaller error when a small learning rate is chosen. On the contrary, Analog SGD converges with a much larger error, which does not decrease as the learning rate decreases. This result demonstrates the discrepancy between the theory of digital SGD and the performance of Analog SGD . More details are deferred to Appendix I.

2.2 Training dynamic on analog devices

Compared to digital devices, the key feature of analog devices is analog signal . The input and output of analog arrays are analog signals, which are prone to be perturbed by noise, including read noise and input/output noise [21, 28]. Moreover, real training typically involves the utilization of mini-batch samples, which also introduces noise. Besides the data noise, another notable feature of analog accelerators is that the model weight is represented by material conductance.

Pulse update. To change the weights in analog devices, one needs to send an electrical pulse to the resistive element, and the conductance will change by a small value, which is referred to as pulse update [15]. To apply an update ∆ w to the weight w , using the pulse update needs to send a series of pulses to the resistive element, the number of which is proportional to the update magnitude | ∆ w | . Since the increments responding to each pulse are small typically, we can regard the change in conductance as a continuous process. Consequently, common operations involving significant weight changes, like copying the weight, are expensive in analog accelerators. In contrast, gradient-based algorithms typically update small amounts at each iteration, rendering the pulse update extremely efficient. Figure 2 presents the weight change on AIMC devices with pulse number.

Asymmetric update. Even though the pulse update is performed efficiently inside AIMC accelerators, it suffers from a phenomenon that we refer to as asymmetric update . This means that if we apply

+

-

number of pulses

weight      (conductance)

+

-

number of pulses

weight      (conductance)

the change ∆ w > 0 and ∆ w < 0 on the same weight w , the amount of weight change will be different. Considering the weight W k at time k and the expected update ∆ W ∈ R D , we express the asymmetric update as W k +1 = U ( W k , ∆ W ) with the update function U : R × R → R defined by 1

where q + ( · ) and q -( · ) : R → R + are up and down response factors, respectively. The response factors measure the ideality of analog tiles. In the ideal situation, q + ( w ) = q -( w ) ≡ 1 (see Figure 2, left), and analog algorithms have the same numerical behavior of the digital ones. Defining the symmetric and asymmetric components as F ( w ) := 1 2 ( q -( w ) + q + ( w )) and G ( w ) := 1 2 ( q -( w ) -q + ( w )) , the update in (6) can be expressed in a more compact form U ( w, ∆ w ) = w +∆ w · F ( w ) -| ∆ w | · G ( w ) . For simplicity, assume that all the coordinates of W use the same update rule, and the analog update can be written as

where | · | and ⊙ represent the coordinate-wise absolute value and multiplication, respectively. Note that in (7), the ideal weight update ∆ W is algorithm-specific: ∆ W is the gradient in Analog SGD while it is an auxiliary weight (c.f. in Section 4) in Tiki-Taka .

Remark 1 (Physical constraint) . It is attempting to scale the ∆ w by q + ( w ) or q -( w ) to cancel the effect of asymmetric update in (6) dynamically. However, this is impractical since it is hard to implement the reading and scaling at the same time on analog tiles [22].

̸

Symmetric point. The asymmetric update makes up and down responses different almost everywhere, i.e. q + ( w ) = q -( w ) for almost any w . If a point w ⋄ satisfies q + ( w ⋄ ) = q -( w ⋄ ) and G ( w ) = 0 , w ⋄ is called a symmetric point . With loss of generality, the response factor is defined so that F ( w ⋄ ) = 1 . Therefore, near the symmetric point, the update ∆ w can be accurately applied on w , i.e., U ( w ⋄ , ∆ w ) ≈ w ⋄ +∆ w . If all the coordinates of matrix W k hover around the symmetric point, the analog devices can exhibit performance that resembles the digital ones. In the next section, we will show that the weight is biased toward its symmetric point.

Asymmetric linear device. Although our unified formulation (7) can capture the response behaviors of different materials, this paper mainly focuses on the behaviors of the asymmetric linear device (ALD), similar to the setting in [19]. ALD has a positive parameter τ which reflects the degree of asymmetry and its response factors are written as linear functions q + ( w ) = 1 -( w -w ⋄ ) /τ, q -( w ) = 1 + ( w -w ⋄ ) /τ . Consequently, ALD has F ( w ) = 1 , G ( w ) = ( w -w ⋄ ) /τ , and symmetric point w ⋄ ; see Figure 2, right. Even though ALD is a simplified device model, it is representative enough to

reveal the key properties and difficulties of gradient-based training algorithms on analog devices. If not otherwise specified, w ⋄ is always 0 for simplicity. In summary, the update of ALD is expressed as

where the first equation is the update of one coordinate while the second one stacks all the elements together. Replacing ∆ W with noisy gradient α ( ∇ f ( W k ) + ε k ) reaches the dynamic (3).

2.3 Saturation, fast reset, and bounded weight

Based on the ALD dynamic (8), we study the properties of analog training, which serve as the stepping stone of our analyses. Recall that the asymmetric update leads to different magnitudes of increase and decrease. The following lemma characterizes the difference between two directions.

Lemma 1 (Saturation and fast reset) . For ALD with a general w ⋄ , the following statements are valid

The proof is deferred to Section E.1. Remarkably, Lemma 1 holds for any algorithm with any update ∆ w , and it reveals the impact of the asymmetric update. In principle, Lemma 1 can be written as | w k +1 -w k | = | 1 ±| w k -w ⋄ | /τ | · | ∆ w | , where the symmetric point w ⋄ = 0 is omitted.

When w k is not at the symmetric point w ⋄ = 0 , the update is scaled by a factor. When w k lies around the symmetric point, | w k +1 -w k | ≈ | ∆ w | , where all updates are applied to the weight and hence the analog devices closely mimic the performance of digital devices. When w k moves away from w ⋄ , i.e. sign (∆ w ) = sign ( w k ) , the update becomes none. When w k gets closer to ± τ , nearly no update can be applied. This phenomenon is called saturation ; see also [22]. On the contrary, w k changes faster when it moves toward w ⋄ , which is referred to as fast reset .

Because of the saturation property, the weight on the analog devices is intrinsically bounded, which will be helpful in the later analysis. The following theorem discusses the bounded property.

Theorem 1 (Bounded weight) . Denote ∥ W ∥ ∞ as the ℓ ∞ norm of W . Given ∥ W 0 ∥ ∞ ≤ τ and for any sequence { ∆ W k : k ∈ N } , which satisfies ∥ ∆ W k ∥ ∞ ≤ τ , it holds that ∥ W k ∥ ∞ ≤ τ, ∀ k ∈ N .

The proof is deferred to Section E.2. Theorem 1 claims that the weight is guaranteed to be bounded, even without explicit projection, which makes analog training different from its digital counterpart. Similar to Lemma 1, Theorem 1 does not depend on any specific analog training algorithm.

3 Performance Limits of Analog Stochastic Gradient Descent

After modeling the dynamic of analog training, we next discuss the convergence of Analog SGD . As shown by Lemma 1, the update is slow near the boundary of the active region. The weight is expected to stay within a smaller region to avoid saturation, which necessitates the following assumption.

Assumption 4 (Bounded saturation) . There exists a positive constant W max < τ such that the weight W k is bounded in ℓ ∞ norm, i.e., ∥ W k ∥ ∞ ≤ W max . The ratio W max /τ is the saturation degree.

Assumption 4 requires that W k is bounded inside a small region, which is a mild assumption in real training. For example, one can apply a clipping operation on w k to ensure the assumption. In Appendix E.2, we show that Assumption 4 provably holds under the strongly convex assumption. It is worth pointing out that Assumption 4 implicitly assumes there are critical points in the box { W : ∥ W ∥ ∞ ≤ τ } . Otherwise, the gradient will push the weight to the bound of the box and it becomes possible that W max < ∥ W k ∥ ∞ ≤ τ .

Intuitively, without the asymmetric bias, the weight is stable near the critical point. In contrast, with asymmetric bias, the noisy gradient that pushes w k toward its symmetric point is amplified by fast reset, while the one that drags w k away from 0 is suppressed by saturation (c.f. Lemma 1). Consequently, the weight w k is attracted by 0 , which prevents the stability of Analog SGD around the critical point. We characterize the convergence of Analog SGD in the following theorem.

Theorem 2 (Convergence of Analog SGD ) . Under Assumption 1-4, if the learning rate is set as α = √ f ( W 0 ) -f ∗ σ 2 LK and K is sufficiently large such that α ≤ 1 L , it holds that

where S K denotes the amplification factor given by S K := 1 K ∑ K k =0 ∥ W k ∥ 2 ∞ /τ 2 1 -∥ W k ∥ 2 ∞ /τ 2 ≤ W 2 max /τ 2 1 -W 2 max /τ 2 .

The proof of Theorem 2 is deferred to Appendix G.1. Theorem 2 suggests that the average squared gradient norm is upper bounded by the sum of two terms: the first term vanishes at a rate of O ( √ σ 2 /K ) which also appears in the SGD's convergence bound (5); the second term contributes to the asymptotic error of Analog SGD , which does not vanish with the total number of iterations K ; that is, lim sup K →∞ 1 K ∑ K -1 k =0 E [ ∥∇ f ( W k ) ∥ 2 ] ≤ σ 2 S ∞ exist.

Impact of saturation/asymmetric update. The saturation degree W max /τ affects both convergence rate and asymptotic error. The ratio is small if W k remains close to the symmetric point or τ is sufficiently large. The exact expression of S K depends on the specific noise distribution, and thus is difficult to reach. However, S K reflects the saturation degree near the critical point W ∗ when W k converges to a neighborhood of W ∗ . Intuitively, W k ≈ W ∗ implies S K ≈ ∥ W ∗ ∥ 2 ∞ /τ 2 1 -∥ W ∗ ∥ 2 ∞ /τ 2 . Therefore, if the critical point is near the symmetric point, the asymptotic error S K could be small. The asymmetric update has a negative impact on both rate and error. It slows down the convergence of SGD by a factor 1 / (1 -W 2 max /τ 2 ) , and, meanwhile, a smaller τ increases the asymptotic error.

To demonstrate the asymptotic error in Theorem 2 is not artificial, we provide a lower bound next.

Theorem 3 (Lower bound of the error of Analog SGD ) . There is an instance which satisfies Assumption 1-4 such that Analog SGD generates a sequence { W k : k = 0 , 1 , · · · , K } which satisfies

The proof of Theorem 3 is deferred to Appendix G.2. Theorem 3 implies that σ 2 S K in the right-hand side (RHS) of (9) can not be improved and therefore the lower bound of the asymptotic error. It demonstrates that the presence of asymptotic error is intrinsic and not an artifact of the convergence analysis. The nonzero asymptotic error also reveals the fundamental performance limits of using Analog SGD for analog training, pointing out a new venue for algorithmic development.

4 Eliminating Asymptotic Error of Analog Training: Tiki-Taka

Building upon our understanding on the modeling of gradient-based analog training in Section 2 and the asymptotic error of Analog SGD in Section 3, this section will be devoted to understanding means to overcome such asymptotic error in analog training.

We will focus on our study on a heuristic algorithm Tiki-Taka that has shown great promise in practice [22]. The key idea of Tiki-Taka is to maintain an auxiliary array to estimate the true gradient. To be specific, Tiki-Taka introduces another analog device, P k , besides the main one, W k . At the initialization phase, P k is initialized as 0 . At iteration k , the stochastic gradient is computed using the main device W k and is first applied to P k . Since P k is also an analog device, the change of P k still follows the dynamic (8) by replacing W k with P k and ∆ W with ∇ f ( W k ) + ε k , that is

where β is a learning rate. After that, the value P k +1 is read and transferred to the main array W k via

Tiki-Taka performs recursion (TT-P) and (TT-W) alternatively until it converges. Empirically, Tiki-Taka outperforms Analog SGD in terms of the final accuracy. However, Tiki-Taka is a heuristic algorithm, and there are no convergence guarantees so far. In this section, we demonstrate that the improvement of Tiki-Taka stems from its ability to eliminate asymptotic errors.

Stability of Tiki-Taka . As explained in Section 3, the gradient noise contributes to the asymptotic error of Analog SGD . To eliminate the error, the idea under Tiki-Taka is to reduce the noise impact. To see how Tiki-Taka reduces the noise, consider the case where W k is already a critical point and P k is initialized as 0 , i.e. ∇ f ( W k ) = P k = 0 . After one iteration, the weight W k drifts because of the noise. For Analog SGD , the expected drift is

In contrast, Tiki-Taka updates the auxiliary array by P k +1 = P k + βε k -β τ | ε k | ⊙ P k = βε k , which implies E [ P k +1 ] = 0 and E [ | P k +1 | ] = β E [ | ε k | ] . After the transfer, its expected drift is

̸

Comparing (12) with (11), it can be observed that Tiki-Taka improves the expected drift from O ( α ) to O ( αβ ) . With sufficiently small β , Tiki-Taka controls the drift and makes the weight stay at the critical point. For a more generic scenarios, P k = 0 . However, it is worth noting that

̸

which makes P k back to 0 when E [ | ε k | ] = 0 . Therefore, by controlling the drift, the Tiki-Taka algorithm manages to stay near a critical point.

̸

Note that from (13), the stability of P k relies on the presence of noise, i.e. E [ | ε k | ] = 0 . In addition to the upper bound on the noise in Assumption 3, a lower bound for the noise is also assumed.

Assumption 5 (Coordinate-wise i.i.d. and non-zero noise) . For any k ≥ 0 and i, j ∈ I , [ ε k ] i and [ ε k ] j are i.i.d. from a distribution D c which ensures E [ ε k ] i ∼D c [ ε k ] = 0 . Furthermore, there exists σ > 0 and c > 0 such that E [ ε k ] i ∼D c [[ ε k ] 2 i ] ≤ σ 2 /D and E [ ε k ] i ∼D c [ | g +[ ε k ] i | ] ≥ cσ for any g ∈ R .

Intuitively, Assumption 5 requires the non-zero noise, which is mild since the random sampling and the physical properties of analog devices always introduce noise. The factor D in the denominator makes it consistent with Assumption 3. We discuss this assumption in more detail in Appendix D.2.

Theorem 4 (Convergence of Tiki-Taka ) . Suppose Assumption 1-5 hold and the learning rate is set as α = O (1 / √ σ 2 K ) , β = 8 αL . It holds for Tiki-Taka that the expected infinity norm P k is upper bounded by E [ ∥ P k +1 ∥ 2 ∞ ] ≤ P 2 max := 41 L 2 τ 4 D c 2 σ 2 . Furthermore, if σ 2 and D are sufficiently large so that 33 P 2 max /τ 2 < 1 it is valid that

̸

The proof of Theorem 4 is deferred to Appendix H. Theorem 4 first provides the upper bound for the maximum magnitude of P k that decreases as the variance σ 2 increases or τ decreases. This observation is consistent with (13), which implies the P k tends to zero when β τ E [ | ε k | ] = 0 . Ensuring stability during the training requires the noise to be sufficiently large to render the saturation degree of P max /τ sufficiently small. In addition, the condition 33 P 2 max /τ 2 < 1 requires the D to be sufficiently large, which is easy to meet when training large models.

Convergence rate. Theorem 4 claims that Tiki-Taka converges at the rate O ( √ σ 2 L K 1 1 -33 P 2 max /τ 2 ) . Therefore, we reach the conclusion that lim sup K → inf 1 K ∑ K -1 k =0 E [ ∥∇ f ( W k ) ∥ 2 ] = 0 and Tiki-Taka eliminates the asymptotic error of Analog SGD . Furthermore, Tiki-Taka improves the factor 1 / (1 -W 2 max /τ 2 ) in Analog SGD 's convergence (c.f. (9)) to 1 / (1 -33 P 2 max /τ 2 ) , wherein W 2 max /τ 2 and P 2 max /τ 2 are the saturation degrees in fact. Notice that P k tends to 0 as indicated

0

1000

2000

Gradient Computation (k)

10

4

10

3

10

2

f

(

Wk

)

f

*

=1.0

Digital SGD Dynamic

Analog SGD Dynamic

Analog SGD Simulation

0

1000

2000

Gradient Computation (k)

=1.5

0

1000

2000

Gradient Computation (k)

=2.0

0

1000

2000

Gradient Computation (k)

=2.5

by (13) while W max does not because 0 is usually not a critical point. Therefore, it usually has P 2 max /τ 2 ≪ W 2 max /τ 2 in practice, implying faster convergence. The convergence matches the lower bound O ( √ σ 2 L/K ) for general stochastic non-convex smooth optimization [24] up to a constant.

5 Numerical Simulations

In this section, we verify the main theoretical results by simulations on both synthetic datasets and real datasets. We use the PYTORCH to generate the curves for SGD in the simulation and use open source toolkit AIHWKIT [27] to simulate the behaviors of Analog SGD ; see github.com/IBM/aihwkit . Each simulation is repeated three times, and the mean and standard deviation are reported. More details can be referred to in Appendix I. The code of our simulation implementation is available at github.com/Zhaoxian-Wu/analog-training .

0

200

400

Number of Gradient Computation (

k

)

10

5

10

4

10

3

10

2

10

1

f

(

W k

)

f

*

=2

=3

=5

=10

=20

=50

SGD

0

50

100

150

200

Number of Gradient Computation (

k

)

10

2

10

1

f

(

W k

)

f

*

2 =0.001

2 =0.01

2 =0.1

2 =1.0

0

20

40

Number of Gradient Computation (k)

10

2

10 0

10 2

10 4

f

(

W k

)

f

*

5.1 Verification of the analog training dynamic

To verify that the proposed dynamic (3) characterizes analog training better than SGD dynamic (2), we conduct a numerical simulation on a least-squares task, and compare Analog SGD implemented by AIHWKIT, the digital and analog dynamics given by (2) and (3), respectively; see Figure 3. The results show that the proposed dynamic provides an accurate approximation of Analog SGD .

5.2 Ablation study on the asymptotic training error

We verify some critical claims about the asymptotic error of Analog SGD on a least-squares task.

Impact of τ . To verify Theorem 2 that the error diminishes with a larger τ , we assign a range of value τ and plot the convergence of Analog SGD . The result is reported on the Left of Figure 4. When τ is small, the asymmetric bias introduces a notable gap between Analog and Digital SGD. As τ increases, the gap diminishes. The result demonstrates the asymptotic error decreases as τ increases.

Impact of σ 2 . To verify the asymptotic error is proportional to σ 2 , we inject noise with different variances. The result is reported in the middle of Figure 4. The result illustrates that the asymptotic error increases as the noise increases.

0

20

40

60

80

Epoch

20

40

60

80

100

Test Accuracy

=0.7

=0.78

=0.8

Analog SGD

Tiki-T aka

Digital SGD

78

80

97.0

97.5

0

20

40

60

80

Epoch

20

40

60

80

100

Test Accuracy

=0.6

=0.7

=0.8

Analog SGD

Tiki-T aka

Digital SGD

78

80

98.75

99.00

99.25

τ

= 0

.

70

τ

= 0

.

78

τ

= 0

.

80

τ

= 0

.

60

τ

= 0

.

70

τ

= 0

.

80

D SGD

98.15

±

0.04

99.16

±

0.08

A SGD

11.35

±

0.00

97.71

±

0.63

97.78

±

0.36

88.99

±

0.89

92.15

±

0.48

93.45

±

0.46

TT

97.64

±

0.17

97.62

±

0.05

97.50

±

0.11

98.84

±

0.07

98.87

±

0.04

98.86

±

0.07

Impact of the initialization . To demonstrate the asymptotic error is not artificial, we perform Analog SGD from different initializations. The result is reported on the right of Figure 4. The result illustrates that Analog SGD converges to a similar location regardless of the initialization. The smooth convergence curve ensures the error comes from bias instead of limited machine precision. Therefore, the asymptotic error is intrinsic and independent of the initialization.

5.3 Analog training performance on real dataset

We also train vision models to perform image classification tasks on real datasets.

MNIST FCN/CNN. We train Fullyconnected network (FCN) and convolution neural network (CNN) models on MNIST dataset and see the performance of Analog SGD and Tiki-Taka under various τ . The results are reported in Figure 5. By reducing the variance, Tiki-Taka outperforms Analog SGD and reaches comparable accuracy with digital SGD. On both

of the architectures, the accuracy of Tiki-Taka drops by < 1% . In the FCN training, Analog SGD achieves acceptable accuracy on τ = 0 . 78 and τ = 0 . 80 but converges much more slowly. In the CNN training, the accuracy of Analog SGD always drops by > 6% .

CIFAR10 Resnet. We also train three Resnet models with different sizes on CIFAR10 dataset. The last layer is replaced by a fully-connected layer mapped onto an analog device with parameter τ = 0 . 8 . The results are shown in Table 2. In this task, Analog SGD does not suffer from a significant accuracy drop but is still worth that Tiki-Taka , A surprising observation for analog training is that both Analog SGD and Tiki-Taka outperform Digital SGD. We conjecture it happens because the noise introduced by analog devices makes the network more robust to outlier data.

6 Conclusions and Limitations

This paper points out that Analog SGD does not follow the dynamic of digital SGD and hence, we propose a better dynamic to formulate the analog training. Based on this dynamic, we studies the convergence of two gradient-based analog training algorithms, Analog SGD and Tiki-Taka . The theoretical results demonstrate that Analog SGD suffers from asymptotic error, which comes from the noise and asymmetric update. To overcome this issue, we show that Tiki-Taka is able to stay in the critical point without suffering from an asymptotic error. Numerical simulations demonstrate the existence of Analog SGD 's asymptotic error and the efficacy of Tiki-Taka . One limitation of this work is that the current analysis is device-specific that applies to asymmetric linear device. While it is an interesting and popular analog device, it is also important to extend our convergence analysis to more general analog devices and develop other device-specific analog algorithms in future work.

References

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Norm Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, et al. TPU v4: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings. In Annual International Symposium on Computer Architecture , pages 1-14, 2023.

Hadi Esmaeilzadeh, Adrian Sampson, Luis Ceze, and Doug Burger. Neural acceleration for general-purpose approximate programs. In 45th annual IEEE/ACM international symposium on microarchitecture , pages 449-460. IEEE, 2012.

Dharmendra S Modha, Filipp Akopyan, Alexander Andreopoulos, Rathinakumar Appuswamy, John V Arthur, Andrew S Cassidy, Pallab Datta, Michael V DeBole, Steven K Esser, Carlos Ortega Otero, et al. Neural inference at the frontier of energy, space, and time. Science , 382(6668):329-335, 2023.

An Chen. A comprehensive crossbar array model with solutions for line resistance and nonlinear device characteristics. IEEE Transactions on Electron Devices , 60(4):1318-1326, 2013.

Abu Sebastian, Manuel Le Gallo, Riduan Khaddam-Aljameh, and Evangelos Eleftheriou. Memory devices and applications for in-memory computing. Nature Nanotechnology , 15:529544, 2020.

Wilfried Haensch, Tayfun Gokmen, and Ruchir Puri. The next generation of deep learning hardware: Analog computing. Proceedings of the IEEE , 107(1):108-122, 2019.

Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE , 105(12):2295-2329, 2017.

Zihan Zhang, Jianfei Jiang, Yongxin Zhu, Qin Wang, Zhigang Mao, and Naifeng Jing. A universal RRAM-based DNN accelerator with programmable crossbars beyond MVM operator. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 41(7):20942106, 2021.

Abu Sebastian, Manuel Le Gallo, and Vijay Narayanan. IBM research's latest analog AI chip for deep learning inference. https://research.ibm.com/blog/ analog-ai-chip-inference , 2023.

Stefano Ambrogio, Pritish Narayanan, Hsinyu Tsai, Robert M Shelby, Irem Boybat, Carmelo Nolfo, Severin Sidler, Massimo Giordano, Martina Bodini, Nathan CP Farinha, et al. Equivalentaccuracy accelerated neural-network training using analogue memory. Nature , 558(7708):60, 2018.

Geoffrey W Burr, Robert M Shelby, Abu Sebastian, Sangbum Kim, Seyoung Kim, Severin Sidler, Kumar Virwani, Masatoshi Ishii, Pritish Narayanan, Alessandro Fumarola, et al. Neuromorphic computing using non-volatile memory. Advances in Physics: X , 2(1):89-124, 2017.

J Joshua Yang, Dmitri B Strukov, and Duncan R Stewart. Memristive devices for computing. Nature nanotechnology , 8(1):13, 2013.

Tayfun Gokmen and Yurii Vlasov. Acceleration of deep neural network training with resistive cross-point devices: Design considerations. Frontiers in neuroscience , 10:333, 2016.

Shubham Jain et al. Neural network accelerator design with resistive crossbars: Opportunities and challenges. IBM Journal of Research and Development , 63(6):10-1, 2019.

S. R. Nandakumar, Manuel Le Gallo, Irem Boybat, Bipin Rajendran, Abu Sebastian, and Evangelos Eleftheriou. Mixed-precision architecture based on computational memory for training deep neural networks. In IEEE International Symposium on Circuits and Systems , pages 1-5, 2018.

S. R. Nandakumar, Manuel Le Gallo, Christophe Piveteau, Vinay Joshi, Giovanni Mariani, Irem Boybat, Geethan Karunaratne, Riduan Khaddam-Aljameh, Urs Egger, Anastasios Petropoulos, Theodore Antonakopoulos, Bipin Rajendran, Abu Sebastian, and Evangelos Eleftheriou. Mixedprecision deep learning based on computational memory. Frontiers in Neuroscience , 14, 2020.

Malte J Rasch, Fabio Carta, Omebayode Fagbohungbe, and Tayfun Gokmen. Fast offset corrected in-memory training. arXiv preprint arXiv:2303.04721 , 2023.

Nanbo Gong, Malte Rasch, Soon-Cheon Seo, Arthur Gasasira, Paul Solomon, Valeria Bragaglia, Steven Consiglio, Hisashi Higuchi, Chanro Park, Kevin Brew, et al. Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. In IEEE International Electron Devices Meeting , 2022.

Sapan Agarwal, Steven J Plimpton, David R Hughart, Alexander H Hsia, Isaac Richter, Jonathan A Cox, Conrad D James, and Matthew J Marinella. Resistive memory device requirements for a neural algorithm accelerator. In International Joint Conference on Neural Networks , pages 929-938. IEEE, 2016.

Tayfun Gokmen and Wilfried Haensch. Algorithm for training neural networks on resistive device arrays. Frontiers in Neuroscience , 14, 2020.

Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review , 60(2):223-311, 2018.

Yossi Arjevani, Yair Carmon, John C Duchi, Dylan J Foster, Nathan Srebro, and Blake Woodworth. Lower bounds for non-convex stochastic optimization. Mathematical Programming , 199(1-2):165-214, 2023.

Tayfun Gokmen. Enabling training of neural networks on noisy hardware. Frontiers in Artificial Intelligence , 4:1-14, 2021.

Murat Onen, Tayfun Gokmen, Teodor K Todorov, Tomasz Nowicki, Jesús A Del Alamo, John Rozen, Wilfried Haensch, and Seyoung Kim. Neural network training with asymmetric crosspoint elements. Frontiers in artificial intelligence , 5, 2022.

Malte J Rasch, Diego Moreda, Tayfun Gokmen, Manuel Le Gallo, Fabio Carta, Cindy Goldberg, Kaoutar El Maghraoui, Abu Sebastian, and Vijay Narayanan. A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays. IEEE International Conference on Artificial Intelligence Circuits and Systems , pages 1-4, 2021.

Malte J Rasch, Charles Mackin, Manuel Le Gallo, An Chen, Andrea Fasoli, Frédéric Odermatt, Ning Li, S. R. Nandakumar, Pritish Narayanan, Hsinyu Tsai, et al. Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators. Nature Communications , 14(1):5282, 2023.

Weier Wan, Rajkumar Kubendran, Clemens Schaefer, Sukru Burc Eryilmaz, Wenqiang Zhang, Dabin Wu, Stephen Deiss, Priyanka Raina, He Qian, Bin Gao, et al. A compute-in-memory chip based on resistive random-access memory. Nature , 608(7923):504-512, 2022.

Riduan Khaddam-Aljameh, Milos Stanisavljevic, J Fornt Mas, Geethan Karunaratne, Matthias Braendli, Femg Liu, Abhairaj Singh, Silvia M Müller, Urs Egger, Anastasios Petropoulos, et al. HERMES core-a 14nm CMOS and PCM-based in-memory compute core using an array of 300ps/LSB linearized CCO-based adcs and local digital processing. In 2021 Symposium on VLSI Circuits , pages 1-2. IEEE, 2021.

Chengxin Xue, Yencheng Chiu, Tawei Liu, Tsungyuan Huang, Jesyu Liu, Tingwei Chang, Huiyao Kao, Jinghong Wang, Shihying Wei, Chunying Lee, et al. A CMOS-integrated computein-memory macro based on resistive random-access memory for AI edge devices. Nature Electronics , 4(1):81-90, 2021.

Laura Fick, Skylar Skrzyniarz, Malav Parikh, Michael B Henry, and David Fick. Analog matrix processor for edge AI real-time video analytics. In IEEE International Solid-State Circuits Conference , volume 65, pages 260-262. IEEE, 2022.

Pritish Narayanan, Stefano Ambrogio, A Okazaki, Kohji Hosokawa, Hsinyu Tsai, A Nomura, T Yasuda, C Mackin, SC Lewis, A Friz, et al. Fully on-chip MAC at 14nm enabled by accurate row-wise programming of PCM-based weights and parallel vector-transport in duration-format. In Symposium on VLSI Technology , pages 1-2. IEEE, 2021.

Peng Yao, Huaqiang Wu, Bin Gao, Jianshi Tang, Qingtian Zhang, Wenqiang Zhang, J Joshua Yang, and He Qian. Fully hardware-implemented memristor convolutional neural network. Nature , 577(7792):641-646, 2020.

Benjamin Scellier, Maxence Ernoult, Jack Kendall, and Suhas Kumar. Energy-based learning algorithms for analog computing: a comparative study. In Advances in neural information processing systems , 2023.

Hadjer Benmeziane, Corey Lammie, Irem Boybat, Malte J Rasch, Manuel Le Gallo, Hsinyu Tsai, Ramachandran Muralidhar, Smail Niar, Ouarnoughi Hamza, Vijay Narayanan, et al. Analognas: A neural network design framework for accurate inference with analog in-memory computing. In IEEE International Conference on Edge Computing and Communications , pages 233-244. IEEE, 2023.

Logan G Wright, Tatsuhiro Onodera, Martin M Stein, Tianyu Wang, Darren T Schachter, Zoey Hu, and Peter L McMahon. Deep physical neural networks trained with backpropagation. Nature , 601(7894):549-555, 2022.

Ali Momeni, Babak Rahmani, Benjamin Scellier, Logan G Wright, Clara C McMahon, Peter L andWanjura, Yuhang Li, Anas Skalli, Natalia G. Berloff, Tatsuhiro Onodera, Ilker Oguz, Francesco Morichetti, Philipp del Hougne, Manuel Le Gallo, Abu Sebastian, Azalia Mirhoseini, Cheng Zhang, Danijela Markovi´ c, Daniel Brunner, Christophe Moser, Sylvain Gigan, Florian Marquardt, Aydogan Ozcan, Julie Grollier, Andrea J Liu, Demetri Psaltis, Andrea Alù, and Romain Fleury. Training of physical neural networks. arXiv preprint arXiv:2406.03372 , 2024.

Demetri Psaltis, David Brady, Xiang-Guang Gu, and Steven Lin. Holography in artificial neural networks. Nature , 343(6256):325-330, 1990.

Tyler W Hughes, Ian AD Williamson, Momchil Minkov, and Shanhui Fan. Wave physics as an analog recurrent neural network. Science advances , 5(12):eaay6946, 2019.

Alexander N Tait, Thomas Ferreira De Lima, Ellen Zhou, Allie X Wu, Mitchell A Nahmias, Bhavin J Shastri, and Paul R Prucnal. Neuromorphic photonic networks using silicon photonic weight banks. Scientific reports , 7(1):7430, 2017.

Sebastian U Stich. Unified optimal analysis of the (stochastic) gradient method. arXiv preprint arXiv:1907.04232 , 2019.

Ahmed Khaled and Peter Richtárik. Better theory for SGD in the nonconvex world. Transactions on Machine Learning Research , 2022.

Yury Demidovich, Grigory Malinovsky, Igor Sokolov, and Peter Richtárik. A guide through the zoo of biased sgd. arXiv preprint arXiv:2305.16296 , 2023.

Tianbao Yang, Qihang Lin, and Zhe Li. Unified convergence analysis of stochastic momentum methods for convex and non-convex optimization. arXiv preprint arXiv:1604.03257 , 2016.

Yanli Liu, Yuan Gao, and Wotao Yin. An improved analysis of stochastic gradient descent with momentum. Advances in Neural Information Processing Systems , 33:18261-18271, 2020.

Kun Yuan, Bicheng Ying, and Ali H Sayed. On the influence of momentum acceleration on online learning. The Journal of Machine Learning Research , 17(1):6602-6667, 2016.

Vien Mai and Mikael Johansson. Convergence of a stochastic gradient method with momentum for non-smooth non-convex optimization. In International conference on machine learning , pages 6630-6639. PMLR, 2020.

Nanbo Gong, T Idé, S Kim, Irem Boybat, Abu Sebastian, V Narayanan, and Takashi Ando. Signal and noise extraction from analog memory elements for neuromorphic computing. Nature communications , 9(1):2102, 2018.

Geoffrey W Burr, Matthew J BrightSky, Abu Sebastian, Huai-Yu Cheng, Jau-Yi Wu, Sangbum Kim, Norma E Sosa, Nikolaos Papandreou, Hsiang-Lan Lung, Haralampos Pozidis, Evangelos Eleftheriou, and Chung H Lam. Recent Progress in Phase-Change Memory Technology. IEEE Journal on Emerging and Selected Topics in Circuits and Systems , 6(2):146-162, 2016.

Manuel Le Gallo and Abu Sebastian. An overview of phase-change memory device physics. Journal of Physics D: Applied Physics , 53(21):213002, 2020.

Jun-Woo Jang, Sangsu Park, Geoffrey W Burr, Hyunsang Hwang, and Yoon-Ha Jeong. Optimization of conductance change in Pr 1 -x Ca x MnO 3 -based synaptic devices for neuromorphic systems. IEEE Electron Device Letters , 36(5):457-459, 2015.

Jun-Woo Jang, Sangsu Park, Yoon-Ha Jeong, and Hyunsang Hwang. ReRAM-based synaptic device for neuromorphic computing. In IEEE International Symposium on Circuits and Systems , pages 1054-1057, 2014.

Seokjae Lim, Myounghoon Kwak, and Hyunsang Hwang. Improved synaptic behavior of CBRAM using internal voltage divider for neuromorphic systems. IEEE Transactions on Electron Devices , 65(9):3976-3981, 2018.

Elliot J Fuller, Scott T Keene, Armantas Melianas, Zhongrui Wang, Sapan Agarwal, Yiyang Li, Yaakov Tuchman, Conrad D James, Matthew J Marinella, J Joshua Yang, Alberto Salleo, and A Alec Talin. Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing. Science , 364(6440):570-574, 2019.

Murat Onen, Nicolas Emond, Baoming Wang, Difei Zhang, Frances M Ross, Ju Li, Bilge Yildiz, and Jesús A Del Alamo. Nanosecond protonic programmable resistors for analog deep learning. Science , 377(6605):539-543, 2022.

Yulong Li, Seyoung Kim, Xiao Sun, Paul Solomon, Tayfun Gokmen, Hsinyu Tsai, Siyu Koswatta, Zhibin Ren, Renee Mo, Chun Chen Yeh, et al. Capacitor-based cross-point array for analog neural network with record symmetry and linearity. In 2018 IEEE Symposium on VLSI Technology , pages 25-26. IEEE, 2018.

Seyoung Kim, Teodor Todorov, Murat Onen, Tayfun Gokmen, Douglas Bishop, Paul Solomon, Ko-Tao Lee, Matt Copel, Damon B Farmer, John A Ott, et al. Metal-oxide based, CMOScompatible ECRAM for deep learning accelerator. In IEEE International Electron Devices Meeting , pages 35-7. IEEE, 2019.

Stefano Fusi and LF Abbott. Limits on the memory storage capacity of bounded synapses. Nature neuroscience , 10(4):485-493, 2007.

Jacopo Frascaroli, Stefano Brivio, Erika Covi, and Sabina Spiga. Evidence of soft bound behaviour in analogue memristive devices for neuromorphic computing. Scientific reports , 8(1):1-12, 2018.

Paiyu Chen, Binbin Lin, I-Ting Wang, Tuohung Hou, Jieping Ye, Sarma Vrudhula, Jae-sun Seo, Yu Cao, and Shimeng Yu. Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. In IEEE/ACM International Conference on Computer-Aided Design , pages 194-199. IEEE, 2015.

Lisha Chen, Heshan Devaka Fernando, Yiming Ying, and Tianyi Chen. Three-way trade-off in multi-objective learning: Optimization, generalization and conflict-avoidance. In Advances in Neural Information Processing Systems , 2023.

Andreas Winkelbauer. Moments and absolute moments of the normal distribution. arXiv preprint arXiv:1209.4340 , 2012.

Milton Abramowitz and Irene A Stegun. Handbook of mathematical functions with formulas, graphs, and mathematical tables , volume 55. US Government printing office, 1948.

NIST Digital Library of Mathematical Functions . https://dlmf.nist.gov/ , Release 1.1.12 of 2023-12-15. F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain, eds.

Tayfun Gokmen, Murat Onen, and Wilfried Haensch. Training deep convolutional neural networks with resistive cross-point devices. Frontiers in neuroscience , 11:538, 2017.

Supplementary Material for 'Towards Exact Gradient-based Training on Analog In-memory Computing'

Table of Contents

A Literature Review

This section briefly reviews literature that is related to this paper, as complementary to Section 1.

Inference and other applications of analog devices. Before designing hardware for training, a series of AIMC prototypes focus on accelerating the inference phase [29, 30, 31, 32, 33, 12, 34]. The state-of-the-art result shows that the analog inference is capable of reaching comparable accuracy with digital inference [28]. Besides analog model training and inference, researchers also managed to exploit the advantages of analog devices to facilitate the machine learning task. For example, energy-based learning algorithms have been studied in [35], and the neural architecture search on analog devices has been studied in [36].

Relation with physical neural network. Our work is also in spirit related to the recent works on physical neural networks (PNNs) [37, 38]. But we are particularly interested in AIMC hardware with resistive crossbar arrays, while PNN is a generic concept of implementing neural networks via a

physical system. Various hardware is capable of supporting PNN, such as holographic grating [39], wave-based systems [40], and photonic networks [41], to name a few.

Convergence analysis of gradient-based algorithms. In the digital domain, a series of literature has discussed the convergence of gradient-based algorithms. As the basis of neural model training, much work focuses on the convergence of SGD [42, 43, 44, 23] and its variety stochastic gradient descent with momentum (SGDM) [45, 46, 47, 48]. Notice that the difference between digital SGD (2) and Analog SGD (3) lies on the asymmetric bias term, which implies Analog SGD can be regarded as a digital SGD with bias. From this perspective, the convergence of Analog SGD (c.f. Theorem 2) is similar to that of the digital bias-SGD counterparts. However, the results in this paper are still challenging and non-trivial, especially in the convergence of Tiki-Taka , because the updates of both W k and P k involve asymmetric bias.

Nowadays, gradient-based algorithms demonstrate their powerful capabilities in model training. In fact, both SGD and SGDM have reached the theoretical lower bound of the convergence rate, √ σ 2 /K [24]. This lower bound is a generic result for any stochastic gradient-based algorithms, which means that the convergence of gradient-based analog algorithms, including Analog SGD and Tiki-Taka , is also subject to it.

B Implementation of Asymmetric Updated by Pulse Update

The proposed dynamic (3) of Analog SGD in Section 2 is inspired by the empirical studies in [22] and [19]. However, the proposed dynamic (3) is only an approximation because we can not directly apply an arbitrary increment ∆ w on the weights. Instead, we need to send a series of pulses to analog devices, which leads to the change on the weight. In this section, we introduce the implementation of analog update U ( · , · ) in (6) by pulse update and analyze the error of pulse update implementation.

Implementation. Similar to U ( · , · ) , we introduce the update function U p ( · , · ) : R ×{ + , -} → R for the pulsed update, defined by

where ∆ w min > 0 is the response step size determined by devices. Since ∆ w min is the minimum change of weight, it is sometimes called the resolution or granularity of the devices. Given the initial weight w , the updated weight after receiving one pulse is U p ( W k , s ) where the update sign s = + if ∆ w ≥ 0 and s = -otherwise.

The response step size ∆ w min is usually known by physical measurement. In order to approximate U ( w, ∆ w ) , analog device first computes the pulse series length (bit length) by

which ensures

After that, a total of BL pulses are sent to the analog device, forcing the weight to become

which implement the analog update U ( w, ∆ w ) .

Error analysis. Directly expending the (18) yields

which indicates the difference between U ( w, ∆ w ) and U BL p ( w,s ) are higher order infinitesimal. As a concrete example, the ∆ w min is set as 0 . 002 [49] or 0 . 0949 [20] for ReRAM devices.

For asymmetric linear device (ALD, defined in Section 2), it holds that | q ′ + ( w ) | = | q ′ -( w ) | = 1 τ and q ′′ + ( w ) = q ′′ -( w ) = 0 . Consequently, the difference between U ( w, ∆ w ) and U BL p ( w,s ) is

Since ∆ w min and ∆ w = O ( α ) are usually small, the error can usually be ignored.

Significance. The significance of the analysis in this section is that it explains how to estimate the response factors q + ( · ) and q -( · ) in (3). In (15), it shows that they are, in fact, the response factors of the given devices, which can be measured by physic measures [49]. In Section C, we showcase various capable base materials for analog devices and their response factors.

C Examples of Analog Devices

In AIMC accelerators, weights of models can be represented by the conductance of based materials, such as PCM [50, 51], ReRAM [52, 53], CBRAM [54, 55], or ECRAM [56]. In this section, we showcase a spectrum of alternative devices for analog training.

C.1 Example 1: Linear step device (Soft bounds devices)

A large range of devices, including ReRAM [49, 20], Capacitor [57], EcRAM [57, 56], EcRamMO [58], have linear response factor

where τ min < 0 < τ max and w ⋄ are constants, whose meaning will be explained later. With these response factors, the function F ( · ) and G ( · ) can be written as

ALD used in the paper is a special linear step device with τ max = τ and τ min = -τ .

C.2 Example 2: Power step device

For power step device, q + ( w ) and q -( w ) are power function with respect to W , which models implements synapse model [59, 60].

where γ + and γ -are parameters determined by materials.

C.3 Example 3: Exponential step device

In some ReRAM and CMOS-like devices [61, 21], the response factors are captured by an exponential function, i.e. Exponential update step or CMOS-like update behavior.

In Figure 6, the curve of response factors and F ( · ) and G ( · ) are illustrated.

4

2

0

2

4

w

0.0

0.5

1.0

1.5

2.0

2.5

Response Factor

Linear Step Device

4

2

0

2

4

w

0.0

0.5

1.0

Power Step Device

4

2

0

2

4

w

0.00

0.25

0.50

0.75

1.00

Exponential Step Device

min

q

+(

w

)

q

(

w

)

4

2

0

2

4

w

1.0

0.5

0.0

0.5

1.0

1.5

Sym/Asym components

4

2

0

2

4

w

0.4

0.2

0.0

0.2

0.4

4

2

0

2

4

w

1.0

0.5

0.0

0.5

1.0

F

(

w

)

F

(

w

)

G

(

w

)

min

D Verification of Assumptions

D.1 Verification of Assumption 4 under Strongly Convex Objective

This section proves the weight W k is strongly bounded when the loss function is strongly convex. We prove the result for both Analog GD ( ε k = 0 ) and Analog SGD with ε k possessing some special structure. Before proving this, we introduce strongly convex assumption.

Assumption 6 ( µ -strongly convex) . The objective f ( W ) is µ -strongly convex, i.e.

Assumption 6 ensures the critical point W ∗ (and hence optimal point) is unique.

D.1.1 Bounded Saturation of Analog GD

The following theorem provides a sufficient condition under which the Assumption 4 holds.

Theorem 5 (Bounded saturation of Analog GD) . Suppose Assumptions 1 and 6 hold and define

The sequence { W k } generated by Analog GD satisfies ∥ W k ∥ ∞ ≤ W max .

Theorem 5 claims that if both L µ ∥ W 0 -W ∗ ∥ and ∥ W ∗ ∥ ∞ are sufficiently small, which means the optimal point W ∗ is located inside the active region and W 0 is properly initialized, the saturation degree ∥ W k ∥ ∞ /τ can be sufficiently small as well, even though no projection operation is applied.

Proof of Theorem 5. Given f ( · ) is L -smooth (Assumption 1), we know from (57) that

where the second inequality comes from the bounded weight (Theorem 1). Consequently, we reach the conclusion that f ( W k ) ≤ f ( W 0 ) , ∀ k ∈ N . The strongly convex assumption (Assumption 6) claims that

Therefore, triangle inequality ensures that

which completes the proof.

Remark 2 (Bounded weight of digital GD) . Theorem 5 also holds for digital GD, which can be achieved by forcing τ in inequality (29) to infinite.

In high-dimensional cases ( D > 1 ), Theorem 5 does not exclude the situation ∥ W k ∥ ∞ > ∥ W ∗ ∥ ∞ , which is unfavorable in the example of Analog SGD 's lower bound (c.f. Section G.2). To facilitate the proof of the lower bound, we consider the scalar case ( D = 1 ). To distinguish between vectors and scalars, we use the notation w k (and w ∗ ) instead of W k (and W ∗ ) in the next theorem.

Theorem 6 (Bounded saturation of Analog GD with scaler weight) . Suppose Assumptions 1 and 6 hold and w k ∈ R is a scalar. If the learning rate α ≤ 1 2 L , the following statements hold true

Consequently, the sequence { w k } generated by Analog GD satisfies the following relation | w k | ≤ w max := min {| w 0 | , | w ∗ |} .

Theorem 6 asserts that { w k } converges monotonically from w 0 to w ∗ . If | w 0 | ≤ | w ∗ | , we have w max = | w ∗ | , which improves the bound in Theorem 5.

Proof of Theorem 6. Expanding the objective f ( · ) in a Taylor series around w yields that 2

where ˜ µ ( w ) depends on w and satisfies µ ≤ ˜ µ ( w ) ≤ L because of the smoothness (Assumption 1) and strong convexity (Assumption 6) assumptions. Consequently, the gradient on w can be written as

with which the gradient descent has the following property

Using this inequality and manipulating ∥ w k +1 -w ∗ ∥ 2 as

where the ( a ) uses Cauchy's inequality. Noticing Theorem 1 claims ∥ w k ∥ ∞ τ ≤ 1 and the learning rate is chosen as α ˜ µ ( w k ) ≤ αL ≤ 1 , we have ∥ w k +1 -w ∗ ∥ 2 ≤ ∥ w k -w ∗ ∥ 2 .

In addition, Lemma 1 claims that

where the second inequality holds because Theorem 1 and smoothness assumption (Assumption 1 guarantee ∥ 1 + w k /τ ∥ ≤ 2 and ∥∇ f ( w k ) ∥ ≤ ∥ w k -w ∗ ∥ , respectively. This fact implies that

w k +1 -w ∗ has the same sign with w k -w ∗ . Consequently, inequality (36) and (37) show that in case 1, if w k ≥ w ∗ , we have

Keeping using this inequality reaches the result. Case 2 can be proved by the similar method.

D.1.2 Bounded Saturation of Analog SGD

The following theorem provides a sufficient condition under which the Assumption 4 holds. In the section, we show that if the noise has special structures, the saturation degree ∥ W k ∥ ∞ /τ is bounded.

Assumption 7 (Random sample noise) . The noise can be written as ε = ∇ f ( W ; ξ ) - ∇ f ( W ) , where ξ is a random variable sampled from an underline data distribution D f , f ( W ; ξ ) is a function of W and ξ , and the gradient ∇ f ( W ; ξ ) is taken over W .

Assumption 7 holds if the noise comes from the random sampling from a series of functions.

Theorem 7 (Bounded saturation of Analog SGD ) . Suppose Assumption 7 holds and each f ( W ; ξ ) is L -smooth (Assumptions 1) and µ -strongly convex (Assumptions 6) with respect to W . Denote the minimum of f ( W ; ξ ) as W ∗ ξ . If the supremum sup ξ {∥ W ∗ ξ ∥ ∞ } exists, define

The sequence { W k } generated by Analog SGD satisfies ∥ W k ∥ ∞ ≤ W max uniformly.

Given the strong convexity with respect to W , f ( W ; ξ ) has unique minimum, and hence W ∗ ξ is welldefined. Similarly, the supremum sup ξ {∥ W 0 -W ∗ ξ ∥} exists since sup ξ {∥ W ∗ ξ ∥ ∞ } exists. Theorem 7 claims that the saturation degree is bounded given the noise comes from random sampling over a series of smooth and strongly convex functions. Similar to Theorem 5, Theorem 7 also holds for digital GD by forcing τ in inequality (29) to infinite. The proof of Theorem 7 is inspired by that of [62, Lemma 1].

Proof. According to Theorem 5, it holds that

Therefore, triangle inequality ensures that

which completes the proof.

Similar to Theorem 6, we can improve the bound when the weight is scalar ( D = 1 ). The notation w k (and w ∗ ) instead of W k (and W ∗ ) are adopted here to indicate the scalar situation.

Theorem 8 (Bounded saturation of Analog GD) . Suppose Assumption 7 holds and for any ξ , f ( w ; ξ ) is L -smooth (Assumptions 1) and µ -strongly convex (Assumptions 6) with respect to w . Denote the minimum of f ( w ; ξ ) as w ∗ ξ . If the supremum sup ξ {∥ w ∗ ξ ∥} exists, it holds for any k ∈ N that

Proof of Theorem 8. The theorem is proved by induction. The statement holds trivially at k = 0 . Suppose (42) holds for k . If W k ≥ w ∗ , Theorem 6 guarantees that w k ≥ w k +1 ≥ w ∗ ξ and hence

On the contrary, if w k ≤ w ∗ , Theorem 6 guarantees that w k ≤ w k +1 ≤ w ∗ ξ and hence

which implies w k +1 still satisfies (42). Now the conclusion is reached.

D.2 Verification of Assumption 5: Non-zero Property of Gaussian Noise

This section verifies Assumption 5. In principle, Assumption 5 requires the expectation that noise is non-zero. To see that, note that when the probability density function of D is non-zero at only one point, c equals zero. On the contrary, large c is at the other side of the spectrum, which appears when the probability density function is relatively 'uniformly distributed' throughout the probability space. In analog acceleration devices, it is mild because noise is introduced during computation.

Generally, the estimation of c can be challenging because it involves the computation of the raw momentum E ε [ | g + ε | ] of the noise. Scarifying the rigorousness a bit but providing an intuitive result, we get the parameter c by considering an approximation of this raw momentum and deducing a bound for this approximation.

In this section, we demonstrate that for Gaussian noise ε ∼ N (0 , σ 2 ) , c can be selected as √ 2 π . For any g , it holds that

Because g + ε is Gaussian random variable with mean g and variance σ 2 , the second term in the RHS has closed form [63] given by

In the equation above, 1 F 1 ( · ; · ; · ) is used to denote Kummer's confluent hypergeometric functions. It has been shown [64, 13.1.5] that when z ≤ 0 , 1 F 1 ( · ; · ; · ) has the asymptotic property

where Γ( · ) is Gamma function. When z is a small, 1 F 1 ( a ; b ; z ) has the following approximation for any a, b ∈ R [65, Sec 11.2]

Let z = -g 2 2 σ 2 . On the one hand when σ ∼ 0 , z is large and (47) with a = -1 2 and b = 1 2 asserts that

On the other hand, when σ →∞ , z is small and (48) provides that

In conclusion, choosing c = √ 2 π yields E ε [ | g + ε | ] ≿ √ 2 π σ . Here ≿ is used to indicate the lower bound holds for an approximation of the left-hand side (LHS).

E Proof of Analog Training Properties

E.1 Proof of Lemma 1: Saturation and Fast Reset

Lemma 1 (Saturation and fast reset) . For ALD with a general w ⋄ , the following statements are valid (Saturation) If sign (∆ w ) = sign ( w k ) , it holds that | w k +1 -w k | = | 1 -| w k -w ⋄ | /τ | · | ∆ w | . (Fast Reset) If sign (∆ w ) = -sign ( w k ) , it holds that | w k +1 -w k | = | 1 + | w k -w ⋄ | /τ | · | ∆ w | .

Proof of Lemma 1. The proof can be completed by simply extending the LHS of

Using the fact that w ⋄ = 0 completes the proof.

E.2 Proof of Theorem 1: Bounded Weight

This section proves that the weight W k is bounded.

Theorem 1 (Bounded weight) . Denote ∥ W ∥ ∞ as the ℓ ∞ norm of W . Given ∥ W 0 ∥ ∞ ≤ τ and for any sequence { ∆ W k : k ∈ N } , which satisfies ∥ ∆ W k ∥ ∞ ≤ τ , it holds that ∥ W k ∥ ∞ ≤ τ, ∀ k ∈ N .

Proof of Theorem 1. Without causing confusion, omit the superscript k and denote the i -th coordinate of W k and ∆ W k as w i and g i , respectively. It holds that ∥ W k ∥ 2 ∞ < τ and | g ij | ≤ τ

Taking square root on both sides completes the proof.

F Proof of Analog Gradient Descent Convergence

In Section 3, we show that the asymptotic error is proportional to the noise variance σ 2 . The following theorem prove that the Analog GD converges to a critical point.

Theorem 9 (Convergence of Analog GD) . Under Assumption 1, 2 and 4, if the learning rate is set as α ≤ 1 L , it holds that

A surprising conclusion of Theorem 9 is that the asymmetric update actually will not introduce any asymptotic error in training. It also indicates that the asymptotic error comes from the interaction between asymmetric update and noise from another perspective.

Proof of Theorem 9. The L -smooth assumption (Assumption 1) implies that

where the equality comes from

The third term in the RHS of (54) can be bounded by

Substituting (56) back into (54) and cooperating with Assumption 4 yield

Rearranging and averaging (57) for k from 0 to K -1 deduce that

Noticing Assumption 2 claims that f ( W k ) ≥ f ∗ , we complete the proof.

G Proof of Analog Stochastic Gradient Descent Convergence

G.1 Proof of Theorem 2: Convergence of Analog SGD

This section provides the convergence guarantee of Analog SGD under non-convex assumption on asymmetric linear devices.

Theorem 2 (Convergence of Analog SGD ) . Under Assumption 1-4, if the learning rate is set as α = √ f ( W 0 ) -f ∗ σ 2 LK and K is sufficiently large such that α ≤ 1 L , it holds that

where S K denotes the amplification factor given by S K := 1 K ∑ K k =0 ∥ W k ∥ 2 ∞ /τ 2 1 -∥ W k ∥ 2 ∞ /τ 2 ≤ W 2 max /τ 2 1 -W 2 max /τ 2 .

Proof of Theorem 2. The L -smooth assumption (Assumption 1) implies that

where the second inequality comes from the assumption that noise has expectation 0 (Assumption 3)

and the following inequality

With the learning rate α ≤ 1 2 L and bounded variance of noise (Assumption 3), (59) becomes

Cooperated with Assumption 3, the last term in the RHS of (59) can be bounded by

where the last inequality comes from Assumption 3

Plugging (62) back into (59) yields

Assumption 4 ensures that 1 -∥ W k ∥ 2 ∞ /τ 2 ≥ 1 -W max /τ 2 > 0 , which enables dividing both side of (64) by 1 -∥ W k ∥ 2 ∞ /τ 2 . Taking expectation over all ε k , ε k -1 , · · · , ε 0 and averaging for k from 0 to K -1 deduce that

where the second inequality uses Assumption 2 and the last equality chooses the learning rate as α = √ f ( W 0 ) -f ∗ σ 2 LK . The proof is completed.

G.2 Proof of Theorem 3: Lower Bound of Analog SGD

This section provides the lower bound of Analog GD under non-convex assumption on noisy asymmetric linear devices.

Theorem 3 (Lower bound of the error of Analog SGD ) . There is an instance which satisfies Assumption 1-4 such that Analog SGD generates a sequence { W k : k = 0 , 1 , · · · , K } which satisfies

The proof is completed based on the following example.

(Example) Consider an example where all the coordinates are identical, i.e. W k = w k 1 for some w k ∈ R where 1 ∈ R D is the all-one vector. Define W ∗ = w ∗ 1 where w ∗ ∈ R is a constant scalar and a quadratic function f ( W ) := L 2 ∥ W -W ∗ ∥ 2 whose minimum is W ∗ . Initialize the weight on W 0 = W ∗ and choose the learning rate α ≤ min { 1 2 L , 1 µ +6 σ/ ( τ √ D ) } . Furthermore, consider the noise defined as follows,

where random variable ξ k ∈ R is sampled by

As a reminder, it is always valid that | w k | = ∥ W k ∥ ∞ ≤ τ (c.f. Theorem 5) and 0 ≤ p k ≤ 1 . Therefore, the noise distribution is well-defined. Furthermore, without loss of generality 3 , we assume | w ∗ | ≤ τ 4 and σ ≤ τL √ D 4 √ 3 .

Since all the coordinates are identical, Analog SGD can be regarded to train a scalar weight at each coordinate. Furthermore, with the definition

the noise ξ k satisfies Assumption 7. Therefore, even though w ∗ ξ k are time varying, we can regard w k as the initial point and only consider w k +1 . By this way, the conditions of Theorem 6 and Theorem 8 are met, and their claim for w k +1 is valid.

To prove the lower bound, we next introduce several necessary lemmas to facilitate the proof of Theorem 3. The first one is used to verify Assumption 4.

Lemma 2. Suppose | w ∗ | ≤ τ 4 and σ ≤ τL √ D 4 √ 3 . Define W max := τ 2 . The sequence { W k : k ∈ N } generated by Analog SGD (3) on the example above satisfies

Proof of Lemma 2. The statement can be proved by induction. When k = 0 ,

To prove (69) for k +1 , Theorem 8 asserts that

or equivalently

According to the triangle inequality, ∥ W k +1 ∥ ∞ is still bounded by

Therefore, (69) is confirmed for k +1 and the proof of Lemma 2 is completed.

Noise ξ k defined by (67) is time-varying since W k is changing. The following lemma provides an upper bound for the variation of noise with respect to W k .

Lemma 3. Suppose ∥ W k ∥ ∞ ≤ W max ≤ τ 2 . The following statements are always valid

Proof of Lemma 3. Define the functions

with which ξ + k and ξ -k can be written as

The derivatives of g + ( x ) and g -( x ) is

whose norms are upper bounded by

Therefore, both g + ( x ) and g -( x ) are Lipschitz continuous and hence

The proof of Lemma 3 is then completed.

In the proof of Theorem 3, the most tricky part in the recursion (3) is the gradient wrapped by absolute value |∇ f ( W k ) + ε k | . Fortunately, the expectation E [ |∇ f ( W k ) + ε k | ] can be expressed explicitly in the example above, which is demonstrated by the following lemma.

Lemma 4. Suppose the learning rate is chosen such that α ≤ min { 1 2 L , 1 µ +6 σ/ ( τ √ D ) } and (74) are valid, it holds that

Proof of Lemma 4. The proof of Lemma 4 closely relies on the following inequality

or equivalently, the gradient [ ∇ f ( W k )] i = L ([ W k ] i -w ∗ ) satisfies

By (85), we claim the signs of [ ∇ f ( W k )] i + ξ + k and [ ∇ f ( W k )] i + ξ -k never change during the training and thus the absolute value can be removed

Accordingly, E [ |∇ f ( W k ) + ε k | ] can be written as

which is the result. Therefore, the rest of proof shows (84) is valid.

Verification of (84) . The statement can be proved by induction. When k = 0 , (84) holds naturally since W 0 = W ∗ .

To prove (84) for k +1 , we consider ε k = ξ + k 1 and ε k = ξ -k 1 separately, and the conclusion holds for both cases.

Case 1: ε k = ξ + k 1 = σ √ D √ 1 -p k p k · 1 . From the induction assumption, it is valid that

Therefore, Theorem 6 asserts that [ W k +1 ] i ≤ [ W k ] i and hence we have p k +1 ≥ p k , ξ + k +1 ≤ ξ + k and ξ -k +1 ≤ ξ -k . Consequently, the second inequality of (84) can be reached by Theorem 8

To obtain the other inequality, notice that

where ( a ) comes from Theorem 6, (b) comes from Lemma 3 and inequality (37).

The choice of learning rate implies that

from which and (91) we reach that

Combining (94) and (90) reaches (84) in case 1.

Case 2: ε k = ξ -k 1 = -σ √ D √ p k 1 -p k · 1 . Noticing the permutation invariant between p k and 1 -p k in the noise definition (67), we find that the proof for case 2 is similar to that of case 1. Therefore, the proof of case 2 is omitted here.

In conclusion, (84) still holds for k +1 and (84) is verified. Now the proof of Lemma 4 is completed.

After providing the necessary lemmas, we begin to prove Theorem 3.

Proof of Theorem 3. Consider the example constructed above. Before deriving the lower bound, we demonstrate Assumption 1-4 hold. It is obvious that ∇ f ( W ) = L ( W -W ∗ ) satisfies Assumption 1 and f ( W ) ≥ f ∗ := 0 satisfies Assumption 2. In addition, Assumption 3 could be verified by noticing (66) impplies E ε k [ ε k ] = 0 and E ε k [ ∥ ε k ∥ 2 ] ≤ σ 2 . Assumption 4 is verified by Lemma 2.

Now we can derive the lower bound. Utilizing Lemma 4 and manipulating the recursion (3), we have the following result

∗

Multiplying the both size of (95) by L and plugging in the equation ∇ f ( W k ) = L ( W k -W ) yield (96)

where the last equality uses W k = w k 1 . Recall the probability p k = 1 2 (1 -w k τ ) , we have

Substitute them back into (96)

where the first equality utilizes | w k | 2 = ∥ w k 1 ∥ 2 ∞ = ∥ W k ∥ 2 ∞ . Reorganizing (98), we obtain

It is worth noticing that the second term in the RHS of (99) is in the order of O ( α )

where ( a ) is Lemma 1 and ( b ) comes from ∥∇ f ( W k ) ∥ = L ∥ W k -W ∗ ∥ ≤ L ( ∥ W k ∥ + ∥ W ∗ ∥ ) ≤ L √ D ( ∥ W k ∥ ∞ + | w ∗ | ) . Therefore, the gradient can be rewritten as

Taking the square norm of the gradient and averaging for k from 0 to K -1 we obtain

where the following equality is applied

The proof of Theorem 3 is thus completed.

H Proof of Tiki-Taka Convergence

This section provides the convergence guarantee of the Tiki-Taka under the non-convex assumption. Theorem 4 (Convergence of Tiki-Taka ) . Suppose Assumption 1-5 hold and the learning rate is set as α = O (1 / √ σ 2 K ) , β = 8 αL . It holds for Tiki-Taka that the expected infinity norm P k is upper bounded by E [ ∥ P k +1 ∥ 2 ∞ ] ≤ P 2 max := 41 L 2 τ 4 D c 2 σ 2 . Furthermore, if σ 2 and D are sufficiently large so that 33 P 2 max /τ 2 < 1 it is valid that

H.1 Proof of Tracking Lemma

Before the proving Theorem 4, we first introduce two useful lemmas.

Lemma 5 (Tracking Lemma) . Under Assumption 1 and 3, it holds that

To make P k +2 sufficiently close to the true gradient, ∥ P k +1 ∥ 2 should be small.

Proof of Lemma 5. According to the update rule of P k , we have

With Assumption 3, the first term of (103) can be bounded via the same technique as (63), that is

where the last inequality holds because

The third term in the RHS of (103) can be manipulated by the variance decomposition and consequently can be bounded by

Combining (104) and (106) with (103) provides that

In the inequality above, ( a ) sets u = β/ 2 ; ( b ) holds because

and

and ( c ) comes from

where the last inequality holds because β ≤ 2 τ 2 D βσ 2 . Now we get the claim.

H.2 Proof of Weight Decay Lemma

Lemma 6 (Weight Decay Lemma) . Suppose Assumption 1, 3 and 5 hold. If β satisfies

it holds that

Proof of Lemma 6. The proof begins from the manipulation of the expected square norm of P

k +2 2

For the sake of simplicity, below we use [ g k +1 ] i := [ ∇ f ( W k +1 )] i to denote the i -th coordinate of the gradient. According to Assumption 5, the first term in the RHS of (112) can be bounded by

where the last inequality holds because of the selection of β . To bound the third term in the RHS of (112), we use the unbiased property of E ε k +1 [ ε k +1 ] = 0 (Assumption 3) and ∥ P k ∥ ∞ ≤ τ (Theorem 1), which lead to

Plugging (113) and (114) into (112) yields

Notice that the second term in the RHS of (115) can be upper bounded by Assumption 3 and inequality (63), and the third term can be bounded by Young's inequality

Inequality (63), (116) and the condition β ≤ 2 3 cσ lead to the result

Applying the inequality

on (117) completes the proof.

H.3 Proof of Bounded P k

Lemma 7 (Bounded Saturation of P k ) . Suppose Assumption 1, 3 and 5 hold. If β satisfies

it holds that and further

Proof of Lemma 7. Let i ∗ := arg max i | [ P k +2 ] i | . We expand the expected square norm of P k +2 by

For the sake of simplicity, below we use [ g k ] i := [ ∇ f ( W k )] i to denote the i -th coordinate of the gradient. According to Assumption 5, the first term in the RHS of (122) can be bounded by

where the last inequality holds because of the selection of β . To bound the third term in the RHS of (122), we use the unbiased property of E [ ε k ] i ∗ [[ ε k ] i ∗ ] = 0 of Assumption 5 and ∥ P k +1 ∥ ∞ ≤ τ in Theorem 1, which lead to

Plugging (123) and (124) into (122) yields

where the last inequality comes from Assumption 5 and variance decomposition (63). Notice that the second term in the RHS of (125) can be upper bounded by Assumption 3 and inequality (63), and the third term can be bounded by Young's inequality

Inequality (126) and the condition β ≤ 2 3 cσ lead to the result

where the last inequality holds because the parameter is chosen as β ≤ L 2 τ 3 D 3 cσ 3 . Telescoping over 0 to K -1 and using the fact that ∥∇ f ( W k ) ∥ 2 ∞ ≤ ∥∇ f ( W k ) ∥ 2 ≤ 4 L 2 τ 2 D , we achieve that

where the last inequality holds due to the fact that P 0 = 0 . Now we complete the proof.

H.4 Proof of Tiki-Taka Descent Lemma

Lemma 8 (Descent Lemma) . Under Assumption 1, it holds that

Proof of Lemma 8. The L -smooth assumption (Assumption 1) implies that

where the equality comes from

From the update rule of Tiki-Taka and bounded saturation assumption (c.f. Assumption 4), the third term in the RHS of (129) can be bounded by

Substituting (131) back into (129) yields

which completes the proof.

H.5 Proof of Theorem 4: Convergence of Tiki-Taka

With the lemmas above, we can begin the proof of Theorem 4.

Proof of Theorem 4. The statement that E [ ∥ P k +1 ∥ 2 ∞ ] ≤ P 2 max := 41 L 2 τ 4 D c 2 σ 2 has been shown in Lemma 7. Therefore, it suffices to prove the convergence rate.

The proof begins by defining a Lyapunov function

Notice that the last term of V k has the iteration

where (a) uses ∥ x + y ∥ 2 ≤ 1 1 -u ∥ x ∥ 2 + 1 u ∥ y ∥ 2 for any vector x, y ∈ R D and scalar 0 < u < 1 ; (b) follows Lemma 7, bounded weight (c.f. Theorem 1), and Assumption 1; (c) specifies u = βcσ 2 τ and the inequalities 1 -βcσ/τ 1 -u ≤ 1 -βcσ τ + u and 1 1 -u ≤ 2 .

By inequality (134), Lemma 5, 6 and 8, we have

Rearranging inequality (135) above, we have

where the second inequality holds because the selection of α and β as follows implies the coefficients of the third to fifth term of RHS are greater than 0

deduce that

where the last inequality holds when D is sufficiently large. The proof is completed.

I Simulation Details and Additional Results

This section provides details about the experiments in Section 2.1 and 5. The analog training algorithms, including Analog SGD and Tiki-Taka , are provided by the open-source simulation toolkit AIHWKIT [27], which has Apache-2.0 license; see github.com/IBM/aihwkit . We use Softbound device provided by AIHWKIT to simulate the asymmetric linear device (ALD), by setting its upper and lower bound as τ . The digital algorithm, including SGD, and dataset used in this paper, including MNIST and CIFAR10, are provided by PYTORCH, which has BSD license; see https://github.com/pytorch/pytorch .

Hardware. We conduct our experiments on an NVIDIA RTX 3090 GPU, which has 24GB memory and maximum power 350W. The simulations take from 30 minutes to one hour, depending on the size of the model and the dataset.

Statistical Significance. The simulation reported in Figure 5 is repeated three times. The randomness originates from the data shuffling, random initialization, and random noise in the analog device simulator. The mean and standard deviation are calculated using statistics library.

I.1 Least squares problem

In Section 2.1 and 5.1, we considers the least squares problem on a synthetic dataset and a ground truth W ∗ ∈ R D , whose elements are sampled from a Gaussian distribution with mean 0 and variance σ 2 data . Consider a matrix A ∈ R D out × D of size D = 40 and D out = 100 whose elements are sampled from a Gaussian distribution with variance σ 2 A . The label b ∈ R D out is generated by b = AW ∗ where W ∗ are sampled from a standard Gaussian distribution with σ 2 W ∗ .

The problem can be formulated by

2

Taking expectation, averaging (138) over k from 0 to K -1 , and choosing the parameter α as

1.0

0.5

0.0

0.5

1.0

Weight Values

0

2

4

6

8

Frequently

Digital SGD

Tiki-T aka

0.0

0.2

0.4

0.6

0.8

1.0

Saturation

0

5

10

15

20

Frequently

Tiki-T aka

During the training, the noise with variance level σ 2 g is injected into the gradient. For digital SGD and the proposed Analog SGD dynamic, we add a Gaussian noise with mean 0 and variance σ 2 g to the gradient. For Analog SGD in AIHWKIT, we add a Gaussian noise with mean 0 and variance σ 2 g /s 2 A into W when computing the gradient. The constant s A is defined as the mean of the singular value of A , which is introduced since the noise on W is amplified by A ⊤ A , whose impact is approximately proportional to A 's singular value.

In Figure 1, the response step size ∆ w min =1e-4 while τ = 3 . The maximum bit length is 800. The variance are set as σ 2 data = 0 . 30 2 , σ 2 A = 1 . 00 2 , σ 2 W ∗ = 0 . 45 2 , σ 2 g = (0 . 01 /D ) 2 .

In Figure 3, the learning rate is set as α = 3e-2. The response step size ∆ w min is set so that the number of states is always 300, i.e. 2 τ/ ∆ w min = 300 , when the τ is changing. The maximum bit length is 300. The variance are set as σ 2 data = 0 . 30 2 , σ 2 A = 0 . 50 2 , σ 2 W ∗ = 0 . 30 , σ 2 g = 0 . 10 .

I.2 Classification problem

We conduct training simulations of image classification tasks on a series of real datasets. The gradient-based analog training algorithms, including Analog SGD and Tiki-Taka , are implemented by AIHWKIT. In the real implementation of Tiki-Taka , only a few columns or rows of P k are transferred per time to W k to balance the communication and computation. In our simulations, we transfer 1 column every time. The response step size is ∆ w min = 1e-3.

The other setting follows the standard settings of AIHWKIT, including output noise (0.5 % of the quantization bin width), quantization and clipping (output range set 20, output noise 0.1, and input and output quantization to 8 bits). Noise and bound management techniques are used in [66]. A learnable scaling factor is set after each analog layer, which is updated using digital SGD.

3-FC / MNIST. Following the setting in [15], we train a model with 3 fully connected layers. The hidden sizes are 256 and 128. The activation functions are Sigmoid. The learning rates are α = 0 . 1 for SGD, α = 0 . 05 , β = 0 . 01 for Analog SGD or Tiki-Taka . The batch size is 10 for all algorithms.

CNN / MNIST. We train a convolution neural network, which contains 2-convolutional layers, 2-max-pooling layers, and 2-fully connected layers. The activation functions are Tanh. The learning rates are set as α = 0 . 1 for digital SGD, α = 0 . 05 , β = 0 . 01 for Analog SGD or Tiki-Taka . The batch size is 8 for all algorithms.

Resnet / CIFAR10. We train different models from Resnet family, including Resnet18, 34, and 50, The base model is pre-trained on ImageNet dataset. The learning rates are set as α = 0 . 15 for digital SGD, α = 0 . 075 , β = 0 . 01 for Analog SGD or Tiki-Taka . The batch size is 128 for all algorithms.

I.3 Verification of bounded saturation

To further justify the Assumption 4, we visualize the weight distribution of the trained 3-FC model; see Figure 7. The results show that despite the absence of projection or saturation, the weights trained by digital SGD are bounded, which means that it is always possible to find an τ to represent all the weights in analog devices without being clipped. In the right of Figure 7, W max for Analog SGD could be chosen as 0 . 2 τ . Without constraint on the W max , Tiki-Taka has a large W max .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper and important assumptions and limitations.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discussed in the Conclusion section.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All have been clearly listed.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We describe all the implementation details, including datasets, models, and hyperparameters of our simulations in Appendix I.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code of our simulation implementation is available at github.com/ Zhaoxian-Wu/analog-training .

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe all the implementation details, including datasets, models, and hyperparameters of our simulations in Appendix I.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The factors of variability, the method for calculating the error bar, the assumption on the distributions, and the type of error bar are stated in Appendix I.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The computation resource requirements and estimated running time are clearly stated in Appendix I.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: there is no societal impact of the work performed.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer:[NA]

Justification: The paper poses no such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We explicitly state the license of the code and data used in Appendix I.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: the paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: the paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA] .

Justification: the paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis is a theoretical paper about training analog systems based on resistive memories. The paper tackles specifically the problem of weight update asymmetry in such resistive devices. The paper develops a model for the weight dynamics under “Analog SGD”, taking into account weight update asymmetry, and it shows that Analog SGD does not converge, due to an asymptotic error. The paper then shows that the Tiki-Taka algorithm, developed in prior works for such analog devices, has better convergence properties. Tiki-Taka eliminates the asymptotic error and converges to a critical point. Simulations corroborate the theory.\n\nSTRENGTHS:\nThe topic tackled by the paper is very important. Analog computing could lead to energy efficiency gains of several orders of magnitude for both inference and training, compared to digital neural networks on GPUs. Developing appropriate theoretical models of analog computing to better understand the behavior of these analog systems is of utmost importance. To my knowledge, this paper is the first attempt to study the convergence properties of analog SGD.\n\nThe paper is clear and the study is thorough.\n\nWEAKNESSES:\nThe theoretical analysis rests on a set of models and assumptions. Thus, the applicability of the conclusions rests on the validity of all these models/assumptions. In practice, I expect none of these assumptions to perfectly capture the real behavior of the system. For example, Figure 2 hints that the analog SGD dynamics (Eq3) better fits real behaviors than the digital SGD dynamics (Eq2), but the matching still seems to be far from perfect.\nThis, of course, is not a problem, however, what is more problematic is that there does not seem to be any discussion in the paper about which assumptions/models are accurate and widely accepted, and which ones less accurately capture empirical data or are still debated by the community.\n\nThe paper implicitly assumes that backpropagation (BP) is used for computing the weight gradients. However, a large (and growing) group of researchers in neuromorphic computing (or ”physical neural networks”) think that BP is not the best fit for analog devices, and they are exploring alternative training methods. See e.g. Ref [1] below for a very recent review of such algorithms. In particular, Ref [1] discusses several algorithms that extract or estimate the weight gradients in broad classes of analog systems. My understanding is that it wouldn’t take too much time and effort to extend the present study to include these other gradient-descent-based algorithms. In my view, including these other methods would add a lot of value to the paper, by broadening the potential impact of the work.\n\nReference:\n[1] Momeni, Ali, et al. \"Training of Physical Neural Networks.\" arXiv preprint arXiv:2406.03372 (2024).\n\n\nMinor remark. One claim in the introduction is not properly referenced. Specifically, where do the figures of $2.4 million and $4.6 million for training LLaMA and GPT-3 come from? The references provided are the LLaMA and GPT-3 papers, which do not provide these figures, if I am not mistaken.\n\nQUESTIONS:\nIs the model of Eq6 for weight update asymmetry a widely accepted model? Does it match empirical data? Similarly, are the forms of q+ and q- for ALD, written at lines 174-175, widely accepted and/or corroborated by data?\n\nDoes the AIHWKit very faithfully simulate real analog devices? What are the strengths and limitations of this tool?\n\nThe paper is concerned with analog resistive memories where the weights are implemented as conductances (which are positive, I suppose), but on Figure 6 (Appendix L), I see that the weight values can be negative. Lemma 1 also indicates that w_k can be either positive or negative. Could you clarify this point?\n\nI understand that the RHS of Eq.5 can be arbitrarily small by choosing alpha = sqrt(1/K) and letting K -> infinity. But, unless I am mistaken, this does not imply that the sequence grad f(W_k) converges to zero. Can we conclude from Eq5 that W_k converges? If not, why is Eq.5 important for studying the convergence of (digital) SGD?\n\nI am not sure to understand why Theorem 2 is important. To my understanding, the important result is Theorem 3, which shows that analog SGD does not converge, contrary to digital SGD (Eq.5). Could you clarify this point?\n\nIs the theory of analog SGD and Tiki-Taka provided in this paper limited to backpropagation? Or could it be used with other methods that compute weight gradients, e.g. Equilibrium Propagation and/or other algorithms presented in Ref [1] above? Similarly, could the AIHWKit be employed with these other training algorithms?\n\nLIMITATIONS:\nThe applicability of the theoretical results depends on the validity of the underlying assumptions – see my comment above. To be clear, I am supportive of the methodology followed by the authors. However, given that the overall model will be at most as accurate as the least accurate assumption, it would be extremely useful to know which assumptions/models we can trust 100% (i.e. are well accepted), and which ones are still debated / less supported by experiments. This would help future works identify what are the main “bottlenecks” of the study, to investigate those in depth and perhaps further improve the model. For instance, can we really trust Eq3 as “the right model” of weight dynamics? Or should one merely think of it as a better model than Eq. 2?", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-27"}

comments: SUMMARY:
The paper introduces the training on the analog in-memory accelerators with SGD. The traditional analog SGD algorithm suffers from inexact convergence due to asymmetric updates/gradient noise. The paper shows theoretical foundations for gradient-based training on analog devices. The authors propose Tiki-Taka, which reflects the device model to the Analog SGD. It shows better empirical performance, e.g., it can eliminate asymptotic errors and converge to critical points. They show the theoretical and empirical efficacy of the proposed approaches in overcoming training limitations on the analog devices.

STRENGTHS:
- The paper is well organized and easy to follow.
- Concrete theoretical foundings on the proposed algorithm.
- Benchmark sets and the target network look reasonable, and the result also sound and look promising.
- Showing cases on various analog devices like ReRAM, etc.

WEAKNESSES:
- It'd be great if authors present how computations (ops) can be realized on the analog devices.
- Needs comparison to other analog SGD methods, like [16] or [22].

QUESTIONS:
- Floating-point operations are not trivial on analog devices. How can they be implemented? Additionally, for SGD training, how are analog devices more efficient compared to digital devices?
- How does it scale to large batch sizes?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

comments: SUMMARY:
This paper presents a theoretical framework for gradient-based training on analog devices. This work first identifies the non-convergence problem of Analog SGD, which stems from asymptotic errors due to asymmetric updates and gradient noise and then presents a convergence analysis of Tiki-Taka, demonstrating its ability to accurately converge to a critical point, thereby eliminating the asymptotic error.

STRENGTHS:
-- The proof the convergence of Analog SGD and showing that noise and asymmetric updates together lead to its asymptotic error. 

-- Demonstration of Tiki-Taka algorithm precisely converging to the critical point by mitigating the drift caused by asymmetric bias and noise.

-- Empirical simulations using both synthetic and real datasets to confirm the presence of asymptotic error in Analog SGD and to show that Tiki-Taka outperforms Analog SGD.

WEAKNESSES:
-- The experimental results were obtained on rather small datasets. It'd be great to include the results on ImageNet.

-- The experiments were limited to one type of networks (i.e., resnet) on CIFAR dataset. How does it generalize on other networks such as Transformers, or mobile CNNs?

QUESTIONS:
See weaknesses

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-16