Hardware-restriction-aware training (HRAT) for memristor neural networks
Authors: Zhimin Tang
Rujie Zhao
Linkai Luo
Haibo Wang
Chao Lu
Venue: Submitted to ICLR 2023 (2023)
Keywords: Neuromorphic computing
Memristor
Neural network training
Hardware restrictions

Abstract: Memristor neural network (MNN), which utilizes memristor crossbars for vector-matrix multiplication, has huge advantages in terms of scalability and energy efficiency for neuromorphic computing. MNN weights are usually trained offline and then deployed as memristor conductances through a sequence of programming voltage pulses. Although weight uncertainties caused by process variation have been addressed in variation-aware training algorithms, efficient design and training of MNNs have not been systematically explored to date. In this work, we propose Hardware-Restriction-Aware Training (HRAT), which takes into account various non-negligible limitations and non-idealities of memristor devices, circuits, and systems. HRAT considers MNN's realistic behavior and circuit restrictions during offline training, thereby bridging the gap between offline training and hardware deployment. HRAT uses a new batch normalization (BN) fusing strategy to align the distortion caused by hardware restrictions between offline training and hardware inference. This not only improves inference accuracy but also eliminates the need for dedicated circuitry for BN operations. Furthermore, most normal scale signals are limited in amplitude due to the restriction of non-destructive threshold voltage of memristors. To avoid input signal distortion of memristor crossbars, HRAT dynamically adjusts the input signal magnitude during training using a learned scale factor. These scale factors can be incorporated into the parameters of linear operation together with fused BN, so no additional signal scaling circuits are required. To evaluate the proposed HRAT methodology, FC-4 and LeNet-5 on MNIST are firstly trained by HRAT and then deployed in hardware. Hardware simulations match well with the offline HRAT results. We also carried out various experiments using VGG-16 on the CIFAR datasets. The study shows that HRAT leads to high-performance MNNs without device calibration or on-chip training, thus greatly facilitating commercial MNN deployment.

Under review as a conference paper at ICLR 2023
HARDVARE-RBSTRRCTION-AAAAA TRAINING (HRAT) FOR MEMRISTOR NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Memristor neural network (MNN), which utilizes memristor crossbars for vector- matrix multiplication, has huge advantages in terms of scalability and energy ef- ficiency for neuromorphic computing. MNN weights are usually trained offline and then deployed as memristor conductances through sequence of program- ming voltage pulses. Although weight uncertainties caused by process variation have been addressed in variation-awe aworr training algorithms, efficient design and training of MNNs have not been systematically explored to date. In this work, we propose
Training (HRAT), which takes into ac- count various non-negligible limitations and non-idealities of memristor devices, circuits, and systems. HRAT considers MNN's realistic behavior and circuit re- strictions during offline training, thereby bridging the gap between offline training and hardware deployment. HRAT uses new batch normalization (BN) fusing strategy to align the distortion caused by hardware restrictions between offline training and hardware inference. This not only improves inference accuracy but also eliminates the need for dedicated circuitry for BN operations. Furthermore, most normal scale signals are limited in amplitude due to the restriction of non- destructive threshold voltage of memristors. To avoid input signal distortion of memristor crossbars, HRAT dynamically adjusts the input signal magnitude dur- ing training using learned scale factor. These scale factors be incorporated into the parameters linear operation together with fused BN, so no additional signal scaling circuits required. To evaluate the proposed HRAT methodology, FC-4 and LeNet-5 on MNIST are firstly trained by HRAT and then deployed in hardware. Hardware simulations match well with the offline HRAT results. We also carried out various experiments using VGG-16 16 on the CIFAR datasets. The study shows that HRAT leads to high-performance MNNs without device calibra- tion or on-chip training, thus greatly facilitating commercial MNN deployment.
INTRODUCTION
Memristor neural network (MNN) has emerged increasingly feasible option alleviate the scalability and energy efficiency challenges in neuromorphic computing. While several small-scale MNNs have been prototyped Li (2018); Yao et al. (2020); Wan et al. (2022), efficient design and training of MNNs require an in-depth understanding of various restrictions from device, circuit, and system perspectives. These hardware restrictions include weight uncertainty noise caused by mem- ristor variability and a limited number of programming pulse cycles to tune memristor conductance (e.g., 500 Yao al. (2020)), weight quantization noise due to limited states of memristor conduc- tance (e.g., 5- and 4-bit in Yao et al. (2020); Wan et al. (202))) non-destructive threshold voltage of memristors Joo et al. (2010), limited output swing operational amplifiers Karki (201)), and bias quantization noise from mniterresolutin
eegital-to-analoy converters (DACs). These hardware strictions collectively reduce the accuracy of MNN inference. Ignoring these hardware restrictions during software offline training may result in poor inference or even functional failure.
As critical step in network training, batch normalization (BN) can accelerate training convergence loffe & Szegedy (2015). The scale and shift operations of BN can be merged into the previous linear operation (e.g., fully connected or convolutional layer) after training. In this way, the hardware complexity and cost of MNNs are alleviated, as BN does not require explicit memristor crossbars in

---

Under review as a conference paper at ICLR 2023
on-chip deployment. We envision that the aforementioned hardware restrictions have significant impact on BN fusion (also known as BN folding) in MNNs. For example, bias distortion caused by DACs and the limited output swing of operational amplifiers should be considered during BN fusion.
Although several BN fusing strategies Jacob et al. 2018); Krishnamoorthi (0018); ; Py Torch (002);; Wan al. 2022) have been reported for quantization-awart training, unfortunately. dedicated BN fusion strategies for MNN training and hardware deployment has so far received little attention. As a result, it imperative develop sardwwre-restriction--- -aware BN fusing strategies to align signal distortion caused by hardware restrictions before and after BN fusion in MNNs.
In this work, we propose a Reeraar--Reettiiion---eeeee Aware Training (HRAT) method, which takes into account various non -egliggie restrictions and non-idealities from device, circuit, and system per- spectives. HRAT considers realistic behavior and hardware restrictions of MNNs during offline training, thereby bridging the gap between offline training and hardware deployment. The key con- tributions of this work are summarized as follows:
We model various hardware restrictions of MNNs and integrate them into training, enabling harddwarerestrictionnaww training (HRAT)). HRAT uses new BN fusing strategy to align the instrictiio-induccer distortion between offline training and hardware inference. This not only improves inference accuracy but also eliminates the need for dedicated circuitry for BN operation. To avoid input signal distortion of memristor crossbars, HRAT dynamically adjusts the signal magnitude during training using a learned scale factor. These scale factors can be incorporated into the parameters of linear operation together with fused BN, so no additional signal scaling circuits are required.
We conduct various experiments on baseline networks (FC-4, LeNet-5, and VGG-16) on datasets (MNIST, CIFAR-1O, and CIFAR-100) to demonstrate the performance of HRAT. To evaluate the proposed HRAT methodology, FC-4 and LeNet-5 are firstly trained by HRAT and then deployed in hardware. Hardware simulation results match well with the offline HRAT results, indicating that HRAT can bridge the gap between offline train- ing and hardware deployment. To investigate the effectiveness of HRAT on large-scale networks, we conduct experiments using VGG-16 the CIFAR datasets. Experimen- tal results demonstrate that HRAT can lead to state-of-the-ar MNNs without performing prohibitively expensive and time-consuming on-chip retraining, enabling low-cost high- performance MNNs for large-scale commercialization of neuromorphic systems.
2 RELATED WORK
Variation-aware MNN training. Prior work on MNN offline training has predominately focused on variation-awan training. Liu et al. (015) address memristor conductance variability (i.e., weight uncertainties) by introducing an additional term called "penalty of variation" into the training con- straints. Then, upper bound for the variation penalty is estimated and used during training. This method improves the tolerance of trained weights to memristor variability by applying tighter train- ing constraints. Zhu et al. (2020) propose statistical training to deal with MNN weight uncertain- ties. Trained weights are modeled as linear functions of random variables, representing memristor variability. Subsequent network computations are modified to propagate the effects of weight uncer- tainties. The effectiveness of statistical training has only been validated on small MNNs. Gao et al.) (202T) propose variation- aware MNN training framework, which develops analytical model for weight uncertainties and uses it as a constraint during training. YYang et al. (022)) propose stochastic- noise-aware training to inject stochastic noise during training. Stochastic noise includes memristor programming noise (i.e., memristor conductance variability), thermal noise, shot noise, and random telegraph noise. Mao et al. (2022) propose aefect-aware training to account for the effects of mem- ristor conductance variability, relaxation, and failure during training. Büchel et al. (202)) address the issue of memristor conductance variability via adversarial regularization. In order enhance the MNN robustness to memristor programming noise, weight space is attacked by adding Gaussian noise Murray & Edwards (1994) to parameters during training. Waan et al. (2022) propose noise- resilient training to inject Gaussian noise into MNN weights during the forward pass. In summary, the aforementioned MNN training methods mainly improve noise immunity to memristor conduc- tance (i.e., weight) uncertainties and address weight mismatch between offline training and on-chip deployment. However, other non noglnegiii MNN hardware restrictions, such as the non-destruetive threshold voltage of memristors, have not been incorporated into training.
2

---

Under review as a conference paper at ICLR 2023
BN fusing strategies for quantization-aware training. Several BN fusing strategies have been proposed for quamtization-awar training (Figure Sn Appendix for details). By incorporating batch normalization into linear operation (e.g.. fully connected layer or convolutional layer), these strate- gies convert W (i.e., weight of linear operation) into W 7/ 7//vo=+ + (ie., fused weight). Here y is a learnable scale factor in BN, o² is output variance of linear operation, and is small constant added to prevent the divide by zero error. Wan et al. (2022) implement BN fusion in a straightfor- ward manner [offe & Szegedy (2015). Jacob et al. 2018) introduce fused linear operation at the current batch scale. During training, the statistics (i.e., mean and variance) of the current batch are extracted before performing the fused linear operation. Then, these statistics are used to update the Moving Average (MA) statistics and generate fused weight and bias. In this way, fused linear oper- ation is performed using the obtained fused weight and fused bias. Krishnamoorthi (2018) introduce fused linear operation at the MA scale with a correction for the current batch scale. During train- ing, the output of fused linear operation is corrected from the MA scale to the current batch scale by multiplying Vo + Vo2 + e, where o the output variance of linear operation extracted from current batch. Thus, an additional linear operation is required to compute the current batch statistics, which is used to update the MA statistics and correct the fused scale of linear operation. PyTorch] (2022) introduces fused linear operation at the MA scale with plain BN and a correction to the unfused scale. In order to apply a plain BN, the scale of fused linear operation is corrected to the the unfused scale by multiplying va² + t//.. Since the strategies Jacob al. 2018); Krish- namoorthi (2018) require two linear operations (i.e., fused linear and linear), they are much more computationally expensive than (rrTorch(202) whose strategy involves one linear operation.
While these existing BN fusing strategies help align the distortion caused by weight quantization, the hardware restrictions of MNN systems lead to more aspects of distortion during BN fusion. For example, since fused bias and outputs of fused linear operation are represented as voltages in MNNs, effective BN fusion for MNNs needs to ensure that they conform to the limited output swing of operational amplifiers. Otherwise, voltage saturation and clamping of fused bias or outputs of fused linear operation will cause distortion and inference degradation. Therefore, a BN fusing strategy should align the distortion caused by hardware restrictions between training and hardware inference.
MNN HARDWARE DEPLOYMENT AND RESTRICTIONS
An MNN consists of multiple interconnected network layers. The hardware schematic for one layer plotted in Figure [(a) where a memristor crossbar implements a layer of synapses, and each offline- traine weight is achieved by the difference in memristor conductance between two differen- tial memristors Prezioso et al. (2015): [L et al.] (2018); : Yao al. (2020); Wan et al. (2022), allowing the realization of positive and negative weight. Each offline-trained bias is downloaded into register circuit, which controls a digital-to-analor converter (DAC) for providing a bias voltage to a neuron summation circuit. The neuron summation circuit and activation circuit work together to generate an output voltage Krestinskaya et al. (2019). Figure [(b]]soww the measured current-voltage sweep curve of memristors Yan et al. (2019). Memristors have two distinct modes of operation: safe op- eration mode, in which memristor conductance remains unchanged from its previously programmed value, and conductance programming mode, in which a series of programming pulse voltages are applied to a memristor until its conductance approaches its expected value. Since MNNs should run in the safe operation mode during inference, the input voltage across memristors cannot exceed the upper voltage limit of its safe operation region, such as 0.2V in Yan et al. (011)). This upper voltage limit can be viewed as the non- non-destrecttvv threshold voltage Jo et al. (010). VrH, below which the previously programmed memristor conductance (i.e., offline-trained weight) does not change dur- ing inference. The non-destruetive threshold voltage varies with the material, fabrication process, and physical structure of memristors. To ensure proper inference, the input voltage clamp circuit in Figure should restrict voltages across the memristor crossbar with [-VH,, VrHl.
Figure I(c) shows the schematic of a neuron summation circuit, which consists of transimpedance amplifiers and fully differential amplifier to convert and scale the current difference (i.e., I+ I-) to voltage and then add the DAC generated bias voltage. To avoid the inconvenience caused by dual power supply, the neuron summation circuit is designed to operate with a single positive power supply and set the signal ground level of the neuron summation circuit to be at the half of the supply
3

---

Under review as a conference paper at ICLR 2023
mxn memristor crossbar
V1
600
Mini
Inputs
400
Current(uA)
200
Via
11's
V
-200
Register
safe
Vasst
Σ
Visa
E
V
Σ
operation
400
DAC
Vs
Vsu
region
ReLU Activation Circuit
-600
Sbal....io
Vatue
-0.4
-0.2
Vor
Outputs
Vuz
V
Voltage(V)
(a) Hardware schematic of MNN layer for forward pass. A crossbar implements layer synapses. Each offline-trained weight is achieved by two differential memristors.
(b) Measured current and voltage sweep curve of memristors Yan et (019
Vout
Saturation
VDD
R
W
Vss
VsG
XI
W
V
r
WW
x2
X3
Normal O peration
hais
Saturation
Region
Vin
SG
(c) A neuron summation circuit. It uses resistors and operational am- plifiers scale the current difference of differential memristors, and then adds a bias voltage from a digital-to-and risital-to-analee to-alo converter (DAC).
(d) Output voltage swing of ampli- fiers and DACs is limited between OV and VDD (the supply voltage).
Figure 1: MNN hardware overview and deployment restrictions.
voltage (i.e., VsG= VDD/2 in Figure I(c)). In this way, all node voltages are between OV and VDD (the supply voltage), and the voltages VVINI--ViNm) in Figure T(a) will be clamped into [Vsa VTH, Vsa + VTH). In addition to the memristor threshold voltage restriction described above, MNN devices, circuits, and systems suffer from the following restrictions and non-idcalities:
Weight quantization noise. Offline-trained weight is implemented by memristor conductance in MNN. Memristor is typically programmed several conductance states, resulting in limited weight bitwidth (i.e.. 5 bits Yao et al. (2020) and 4 bits Wan et al. (2022)) and weight quantization noise.
Weight uncertainty noise. Due to physical mechanisms (e.g., device relaxation Mao et (2022); Wan et al. (202)) and limited number of programming pulse cycles (e.g., 500 in Yao et al. (2020)), memristor conductance manifests significant variations, leading considerable mismatch
from dffline-trained weight Büchel et al. (2022).
Limited output swing of operational amplifiers and DACs. The outputs of the amplifier or DAC can only swing within the power supply range (assuming rail-to-rail circuit topologies are used). The circuit outputs will be clamped the ground VDD level when the intended signal values exceed the output swing range as shown in Figure I(d) Such hardware restriction should be considered during training.
Bias quantization noise of finite-resolution DACs. DAC circuits are widely used for bias voltage generation due to their high precision and great flexibility. The output resolution of a DAC (usually specified in bits) represents the smallest output increment that can be produced. When bias voltages are generated by finiterresolution DACs, bias quantization noise potentially degrades the inference performance.

---

Under review as a conference paper at ICLR 2023
4 MARDWARRERESTRICTION-AAAAA TRAINING (HRAT)
In this section, we present Senddware-Restrictoon---a Training (HRAT), which takes into account various non-negligible restrictions of memristor devices, circuits, and systems. Figure 2(a) and 2 illustrate the HRAT process for layer without BN and with the proposed BN fusing strategy, respectively. In HRAT, weight parameters are quantized to mimic the process of using a limited number of pulse cycles to program memristor conductance; bias parameters quantized to mimic bias quantization noise caused by the finite-resolution of DACs; process variation of memristor devices is mimicked by adding weight uncertainty noise, and the limited output swing of operational amplifiers and DACs is mimicked by a clamp function. Furthermore, trainable scale factor is added each layer. this way, the output signal magnitude of each network layer can be adjusted to proper range. Key aspects of HRAT will be described in detail in the following subsections.
[-Ve,
YiN
VDD: supply voltage for MNN
5: learnable scale factor Vna non-destructive threshold voltage of memristor
Clamp
Add
Clamp
[-V. Vdl
[-Vo Vdl
Vg: maximum allowable positive signal level, =s VOD/2
Quant
Clamp
ReLU
Vec minimum allowable negative signal level, "(-VDD/2)
[0, Vel
Fused Bias
Yir, w/o Fused Blas
+
Div
Factor
Separate
Output [o, VDD/2]
Yie
Output VDD/2]
Batch Norm
Update MA Mean & MA Var
[0, Val
ReLU
Add
Bias
yw/o Bias
[-Vo Vd]
Current unfused scale
Clamp
Scale
Yis w/o Fused Blas
MA fused scale
clamp
Add
Fused Linear
+
(-Vo, Vel
Quant
Clamp
Add
Weight Uncertainty Noise
[-Vra, Vni]
Blas
Linear
MA fused scale
[-Vou. Vnd
Add
Weight Uncertainty Noise
Clamp
Fused Weight
Clamp
Quant
Fuse BN
MA Mean & MA Var
Input
Weight
Input Weight
(a) HRAT for network layer without BN
(b) HRAT for network layer with BN fusion
Figure 2: Hardwarr-restrrction--aaa training (HRAT) for MNNs.
4.1 PARAMETER-NOISE-AWWER TRAINING
Instead of training MNNs at full precision, their offline-trained weights are typically quantized to finite discrete states (i.e., 5 bits Yao al. (2020) and bits Mao et al. (2022); Wan al. (2022) As described in Appendix | we uniformly quantize weight parameters and symmetrically clamp the states finite number. To model the weight uncertainty and improve the noise robustness of MNNs, we follow Murray & Edwards (1944); Wan et al. (2022) and inject Gaussian noise to weight parameters during forward pass. Thus, weight quantization noise and weight uncertainty noise are added to weight parameters during training. Similarly, due to the limited resolution of DACs, DAC outputs are linearly quantized. Quantization noise is injected into the bias parameter during training- Estima-
To solve the non oon-differentiaiion issue of quantization function, we use the Straight-Throur tor (STE) trick to relax the quantization function of weight and bias parameters. The perception of weight and bias noise during training is called parameter-noise-ant resulting neural networks are insensitive to MNN deployment restrictions of weight quantization, weight uncertainty, and bias quantization noise. While other methods may perform better in ad- dressing parameter noise (e.g., adversarial (raining Büchell Büchel et al. (2022)). simply injecting parameter noise is more computation efficient and sufficient for maintaining MNN accuracy.
training, through which the
5

---

Under review as a conference paper at ICLR 2023
4.2 NON-DESTRUCTIVE THRESHOLD VOLTAGE OF MEMRISTORS
During inference, memristor should operate at or below its non-destructive threshold voltage to avoid conductance drift. As the threshold voltage is usually low (e.g.. 0.1V Yan et al. (2018), 0.2V Yan et al. (2019), 0.2V et al. (2020)). the magnitude of most imput signals to memristor crossbars will be chopped by the input clamp circuit in Figure I(a)), causing significant signal distortion. Therefore, HRAT clamps the signal magnitude across memristor crossbars to [- --VHH, VrH) during training to correspond to the input clamp circuit in hardware MNNs.
4.3 SIGNAL MAGNITUDE SCALING
Note that all output voltages of operational amplifers and DACs in Figure Jare restricted between O and VDD. By setting the signal ground level Vag= VDD/2 in the neuron summation circuit of Figure [(c) the equivalent signal swing range is [- VDD/2. VDD/2]. Then, by introducing a learnable scale factor for each network layer in the proposed HRAT, the equivalent signal swing range at these hardware nodes is clamped to [- s VDD/2, s- VDD/2]. Overall, HRAT divides output signals of network layer by scale factor and attempts to linearly compresses them within the output swing of operational amplifiers and DACs. During inference, these scale factors can be incorporated into the parameters of linear operation together with fused BN, so no additional signal scaling circuits are required.
4.4 BN FUSING STRATEGY
Training an MNN with or without BN layers makes a big difference. The training process for a network layer without BN operation is illustrated Figure ((a) which takes into account the above hardware restrictions. Regarding training a network layer with BN operation, we propose new BN fusing strategy to align the eestriction-induce distortion between offline training and hardware inference. Referring to Porch 2022), this new BN fusing strategy performs one linear operation. As illustrated in Figure 2(b) the fused weight first obtained using the MA statisties, and then the weight quantization noise and weight uncertainty noise are injected into the fused weight. Next, after performing the fused linear operation, the output is corrected to the unfused scale by multiplying Va² + e/y. Because the corrected output is on the unfused scale, normal BN can be performed to obtain the result yEn and update the MA statistics. Since the calculation of yBN does not consider hardware restrictions, YBN is separated into two terms: fused bias b fused and BN result without fused bias 'YEN bfused" Next, bias quantization noise is added into the b fused- Then, quantized fusedd. 'YBN bjused" and their sum are clamped to the range of [- 8. VDD/2, 8 VDD/2). respectively. After passing ReLU activation function and dividing scale factor s, the final output of HRAT for network layer is limited to [0, VDD/2].
5 BASELINE NETWORK ARCHITECTURES AND EXPERIMENTAL SETUP
We choose four-layer fully connected NN (FC-4), LeNet-5 LeCun et al. (9988. and VGG-16 Simonyan & Zisserman (203)). as our baseline architectures. We design the miniature model FC-4 for rapid verification of the proposed HRAT algorithm. FC-4 consists of three hidden layers and one classification layer. The three hidden layers have 512, 128, and 32 nodes respectively. LeNet- 5 and VGG-16 are implemented from the original papers. only slightly different to accommodate different datasets. For experiments on the CIFAR datasets, the number of features in the hidden layers of LeNet-5 is increased by a factor of 5. For fast convergence and better performance, we also implement batch normalization (BN) layers in each bascline model. Features are normalized via BN except for the fully connected layers in VGG-16. The default supply voltage of MNNs is 3V. Memristor behaviors (e.g., the non-destructive threshold voltage of 0.2V and conductance tuning range of [2μS, 20μuS) are obtained from the experimental results Yao et al. (2020). Offline weights are initialized and limited to the range of [-1,1]. After HRAT, offline-trained weights are transformed to the memristor conductance values for hardware deployment. Weight noise follows Gaussian distribution and is generated according the weight range, for example, std=0.1 means that the standard deviation of weight noise is equal to 10% of the entire weight range. We use 40 test runs statistically measure the inference performance of these baseline architectures.

---

Under review as a conference paper at ICLR 2023
6 EXPERIMENTAL RESULTS AND DISCUSSION
6.1 FC-4 AND LENET-S ON MNIST
the mean inference accuracy of FC-4 on MNIST for several combinations of weight noise and weight bitwidth. An inference accuracy of 98.60% is obtained using the software bench- mark models (i.e., floating-point or 8-bit quantized weights without MNN hardware restrictions). Mean accuracies of 96.05% and 97.82% achieved for HRAT without and with signal strength scaling, respectively. The 1.77% accuracy difference reflects the importance of performing signal magnitude scaling in HRAT. If on-chip retraining performed after HRAT, the mean accuracy rises from 97.82% to 98.40% which is very close to the software benchmark result (i.e.. 98, 60%). Al- though on-chip retraining has been demonstrated in small-scale MNNS Li al. (2018); Wang et al. 2019); Yao et al. (2020) to recover the accuracy drop caused by hardware non-idealities, such ap- proaches require complex analog backpropagation learning circuitry Krestinskaya et al. (201 b), making them unsuitable for cost-effective hard implementation. Figure plots the standard deviation (std) of inference accuracy for FC-4 on MNIST. HRAT results in an average variance of in- ference accuracy of 0.39, which drops 0.07 after performing on-chip retraining. Figure (3C) plots the mean inference accuracy of LeNet-5 on MNIST for several combinations of power supply volt- age VDD and memristor threshold voltage VTH. assuming zero weight noise and 8-hit quantized weights. Thanks to signal magnitude scaling, HRAT is insensitive to the choice of power supply voltage and memristor threshold voltage, and hence achieves a mean accuracy of 98. 84.. Figure shows very close inference accuracy of FC-4 and LeNet-5 models on MNIST with HRAT.
Figure
101
Shandsed tantetts of tafresice Aonmmny 3CC4 on 30N1S
Xeen Infrence horuray T0-4 Model MNSST
(Sertingpott wragata)
égual stcrngga soulng
beachanzzk modil (6-ta quantiall wiglls)
100
MIN [IMAT w/o
woilin)
MONN (HART to- recmininng
MANN (EILA1)
MESN [BART vit on ship oenining
06
06
16
solee mnleril tii
sed- Jedda
molee Dhydla
ander anleeti 1il
oodert 2ut
Weight Nuse
(Stundud Deviation)
Weight Quantication Blwitth
Weight Nuie:
(Staniand Deviation) Hieght Quanization Bitwish
(a) Mean inference accuracy of FC-4 on MNIST
(b) Inference variance of MNN FC-4 on MNIST
Moss Inferuee Accunacy LaStes Mode in 52539
311
KESN THECAT MON (MRAAT)
59
HRAT)
L-Net-3)
18
87.
Aocusuy Compation
45.5
pri extt)
jne ary)
gee uanvy
tired DV aer-auy
(WV cummy
indeetl iodettarsss riaa
sntesl Gibl
Andey GiM
andes de dib
pV:
(Poner Swptt Vollage Soo Thnddal Vallage of Menaiitorn
Wrighe Moke Saddand Devacing) Weigle Quaoniaction Brwidd
(c) Mean inference accuracy of MNN L on
(d) Approximate inference accuracy of FC-4 and
MNIST for several combinations of VDD and VTH
LeNet-5 models on MNIST with HRAT
Figure 3: Inference performance of MNN FC-4 and LeNet-5 models on MNIST.
7

---

Under review as a conference paper at ICLR 2023
To validate the proposed HRAT, offline trained FC-4 and LeNet-5 models are implemented in hard- ware circuits and simulated using the Cadence Spectre tool. Fully-connected layers are implemented according to Figure [(a] (Figure 6 in Appendix for details). For convolutional layers, each filter kernel is implemented by sub-circuit similar to a fully-connected layer. The filter sub-cireuit is in- stantiated multiple times, so all neuron convolutions of layer are computed simultaneously. ReLU activation and max pooling are also implemented by dedicated analog circuits. BN fused weights and biases, along with scale parameters obtained from HRAT are transformed to memristor con- ductance values, DAC outputs, and amplifier gains in these analog circuits. To reduce hardware simulation time, we use macro model of operational amplifiers to capture realistic behaviors, such as limited output swings, finite voltage gain, limited bandwidth, etc. Then, output voltages of each neuron obtained from hardware circuit simulations are compared with their corresponding offline HRAT results. We find hardware simulation matches with offline HRAT. (Appendix A.4 for details)
6.2 VGG-16 ON CIFAR DATASETS
Figure 4(a) plots the mean inference accuracy of VGG-16 on CIFAR-100 for several combinations of weight noise and weight bitwidth. The software benchmark models (i.e., floating-point weights or 8-bit quantized weights) achieve inference accuracies of 68.59% and 66.27%, respectively. Note that software benchmark models are not affected by hardware non-idealities. At a weight noise level of std 0.025, HRAT with 6-bit quantized weight leads to the highest accuracy of 62.85%. When the weight noise level is std = 0.05, HRAT with 4-bit quantized weight leads to the highest accuracy of 57.94% which is only 8.33% lower than the software benchmark model (i.e., 66.27%). Note std=0.05 means that the standard deviation of Gaussian weight noise is equal to 5% of the entire weight range, this weight noise level is significant. These results demonstrate that HRAT is robust to weight noise disturbance. Figure 4(a) also reveals that, for given weight noise level, there is an optimal weight bitwidth to balance the trade-off between noise immunity and expressiveness of MNN. Lower bitwidth means higher quantization noise is injected into weight during training, thus exhibiting stronger noise immunity during inference. However, lower bitwidth is not always better, because the expressive power of the model is limited Yoon et al. (2022). This explains why the to be lower at stronger noise levels. Furthermore, Figure 4(a) shows that if the signal magnitude scaling is not performed in HRAT, training cannot converge at all. Compared to Figure [(a)] where a 1.77% drop in accuracy for FC-4 trained by HRAT without signal magnitude scaling. we can see that signal magnitude scaling is more critical and indispensable in large-scale MNNs. As shown in Figure 4(a) if VGG-16 is retrained on-chip after HRAT, the best accuracy rises to 65.93% and 65. 55.88% for the weight noise levels of std=0.025 and std=0.05, respectively. Both accuracies are close to the software benchmark result (i.e., 66.27%) Figure H[aaalso shows that the optimal weight bitwidth for on-chip retraining is 8. This is because on-chip retraining is done individually for each deployed MNN, instead of statistically like HRAT. Therefore, for each VGG-16 model under on-chip retraining, weight uncertainty caused by memristor variations becomes deterministic rather than stochastic. As a result, when training model with deterministic weight uncertainty, the higher the weight bitwidth, the better the retraining result.
optimal weight bitwidth in Figure4(aj
Figure 4bb plots the variance of inference accuracy for V6G- on CIFAR-100. As lower bitwidth is more robust to noise disturbance, HRAT with lower bitwidth leads to less variance in inference accuracy. For HRAT with on-chip retraining, since retraining is performed individually with deter- ministic weight uncertainty, higher bitwidth results in less variance in inference accuracy.
The effect of DAC resolution on HRAT is simulated and depicted in Figure44(c) and 4(d) where the performance of software benchmark models is also plotted for comparison. Both figures demonstrate that the use of inite-resolution of DACs has big impact on inference. If an ideal DAC (i.e., infinitely small resolution) is used to generate bias, an increase in weight noise reduces the inference accuracy. For a fixed weight noise level, the reduction in DAC resolution significantly deteriorates inference. Figure ([e shows that very close results can be obtained using 14-bit or ideal DACs (i.e., infinitely small resolution) for bias generation. To maintain high inference accuracy, DAC resolution for VGG-16 on CIFAR-10 10 and CIFAR- needs to be no lower than 9 or 12 bits, respectively.
To investigate the effect of using learnable scale factors on HRAT, Figure 4() plots the inference accuracy curves of VGG-16 over 500 epochs on CIFAR-100. Compared with simulation curves using fixed scale factors in HRAT, the learnable scale factors help VGG-16 16 to converge with 2.42% and 4.71 % higher accuracy for ideal DACs and 11t DACs, respectively.
8

---

Under review as a conference paper at ICLR 2023
Most Infeere Nocosey of Vo 18 Model in CIPAR 100
beaderack (Srucagentc wriglas)
Rasfall Dolscn d'eleee focemn VDD0 CIPVO
I-Oou beadeaak (nudal (H4B gueetian vrigles
5005 USBAT ca-dip miraisiaa!
00
HONK (ITRAT)
wis wainn)
70
(IRRT ouillp mereking)
06
-0
10
selecd) doathe
(eS) yai
eviecc (00) shil told
day yha
sullers (5) badt
tolle (0) 4ab
Sed
order ood)
selee 12
Sellugf 80j
Weigh Nowe (Standed Deviation) + Weight Quannination Brrwithh
Weight Nosne
(Stund Deviation)
Wigglt
Quaationtion Bitwiith
(a) Mean accuracy of VGG-16 on CIFAR-100
(b) Inference variance of VGG-16 on 100
Mon Infermmre Ansuney of VCC- in Modil CIEAR-L0
Moon Inferczee Accuray at VOG Model
00
00
Adboss lendceark teccil (lacctppccat widde
lenekancck soold
-000 ((BAA)) sone 6. akkal DMC be graneike)
20
-100 (SILAT, wiih wntes and 2.777, Ieaal DMC Se lose prosecte))
-1888 (MEATT weight onta 2,00 DAC bou gramscio))
NIN5 (SILATT might anter 316,
true
M0N (HILAT: maght scec Duzts abod DAC Tea gerrsinn)
peernies)
MBN (HILAT.
D1520 5-12 DAC his gesratioo)
Whight Quancieation Hivvilth
Weight Quaatization Bltxidt
(c) VGG- 16 accuracy vs. bitwidth on CIFAR-10 10
(d) VGG- 16 accuracy vs. bitwidth on CIFAR-100
12
12
3
720
12
-2
IRLCT 34D0
16
12
34
12
303 Epoch
301
DAC Tewolation
DAC Tomalation
Epoch
(c) Effect of DAC resolution on HRAT
(f) Effect of learnable scale factor 8 on HRAT
Figure 4: Inference performance of MNN VGG-16 models on CIFAR datasets.
7 CONCLUSION
training (HRAT) for memristor neural networks (MNNs) to consider non-negligible hardware restrictions. HRAT integrates various hardware restrictions, adopts a new BN fusing strategy, and dynamically adjusts signal magnitude to avoid distortion. The simulation results of MNN hardware implementation match well with HRAT results, validating that HRAT successfully mimics the realistic behavior and hardware restrictions of MNNs during offline training. Experimental results also demonstrate that HRAT can lead to state-of of-e-of-theear MNNs with- out performing prohibitively expensive and time-consuming on-chip retraining, enabling low-cost high-performance MNNs for large-scale commercialization of neuromorphic computing systems.
We propose

---

Under review as a conference paper at ICLR 2023
REFERENCES
Julian Büchel, Fynn Faber, and Dylan Muir. Network insensitivity to parameter noise via adver- sarial regularization. In ICLR, 2022.
Di Gao, Grace Li Zhang, Xunzhao Yin, Bing Li, Ulf Schlichtmann, and Cheng Zhuo Reli- able memristor-based neuromorphic design using variation-aa defect-aware training. 2021 IEEF/ACM International Conference On Computer Aided Design (ICCAD), pp. 1-9. IEEE, 2021.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In JCML, 2015.
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arite artt--rriiitei-onlyyy inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2704-2713 2018.
Sung Hyun Jo, Ting Chang, Idongesit Ebong, Bhavitavya B Bhadviya, Pinaki Mazumder, and Wei Lu. Nanoscale memristor device as synapse in neuromorphic systems. Nano letters, 10(4): 1301, 2010.
Jim Karki. Understanding operational amplifier specifications. Technical report, Texas Instruments, July 2021.
Olga Krestinskaya, Khaled Nabil Salama, and Alex Pappachen James. Analog backpropagation learning circuits for memristive crossbar neural networks. In 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1-5, IEEE, 2018a.
Olga Krestinskaya, Khaled Nabil Salama, and Alex Pappachen James. Learning memristive neu- ral network architectures using analog backpropagation circuits. IEEE Transactions Circuits and Systems I: Regular Papers, 66(2):719-733 2018b.
Olga Krestinskaya, Alex Pappachen James, and Leon Ong Chua. Neuromemristive circuits for edge computing: A review. IEEE transactions on neural networks and learning systems, (1(1):4-23, 2019.
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08344 2018.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, 86(11):2278-23) 2324,, 1998.
Can Li, Daniel Belkin, Yunning Li, Peng Yan, Miao Hu, Ning Ge, Hao Jiang, Eric Montgomery, Peng Lin, Zhongrui Wang, et al. Efficient and self-adaptive in-situ learning in multilayer memris- neural networks. Nature communications, 9(1):1-8, 2018.
Beiye Liu, Hai Li, Yiran Chen, Xin Li, Qing Wu, and Tingwen Huang. Vortex: Variation-aware training for memristor x-bar. In Proceedingy of the 52nd Annual Design Automation Conference, pp. 1-6, 2015.
Ruibin Mao, Bo Wen, Mingrui Jiang, Jiezhi Chen, and Can Li. Experimeentally-aaidaae crossbar model for defect-aware training of neural networks. IEEE Transactions on Circuits and Systems II: Express Briefs, 69(5):2468-244 24722 2022.
Alan F Murray and Peter J Edwards. Enhanced mip performance and fault tolerance resulting from synaptic weight noise during training. IEEE Transactions on neural networks, 5(5):792-802, 1994.
Mirko Prezioso. Farnood Merrikh- Merrikh-Bayat, BD Hoskins, Gina C Adam, Konstantin K Likharev, and Dmitri B Strukov. Training and operation of an integrated neuromorphic network based on metal- oxide memristors. Nature, (6150))))) 2015.
PyTorch. Pytorch qat module. ht http:////....a.o.oooo/////////// 09-22.
http:////////oeeee///oooouuuo/// 2022. Accessed: 2022-
10

---

Under review as a conference paper at ICLR 2023
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
Weier Wan, Rajkumar Kubendran, Clemens Schaefer, Sukru Bure Eryilmaz, Wenqiang Zhang, Dabin Wu, Stephen Deiss, Priyanka Raina, He Qian, Bin Gao, et al. A compute- -nmputy----mmmory chip based resistive random-access memory. Nature, 608(7923).504-512, 2022.
Zhongrui Wang, Can Li, Peng Lin, Mingyi Rao, Yongyang Nie, Wenhao Song, Qinru Qiu, Yunning Li, Peng Yan, John Paul Strachan, al. In situ training of feed-forward and recurrent convolu- tional memristor networks. Nature Machine Intelligence, 1(9):434-442, 2019.
Xiaobing Yan, Lei Zhang, Huawei Chen, Xiaoyan Li, Jingjuan Wang, Qi Liu, Chao Lu, Jingsheng Chen, Huaqiang Wu, and Peng Zhou. Graphene oxide quantum dots memristors with progres- sive conduction tuning for artificial synaptic learning. Advanced Functional Materials, 114(28); 1803728, 2018.
Xiaobing Yan, Hui Li, Lei Zhang. Chao Lu, Jianhui Zhao, Zhenyu Zhou, Hong Wang, Jingjuan Wang, Xiaoyan Li, Yifei Pei, et al. Density effects of graphene oxide quantum dots on character- istics zrO, 5hf0. 5o2 film memristors. Applied Physics Letters, 114(16): 162906, 2019.
Xiaoxuan Yang, Syrine Belakaria, Biresh Kumar Joardar, Huanrui Yang, Janardhan Rao Doppa, Partha Pratim Pande, Krishnendu Chakrabarty, and Hai Helen Li. Multi-objective optimization of reram crossbars for robust dnn inferencing under stochastic noise. In 2021 IEEE/ACM Interna- tional Conference On Computer Aided Design (ICCAD). pp. 1-9. IEEE, 2021.
Peng Yao, Huaqiang Wu, Bin Gao, Jianshi Tang, Qingtian Zhang, Wenqiang Zhang, J Joshua Yang, and He Qian. Fully hardware-implemented memristor convolutional neural network. Nature, 577 (7792):641-666, 2020.
Jaehong Yoon, Geon Park, Wonyong Jeong, and Sung Ju Hwang. Bitwidth heterogeneous federated learning with progressive weight dequantization. arXiv preprint arXiv:2202. 1453, 2022.
Ying Zhu, Grace Li Zhang, Tianchen Wang, Bing Li, Yiyu Shi, Tsung-Y Yi Ho, and Ulf Schlichtmann. Statistical training for neuromorphic computing using memristor-based crossbars considering pro- cess variations and noise. In 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1590-1593. IEEE, 2020.
A APPENDIX
A.1 WEIGHT QUANTIZATION
To simplify the conductance programming of memristor crossbars, weights are quantized into finite number of states. Low-precision weight makes the model insensitive to deployment noise. Lower bitwidth means higher quantization noise is injected weight during training. Because weight quantization is deterministic, only uncertainty noise causes weight distortion after on-chip deployment. Lower bitwidth does not always lead to better results, since it limits the representative power of neural networks.
In this work, uniformly quantize weights using the following quantization function:
W A AA)
Q(W: bic) Cow
where A a trainable parameter of the quantization function to denote the quantization step size, L-T denotes the rounding operation, bw is given bitwidth, and C.w(.)) clamps quantized values into the range of [-(2hw-1 weight and quantization function cannot be trained through gradient-based optimizers. To address this issue, Eq. (1) is converted into a differentiable form:
1). A, (2 - 1) A]. Due to the non-differentiation of rounding operation,
Q W: bw) = Cow (( + 8rounding) A)
(2)

---

Under review as a conference paper at ICLR 2023
where the rounding noise Grounding W noise, so the gradient of the non-differential Grounding is not calculated during back proppppppggttion. Estimator (STE) trick. Thus, the gradient of W
X. We assume that Grounding is an independent
A
This method is also known as the Straight-Thre and A is obtained through the back-propan proppaaonnattion algorithm. Therefore, we can easily train memristor neural networks with gradient-based optimizers.
A.2 EXISTING BN FUSING STRATEGIES
linear layer with BN is expressed as:
Wx +
(3)
and
148 Vog + + 3
UBN 7
(4)
Where æ is the input, W is the weight parameter, b the bias term, y is the learnable scale factor of BN, ,3 is the bias term of BN, μg mean output of the current batch, and o is the variance of the current batch. During inference, MA statistics and o² are used for normalization, For full precision software models, BN is fused into the linear operation via combining (3) and (4) Then, the inference can be calculated with fused parameters (fused weight and fused bias) as:
7 UBN V02 4E
Y
Wx 3+
(b 11)
(5)
V02 +
where the fused weight W fused Va2+c W. and the fused bias bossed B Var+e p.).
During training, existing BN fusing strategies are illustrated in Figure S Figure 5(a) shows that
instead of fusing BN during training. BN is fused during deployment according to Eq. ( Wan ct al. (2022) train their MNN in this way. Figure ((b) [(c) S(d) illustrate the training schemes for quantization-wwar training (QAT). They assume that the quantization effect of fused bias can be ignored, so only consider the distortion caused by the fused weight quantization. During inference, only fused linear operation needs to be performed using fused weight and bias obtained from the MA statistics.
Output Yas
Fused Blas
Current fused scale
Vie w/o Fused Bias
Current fused scale
Output Yaa
Output Yaa
Scale
Yie w/o Fused Bias
Batch Norm
Update MA
Add
fused
Y
Mean MA Var
Output
Yus Fused Bias
Current fused scale
Fused Linear
Add
Bias
Current fused scale
Fused Blas
yw/o Bias
Batch Norm
Fused Linear
Quant
Current
MA fused scale
Scale
y
Quant
Fused Weight
Masw/o Bias
Blas
Add
Current fused scale Fused Weight
fused
Fused
Linear
Fuse BN
Mean & MA Var
Update MA
Quant
Update
f MA fused scale Weight
Moan MA Var
Mean & Var
Mean Var
Quant
y
Mean & MA Var
Linear
Linear
Input
Weight
Input Weight Bias
Weight Blas
Input
Weight
(a) Wo fusing BN
(b) Jacob et al. (2018)
(c) Krishnamoorthi (0018)
(d) PyTorch
Figure 5: Existing BN fusing strategies in literature.
12

---

Under review as a conference paper at ICLR 2023
A.3
FORMULATIONS OF HRAT
A.3.1
TRAINING WITHOUT BN
As illustrated in Fig. ( training fully-connected layer without BN is expressed as:
VeLUUCC, ((O(W; (b))) + W noise ) Cvr (x) + 2(b)))
y
(6)
where Cv. (-) clamps input to the scaled signal range of [-Ve, Vel, Ve generated from a Gaussian distribution to model memristor variations, $ is learnable scale factor. @(b) quantizes b according to the bitwidth of DAC and the scaled signal range of [-Vo, Vel. In fact, we use uniform noise instead of quantization noise to introduce randomness.
VDDD22, 2, W noise is
After training, weight is divided by to obtain scaled weight. Since memristor usually has certain conductance range to tune (e.g., (2 20μS] Yao et al. (2020), ) scaled weight is transformed to memristor conductance by multiplying a converting factor s, as:
QW bw)
W deploy Sc
Bias is divided by to fuse the scale factor.
Q(b)
deploy
(8)
8
Then, can program W deploy to memristor crossbars and program deepooy to DACs. Since weight is transformed to the memristor conductance range, an additional signal amplification factor 1/s. should be taken into account. Therefore, the inference is expressed as:
y ReI.U(CvDD/2 (CvDD/2 (1/8e (Wdeploy W Wnoise Cyoise)CVra (x:) beeepoy))
A.3.2 TRAINING WITH THE PROPOSED BN FUSING STRATEGY
Figure((b) fully-connecteds layer trained with the proposed BN fusing strategy. Deployment noise is applied on fused weight, through introducing factor T
Vož+c' we fuse weight as:
W fused
W
(10)
In order to apply a normal BN after fused linear operation, we correct the output to original scale by dividing Vor and then the bias term is added to the output of original scale. Before applying BN, have:
Q fused: bw) + W Inoiss))Vrrn
(x) + Q(b)
(11)
The result VBN is obtained by applying a normal BN according to Eq. (4). The above steps only address fused weight, other hardware restrictions are still not integrated. order quantize fused bias and clamp the output range, we separate yBN to fused bias bfused and ŪBN.
bused B B+
bb
4)
(12)
va2
+
OBN JBN buused
(13)
Then, other hardware restrictions are applied based on the illustration in Figure 2(b) After training, BN and s are fused into previous linear operation. So we have
2
Vo W: bw)
W Weeploy 8
(14)
Q (bsuedd)
bdeploy
(15)
Finally, perform hardware deployment, similar the steps in Section A.3.1
13

---

Under review as a conference paper at ICLR 2023
A.4 CIRCUIT SIMULATION RESULTS
Figure shows the hardware operation of a fully-connected layer. The input signal is clamped to avoid excessive voltage across memristors. Linear operation is executed via memristor crossbar, where weights are programmed into memristor conductance values. The output of linear operation, along with bias voltages from DACs, are passed neuron summation circuit and activation circuit for processing. Table [ summarizes the difference in results between offline HRAT and hardware circuit simulation for all nodes across all layers of LeNet-5. The second column lists the number of outputs at each layer. We use the MNIST dataset with an input dimension of 28 x 28 x 1 for this experiment. Hence, the output dimension of the first convolutional layer is 24 24 x 6. The last three columns summarize the maximum, mean, and standard deviation of the error values for each network layer. demonstrates that circuit simulation results closely match the offline HRAT results.
Output IVDD//2, VDD]
ReLU Activation Circuit
[o, VDDJ
Blas voltages from
[o, VDD
Linear operation memristor crossbar, where weight are programmed Into
uncertaintiess
M VustNuu
Input Clamp Circuit
Input
Figure 6: Schematic of fully-connected layer for memristor neural network.
Table 1: All-layer output comparison of LeNet-5 between offline HRAT and hardware simulation.
network layer
# of outputs
Maximum Error
Mean error
Error variance std
conv. layer 3456
1.40
10
1.24
10
1.85 x 10
conv. layer
1024
3.56 10 10-3
3.97
10-1
5.82 10
fc layer
fc layer 120
2.29 x 10
3.33
10
5.15 x 10 4
84
4.10
10" 10-3
7.83
10
1.07 10
fc layer
10
8,00 x 10
1.59
10
2.73 x 10
A.5 SIMULATION RESULTS OF VGG-16 ON CIFAR DATASETS
Table 2 lists the mean inference accuracy of VGG-16 on CIFAR datasets, when four weight noise levels (std=0, std=0.025, std=0.5, and std=0.075) are present. VGG-16 is either trained by HRAT, or by HRAT followed by on-chip retraining. The optimal weight bitwidth is selected to report the highest inference accuracy for each case. The inference accuracy of HRAT slowly drops as the weight noise level increases. At a strong weight noise level (std=0.05), HRAT achieves a high accuracy of 87. 37.24% for CIFAR-10 dataset. If on-chip retraining is applied after HRAT, the inference accuracy is close to the baseline (i.e., weight noise std=0) results.
Table 2:
Performance comparison among various weight noise levels for VGG-16 on CIFAR datasets, assuming very high-resolution DACs used for bias generation.
CIFAR-10
CIFAR-100
Weight Noise (std)
HRAT
HRAT+retrain
Diff
HRAT
HRAT+retrain
Diff
0.0 (baseline)
91.48
91.48
0.0
66.27
66.27
0.0
0.025
90.33
91.30
0.97
62.85
65.93
3.08
0.05
87.24
90.72
3.48
57.94
65.88
7.94
0.075
84.70
90.70
6.00
55.64
66.10
10.46
14


Reviews:
[{"review_id": "ccLx82xgr3r", "reviewer": "Reviewer_dKMU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce a batch normalization fusing strategy to align the distortion between offline training and hardware inference for Memristor neural networks (MNNs). Through a learnable scaling factor, the proposed training algorithm can adjust the magnitude of input signal adaptively. The evaluation is conducted on various models and datasets. The comparison shows that the signal strength scaling plays an important role in recovering the accuracy.", "summary_of_the_review": "As mentioned above, more clarification and evaluation are required to make this paper more convincing.", "strength_and_weaknesses": "Pros:\n\n++ This paper targets an interesting topic and could benefits the community of efficient community.\n\n++ The experiments on various models and datasets show the effectiveness.\n\nCons:\n\n-- The proposed hardware-restriction-aware training seems to adopt the major framework of previous work [1,2,3], such as non-destructive threshold voltage of memristors and parameter-noise-aware training. It is difficult to tell apart the contribution of the authors and the preliminaries.\n\n-- MNNs can achieve better energy efficiency, however, there is no relevant evaluation of the trade-offs between accuracy and efficiency compared with other efficient networks in the experiment.\n\n-- Lack of comparison with other SOTA basslines of MNNs, such as [4,5], in Figure 3 and 4. \n\n-- Although the author highlights the design of BN fusing, it is difficult to see its superiority besides signal strength scaling.\n\n\n[1]. Yan et al. Density effects of graphene oxide quantum dots on characteristics of zr0. 5hf0. 5o2 film memristors. Applied Physics Letters, 114(16):162906, 2019.\n\n[2]. Jo et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano letters, 10(4):1297– 1301, 2010.\n\n[3]. Wan et al. A compute-in-memory chip based on resistive random-access memory. Nature, 608(7923):504–512, 2022.\n\n[4]. Yao et al. Fully hardware-implemented memristor convolutional neural network. Nature, 577 (7792):641–646, 2020.\n\n[5]. Wang et al. In situ training of feed-forward and recurrent convolutional memristor networks. Nature Machine Intelligence, 1(9):434–442, 2019.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "UJlp9TNRyb", "reviewer": "Reviewer_T82t", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper advocates for an offline training mechanism for memristor devices and proposes a customized training approach (employing noise-aware training and batch normalization) for such devices. The results across a set of small-sized NNs show that using BN and this hardware-restriction-aware training improves the final model accuracy.", "summary_of_the_review": "The paper shows the feasibility of employing noise-aware/hardware-limitation-aware training for memristor circuits and show limited set of results for few small-sized NNs. While I agree that exploring non-conventional devices for future generation of NN accelerators is crucial, but I am not convinced that the paper brings any additional insights to the existing literature in terms of training under such hardware constraints. ", "strength_and_weaknesses": "**Strength**\n\n- The paper nicely reviews some of the prior work on memristor training and assembles the existing methods into an end-to-end framework for training.\n\n- Results across few small-sized NNs show that the suggested training approach could increase the final model performance.\n\n\n**Weaknesses**\n\n- While the paper advocates for an offline-training method, the trade-offs between an online and offline (or possibly hybrid approach) is not neither well-studied, nor justified.\n\n- While there are few work that have explored and analyzed memristor training for semi-large models, the paper lacks showing the generality and scalability of their solution for these models.\n\n- Using the same noise distribution during training and deployment is not practical. This is because not only each circuit has its own non-idealities but also these non-idealities change as circuit ages.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "NldrrJblaV", "reviewer": "Reviewer_Xqdg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to adapt the neural network training to the non-idealities imposed by the memristor crossbars. The paper proposes hardware-Restriction-Aware Training (HRAT) which takes into account the finite-resolution of DACs, process variation of memristor devices, etc.\nFor each of the non-idealities, the paper proposes a counterpart layer to account for the non-ideality to be considered in the weights during inference.\nThe paper provides experimentation on FC-4 and LeNet-5 on MNIST, VGG-16 on CIFAR to show that it can better mimic the realistic behavior of the hardware compared to naive training.", "summary_of_the_review": "This paper aims to adapt the neural network training to the non-idealities imposed by the memristor crossbars. The paper proposes hardware-Restriction-Aware Training (HRAT) which takes into account the finite-resolution of DACs, process variation of memristor devices, etc. For each of the non-idealities, the paper proposes a counterpart layer to account for the non-ideality to be considered in the weights during inference. The paper provides experimentation on FC-4 and LeNet-5 on MNIST, VGG-16 on CIFAR to show that it can better mimic the realistic behavior of the hardware compared to naive training.\n\nThe paper definitely works on an important topic that aims to bridge the gap between the simulation and the real hardware implementation of MNNs. The research direction of the paper has the potential to enable a significantly more efficient neural execution.\n\nAnother point is that the paper does a fine job describing the problem. Considering the fact that many people in the ML community might not be familiar with the technological advances in the hardware, this paper can serve a good starting point.\n\nHowever, the paper has several issues. First of all, the paper seems to be ignoring the huge body of works in the computer architecture and design automation community. The community has looked into various ways to mitigate the non-idealities in the hardware through similar approaches. The paper currently compares against a naive baseline. However, it should compare against some of the relevant works cited below. In fact, it seems that the paper also misses out on some relevant works including:\n* Shafiee, Ali, et al. \"ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars.\" ACM SIGARCH Computer Architecture News 44.3 (2016): 14-26.\n* Ghodrati, Soroush, et al. \"Mixed-Signal Charge-Domain Acceleration of Deep Neural Networks through Interleaved Bit-Partitioned Arithmetic.\" Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques. 2020.\n\nEven if we ignore the fact that the paper does not make any novel contributions, the paper only experiments on small scale networks which make it far from generalizable. ", "strength_and_weaknesses": "+ Works on an important problem of closing the gap between the ideal vs memristor-based neural execution.\n+ Does a fine job explaining the problem\n\n- No novelty.\n- Experiments are only provided in small scale networks which do not seem representative.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HIE5K8sHQd", "reviewer": "Reviewer_SYfm", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a new training methodology, HRAT, for neural networks to be deployed on Memristor (RRAM) based hardware. The presented methodology accounts for non-idealities such as Quantization error for weights and limited output swing of operational amplifier circuits. Additionally, the methodology also covers training networks with fused batch normalization (fuse BN step with previous linear operation). The authors account for additional errors potentially introduced in different steps involved in their fused BN approach. Lastly, the authors present results showing models trained with HRAT demonstrate high accuracy despite hardware limitations for small scale networks. They also highlight the scalability of HRAT to larger networks showing good inference accuracy despite large quantization noise ratios. They also highlight the effectiveness of the scale factor parameter integral to HRAT methodology that enables high inference accuracy.", "summary_of_the_review": "The authors have done a good job explaining the key ideas and running experiments to demonstrate their effectiveness. Adding a few more baseline points to the comparisons might help the authors better justify the impact of their proposed technique. The trainable s-factor parameter is very useful as it allows authors to selectively tune out the noisiest blocks in the hardware implementation of their design. The key hardware knowledge used by the authors to model all sources of error in MNNs is another useful contribution of the paper. ", "strength_and_weaknesses": "The authors have done a good job of explaining the context of the problem (Section 3) and the prior art (Section 2). Additionally, Section 6 presents a useful set of experimental results that help understand the proposed methodology and its limitations (on-chip training results included). Especially, Section 6.2 presents useful explanation of the analysis on large network which helps explain the inference accuracy trends reported in Figure 3 and 4. \n\nIn Section 6.1 the authors mention that HRAT simulated results match the voltages measured in memristor hardware simulations to a good accuracy. While Table 1 in Appendix A.4 captures this correlation, it would be helpful if the authors added relative accuracy metrics instead of absolute. Depending on the layer in question and features involved the same absolute magnitude of error can have a large range of impact, please add % error stats instead. Also, can the authors explain why the results observed for FC-4 and LeNet-5 in terms of HRAT to HW correlation will hold for larger networks? \n\nFigure 3 highlights the superior training accuracy of HRAT and its variants, it would be useful to have a comparison against other training approaches for these data points as it would reveal the additional accuracy improvement enabled by HRAT. \n\nSection 6.2, the authors mention that std=0.05 is a significant noise range for which HRAT suffers an ~8% reduction on accuracy compared to software trained model. This assertion needs two qualifiers, can the authors add more details justifying the claim that 5% is a typical/large value for the observed noise and secondly, 8% loss in accuracy can be significant depending on the actual application involved. Specially considering that the alternative of on-chip retraining though costly can give very high accuracy with low variance. It would again benefit the discussion to add other approaches for MNN training that don’t involve on-chip training and highlighting HRAT’s accuracy improvement over them.\n\nSection 6.2 explains that limited DAC resolution plays an important role in dictating the accuracy loss in HRAT. Trainable s-factor scales down this noise significant and allows selecting the sweet spot of weight quantization for given noise levels for best accuracy. Basic question for the authors, does the limited DAC resolution scenario have no-impact in the case of on-chip training? If it does and HRAT still achieves comparable accuracy it might be helpful to add that in Figure 4. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject