There are several 'INTRODUCTIONs' below. Would it be better to have sub-lists under 'INTRODUCTION', or is it better to write without sub-lists?

INTRODUCTION
Deep neural networks (DNNs) have achieved impressive performance in
various applications such as image recognition [1], natural language processing
[2], and VR/AR devices [32]. Recent studies have attempted to utilize DNN
models on edge devices. For efficient edge computing, it is necessary to reduce
the amount of memory and computation of the model as well as process model
inference efficiently.
For efficient edge computing, two conditions must be satisfied. The first is
that DNN models need to be lightweight so that they can be loaded into a limited
memory. The second is that it is necessary to quickly process the inference of
the deep-learning model for user convenience.
Various techniques are being studied to reduce the overhead of deeplearning
models, one of which is deep-learning quantization algorithms. A
quantization algorithm is a technique that maps the parameters of a deep
learning model with floating-point precision to low precision to reduce the
amount of memory and computation [3]–[14],[24],[29],[30].
In addition, to achieve fast inference in deep learning models, research on
dedicated hardware (HW) accelerators that efficiently process the multiplyand-
accumulate (MAC) operations of deep-learning algorithms through
parallel processing is being conducted based on various HW platforms, such as
FPGA and ASIC [15]–[27].
However, prior quantization algorithms and accelerators have not been cooptimized,
resulting in inefficient operations [11],[12]. Some sections of the
quantization process still require floating-point computation, which requires
complex high-precision computational logic when processed by accelerators.
Furthermore, to store all the intermediate computational results, the resources
required to design the accelerator have increased.
To overcome these problems, I propose a new quantization algorithm called
element-wise partial product quantization, along with a new HW accelerator
optimized for this technique. The size of the output partial product activation
results is reduced by applying quantization preferentially to element-wise
computational results. As the size of the intermediate values decreases, the
accelerator, which stores these values, can handle them with only a small
amount of resources. I have also replaced all high-precision computational
processes in the quantization algorithm with dyadic numbers so that they can
be easily processed in HW accelerators.
Experimental results show that DNN inference can be performed with
efficient integer and bit operations with a loss in accuracy of only 3%–4% and
reductions in the number of flip-flops, look-up tables (LUTs), and
inputs/outputs (I/Os) required for the accelerator design of 13.5%, 7.5%, and
15%, respectively.

Chapter 1. INTRODUCTION
1. Dissertation introduction
Over the past decade, artificial neural network-based deep learning
technology has been successfully applied in diverse fields. However, as
networks become deeper and broader, real-world solutions require
consideration of the computational cost. For example, a representative
autoregressive language model, GPT-4 [1], increases the number of
parameters to 175 billion, thereby significantly amplifying the computational
burden. A variety of studies on neural network compression have been
conducted with minimal performance degradation to alleviate these problems.
Through aggressive reduction of parameters to a data width of 1-bit at the
expense of considerable accuracy loss, binarized neural networks (BNNs)
demonstrate significant benefits in terms of memory footprint and
computational speed. Various studies have addressed the accuracy loss of
BNNs, such as XNOR-Net and Bi-real [2-3]. Nonetheless, there is still an
inherent limit in improving BNN performance through parameter processing or
modulation.
Knowledge distillation (KD) is a widely applicable technique for
compressing neural networks [4]. The key idea behind KD is to supervise the
student network by imitating the teacher network via soft probabilities, which
exposes more information than the class label and helps the student network
learn. KD performance is primarily determined by the different characteristics
of the teacher and student networks, such as the data widths, topologies, and
hyperparameter configuration.
Denser knowledge can be acquired as the depth of the teacher network
increases, whereas as a soft label approaches a hard label, it becomes too
taxing for the student network to emulate the teacher network owing to
insufficient capacity [5-6]. A low-bit network has been applied to the student
network to boost the KD compression efficiency [2,7-8]. However, in KD,
where harmony between the teacher and student networks is emphasized, the
quantization gap between the two networks causes negative side effects.
Furthermore, the data width parameter is an essential consideration because
the difference in performance between the two networks is closely related to
the number of recognizable classes.
Additionally, the optimizer algorithm utilized in BNN training calculates the
difference between the predicted value of the network and the ground-truth
calculated from the forward path as the loss function. The optimizer then
determines the direction and size of the weight update. However, because the
weights of BNNs are limited to -1 and 1, only 512 (2^9) cases can be
represented in a weight of 3x3. This performance limitation renders the
application of optimizers and weight update policies designed for full precision
parameters to BNNs an over-engineered approach. Currently, the fullprecision
weight-based BNN training method requires additional computation
by updating the value of the unchanged weight due to the sign function with a
high threshold. Furthermore, the frequency of weight updates is high in the
early stages of training, leading to a significant overhead for parameter
changes.
2. Dissertation contribution
In this study, gradient mismatch mitigation in a BNN and a KD composed of
a binarized student network are addressed.
Conventional constant weight clipping (CWC) causes a gradient mismatch
in BNNs because fixed clipping values cannot cope with the dynamics of
weight distribution. Although the dynamic clipping range has been suggested
as an alternative, it has enormous complexity. To address this problem,
trainable clipping values used for easing gradient mismatches were introduced.
Next, we propose a method to alleviate the capacity shortage of binarized
student networks caused by data width difference between the teacher and
student networks. Numerous studies [2,6-7] have revealed that a difference
between teacher and student networks greater than the effective range results
in insufficient knowledge transfer between them. Hence, adaptive scheduling
based on the quantization gap is required to balance the knowledge proportion
of each network. Inspired by this observation, information entropy was
developed to assess the difference between the two networks appropriately.
The growing complexity of deep learning models poses a challenge to the
training process in terms of difficulty and time required. Training time is
influenced by several factors, such as the complexity of the model, the task,
and the available hardware resources. In resource-constrained environments,
the complexity of the model learning process needs to be reduced to mitigate
this challenge. One approach to reducing the computational burden of
backpropagation and weight updating is by fixing the layers of the BNN based
on the sign flip rate (SFR). This approach accelerates the training process
while minimizing the accuracy drop.
The contributions of this study are as follows:
⚫ To mitigate the drawback of CWC that causes gradient mismatch in
BNNs, we utilize a trainable weight clipping function adaptable to
the dynamic weight distribution.
⚫ In online distillation, information entropy-based temperature
scheduling is introduced to overcome the problems caused by a
poorly trained teacher network at the beginning of learning and ii)
a capacity shortage of the student network.
⚫ During BNN training, measuring the rate of sign flips before and
after weight updates can reduce the computational load for updates
that fail to meet the threshold criteria set by the sign function in
the latent weight update process.
⚫ The effectiveness is revealed by aggregating the proposed
approaches, which can be employed in the diverse network models.
Furthermore, the binarized student network applied in the proposed
methods exhibit a top-1 test accuracy comparable to that of the
baseline CNN.
3. Outline of this dissertation
This paper is organized as follows. Section 2 provides an overview of
related studies. Section 3 presents the main algorithm flow and details of the
proposed learnable weight clipping approach. In Section 4, we introduce the
concept of information entropy distance-based knowledge distillation for
quantization gap. Section 5 describes the BNN weight updating rule using
discrete weight property. Finally, Section 6 presents the conclusions drawn
from our study.

Chapter 1. INTRODUCTION
Figure 1. Growth in the number of DNN model parameters over the year.
Deep Neural Networks (DNNs) are powerful artificial intelligence techniques
used in many fields such as image classification, natural language processing,
and autonomous systems. In the image classification, they learn a lot of weight
parameters from large input data and use this knowledge to make predictions or
classification on new data. As this process of replacing the role of neuron
through DNNs gradually develops, a large amount of data is required.
As depicted in Figure 1, the evolution in DNN models is marked by an
exponential increase in their complexity and size. This evolution is primarily
driven by the quest for higher accuracy and the ability to process increasingly
complex datasets.
The expanding size of these models poses significant challenges. Primarily,
the requirement for memory storage and computational power increases
proportionally with model size. As DNNs become deeper and more complex, the
memory needed to store weights, and the computation required for matrix
multiplications during both training and inference, escalate dramatically.
The challenge intensifies when deploying these advanced DNN models on
edge devices, such as smartphones, IoT devices, and embedded systems. These
platforms often have limited memory bandwidth and storage capacity, which
cannot be easily upgraded due to cost, space, or power consumption constraints.
Consequently, executing state-of-the-art DNN models on such resourceconstrained
devices becomes increasingly difficult, leading to performance
bottlenecks.
Therefore, to facilitate the execution of state-of-the-art DNN models on
resource-constrained devices, we propose a model compression technique,
specifically focusing on the compression of model parameters, weights. Among
the prevalent model compression methods, we employ Quantization for model
size reduction and Entropy-based Lossless Compression for data compression.
To generate data conducive to compression, we utilize both Uniform
Quantization for maintaining data uniformity and Non-uniform Quantization to
minimize accuracy loss. Additionally, we implement the Huffman coding
algorithm for data compression. Our approach aims to compress weights in a
manner that maintains model accuracy (top-1 Accuracy) within a 1% loss
3
threshold, thereby enabling the execution of DNN models on resourceconstrained
platforms.
To evaluate our compression algorithm, we compare it against INT8
Quantization and other contemporary state-of-the-art model compression
techniques. Our methodology is tested on DNN models from the ImageNet [3]
dataset, achieving an average compression ratio of 5.84, effectively
compressing the model parameters, or weights, to 6 bits. This approach
proposes a compression algorithm for efficiently executing DNN models on
resource-constrained platforms.

Chapter 1. INTRODUCTION
1.1 Overheads of Inference of Convolutional Neural Networks
The rapid adoption of the Convolutional Neural Network (CNN) has
drastically increased the demand for high computational capabilities. Since the
training and inference processes of the CNN models require the massive
transfer of weight parameters and arithmetic operations on the parameters,
computing for CNN applications shows characteristics of both massive
computational complexity and memory consumption. For example, CNN
inference on image data with a dimension of 3 x 224 x 224 using a relatively
simple AlexNet model results in a computational complexity of 0.729 GFLOPs
and a memory consumption of 0.82GB. The same task with a relatively complex
VGG-16 model results in the computational complexity of 16 GFLOPs and the
memory consumption of 1.46GB [1] [2] [3]. Mitigating the overheads of the
CNN inference tasks has been a crucial goal, especially in low-cost mobile or
embedded platforms with limited computing resources.
2
1.2 Weight Quantization for Model Compression
Weight quantization has been one of the most widely adopted model
compression techniques for relieving both overheads of CNN computation.
Weight quantization expresses weight parameters using data formats with lower
precision. For example, the most popular INT8 quantization scheme transforms
the weight parameters originally stored in the 32-bit single-precision floating
point data format (FP32) into the 8-bit integer data format (INT8). Weight
quantization reduces the memory space taken up by the weight proportionally
to the ratio of the bit-width while replacing complex floating-point operations
with simpler integer operations to mitigate the computational complexity of the
CNN inference tasks. One of the hardware-friendly quantization schemes,
called Power-of-Two (POT) quantization, further reduce the computational
complexity and improve the performance of the CNN inference tasks. The POT
quantization scheme quantizes the weights of filters into power-of-two values,
which significantly reduces the computational complexity by replacing the
multiplication operation with the bit-shift operation [4] [5] [6].
3
1.3 Processing-in-Memory Technique
For the CNN Inference workloads' heavily memory-bound aspects,
Processing-in-Memory (PIM) technique which eliminates data transfer
between the Core Side and the Memory Side has been regarded as a notable
hardware solution for the overheads of CNN Inference. Virtually all types of
memories including DRAM, SRAM, PCM, MRAM, RRAM and even those
implemented as caches in the processors have been utilized as base for PIM
technique.
1.4 Limitations of PIM and POT Quantization Techniques
Yet, both PIM and POT Quantization techniques have shortcomings to be
solved. Most PIM systems usually come with significant amount of area
overhead due to additional hardware such as buffers and ALUs for full support
of compute instructions.
Besides, POT Quantization brings about significant degree of accuracy
drop in CNN inference unless retrained for dozens of epochs, making it unfit for
accuracy sensitive use cases such as medical image classification and security
systems.
4
1.5 Research Goal
In this paper, we leverage the POT quantization to implement a novel PEC
architecture that utilizes shifters for performing multiplication operations at a
low cost. We also introduce a filter-wise POT quantization scheme that
quantizes the weight parameters to the Power-of-Two data format in filterwise
manner to better fit the random weight distribution of the convolution
layers. The filter-wise quantization scheme achieves better accuracy than the
naive model-wise Power-of-Two quantization scheme by quantizing only the
parts of the filters in the neural network.
We also study the impact of re-training on the inference accuracy of the
POT quantized CNN models to determine the sweet spot on the trade-off
between quantization degree and the accuracy.
Besides, we design the framework named Cooperative Acceleration of
CNN Inference in Multicore Processors (i.e., The Cooperative Acceleration).
5
1.6 Outline of the Paper
This paper is structures as follows: In Section II, we review the backgrounds,
limitations, and motivations for our new design in CNN Quantization, Powerof-
Two Quantization, Processing-in-Memory Techniques. In the following
Section III, we describe the key techniques of our acceleration scheme: Basic
In-Cache Processing Architecture, The Filter-Wise Quantization Scheme for
PIM, and the Cooperative Acceleration of CNN Inference. In the next Section
IV, we put detail on Data Placement Schemes for CNN parameters for improved
performance and hardware utilization. We describe the data placement schemes
in both inter-LLC-slice and intra-LLC-slice granularities. In Section V, we
design the detailed microarchitecture of the cooperative acceleration throughout
the whole cache hierarchy. In the following Section VI, we simulate the
Inferences of three CNN models under consideration on our cooperative
acceleration system, and compare its latency, energy consumption and area
consumption. Finally, in Section VII, we summarize the study and verify the
benefits brought on by the Cooperative Acceleration.
The contributions of this work are:
• We show that each convolutional layer in the CNN models show different
sensitivity towards the Power-of-Two quantization.
• We show that Power-of-Two quantization of only the parts of the convolution
filters can quantize the significant amount of weight parameters with small
accuracy drop.
• We show that the L3 cache of off-the shelf CPU can be transformed into a
massively parallel processing unit.
• We show that the filter-wise quantization scheme can improve throughput of
6
PIM-based CNN accelerators
• We show that the Cooperative Acceleration of CNN Inference with between
the Host Core and the PIM-Enabled Cache benefits the system in latency,
throughput, thermal and energy overhead aspects.
A. Abbreviations and Acronyms

 FX - Fixed Point
 POT - Power of Two
 CNN - Convolutional Neural Network
 PEC - PIM Enabled Cache
 STD - Standard Deviation
 SPU - Slice Processing Unit
 SA – Sense Amplifier.