다음은 무엇에 대해서 설명하고 있는가? 뭐라고 검색하면 좋을까?

Chapter 2. BACKGROUND
A. Model compression
Quantization [4] is a key process in deep learning, serving as an essential
approach for compressing models, particularly in the context of Deep Neural
Networks (DNNs) [5]. The purpose of quantization is to reduce the bit precision
of elements such as weights and activations, thereby preserving the efficiency
of the model while substantially decreasing its demands on computation and
memory. This is particularly beneficial for implementing DNNs on devices with
resource-constrained platforms, such as smartphones and edge devices.
The process of quantization involves transforming parameters from higherprecision
formats (for example, 32-bit floating points) to more compact forms
(such as 8-bit or 16-bit integers), resulting in a more efficient model that
benefits from a reduced memory footprint and enhanced processing speeds. In
addition to these benefits, quantization can also contribute to lower energy
consumption since operations with lower precision typically require less power.
There are various strategies for quantization, among which uniform and nonuniform
quantization are the most common.
5
A.1 Uniform quantization.
Among the various compression techniques, uniform quantization is
especially significant for its straightforward approach to reducing the storage
and computational demands of DNNs. It operates by uniformly lowering the bitwidth
of the parameters across the network, compressing the original floatingpoint
precision into a more compact integer representation.
This uniformity is achieved by applying a single scale factor to convert all
high-precision weights and activations into lower-precision formats (32-bit
floating point numbers to 8-bit or 16-bit integers) [6], thereby reducing the
model's size and simplifying its computations.
Figure 2. An example of uniform quantization.
Figure 2. shows an example of uniform quantization. Looking at the
expression range of the value after conversion, it is configured at uniform
6
intervals.
These can simplify the computation and make it hardware-friendly. Uniform
quantization is the increased efficiency in memory usage. By storing uniform
quantized parameters in a lower bit-width format, the memory footprint of a
DNN can be significantly reduced, making the model more suitable for devices
with limited memory capacity. Another benefit is the potential for faster
computational performance. Since low-precision arithmetic operations can be
executed more rapidly, inference times can be considerably shortened, which is
especially beneficial on hardware optimized for low-precision calculations.
7
A.2 Non-uniform quantization.
Figure 3. An example of non-uniform quantization.
Figure 3. shows an example of non-uniform quantization. Unlike uniform
quantization, non-uniform quantization does not apply a single scale factor
across all network parameters. Instead, it allows for different quantization levels
for various parameters to align closely with their specific statistical distribution.
This aligned quantization is important in maintaining the integrity of the original
data distribution, which is often skewed or follows a non-linear scale.
The process of non-uniform quantization involves mapping the highprecision
parameters to a discrete set of values that are not uniformly spaced.
This mapping is sensitive to the underlying data distribution, ensuring that more
prevalent values have finer quantization levels, thus minimizing information loss.
For instance, areas of a neural network where data is densely packed can have
more quantization levels assigned, allowing for a more precise representation
8
of those values.
The advantage of this method is its ability to minimize quantization loss,
particularly in data distributions that are not uniform. By adjusting the
quantization levels to the specific characteristics of the data, non-uniform
quantization can significantly reduce the error introduced during the
compression process. This adjusted approach is especially beneficial for
parameters that have a wide dynamic range or for models where preserving the
exact distribution of the weights and activations is critical for maintaining
performance.
In practice, non-uniform quantization can be more complex to implement
than uniform quantization. It often requires a deeper analysis of the data
distribution within the network and the development of a custom quantization
scheme that can adapt to the unique properties of the data. However, the
potential benefits in terms of reduced quantization error and preserved model
accuracy make it a valuable technique for model compression, particularly in
scenarios where the performance of the quantized model is most important. So,
this paper applies two quantization methods by employing these advantages of
these methods.
This paper examines both symmetric (scale transform) and asymmetric
(affine transform) approaches which assess the data distribution for ease of
compression.
Uniform Quantization can be divided into two ways: Affine transform and
Scale transform [6]. Affine transform is also called asymmetric, and scale
transform is called symmetric.
9
Figure 4. Affine quantization(left) and Scale quantization(right) mapping of
real values to int 8.
Figure 4. shows two methods of affine quantization (left) and scale
quantization (right). First, Affine transform has the following formula:
𝑓(𝑥) = 𝑠 ∙ 𝑥 + 𝑧 (1)
𝑠 =
2𝑏−1
𝛼−𝛽
(2)
𝑧 = −𝑟𝑜𝑢𝑛𝑑(𝛽 ∙ 𝑠) − 2𝑏−1 (3)
𝑥𝑞 = 𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒(𝑥, 𝑏, 𝑠, 𝑧) = 𝑐𝑙𝑖𝑝(𝑟𝑜𝑢𝑛𝑑(𝑠 ∙ 𝑥 + 𝑧), −2𝑏−1, 2𝑏−1 − 1) (4)
𝑥̂ = 𝑑𝑒𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒(𝑥𝑞 , 𝑠, 𝑧) =
1
𝑠
(𝑥𝑞 − 𝑧) (5)
In this formula, 𝛼 𝑎𝑛𝑑 𝛽 represent the minimum (min) and maximum(max)
values of input data, respectively. Applying a single min and max range for all
input values might not be suitable, as it may result in a loss of valuable
information. Therefore, A criterion is needed to introduce a cut-off threshold
and to cut input value within this specified range using the clip function.
For example, as shown in Fig. 4(left)., 𝛼 corresponding to the Max value is
scaled to 127, and 𝛽 corresponding to the Min value is scaled to -128. Z means
zero-point, to accurately express what point the position of zero before
10
conversion corresponds to after quantization. In this way, quantization
considering the distribution of input values is possible.
This approach ensures that important data is maintained during model
compression process while effectively reducing the parameter space for
quantization purposes.
Second, Scale transform is a method in which zero of the input value is
quantized and then accurately corresponds to a value of zero. The formula can
be expressed as follows.
𝑓(𝑥) = 𝑠 ∙ 𝑥 (6)
𝑠 =
2𝑏−1−1
𝛼
(7)
𝑥𝑞 = 𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒(𝑥, 𝑏, 𝑠) = 𝑐𝑙𝑖𝑝(𝑟𝑜𝑢𝑛𝑑(𝑠 ∙ 𝑥), −2𝑏−1 + 1, 2𝑏−1 − 1) (8)
𝑥̂ = 𝑑𝑒𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒(𝑥𝑞 , 𝑠) =
1
𝑠
𝑥𝑞 (9)
Therefore, according to each condition, two methods may be used,
respectively. Quantization can be applied during or after the training process.
Post-training quantization is the most straightforward approach [7], as it only
requires modifying the model's parameters after it has been fully trained.
However, this method