Please, Translate the following English into Korean: 
3 AWQ: ACTIVATION-AWARE WEIGHT
QUANTIZATION
Quantization maps a floating-point number into lower-bit integers. It is an effective method to reduce the model size and inference costs of LLMs (Dettmers et al., 2022; Frantar et al., 2022; Yao et al., 2022; Xiao et al., 2022). In this section, we first propose a weight-only quantization method to improve accuracy without training/regression by protecting more “important” weights. And then develop a data-driven method to search for the optimal scaling that reduces quantization errors (Figure 2).

3.1 Improving LLM Quantization by Preserving 1% Salient Weights
We observe that the weights of LLMs are not equally important: there is a small fraction of salient weights that are much more important for LLMs’ performance compared to others.
Skipping the quantization of these salient weights can help bridge the performance degradation due to the quantization loss without any training or regression (Figure 2(b)).
To verify the idea, we benchmark the performance of quantized LLMs when skipping part of the weight channels in Table 1.
We measured the performance of INT3 quantized models while keeping some ratios of weight channels in FP16.
A widely used method to determine the importance of weights is to look at its magnitude or L2-norm (Han et al., 2015; Frankle & Carbin, 2018).
But we find skipping the weight channels with large norm (i.e., FP16% (based on W)) does not significantly improve the quantized performance, leading to a similar marginal improvement as random selection.
Interestingly, selecting weights based on activation magnitude can significantly improve the performance despite keeping only 0.1%-1% of channels in FP16.
We hypothesize that the input features with larger magnitudes are generally more important.
Keeping the corresponding weights in FP16 can preserve those features, which contributes to better model performance.

Limitations: Despite keeping 0.1% of weights in FP16 can improve the quantized performance without a noticeable increase in model size (measured in total bits), such a mixedprecision data type will make the system implementation difficult.
We need to come up with a method to protect the important weights without actually keeping them as FP16.

3.2 Protecting Salient Weights by Activation-aware Scaling
We propose an alternative method to reduce the quantization error of the salient weight by per-channel scaling, which does not suffer from the hardware inefficiency issue.
Analyzing the quantization error.
We start by analyzing the error from weight-only quantization. Consider a group/block of weight w; the linear operation can be written as y = wx, and the quantized counterpart is y = Q(w)x. Specifically, the quantization function is defined as:
Q(\mathbf{w})=\Delta\cdot\mathrm{Round}(\frac{\mathbf{w}}{\Delta}),\quad\Delta=\frac{\max(|\mathbf{w}|)}{2^{N-1}},\quad(1)

where N is the number of quantization bits, and Δ is the quantization scaler determined by the absolute maximum value. Now consider a weight element w ∈ w, if we multiply w with s > 1 and the inversely scale x, we will have Q(w · s)(x/s), which is:
Q(w\cdot s)\cdot\frac{x}{s}=\Delta^{'}\cdot\text{Round}(\frac{ws}{\Delta'})\cdot x\cdot\frac{1}{s},\quad(2)

where Δ′ is the new quantization scaler after applying s. We empirically find that: (1) The expected error from Round(·) (denoted as RoundErr(·)) does not change: since the round function maps a floating-point number to an integer, the error is roughly uniformly distributed from [0,0.5], resulting in an average error of 0.25; i.e., RoundErr(·) ∼ 0.25. (2) Scaling up a single element w usually does not change the maximum value from the groupw. Therefore we have Δ′ ≈ Δ; (3) As Δ and x are represented in FP16, they have no quantization error. Consequently, the quantization error from equation 1 and 2 can be expressed as
\begin{aligned}\operatorname{Err}(Q(w)x)&=\Delta\cdot\operatorname{RoundErr}(\frac{w}{\Delta})\cdot x\\\operatorname{Err}(Q(w\cdot s)(\frac{x}{s}))&=\Delta'\cdot\operatorname{RoundErr}(\frac{ws}{\Delta'})\cdot x\cdot\frac{1}{s}\end{aligned}\quad(3)

The ratio of the new error to the original error is \frac{\Delta^{\prime}}\Delta\cdot\frac1s\cdot\text{Given }\Delta^{^{\prime}}\approx\Delta\text{ and }s>1, the relative error is smaller for the salient weight w.

To verify the idea, we multiply the 1% salient channels with s > 1 for the OPT-6.7B model, and measure the change in Δ for each group in Table 2.
We find that scaling up the salient channels is quite effective: the perplexity improves from 23.54 for s = 1 (simply RTN) to 11.92 for s = 2.
As s goes larger, the percentage of changed Δ generally gets larger, but the percentage is still quite small for s < 2 (less than 5%); the relative error for the salient channels continues to go smaller as s increases.
Nonetheless, the best PPL actually appears at s = 2.
This is because if we use a very large s, it will increase the relative error for the nonsalient channels when Δ increases (the error of non-salient channels will be amplified by \frac{\Delta^{\prime}}\Delta, and the ratio is larger than 1 for 21.2% of the channels under s = 4), which can damage the model’s overall accuracy.
Therefore, we need to also consider the error from non-salient channels when protecting salient ones.

Searching to scale. To consider both salient and nonsalient weights, we choose to automatically search for an optimal (per input channel) scaling factor that minimizes the output difference after quantization for a certain layer.

Formally, we want to optimize the following objective:
\mathbf{s^*=\arg\min\mathcal{L}(s)}\\\mathbf{s}\\\mathcal{L}(\mathbf{s})=\|Q(\mathbf{W}\cdot\mathrm{diag}(\mathbf{s}))(\mathrm{diag}(\mathbf{s})^{-1}\cdot\mathbf{X})-\mathbf{W}\mathbf{X}\| (4)

Here Q means the weight quantization function (e.g., INT3/INT4 quantization with group size 128), W is the original weights in FP16, and X is the input features cached from a small calibration set (we take a small calibration set from he pre-training dataset in order not to overfit to a specific task). s is a per-(input) channel scaling factor; for s−1 · X, it can usually be fused into the previous operator (Wei et al., 2022b; Xiao et al., 2022). Since the quantization function is not differentiable, we are not able to directly optimize the problem with vanilla backpropagation. There are some techniques relying on approximated gradients (Bengio et al., 2013; Esser et al., 2019), which we found still suffers from unstable convergence. To make the process more stable, we define a search space for the optimal scale by analyzing the factors that will affect the choice of scaling factor. As shown in the last section, the saliency of weight channels is actually determined by the activation scale (thus “activation-awareness”). Therefore, we simply use a very simple search space:
\mathbf{s}=\mathbf{s_X}^\alpha,\quad\alpha^*=\underset{\alpha}{\operatorname*{\arg\min}}\mathcal{L}(\mathbf{s_X}^\alpha) (5)

s_{X} is the average magnitude of activation (per-channel), and we use a single hyper-parameter α to balance between the protection of salient and non-salient channels. We can find the best α by a fast grid search over the interval of [0, 1] (0 means we do not scale; 1 corresponds to the most aggressive scaling in our search space). We further apply weight clipping to minimize the MSE error of quantization. We provide an ablation study on OPT models under INT3-g128 quantization in Table 5; AWQ consistently outperforms round-to-nearest quantization (RTN) and achieves comparable performance as mixed-precision (1% FP16) while being more hardware-friendly. Advantages. Our method does not rely on any regression (Frantar et al., 2022) or backpropagation, which is required by many quantization-aware training methods. It has minimal reliance on the calibration set since we only measure the average magnitude per channel, thus preventing over-fitting (Figure 8). Therefore, our method requires fewer data for the quantization process and can preserve LLMs’ knowledge outside of the calibration set’s distribution. See Section 5.3 for more details.