ClimbQ: Class Imbalanced Quantization Enabling Robustness on Efficient Inferences
Abstract
Quantization compresses models to low bits for efficient inferences which has received increasing attentions.
However, existing approaches focused on balanced datasets, while imbalanced data is pervasive in the real world.
Therefore, in this study, we investigate the realistic problem, quantization on class-imbalanced data.
We observe from the analytical results that quantizing imbalanced data tends to obtain a large error due to the differences between separate class distributions, which leads to a significant accuracy loss.
To address this issue, we propose a novel quantization framework, Class Imbalanced Quantization (ClimbQ) that focuses on diminishing the inter-class heterogeneity for quantization error reduction.
ClimbQ first scales the variance of each class distribution and then projects data through the new distributions to the same space for quantization.
To guarantee the homogeneity of class variances after the ClimbQ process, we examine the quantized features and derive that the homogeneity satisfies when data size for each class is restricted (bounded).
Accordingly, we design a Homogeneous Variance Loss (HomoVar
Loss) which reweights the data losses of each class based on the bounded data sizes to satisfy the homogeneity of class variances.
Extensive experiments on classimbalanced and benchmark balanced datasets reveal that ClimbQ outperforms the state-of-the-art quantization techniques, especially on highly imbalanced data.
Introduction
Convolutional neural networks (CNNs) with high computational complexity inhibits the deployment on resource-limited mobile devices [1, 2, 3, 4, 5].
Quantization has been studied an efficient method for reducing memory storage and accelerating inferences by compressing models to low bits [6, 7, 8, 9, 10, 11, 12].
The existing quantization approaches are generally developed under a common assumption that the data is balanced for separate classes.
However, class-imbalanced data is ubiquitous in the real world.
Despite a realistic problem, quantization on class-imbalanced data has not been studied.
Therefore, in this research, we investigate the issues of quantization on class-imbalanced data.
The exploration results on CIFAR-10-LT (long-tailed CIFAR-10) are depicted in Fig.
Fig.
1a illustrates the data size of each class, where the largest class (indexed 0) has 50 times more data than the smallest class (indexed 9), i.e., an imbalanced dataset.
We then conduct experiments to quantize the classimbalanced data by exploiting a baseline quantization approach, uniform quantization.
Fig.
1b shows that the majority classes (indexed 0 to 3) are quantized to 2 bits with only 2% accuracy loss (compared to the 32-bit model), whereas the minority classes (indexed 8 and 9) obtain 14% accuracy drop.
It indicates that the quantization result is inclined to be dominated by the majority classes, which may result from the heterogeneity of class distributions as presented in Fig. 1c. Fig. 1c manifests that
[Figure]
Fig.
1b depicts the rate of accuracy loss under quantization in comparison to the 32-bit model, where the majority classes have the least accuracy drop, while the accuracy of minority classes decreases the most (as much as 14% accuracy drop for a 2-bit model).
It indicates that the majority classes determine the quantization result due to the fact illustrated in Fig.
1c that the majority classes not only have a large amount of data but also distributed in large variations.
the majority classes always have greater data variations than the minority classes.
Therefore, the quantization results are inclined to be determined by the majority classes, which may lead to a large quantization error and a notable accuracy loss for the minority classes (validated in Fig. 1b).
Motivated by this challenge in quantization, we propose a Class Imbalanced Quantization (ClimbQ) scheme to reduce the differences between class distributions for a robust quantization on imbalanced data.
According to the result as presented in Fig.
1c that the majority classes are distributed more diversely than the minority classes, ClimbQ scales the class distributions, allowing the class variations to be closer.
Afterward, we project data through the new class distributions to the same uniform space for quantization to further decrease the quantization error 1 .
To further ensure that the class distributions after quantization are homogeneous, we analyze the homogeneity of class variances based on Levene’s hypothesis testing in the theory of statistics [13].
As our contribution, we demonstrate that the data size of each class is restricted to satisfy the homogeneity of class distributions (variances) and derive the bounds of the data sizes.
Accordingly, we design a Homogeneous Variance Loss (HomoVar Loss) to rebalance the data of each class.
We place emphases on the (minority) classes with a data size below the lower bound and weight down the loss of the (majority) classes with a larger data size over the upper bound to prevent the majority classes from dictating the quantization results.
Experiments demonstrate that ClimbQ+ (ClimbQ incorporated with HomoVar loss) outperforms the prior works on both imbalanced and balanced datasets.
ClimbQ+ improves the state-of-the-art quantization approaches by 6% for 2-bit ResNet-56, while by 12% for 2-bit MobileNet-V2 on
CIFAR-100-LT in highly imbalanced instances.
Our contributions are summarized as follows:
We develop a novel quantization scheme, ClimbQ , that makes the first attempt to reduce the differences between class distributions for a robust quantization process on imbalanced data.
We derive the bounds of data size for each class to guarantee the homogeneity of class variances.
Accordingly, we propose a HomoVar Loss to reweight the loss on each class in order to prevent the majority classes from dictating the quantization results.
Experimental results demonstrate that ClimbQ+ (ClimbQ with HomoVar loss) has superior improvements over the state-of-the-art quantization on class-imbalanced and benchmark balanced datasets, especially significant for the low-bit models on highly imbalanced data.
Related works
Quantization.
Quantization has received increasing attentions in recent years with the development of the Internet of Things (IoT), meeting the requirements of efficient computing and storage.
HomoVar Loss, ℒ H
Quantization, Q w y ℒ H x k ~ f k : N ( 𝜇 k , σ k 2 )
Class Distribution
Scaling & Generation x’ k ~ f’ k : N ( 𝜇 k , c k 2 σ k 2 ) (a) Class Imbalanced Quantization
[Figure] Estimation (b) Rebalancing for Homogeneity of Variances Q( [Figure]
); n k w(n L ; n U )
QuantizedCNN
FC ŷ ℒ (c) Learning Modules
Q(x’ k )
Quantization-aware training (QAT) is one branch of approaches.
It mainly focuses on learning a clipping range [6, 14, 5, 15] or a data transformation [7, 8, 11, 16] for quantization with a small loss of accuracy.
The well-trained statistics are then stored and utilized in low-bit inferences.
In contrast, Zero-shot quantization (ZSQ) has recently been proposed to incorporate the concept of knowledge distillation, transferring the information from 32-bit models to the quantized networks [17, 10, 18, 19, 20].
They intend to reduce the feature differences between full-bit and low-bit models.
Existing QAT or ZSQ methods learned a single quantization scheme for all data, implicitly assuming that the data follows identical distributions [5, 14, 9, 17, 18, 10, 11, 16].
However, class-imbalanced data is a realistic case in which the class distributions are diverse.
It is visualized in Fig.
1c that the majority classes are always with greater variations and sizes, dominating the quantization results and resulting in a significant quantization error for the minority classes, as seen in Fig.
1b.
Therefore, we aim to develop a robust quantization scheme on imbalanced data with a reduced quantization error.
In particular, this work mainly follows the QAT research instead of ZSQ which exploits additional knowledge from a floating-point model and suffers from higher memory costs.
Class-imbalanced learning.
Class-imbalanced problem poses a great challenge in deep learning, where the minority classes with few data generally have an inferior classification performance to the majority classes [21, 22, 23, 24].
Previous research has studied resampling to balance the classes to address this issue [25, 26, 27, 21].
They focus on collecting more data from the minority classes (i.e., over-sampling) or eliminating some data from the majority (i.e., under-sampling).
Nevertheless, over-sampling increases memory storage and training time, whereas under-sampling raises the over-fitting problem since models are only learned from the sampled data [28].
Another branch of class-imbalanced learning approaches developed is reweighting .
In contrast, they reweight the data losses of each class for balancing instead of sampling, which has no problems of memory consumption and over-fitting.
Prior research suggested focusing more on the minority and assign small losses to the majority [29, 30, 31, 32, 33].
Nevertheless, they are only concerned with class data sizes.
In this research, we design a reweighting loss that leverages not only the data size but also the quantized features to balance data for homogeneity of class variances with a reduced error during quantization.
ClimbQ
Fig.
2 introduces ClimbQ, a novel quantization scheme that considers the heterogeneity of class distributions on class-imbalanced data to reduce quantization errors.
ClimbQ first scales class distributions that enables the new distributions with closer variances for quantization (depicted in
Fig.
2a and detailed in Section 3.1).
In Sec. 3.2, we investigate the homogeneity of class variances based on quantized features.
From the analytical results, we derive the bounds of data sizes that satisfy the homogeneity condition (see blue modules in Fig.
2b and Fig. 2c).
In Sec. 3.3, we design a
HomoVar loss that reweights the data losses of each class according to the quantized data and the class data size (see green modules in Fig. 2c).
3.1 Class imbalanced quantization (ClimbQ)
Existing works quantized data from different classes under the same process, ignoring the heterogeneous class distributions in the real-world imbalanced data.
As visualized in Fig.
1c on CIFAR-10-LT, the majority classes have greater variations than the minority classes, raising the concern that the quantization results incline to be bias toward the majority.
As a result, the minority classes are quantized with inevitable biases that result in a significant accuracy loss (validated in Fig. 1b).
To reduce the quantization error caused by the differences between class distributions, we propose class imbalanced quantization (ClimbQ) , as shown in Fig. 2, which scales the class data to obtain closer distributions (see Sec. 3.1.1) and project them to the same space for quantization (see Sec. 3.1.2).
3.1.1 Class distribution scaling and generation
According to Fig.
1c, majority classes are always distributed in wide ranges, whereas minority classes are distributed in small variations.
Such different class distributions cause massive quantization errors and derive a notable accuracy loss, especially for the minority classes (validated in Fig. 1b).
As a result, we aim to diminish majority class variations while enlarging minority class variations.
First, we denote the network features for N data as X = { x 1 , x 2 , ..., x N } and suppose that the data number for the k -th class is n k , with k = 1 , 2 , ..., K (in total K classes).
Assume the k -th class distribution f k follows the normal distribution denoted as N ( µ k , σ 2 k ) , where µ k and σ k indicate the mean and standard deviation of the distribution 2 .
We scale the k -th class distribution f k : N ( µ k , σ 2 k ) to f ′ k : N ( µ k , ( c k σ k ) 2 ) , where c k is the scale factor defined as 2 − n k (cid:80)
Ki =
1 n i , ∀ k
= 1 , 2 , ..., K 3 .
The design reveals that the (minority) classes with less data have been scaled to larger variances.
The variances of the (majority) classes with a large amount of data are reduced.
In the next subsection, we will develop an efficient quantization method based on the scaled class distributions.
3.1.2 Class distribution quantization
As illustrated in Sec.
3.1.1, data are scaled to N ( µ k , ( c k σ k ) 2 ) based on the class it belongs to.
In the following, we aim to design an efficient quantization process based on the new class distributions.
Recent works adopted and efficient process, the uniform quantization, due to its simplicity and hardware-friendliness in operations on devices [6, 5] 4 .
However, since data distributions are typically not uniformly distributed, the quantization process produces massive errors [34, 35, 7, 5].
Prior research therefore proposed to learn a clipping function, quantization statistics, and other preprocessing procedures to reduce the error [7, 5, 11, 16].
Nevertheless, the proposed preprocessing procedures are complicated and necessarily require additional learning parameters.
Therefore, we propose a one-step transformation without additional learning parameters that allows the class data to be projected to a uniform space with a minimal error under the uniform quantization scheme.
We adopt the distribution function (namely, cumulative distribution function ) as the uniform space projection function.
In the following theorem according to [36], we prove that the data are uniformly distributed after the projection.
Theorem 3.1.
( Proved in Appendix A.1 ) Let f be the continuous probability function of random variable X with its distribution function F , then F ( X ) =
U , where U is the random variable of the standard uniform distribution.
Therefore, by Theorem 3.1, we exploit the distribution function of the new class distributions f ′ k : N ( µ k , ( c k σ k ) 2 ) , ∀ k (determined in Sec. 3.1.1), to project data x k from the k -th class to the uniform space denoted as D k .
The projection function is then formulated as:
[Equation]
where c k is the scale factor introduced and defined in Sec.
Eq.
(1) indicates that the data projection is conditioned on its belonged class and the data size.
After transformed to the uniform space, the data x ′ k :
= D k ( x k ) is then processed by the following uniform quantization function:
[Equation]
where round is the rounding operation, and B represents the quantization bits.
In summary, as depicted in Fig.
2a, the data from different classes are scaled to diminish the differences between class distributions (detailed in Sec.
and then projected to the uniform space for quantization (see Eq.
(1) and Eq.
ClimbQ process is applied in each convolutional layer as presented as the red module in Fig.
2c, i.e., the quantized data Q ( x ′ k ) output from the last layer acts as the input x k for the current layer.
The effectiveness of the scaling on quantization error reduction is demonstrated in Appendix E.
3.2 Rebalancing for homogeneity of variances
In Sec. 3.1, we proposed distribution scaling and uniform space projection to reduce the differences between class distributions for a minimal quantization error.
To ensure homogeneity of the class distributions after quantized through the ClimbQ process, in this subsection, we examine the variances of the quantized features from different classes based on Levene ’s hypothesis testing.
Our main contribution is the derivation of the bounds of the data size for each class to satisfy the homogeneity (illustrated in the blue modules in Fig.
2b and Fig. 2c).
Accordingly, we design a HomoVar loss that focuses on the classes with data sizes out of the bounds (as illustrated in the green modules in Fig.
2b and Fig. 2c).
To prevent the results from being dominated by the majority classes [29, 30, 31, 32], we intend to weight down the losses of those classes whose data size exceeds the upper bound, i.e., the majority classes, while putting more weights on the losses of the minority classes with a smaller data size than the lower bound.
3.2.1 Appropriate class data size estimation
For a systematic analysis of the homogeneity of class distributions following the ClimbQ process (introduced in Sec. 3.1), we refer to Levene ’s hypothesis testing based on the theory of statistical learning [13].
Assume there are total K classes in the dataset.
Two hypotheses are included in
Levene’s testing: null hypothesis H 0 :
σ 21 = σ 22 = ...
=
σ 2 K and alternative hypothesis H a : σ 2 i ̸
=
σ 2 j , for arbitrary i ̸
=
j , i, j = 1 , 2 , ..., K .
The null hypothesis claims the homogeneity of variances, whereas the alternative hypothesis stands for the heterogeneity if any variance is much different from the others.
According to Levene’s hypothesis testing, the null hypothesis is satisfied when the testing statistics
W is small.
The testing statistics W is formulated as the following:
[Equation]
where z ki = | x ki − 1 n k (cid:80) n k i =
1 x ki | is the variation, i.e., the distance of data x ki from the center of the k -th class 5 , ¯ z k · = 1 n k (cid:80) n k i
=
z ki represents the mean variation of the k -th class, and ¯ z ·· = 1 K (cid:80)
Kk
=1 ¯ z k · stands for the mean variation for all data.
In addition, n k and N individually denote the data size of the k -th class and total data size, i.e., N = (cid:80)
Kk =1 n k .
The denominator of W in Eq.
(3) represents the sum of the differences from the within class variations, whereas the numerator shows the total difference from the between class variations.
Thus, we denote the former as S W and the latter as S B in Eq.
The null hypothesis is accepted when W is small, i.e., W < F α ( k − 1 , N − k ) , where F α ( k − 1 , N − k ) is the statistics of F -distribution with degree of freedoms, k − 1 and N − k under the significance level α that manifests the statistical significance on the testing results 6 .
Derived from W < F α ( k − 1 , N − k ) , we prove in the following theorem that each class data size is restricted with a lower bound n Lk and an upper bound n Uk ,
∀ k
= 1 , 2 , ..., K .
It indicates that when class data sizes are not too tiny or excessively large, class variances are homogeneous ( H 0 is satisfied).
On the other hand, when class data sizes are too small or large, significant differences appear between class variances ( H 0 is rejected).
Theorem 3.2.
( Proved in Appendix A.2 )
Given the definitions and the notations in Eq.
(3), if
W < F α ( k − 1 , N − k ) , where F α ( k − 1 , N − k ) is the statistics of F-distribution with degree of freedoms ( k − 1 , N − k ) under the significance level α , then class data size n Lk < n k < n Uk ,
∀ k
= 1 , 2 , ..., K .
The bounds n Lk , n Uk =
[Equation]
where C
[Equation]
Theorem 3.2 demonstrates that if the data size of class n k is in the interval [ n Lk , n Uk ] , H 0 is satisfies, i.e., the class variances are homogeneous.
Otherwise, H 0 is rejected, which reveals that classes with out-of-range data sizes (too minor or too large) may violate the homogeneity of class variances.
Therefore, in the subsequent subsection, we design a loss using the analyzed bounds to reweight the data losses of each classes to adapt the class distributions to satisfy the homogeneity.
3.2.2 Loss for homogeneity of class variances
In the preceding subsection, we derived the bounds of data sizes for each class that satisfies the homogeneity of class variances (see Eq. (3.2)).
Consequently, in this subsection, we propose a reweighted loss, named HomoVar loss , to reweight the classes that may exhibit significant differences from other classes.
We consider three cases on classes: 1) minority classes with a data size below the lower bound, 2) moderated classes with a data size within the required range, and 3) majority classes with a data size exceeding the upper bound.
To avoid the majority from dominating the prediction outcomes, we primarily focus on the minority and downweight the majority (shown in Fig. 1b).
That is, the minority classes are penalized the greatest, followed by the middle and then the majority.
In addition, we evaluate the degree of minority or majority of a class, i.e. how close the class data size is to the nearest bound.
Classes with a minor (enormous) data size far from the derived bounds are weighted more heavily (weighted less).
According to the above insights, we design the HomoVar loss denoted as L H in the following:
[Equation]
where L ki represents the loss on the i -th data of the k -th class, ω k
= 1 − β
[Equation]
| nk
[Equation]
nk | nk − nek | is the weight on the data of the k -th class, β is a constant factor in (0 , 1) that scales the losses on different classes (discussed in Appendix B).
The term | n k − n ek | is used to measure the degree of the minority and majority, where n ek , namely the expected data size, is defined as the nearest bound to the data size if the actual data size n k is out of the range [ n Lk , n Uk ] , otherwise, n k − 1 7 .
When the k -th class is an extraordinarily minor class, the weight ω k is around 1.
Moreover, ω k is near to
[Equation]
when n k is inside the restricted range satisfying the homogeneity of class
[Table]
variances.
Additionally
[Table]
it demonstrates that
[Table]
decreases as the size n k increases, eventually approaching 1 − β when n k is exceedingly large, i.e., a majority class.
If the data size is far beyond the upper bound, ω k approaches zero.
The results are consistent with the ideas behind the design: 1) the weights of minority classes are heavier than those of majority classes, 2) the weights increase as the class data size falls far below the lower bound, and 3) the weights reduce as the class data size exceeds the upper bound.
We analyze and discuss further details in Appendix A.3.
Experiments 4.1 Experiment settings
Class-imbalanced datasets.
We evaluate the effectiveness of ClimbQ and ClimbQ+ (ClimbQ with
HomoVar loss) on the class-imbalanced datasets, Syndigit-LT [37], CIFAR-10-LT [38] and CIFAR100-LT [38].
A parameter setting γ determines the degree of imbalance in the datasets.
γ is the ratio between the data size of the maximum class and the size of the minimum class.
For example, γ = 100 indicates that the data size of the largest class is 100 times larger than the minimum class.
In this paper, we train at imbalance ratios of 10, 50, and 200 and validate on the balanced testing data (i.e., γ = 1 ) to fairly evaluate the performance of each class.
Benchmark balanced dataset.
Although we are primarily concerned with imbalanced data, we compare with the baseline quantization approaches on the benchmark dataset ImageNet-ILSVRC 2012 [39].
The imbalanced ratio is 1 . 77 , which is close to perfectly balanced data ( γ = 1 ).
Architectures.
We use ResNets [40] and MobileNet-V2 [41] as the underlying architectures.
ResNet is a benchmark architecture for quantization research, while MobileNet-V2 is essentially an efficient architecture with lightweight module designs that is also commonly used in image recognition.
Training.
We utilize an NVIDIA Tesla V100 GPU and an NVIDIA GTX 2080Ti for implementation.
ImageNet has a batch size of 512, while Syndigits-LT, CIFAR-10-LT, and CIFAR-100-LT have 128.
The maximum number of training epochs is 200.
The range of the learning rate is from 0.01 to 0.1.
The significance level α referring to Theorem 3.2 is set to 0.05.
The constant factor β in Eq.
(4) is set to 0.999.
More discussions about the setting are detailed in Appendix B. Code is available at https://github.com/tinganchen/ClimbQ.git .
4.2 Comparison results
In the following, we compare the quantization performance of ClimbQ and ClimbQ+ with QAT [8, 15, 11, 16] and ZSQ baselines [19, 18, 17, 10] on the class-imblanced datasets with distinct imbalance ratio settings 8 .
CIFAR-10-LT.
The results of 4-bit and 2-bit quantization of ResNet-20 and MobileNet-V2 on
CIFAR-10-LT are presented in Table 1.
First, we can observe that the prediction accuracy declines with the imbalance ratio γ increases.
In addition, the process of low-bit quantization reduces precision.
[Table]
In particular, most of the compared quantization approaches have a significant accuracy loss for low bit models in highly imbalanced cases, since they quantize on imbalanced data (with heterogeneous class distributions) using the same quantization functions or distillation method, which results in a huge quantization error and a significant loss of accuracy.
In contrast, our ClimbQ and ClimbQ+ quantization schemes are proposed to diminish the heterogeneity in different classes (introduced in
Sec. 3).
Table 1 demonstrates that our proposed methodologies can effectively reduce quantization error and accuracy loss, especially on the lightweight architecture design, MobileNet-V2.
ClimbQ increases the accuracy of baselines under all studied conditions.
In addition, ClimbQ yields a performance increase of up to 10% for 2-bit MobileNet-V2 with γ = 10 and can successfully implemented on the highly imbalanced cases γ = 50 and 200 .
ClimbQ+ can further improve the performance of ClimbQ by as much as 4.4% in absolute accuracy.
The exceptional performance is a result of attempts to reduce the heterogeneity of class variances using quantized features and class data sizes (see the evaluation of homogeneity and the proposed HomoVar loss in Sec. 3.2).
CIFAR-100-LT.
We also study the efficacy of ClimbQ and ClimbQ+ on CIFAR-100-LT, which contains 10 times as many classes as CIFAR-10-LT.
Due to the increasing difficulty of the recognition task, we employ a deeper ResNet-56 model instead of ResNet-20 for quantization.
Table 2 reveals that the accuracy of the compared approaches degrades significantly under highly imbalanced circumstances.
ClimbQ+ enhances the performance of the BatchQuant by 5% to 12% accuracy on 2-bit ResNet-56 and increases 8% to 15% for 2-bit MobileNet-V2, validating the effectiveness of the ClimbQ process (scaling the class variances as introduced in Sec. 3.1) and the HomoVar loss (rebalancing the data size of each class as shown in Sec. 3.2) on quantization error reduction.
Syndigits-LT and ImageNet.
We further evaluate and discuss the performances of ClimbQ and
ClimbQ+ on the imbalanced dataset Syndigits-LT and balanced benchmark ImageNet in Appendix C.
ClimbQ+ on Syndigits-LT in the highly imbalanced case, γ = 200 , achieves a remarkable 80.55% accuracy for 2-bit ResNet-20, i.e., a 22% accuracy improvement over the best baseline result evaluated (only 58.75% accuracy).
Moreover, ClimbQ+ on 2-bit MobileNet-V2 outperforms the state-of-the-art
BatchQuant with a 26% increase in accuracy.
In addition to the imbalanced datasets, we compare our quantization methods against those previously proposed on the balanced benchmark ImageNet.
ClimbQ+ outperforms the previous studies by approximately 3% in terms of accuracy.
More details are discussed in Appendix C.
Ablation study on rebalancing strategies
In Sec. 4, we demonstrated the superior performance of ClimbQ (introduced in Sec. 3.1) and
ClimbQ+ (described in Sec. 3.2) over the baseline quantization approaches, which is particularly with a significant improvement for low-bit and lightweight models in extremely imbalanced scenarios.
In this section, we compare ClimbQ+ to other effective rebalancing methods 9 .
Table 3 demonstrates that
ClimbQ+ has outstanding performances compared to the previous research on rebalancing data sizes.
ClimbQ+ improves absolute accuracy by 4.4% to 22.4% for the 2-bit model on CIFAR-10-LT with γ = 50 and by 2.6% to 25.3% accuracy with γ = 200 .
The extraordinary performances of ClimbQ+ verifies that the HomoVar loss introduced in Sec.
3.2 can effectively reduce more quantization errors
[Table]
and accuracy loss owing to the rebalancing strategy that takes the homogeneity of the class variances (variances of quantized features) into a consideration, rather than only leveraging the class data sizes.
Discussion with potential impacts and limitations
Potential impacts.
ClimbQ with the HomoVar loss aims to rebalance the classes for homogeneity of class variances.
To prevent the majority classes from dominating the quantization results, the rebalancing strategy focuses primarily on the minority classes while scaling down the weights on the majority classes for the homogeneity of class variances, which implies that the prediction results of the minority classes are more dependent on the small amount of data.
Accordingly, they may be much easier to be attacked with a notable performance degradation.
However, recent research has developed the defense strategy against the attacks on imbalanced data [42] which can also be applied to ClimbQ.
Limitations.
ClimbQ mainly focuses on the class-imbalanced data with different class data sizes and distributions.
However, we believe that there are other scenarios of heterogeneous data in the real world that have not been fully studied.
As a result, we will explore and study more research issues of quantization on heterogeneous data in our future works.
Conclusion
In this paper, we study a realistic but not fully studied research problem of quantization on classimbalanced data, where the differences in class distributions lead to a huge quantization error in the minority classes.
To address this issue, we propose ClimbQ to scale the class distributions to diminish the differences between classes for quantization error reduction.
Moreover, we analyze that the homogeneity of class variances under ClimbQ process is subjected to restricted class data sizes.
Therefore, we further propose HomoVar loss to rebalance the classes based on the derived bounds of data sizes to satisfy the homogeneity.
Experiments reveal that ClimbQ/ClimbQ+ outperforms the baseline quantization and rebalance techniques, validating the effectiveness on quantization error reduction by diminishing the differences in class distributions.
Acknowledgment
This work was supported in part by the Ministry of Science and Technology, Taiwan, under grant
MOST 111-2223-E-002 -006 and MOST 111-2221-E-002 -135 -MY3.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Section 6.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sectionand Appendix D. (b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See abstract.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 4.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] We report the best result within 200 training epochs.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4.
(b) Did you mention the license of the assets?
[No] No license of the assets.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] No new assets.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[No] The datasets are public.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[No] The images in datasets do not contain the personal information as we know.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]