A PID Controller Approach for Adaptive Probability-dependent Gradient Decay in Model Calibration
Authors: Siyuan Zhang
Linbo Xie
Venue: NeurIPS 2024 poster (2024)
Keywords: Model calibration
Softmax loss
Gradient decay
PID controller
Supervised learning
Primary area: optimization_for_deep_networks

Abstract: Modern deep learning models often exhibit overconfident predictions, inadequately capturing uncertainty. During model optimization, the expected calibration error tends to overfit  earlier than classification accuracy, indicating distinct optimization objectives for classification error and calibration error. To ensure consistent optimization of both model accuracy and model calibration, we propose a novel method incorporating a probability-dependent gradient decay coefficient into loss function. This coefficient exhibits a strong correlation with the overall confidence level.  To maintain model calibration during optimization, we utilize a proportional-integral-derivative (PID) controller to dynamically adjust this gradient decay rate, where the adjustment relies on the proposed relative calibration error feedback in each epoch, thereby preventing the model from exhibiting over-confidence or under-confidence. Within the PID control system framework, the proposed relative calibration error serves as the control system output, providing an indication of the overall confidence level, while the gradient decay rate functions as the controlled variable. Moreover, recognizing the impact of gradient amplitude of adaptive decay rates, we implement an adaptive learning rate mechanism for gradient compensation to prevent inadequate learning of over-small or over-large gradient. Empirical experiments validate the efficacy of our PID-based adaptive gradient decay rate approach, ensuring consistent optimization of model calibration and model accuracy.

A PID Controller Approach for Adaptive Probability-dependert Gradient Decay in Model Calibration
Siyuan Zhang
Linbo Xie
School of Internet of Things Engineering Jiangnan University Wuxi, China 214122
School of Internet of Things Engineering Jiangnan University Wuxi, China 214122
Abstract
Modern deep learning models often exhibit overconfident predictions, inadequately capturing uncertainty. During model optimization, the expected calibration error tends to overfit earlier than classification accuracy, indicating distinct optimiza- tion objectives for classification error and calibration error. To ensure consistent optimization of both model accuracy and model calibration, we propose a novel wobabblity-ddeeedder
method incorporating function. This coefficient exhibits a strong correlation with the overall confidence level. To maintain model calibration during optimization, we utilize a proportional- integral-derivate (PID) controller to dynamically adjust this gradient decay rate, where the adjustment relies on the proposed relative calibration error feedback in each epoch, thereby preventing the model from exhibiting over-confidence or under-confidence. Within the PID control system framework, the proposed relative calibration error serves as the control system output, providing an indication of the overall confidence level, while the gradient decay rate functions the controlled variable. Moreover, recognizing the impact of gradient amplitude of adaptive decay rates, implement adaptive learning tion to prevent inadequate learning of over-small or over-large gradient. Empirical experiments validate the efficacy of our PID-based adaptive gradient decay rate ap- proach, ensuring consistent optimization of model calibration and model accuracy. The code of implementation is available in http:://gichbb.co///UHIPPPPP
gradient decay coefficient into loss
mechanism for gradient compensa-
1 Inroduction
Model calibration aims to refine the uncertainty distribution of model output, ensuring its faithful reflection of the inherent uncertainty characteristics. Softmax mapping is commonly employed in the baseline learning to establish the output-probability mapping. Yet, with the escalation in model parameterizations, the output uncertainty distribution from over-parameterized models tends to become over-confident [I] Consequently, the baseline strategy, which exclusively depends on Softmax mapping without accounting for calibration characteristics as optimization objectives, fails to achieve perfect model calibration. Particularly in high-risk applications, inadequate model calibration poses heightened safety risks 3
There are three primary strategies for calibrating uncertainty in deep learning models: Bayesian neural networks, post-processing calibration, and training- based model calibration. Bayesian neural networks (BNNs) integrate Bayesian inference into their framework, distinguishing them from tradi- tional neural networks 5). In contrast to conventional neural networks, which yield point estimates,
Linbo Xie is the corresponding author. E-mail: xie wtw.iiooooinnnanndddu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

BNNs offer a probability distribution across potential outputs, enabling uncertainty quantification in predictions [6]. This is accomplished by assigning prior distributions the network's parameters and iteratively updating these distributions using Bayes' theorem as more data is acquired.
The post-processing calibration methods involve establishing the additional ourput-roobbbiliyy re- lationship through supplementary mappings to refine output uncertainty distribution | [ This calibration method avoids disrupting the original model's decision- making pertaining to the primary task, maintaining the original generalization performance [0]. However, a notable drawback emerges: the necessity of additional structure to establish the mapping between outputs and probabilities [ل This task possesses unique characteristics, demanding calibrated properties to construct output- probability mapping structures, optimization goals, and strategies. Unlike typical loss functions and metrics in machine learning, uncertainty is sample-specific; however, validating it individually infeasible [12]. Furthermore, when validated collectively, it fails to fully represent individual sample properties, presenting challenges in calibrating individual sample. This presents a difficulty in calibrating model confidence via post-processing structures
[3
Another approach to model calibration involves integrating various elements into the baseline opti- mization strategy for deep learning 77. This enhancement of elements during the optimization leads to an improvement in the uncertainty distribution of the model output. These methods encompass a range of techniques including pre- training (15), data augmentation [16], label smoothing 17 weight decay M1 early stopping [18), structure sparsity [19), convolutional structure [20], and others. These methods not only bolster the calibration of the model and refine the output uncertainty distribution but also delve into the dynamic attributes of the model optimization, thereby enhancing decision- making. Additionally, they improve the interpretability in the decision-making and optimization process. Furthermore, certain loss functions for model output distribution have been devised based on uncertainty properties. These include MMCE [4), Correctness Ranking Loss [21, CALS (22), Focal loss [23] 24,, and FLSD (25). They jointly address classification accuracy and confidence calibration to mitigate the inclination of the over-confidence and under-confidence.
In the optimization of Softmax-base cross-entropy loss, model accuracy and model calibration represent distinct optimization characteristics [26). Model calibration tends to be overfitting earlier in the optimization compared to accuracy .. To ensure consistent optimization between model accuracy and calibration, it is imperative to achieve high accuracy while maintaining adequate calibration. Our approach introduces a hyperparameter that controls the gradient decay rate within mapping. It indicated negative correlation between the gradient
the Softmax output-probabiin decay rate with increasing instance-level probability and the overall confidence distribution [27]. To achieve consistent optimization of accuracy and calibration, we propose detecting model calibration through validation set in the optimization process. Drawing inspiration from the notable success of proportional-interree -itttnnrll-drriaatiie (PID) controllers in automatic control systems (28), we introduce an Adaptive
Gradient Decay through PID controller approach to calibrate the model. Moreover, considering that bobbbility-depeendee gradient decay rate may impact gradient amplitude, we design a dynamic learning rate mechanism corresponding to the changing gradient decay rate to offset fluctuations in gradient amplitude.
Our main contributions in this work can be summarized as follows: (1) We propose adaptive probability-depedder gradient decay via PID controller. This approach utilizes the feedback mecha- nism from automatic control to detect model calibration and adjust the probability-dependeet gradient decay rate coefficient in Softmax, ensuring consistent optimization of model calibration and accuracy. (2) To counteract fluctuations in gradient magnitude caused by the adaptive probability-depender gradient decay rate, we introduce a dynamic learning rate schedule to follow the dynamic decay rate. (3) Empirical experiments confirm the effectiveness of our approach, achieving improved model calibration while maintaining the model's generalization ability.
2 Problem formulation
2.1 Model calibration
Considering a dataset [(xt, v/ ) C RD x R and classifier ʃ maps x to the outputs ž = 1...., on m classes and k arg max The ground-truth and predicted labels are formulated in one-hot format where ye = and ýA = 1, where e represents the truth class. The associated
confidence score of the predicted label in bascline is p maxx;(2), = I,..., m, where s (-)
2

---

Graflerd
0.8
0.5
0.4
0.3
02
0.1
0.2
0.3
0.4
0.5
0.8
0.9
Probabillty
Figure 1: The gradient magnitude of different gradient decay B with increasing probability Pe-
represents Softmax mapping R" + R.. However, Softmax mapping probabilitiess are not accurately reflected in the properties of model output []]. The probability estimation of modern deep models may show over-confidence under-confidence.
Confidence Calibration Perfect calibration neural network can realized when the confidence score reflects the real probability that the classification is classified correctly. Formally, the perfectly calibrated network satisfied ( vIp - p) = p for all E [0, 1]. However, in practical applications, the sample is divided into M bins {Do b=1 The limited availability of data restricts the ability to accurately estimate the calibration error. According to their confidence scores and the calibration error, an approximation calculated for each bins {D>)b= Ds contains sample with 37 *)
b-1
Average confidence is computed as con (Da) = DDa ΣiED, f and the bin accuracy computed as ace (Db) [D8]
3 ΣieD, (y = Vt). ECE and MCE are calculated follows.
M ECF Σ b=1
Dol
Jace (Db) conf (D&)
MCE max be bE.l..,MM)
lace (Do) conf (Db)
(2)
2.2 Parametric Softmax
Softmax cross-entropy (CE) expressed as
= ---lop log
(3)
Σド=1 e
We introduce two hyperparameters in the Softmax mapping, which is expressed as follows:
eaft
J log
Ejtc eailT + SeaelT
The parametric Softmax cross-entropy can be approximated as the following max function, as shown in (5). Minimizing this function expected that output 2, can be larger than other class outputs Zj.J = 1,..., j,j # which is in line with the logic of the one-versus-all classification decision-maan making cls (x)) max {z, (a)} .j I...,,m.
2//T
lim T-0
log Ejtc etilT Bet./
lim max (log B, Ze/T,j = 1,... m. # 7-00
(5)
The temperature coefficient 7 has been extensively investigated in model calibration. Temperature scaling represents commonly utilized calibration technique in post-processing calibration methods. is regarded soft margin, with its approximation procedure detailed in Eq. (5). Consequently,

---

Cariiennc
Cortderca
Carfornca
(a) 8 = 20
(b) 3 = 10
(c) = 5
Oue Bitere
Our bitic
Conteete
Cortdance
Confoncc
(c) 0.1
(f) 0.01
Figure 2: Confidence and reliability diagrams with ResNet18 on CIFAR-100 (bins 10) In each subplot, the left plot illustrates the sample distribution in individual bins, while the right plot displays the average confidence and accuracy each bin. Ideally, calibration aims for consistency between accuracy and average confidence in each bin. It indicates that a smaller gradient decay rate is associated with more pronounced miscalibration of the model, while larger gradient decay rate mitigates this issue.
CE can be interpreted as a margin- -bsed loss function. Nonetheless, due the distance distortion between input and representation spaces, maximizing the margin in the input space of models is not achieved simultaneously by large margin Softmax. Consequently, dynamic characteristic optimization process tends to be ambiguous.
2.3 Probability-depender gradient decay
Considering the Softmax with the sole hyperparameter 3, the temperature 7 is set to 1.
e0
log
(6)
Ejfc e + Be
Let us first consider the gradient of the Softmax.
ŌJ
Σ e e
(7)
02
Σ e + +3 e²
0J
(8)
0zj
Ee (B
Introducing probabilistic output P5 4 tetm
eti
as an intermediate variable, we obtain
1- De
.
0J
1 1 (3 - 1)pe
(9)
0zj
Pj
. #
1+ 1+ - 1)pe
Fig. [ illustrates how the introduced hyperparameter determines the gradient decay rate as the instance-level probability increases. More empirical experimental results can be found in the Appendix A.2 smaller results in reduced decay rate of gradient amplitude corresponding to probability. Empirical investigations have shown that the magnitude of 3 during optimization determines the average confidence level and consequently impacts the calibration performance of the model. Fig- demonstrates that low gradient decay rate exacerbates the over-confidence in the model's s output, whereas high gradient decay rate can alleviate this issue and yield improved calibration results (27].

---

Desired Value
Validation
Control System
Error
PID Controller
Update Model
Softmax Mapping
RCE
Feedback
Connection
fa(x)
%g
Updute Cradient Decay Rate f
Model *
Output
Optimization
35
Loss Function e²c
2
8f
Learning 1
0
30
Compensation
log Ej e Biee
Figure The framework of PID controller-based adaptive probability gradient decay.
3 Methodology
The probability-depeedent gradient decay rate exhibits negative correlation with the model's average confidence, as shown in Fig. 2 and Appendix(A.2 Higher gradient decay rates result in concave reduction in gradient magnitude the sample confidence increases, thereby yielding smoother distribution of confidence. Although high gradient decay rates can mitigate over-confidence distributions for samples, excessively small or large gradient magnitudes may lead to inadequate optimization. Consequently, we propose an adaptive gradient decay rate by PID controller to ensure model calibration on optimization. Additionally, we propose variable learning rate schedule to adjust the gradient and counterbalance the impact of fluctuating gradient decay rates on gradient magnitude. The whole framework is shown Fig.
3.1 PID controller approach for adaptive probability-dependenl gradient decay
Our approach focuses on regulating the gradient decay rate by monitoring the average confidence and accuracy within each bin of the validation throughout model optimization. This aims to enhance the model's calibration and dynamic properties during optimization. Eq. ① specifies the desired calibration by representing only the absolute value of the difference between average confidence and accuracy within each bin. However, it fails to capture the under-confidence and over-confidence in model calibration. Therefore, we propose Relative Calibration Error (RCE) to reflect over-confidence and inder-confidence:
Du N
RCE
(conf (Db) acc (D:))
(10)
In the control system depicted in Fig. 3 RCE serves the output, with the gradient decay coefficient acting as the controlled variable. The target RCE value is set to O. During each iteration, the model processes the validation set to calculate the RCE. If the RCE is greater than 0, it indicates over-confidence the probability distribution, necessitating an increase in the gradient decay rate during model optimization. Conversely, if the RCE is less than 0, it signifies under-confidence in the probability distribution, prompting a decrease determines the specific adjustment required for the gradient decay coefficient 3. PID controller continually computes an eтror e (t), representing the disparity between the desired optimal RCE and the control system output. then applies correction (P), integral (7), and derivative (D) terms (L). Mathematically, there is:
the gradient decay rate. The PID controller
the system, incorporating proportional
u (t) = Kye (t) + K; l (t) dt + Ka ar
(t)
(11)
5

---

Algorithm 1: PID controller-based adaptive gradient decay
Data: Training set {at Lntrain, ya }== i==;; Validation set {a:tat y/ } {=1
(rannn] N
Neal, Classification model
So: Learning rate X; Batch size Nn; Maximum iteration T maz; PID controller Kp. K2 and Kd--
Result: Classification model fe Initialize 0, 3,
while t < Tmax do
3
# Adjust gradient decay rate 3 by PID controller; RCE E of validation set <- Computing by Eq. 10
5
u (t) t Computing by 11.
Be & Computing by Eq. 12
7
# Adjust learning rate À by gradient compensation;
(1) + Computing by Eq.
A (L) < Computing by Eq. 15
10
# Optimize neural network by gradient-descent:
0 + (1) BJ.
11
30
12 end
where Kp, K₁ and Ka are the gain coefficients the P, I and D terms, respectively. The coefficients Kp. Ki and K2 determine the contributions of present, past and future errors to the current correction. (t) RCE (t) represents the error of the i th optimization epoch. Since 0 < B, the updating step described as follows:
3, = = 1 u(()
(12)
The PID controller is a widely employed feedback control mechanism in engineering and industrial
processes 30 28 It utilized in systems requiring precise control over variables. As depicted in
Fig. 3 the PID controller is utilized in the model optimization to regulate RCE, ensuring balanced model calibration and mitigating overfitting compared to model accuracy within the baseline strategy. As depicted in (I the PID comprises three terms: the proportional (P) term, integral (/) term, and derivative (D) term. Firstly, the proportional (P) action ensures that the controller responds proportionally to the current error. It provides an immediate correction to minimize the RCE. Secondly, the integral (/) action continuously integrates the error over time and adjusts the control signal accordingly, eliminating any steady-state error. It eliminates any steady-state error by gradually reducing the cumulative error to zero. Thirdly, the derivative (D) action anticipates the future behavior of the error by considering its rate of change, damping oscillations and overshoots improve system stability and transient response. PID controllers play a crucial role in maintaining stability, accuracy, and efficiency in ensuring precise model calibration by adjusting bubbability-dependdee gradient decay in model optimization.
3.2 Adaptive learning rate for gradient compensation
Changes in the gobbabiity-ddependee gradient decay rate significantly impact model calibration because varying decay rates change dynamic characteristic the optimization for different samples. While a large gradient decay rate can maintain a similar confidence level in optimizing both high- confidence and low-confidence samples, small decay rate results in curriculum learning sequence where confidence in low-confidence samples increases only when confidence in high-confidence samples reaches certain threshold. Nonetheless, varying decay rates also lead changing gradient magnitudes, thereby influencing model optimization. For instance, small gradient at the outset of optimization could result in insufficient optimization, thereby diminishing the model's generalization. To mitigate the influence of gradient magnitude fluctuations on model optimization, we implement dynamic learning rate to counteract the impact of gradient fluctuations caused by varying (3.
Since output layer. Building on (9), we establish metric to quantify the magnitude of the gradient that specific gradient decay rate
DJ
Ejfc
as = 2
LJ
aJ
can represent the gradient magnitude of the sample in
B
8=

---

Cod Corlaac
Cariiencc
Cortderca
Carfornca
(a) Uncalibrated
(b) Ours
(c) Temp. Scaling
Oue titere
Dun sitice
ACUI
Condenta
Corldance
Confoance
(d) Matrix Scaling
Vector Scaling
(f) Hist. Bin.
Figure 4: Confidence histograms and reliability diagrams for different calibration methods with ResNet35 on CIFAR-100 100. In each subplot, the left plot illustrates the sample distribution in individual bins, while the right plot displays the average confidence and accuracy each bin. Our training calibration can improve performance on confidence estimate.
a (t) = f''
1 Pc
1 (3= - 1) dpe
(13)
Since 0, we obtain
Be 8
B, #1
(t) -
(3.-1)
(14)
0.5
The learning rate, adjusted account for the gradient dynamics over t epoch, modified according to the following formula:
A(I) = ( - 1)x (1 - 1)/
(15)
where A (1) is learning rate of th optimization epoch. The assumption underlying the gradient com- pensation proposed in [3 (5 relies on the uniform distribution of samples within the probability
within the probability range [0, 1] approximates the variation in gradient amplitudes across different gradient decay coefficients B. However, this assumption may hold true during the optimization process, Nonetheless, this simple adaptive learning rate approach can effectively mitigate the issue of excessively small gradients resulting from overly large gradient decay rates. When the magnitude decreases, learning rate increases accordingly to compensate for the Gradient Decay method by
interval [0, 1]. Integrating Zzc
change in gradient amplitude. The whole Adaptive PID controller (PID-AGD) for model calibration is described in Algorithm
4 Empirical experiments
The experimental validation comprises four main components. Firstly, we assess the performance of our algorithm and other calibration methods in terms of model calibration. Secondly, compare the accuracy and model calibration results of our method with those obtained using different loss functions in supervised learning. Thirdly, we conduct ablation experiments the proportional term Kp, integral term KG, and differential term Ka across PID controller. Finally, we conducted the ablation experiment with different optimizers to verify the effectiveness of the adaptive learning rate for gradient compensation.
Train setting The baseline models include ResNet and VGG variants. The datasets comprised SVHN, CIFAR-IO/ 10/100, 102 Flower and Imageem. In CIFAR-I0/1000 the training set contained 40,000 images, with testing and validation sets comprising 10,000 images each. The ratio of training set, validation set and test set in 102 Flower is 2:1:1 respectively. For Tiny- ImageNe,, 100,000 images were used for training, and 10,000 images for testing and validation. SVHN utilized 58,606 training
7

---

Table 1: The calibration performance of different post-hoc calibration methods. The best results are in bold, and relative improvements over 2nd best result in each section are in red. Results are averaged over five runs with different seeds.
Dataset
Model
Metric
Uncalibrated
Hist. Binning
Temp. Scaling
Vector Scaling
TS-AvUC
Ours
ECE
0.160=
0.025
0.033
=0061=0.0:2
0.028:n.004
0.0061J 0.019)
CIFAR-100
88
MCE
0.078
0.059 10:011
0.138 10.u2
0.0521, (.007)
1.0.61
AdaECE
0.160
0.030s0.007
=.061=0.0:1
0.027
0.007TL 0.020)
RCE
0.172 1.172.000
0.0
0.026100001
0.056_0.011
0.035.00005
0 [ 0.015)
CIFAR-100
ResNet35
MCB
0.351
0,053L 0.000)
0.117
0.146.0.0s
1.0631
AdaECE
+.172
0.027
0.053=0.000
0.034=a0.07
0.0141]] 0.013)
RCE
0.025100000
0.03010013
_.073_0.00
0.052.0.012
0.016r] 0.009)
CIFAR-100 100
MCB
0.407
0.1 0.1
0,091 auuzz
0.153.00.00
0.116:00:01
0,156i 0.035)
AdaFCI
0.186.0.009
0.02910012
0.07 100.00
0.052.0.010
0.006 0.023)
ECE 0.240=0.10
0.035
0.029=0.003
0.035=0.000
0.044=0.005
0.013 0.016)
CIFAR-100
VGG16
MCE
0.508
0.042IL 0.002)
0.093
0.040000co
=0000
0 0.04-4::0.00
AdaFCE
0.240
0.02910000
0.035_o.006
0.044_0.005
0.012 0.017)
ECE
0.054=0.0:0
0.011=2.001
0.015
0.014=0.008
0.015=0.006
0.00911 0.002)
CIPAR-10
ResNet35
MCB
0.300
0.255
0, 0
0.07711 0.030)
0.121 =0.001
0.089
AdaFCC
0.054.o.011
0.014.000t
_.013_0.00
0.013.0.005
0.01011 0.003)
ECE
1o.00
0.0161
1.009
0.007
0.010_0.00
0.005 0.002)
SVIN
ResNet18
MCE
0.286:00058
0,251 0.013)
0.313
0.313
0.2641
AdaECE
0.021
0,010=0:005
0.009=0.002
0.013-0.008
0.006 0.003)
ECE
0,101 Lov0ns
0.084
0.086
_.093_
0.075
0.033 0.042)
102 Flower
ResNet50
MCE
0,231
0,180
0.163
0.165
0.132 0.031)
AdaECE
0.100
0.089e0.012
0.098=0.019
0.079=0.001
0,03101 0.048)
ECE 0.144
1.017
0.053 _..03_
L.017.u.00s
0.009| 0.008)
Tiny-ImageNer
ResNet35
MCE
0,236:20.0ta
0.088
0.035
0.093=0.021
0.030 (1005)
0.035t=00006
AdaFCE
1.0431
0.01710001
0.054_9..06
0.016_0.002
0.01011 0.074)
ECE
0.112 +0.011
0.079
#0.008
0.082 =0.007
=.091=0.0L6
0.074 20.010
0.044(L
0.030)
VisDrone
YOLOv3
MCE
0.232
15.033
0.325
20.000
0.178 20.005
0. 0.148 20 000
0.159 10.003
0.136(L
0.012)
AdaECE
0.113
10.012
0.085
10.00s
0.092 10.007
0.078 10.01
0.046(| 0.032)
ECE
0.115 #0.019
0.095
#00011
0.091 +0.015
0.104 20.000
0.092 #0.015
0.077(L 0.01-)
COCO
YOLO.3
MCE
0.224
0,158
10.005
0.158 1.0 007
0. 165 20 o28
0.164( 0.00T)
0.4 130
AdaECE
0.117 +0.018
#0.014
0.105 =0.018
0.094 =0.018
0,078(L 0.015)
images, 14.651 I validation images, and 26,032 testing images. VisDrone contains 5471 training images, 1548 validation images and 3190 testing images. COCO utilized 82,783 training, 40,504 validation, and 40,775 testing images. In all classification experiments, the learning rate, momentum and weight clipping were set to 0.1, 0.9 and Norm=3, respectively. The learning rate decreased to 10% at 40% and 80% of the iterations, with weight decay set 10-4 and total of 200 iterations. For Tiay-ImageNet, the learning rate was set to 0.01 with a batch size of 64. The number of bins in all calibration metric are set 10. P, and D in our method are set 1, 0.1, 1.
4.1 Calibration performance with other calibration methods
The experiments in this subsection aim to validate the efficacy of our proposed calibration method against baseline calibration methods, which employs PID control of the gradient decay rate. We compare our approach with other post-processing calibration methods, including Histogram Binning BI] Temperature Scaling [IO), Vector Scaling, and TS-AvUC [3]. The evaluation metrics employed include BCE (29), MCE, and Adaptive Expected Calibration Error (AdaECE) [12). The datasets primarily consist of 10/100, SVHN 102 Flower, Tiny-ImageNet, VisDrone and COCO, while the models predominantly belong to the VGG, ResNet and YOLO series. The comprehensive experimental results are presented in Table I The results of the visualization of all methods in ResNet35 on data CIFAR- 100 for confidence histograms and reliability diagrams are presented in Fig.
The results presented in Table I demonstrate that all methods effectively enhance the calibration of the model. However, post-procet post-processing calibration methods rely on an optimized independent output-proo propabitobbbblityy mapping, which doesn't alter the optimization process of the original model itself. Consequently, these methods can solely refine the probability distribution of the model output. Our proposed method surpasses other calibration techniques in terms of overall ECE, MCE, and AdaECE. Therefore, these experimental findings underscore the effectiveness of our approach in enhancing model calibration by dynamically adjusting the gradient decay rate during the model optimization.
8

---

Table 2: The calibration performance and accuracy of different objective functions. The best results are in bold, Results are averaged over five runs with different seeds.
CIFAR-10
CIFAR-100
Methods
Models
ACC (S))
ECF
MCF
AdaFCCE
(S))
ECF
MCF
AdaFCF
ResNet18
93.7.1.3
0.041 100010
10.075
0.042100:15
73.61:0.29
0.16010.cos
O. 0.16011.00 0.344_0.00s
Softmax
ResNet35
93,9 30
2.054
0.300
0.0
73.8:3..0
0.1 0.172.0.00
0.
0.172 o23
VGG16
92.1+0.41
0.066=0.022
0.339
0,06840.028
69.2+0.26
0,233+0.064
0,476=0.112
0.236
ResNet18
93.9 40
0.053
0.352=0.072
0.055
0. 0.185 74.2=0.51
0.501 102
0. 183 000
Cosface
ReaNet35
95.6+0.42
0.048=0.012
0.317
0.049
14.6+0.38
0,181+0.005
0,488=0.127
0.178
VGG16
92.7
0.067
0.390
0.068 10.68
71.4
0,238
0.567:2.125
0.233
ResNet18
44,5+10.1
0.038
a.337
0.040
74.1
0.082
0.222
0.085
Conter loss
ResNet35
95.5+0.61
0.043=0.010
0.280=a.039
0,045+0.012
74.2+0.31
0.098
0.250=0.00c
0.101 =0.000
VGG16
43.1+10.1
0.034
0.349
0.034
72.1+0.57
0.216
0.472=0.104 104
00.044
ResNet18
91,9+0.22
0.020=0.00s
0.156:0.03s
0.022+0.007
0.047 22.1=0.26
0.156=0.024
0.049
DCA
ResNet35
92.3:1048 40
0.035
0.186
0.034
13:1
0.067:10.01
0.184_0.05.
0.066:0.mz2
VGG16
90.7.1..8
a.027.0.00s
0.205
10007.10.00
70.9
0.13311.co
0.269.0.000
200000
ResNet18
45.01..41
0.007
10.022
0.078
0.00810..0m
14.3
0.00610.002
0..06_0.018
0.007 10.002
Ours
ResNet35
95.6
0.009
20.002
2.08920.012
0.010 000
75.4
1.0.1
0.063-2001
0.014.0.009
VGG16
92.6a0..0
0.011
0.083=0.031
0.012
71.9=0.35
0.028
0,044=0.003
0.030=0.0100
hpp
444
Alahlla AAw
(a) P:O I:0 D:0
(b) P:1
(c) P:
(d) P:1 I.0.1 D:1
(e) P:1 1:0.01 D:1
Splp
(1) P:1 1:001 D:2
(g) P.1 10.5 D:0.1 (h) P:1 1:0.5 D:0.01 (i) P:O.5 10.5 D:0.1
(i) P:5 1:0.5 D:0.1
Figure 5: Accuracy and ECE of different PID settings with ResNet35 on CIFAR-100. The preceding figures illustrate the testing accuracy and ECE outcomes in the model optimization. Notably, the accuracy appears insensitive to the PID controller configuration. Nonetheless, excessive settings of P, I, and D may compromise the stability of ECE during the model optimization.
4.2 Consistent optimization in supervised learning
Model accuracy and calibration are typically considered as two independent optimization metrics. In the baseline optimization strategy employing cross-entropy as the loss function, the model calibration tend to overfit earlier than the accuracy. To examine the impact of different optimization strategies on model calibration characteristics, Table 2 illustrates these effects. Comparative algorithms such as cosface 32), center loss 33 CE with DCA
34 and softmax-based cross-entropy were utilized, and
their performance in accuracy and uncertainty estimation observed. While cosface, center loss, and our proposed approach, showed improvements in accuracy, the former two algorithms did not consider model calibration, resulting in overconfident predictions. DCA regularity improves model calibration but hurts accuracy. Conversely, our proposed PID-based method with variable gradient decay rate ensures both model accuracy and calibration. This reaffirms the significant influence of probability-deeenden gradient decay rates on model calibration and overall performance.
4.3 Ablation experiments and analysis for PID controller
In the model optimization, both model optimization and gradient coefficient decay need consideration, constituting a bi-level optimization problem. From control system perspective, altering the gradient decay rate modifies the dynamic characteristies of subsequent iterations in the model optimization. However, its impact on model calibration may not be fully apparent in the immediate optimization

---

Table 3: Different optimizer performance in ResNet35 on CIFAR-100, Results are averaged over five runs with different seeds. Adam optimization compromises model accuracy when applied to a dynamic optimization objective using PID controller approach.
SGD
Adam
PID Controller Approach
Gradient Compensation
Accuracy
ECE
AdaECE
73.8%
0.172
0.172
72.5%
0.022
0.023
63.5%
0.023
0.024
74.7%
0.012
0.013
results and may require more iterations for observation. Thus, viewed from this perspective, the entire control system can be regarded having time lag, whereby the controlled variable 3 exhibits a certain delay concerning the RCE. While it may be challenging to mathematically describe the control system, the PID controller serves as a "black-box" controller, leveraging the integral and differential variations of the error, proving highly effective. The ablation experiments are shown in Fig. [5] We conclude that model calibration robust to the choice of PID parameters; however, setting the PID parameters too high can compromise the stability of ECE during model optimization. Based on this, we selected PID settings of 1, 0.1, and for the experiments presented above. On the other hard, in Fig. [5] the accuracy curves with our method all exhibited noticeable jitter, which requires further investigation.
4.4 Ablation experiments of different optimizer
The motivation for proposing the adaptive learning rate for gradient compensation arises from the observation that significant variations in gradient magnitude can negatively impact the model's optimization for classification error. A dynamic learning rate helps maintain relatively stable gradient magnitude. Additionally, the Adam optimizer aids in reducing gradient fluctuations. To verify the novelty of our method, Table 3 presents the performance of various optimizers when applied to our proposed PID controller- based calibration method.
Our experimental results indicate that Adam can indeed provide a more stable gradient and calibration performance, particularly in conjunction with our PID controller approach for model calibration. However, it is notable that Adam results in reduced accuracy, achieving only 63.5% on CIFAR-100 with ResNet35, significantly lower than the baseline accuracy of 73.8%. A key difference arises in the baseline case handled by Adam. In our proposed PID controller method. which adjusts the hyperparameter | during model calibration, the loss function is dynamic. While Adam retains previous gradient information, this can conflict with the current gradient vector direction because the optimization objective is dynamic. In contrast, our compensation method only modifies the learning rate and retains gradient vector direction pertinent to the current loss function. This may explain why the Adam optimizer does not yield better results.
Conclusion
The gradient decay rate plays crucial role in shaping the calibration characteristics and uncertainty distribution of deep learning throughout the dynamic optimization. Our results show a negative correlation between the gradient decay rate with increasing instance-level probability and the overall confidence distribution. This paper introduces a novel optimization approach aimed at regulating the gradient decay rate hyperparameter B, via PID controller. The goal is to achieve perfect model calibration by monitoring the proposed relative calibration error of the validation set. Within this control system framework as shown in Fig. the probabilistic gradient decay rate serves as the controlled variable, while a newly defined re relative calibration error acts as the control system output, mitigating both over-confidence and under-confidence in the model. Additionally, to address fluctuations of gradient amplitude resulting from varying gradient decay rate, a new learning rate compensation mechanism is employed. Empirical validation demonstrates that our proposed adaptive gradient decay rate optimization strategy, facilitated by PID controller, effectively enhances both the accuracy and model calibration in deep learning, ensuring adequate calibration throughout the supervised learning.
10

---

References
[1] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration modern neural networks. In International Conference Machine Learning, pages 1321-1330. PMLR, 2017.
[2] Gustavo Cameiro, Leonardo Zorron Cheng Tao Pu, Rajvinder Singh, and Alastair Burt. Deep learning uncertainty and confidence calibration for the five-class polyp classification from colonoscopy. Medical Image Analysis, 62: 2020.
[3] Jian Xu, Xinxiong Jiang. Siyang Liao, Deping Ke, Yuanzhang Sun, Liangzhong Yao, and Beilin Mao. Probabilistic prognosis of wind turbine faults with feature selection and confidence calibration. IEEE Transactions on Sustainable Energy, 15(1):52-67, 2024.
[4] Dimitrios Milios, Raffaello Camoriano, Pietro Michiardi, Lorenzo Rosasco, and Maurizio Filippone. Dirichlet-based gaussian processes for large-scale calibrated classification. In Advances Neural Information Processing Systems, volume 31, 2018.
[5] Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. In International Conference Learning Representations, 2019.
[6] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pages 1050-1059. PMLR, 2016.
[7] Deng-Bao Wang, Lei Feng. and Min-Ling Zhang. Rethinking calibration of deep neural networks: Do not be afraid overconfidence. Advances in Neural Information Processing Systems, volume 34, pages 11809-11820 11820, 2021.
[8] Meelis Kull, Telmo Silva Filho, and Peter Flach. Beta calibration: well-founded and easily implemented improvement on logistic calibration for binary classifiers. Artificial Intelligence and Statistics, pages 623-631. PMLR, 2017.
[9] Yongqiao Wang, Lishuai Li, and Chuangyin Dang. Calibrating classification probabilities with shape-restricted polynomial regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8):1813-188 2019.
[10] Christian Tomani, Daniel Cremers, and Florian Buettner. Parameterized temperature scaling for boosting the expressive power in post-hoc uncertainty calibration. In European Conference on Computer Vision, pages 555-569. Springer, 2022.
[11] Amir Rahimi, Amirreza Shaban, Ching-An Cheng, Richard Hartley, and Byron Boots. Intra order-preserving functions for calibration of multi-class neural networks. In Advances in Neural Information Processing Systems, volume 33. pages 13456-J 33456-13467, 13467, 2020.
[12] Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models. In Proceedings of the Conference on Empirical Methods Natural Language Processing, pages 1587-1598, 2015.
[13] Ranganath Krishnan and Omesh Tickoo. Improving model calibration with accuracy versus uncertainty optimization. In Advances in Neural Information Processing Systems, volume 33, pages 18237-18248, 2020.
[14] Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. Trainable calibration measures for neural networks from kernel mean embeddings. In International Conference Machine Learning, pages 2805-2814. PMLR, 2018.
[15] Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pages 2712- 2721. PMLR, 2019.
[16] Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances in Neural Information Processing Systems, volume 32, 2019.

---

[17] Aditya Krishna Menon, Ankit Singh Rawat, Sashank J Reddi, Seungyeon Kim, and Sanjiv Kumar. Why distillation helps: a statistical perspective. arXiv preprint arXiv:2005. 10419, 2020.
[18] Linwei Tao, Minjing Dong, Daochang Liu, Changming Sun, and Chang Xu. Calibrating a deep neural network with its predecessors. arXiv preprint arXiv:2302.06245, 2023.
[19] Bowen Lei, Ruqi Zhang, Dongkuan Xu, and Bani Mallick. Calibrating the rigged lottery: Making all tickets reliable. In International Conference on Learning Representations, 2022.
[20] Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 15682-116944 2021.
[21] Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. Confidence- aware learning for deep neural networks. In International Conference on Machine Learning, pages 7034-704 7044. PMLR, 2020.
[22] Bingyuan Liu, Jérôme Rony, Adrian Galdran, Jose Dolz, and Ismail Ben Ayed. Class adaptive network calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages (6070-16079, 2023.
[23] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 2980-2988, 2017.
[24] Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. Calibrating deep neural networks using focal loss. Advances in Neurul Information Processing Systems, volume 33, pages 55288-15299 2020.
[25] Arindam Ghosh, Thomas Schaaf, and Matthew Gormley. Adafocal: Calibration aawbrr adaptive focal loss. In Advances in Neural Information Processing Systems, volume 35, pages 1583-1595, 2022.
[26] Jonathan Wenger, Hedvig Kjellström, and Rudolph Triebel. Non- paramertii calibration for classification. In International Conference on Artificial Intelligence and Statistics, pages 178-190. PMLR, 2020.
[27] Siyuan Zhang and Linbo Xie. Advancing neural network calibration: The role of gradient decay in large-margin softmax optimization. Neural Newworks, 178: 106457, 2024.
[28] Wangpeng An, Haoqian Wang, Qingyun Sun, Jun Xu, Qionghai Dai, and Lei Zhang. A pid controller approach for stochastic optimization of deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8522-8531 8531, 2018.
[29] Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
[30] Kiam Heong Ang, Gregory Chong, and Yun Li. Pid control system analysis, design, and technology. IEEE transactions on control systems technology, 13(4):559-776 2005.
[31] Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In International Conference Machine Learning, volume 1, pages 609-616, 2001.
[32] Feng Wang, Jian Cheng, Weiyang Liu, and Haijun Liu. Additive margin softmax for face verification. IEEE Signal Processing Letters, 25(7):926-930 2018.
[33] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In Computer pages 499-515, 2016.
14th European conference,
[34] Gongbo Liang. Yu Zhang, Xiaoqin Wang, and Nathan Jacobs. Improved trainable calibration method for neural networks on medical imaging classification. In British Machine Vision Conference 2020.
12

---

A Appendix
A.1 Experiments compute resources
For experiments, we utilized compute resources featuring an NVIDIA A100 GPU with PCIe interface and 40GB memory capacity, accompanied by PyTorch version 1.7.0 with CUDA version 11.0. The computational backbone was supported by an Intel Xcon Gold 6278C processor.
A.2 Experiments for fixed gradient decay rate
More empirical results validating the relationship between gradient decay hyperparameter 3 and model calibration are presented in Tab. 4 Tab. aldd Tab. [6] while confidence histograms and relia- bility diagrams are depicted in Figs. 6 These experiments consistently conclude that the gradient decay rate 3 is negatively correlated with the overall confidence level of the model. Specifically, when the value / is small, the gradient decay rate decreases, leading to higher overall confidence in the model output and greater likelihood of overconfident probabilistic output. Conversely, as the value 3 increases, the gradient decay rate increases, resulting in lower overall confidence in the model's 's output and a higher probability of underfitting in the model's 's probabilistic output distribution. Although model accuracy appears somewhat linked to this hyperparameter, conclusive generalizations from theoretical perspective cannot be drawn for the current experimental results.
Table 4: Model calibration of different gradient decay and post-processing calibration. The best results in bold. Results averaged five runs with different seeds. (bins 10)
Gradient decay factor
Dataset
Model
Metric
Vector Sculing
Temp. Scaling
20
0.1
CIFAR-100
ResNet18
MCE
ECE
0.019
0.063
0.139=o.025
0.048=0.006
0.306
0.11 10.011
0.423=00078
0.161=0.00
0.039
0.135
0.064=
0.026:20.005
CIFAR-100
ResNet34
ECE
0.026
0.055=2000s
0.131 20.019
0.182.00.00
0.042:00000 006
0.038:1.006
MCE
0.087
0.162:00033
0.233
0.332:00001
0.13
CIFAR-100
VGG16
ECE
0..20.00009
0. 1013=00011
0.207
0.226 ous
0.030
0.022
MCE
1.311719.022
0.378
0.499
0.556
0.523:1.108
20.011
CIFAR-10
ResNet18
ECE
0.021 13.004
0.025
0.036
0.042.00011
0.01 100.00
0.015
MCE
0,591 10.153
0.268=20025
0.295 068
0.315
20.012
0.08920.000
Tmy-eenageNet
ResNet34
ECE
0.014
0,036=0.008
0.089
0.226=:00056
0.017=0.006
0.019
MCE
0,035 0.035:0.005 :0.005
0.069
0.166600002
0.382
0.036
0.063
Tiny- Muy-IImgeNNe
ResNet50
BCE
0.041
0.013=0.002
0. s0.001
0.188=s0.08g
0.023+0.004
0.027
MCE
0.082:40:011
0,044
0. 0.14920.000
0.377
0.039
0.067
Carfornce
Gorlderz
Carfornce
404
12
(a) 50
(b) 3 =
(c) 3 10
Apcrncy
Accenry
Accinety
(d) 3 5
(c) 8 ==
(f) 8 = 0.1
Figure 6: Confidence histograms and reliability diagrams for gradient decay with ResNet18 on CIFAR-10. 10. (bins = 10)
13

---

(a) = 50
(b) 3 = 20
(c) B 10
Accency
Acceneyy
Accunecy
Conoeace
Gorlderce
Conteenc
(d) 3 5
(c) ==1
f 3 0.1
Figure 7: Confidence histograms and reliability diagrams with ResNet34 on CIFAR-100. (bins 10)
CoR Aarre
(a) B 50
(b) 3 20
(c) B 10
Garfoncce
Corlder
Carfornce
(d) 5
(e) 8
(f) 3 0.1
Figure 8: Confidence histograms and reliability diagrams for gradient decay with VGG16 on CIFAR 100. (bins = 10)
Morre ACCETNNy
Derkircc Accumcc
(a) 8 = 20
(b) 9 = 10
(c) = 5
Orlaaie
(e) 0.1
(1) 0.01
Figure 9: Confidence histograms and reliability diagrams with ResNet50 Tiny Imageem. (bins 10)
14

---

Table 5: The performance of ResNet34 on Tiny-Im Imng-et with different gradient decay. The best results are bold. Results are averaged over five runs with different seeds. (bins 10)
Gradient decay factor
Metric
20
10
0.1
0.01
Top-1 Ace (%)
52.8
53.2
53.8
53.4
53.7
53.6
Top-5 Acc (%)
75.8
75.5
75.1
75.0
74.4
73.2
Training Ace (%)
88.6
88.5
89.7
90.3
90.9
90.5
ECE (bins 10)
0.015
0.034
0.076
0.087
0.224
0.274
MCE (bins = 10)
0.036
0.065
0.151
0.176
0.406
0.518
Table 6: The performance of ResNet50 on Tiny-ImageNet with different gradient decay. The best
results in bold. Results averaged five runs with different seeds. (bins 10)
Gradient decay factor
Metric
20
10
0.1
0.01
Top-1 I Acc (%)
55.9
56.0
56.4
56.3
56.4
56.2
Top-5 Acc (%)
77.7
78.0
77.3
76.6
76.0
74.9
Training Acc (%)
86.4
88.8
90.3
91.9
92.3
91.8
ECE (bins 10)
0.045
0.014
0.046
0.114
0.203
0.249
MCE (bins = 10)
0.084
0.044
0.082
0.151
0.388
0.476
A.3 Limitations and future works
Our work currently lacks theoretical analysis. Although all experimental findings consistently demonstrate the impact of gradient decay rate 3 model calibration, still require theoretical frameworks to explain how the gradient decay rate affects the overall confidence distribution. Our experiments indicate that large gradient decay rates result in similar confidence levels across samples, while smaller rates yield more diseriminatory levels. In essence, smaller decay rates enforce a stringent eurriculum learning sequence, whereby increased confidence in difficult samples only occurs after optimizing easier ones. Consequently, this leads to greater differentiation final confidence distribution of different samples. While these empirical observations are compelling, they lack theoretical substantiation.
Methodologically, our current approach employs PID controller control gradient decay in opti- mization. However, in practice, the effect of gradient decay rate adjustments on model calibration requires large number of epochs manifest changes. Viewed from control systems perspective, this delay indicates a substantial time lag in the control object. However, as neural network optimiza- tion processes defy mathematical description, designing effective controllers becomes inherently challenging. Furthermore, our work also lacks comprehensive statistical analysis the calibrated outputs. Future research should address how alterations in the dynamic gradient decay rate impact the internal optimization process.
Additionally, the temperature coefficient also impacts model calibration. The Softmax with small disperses the inter-class distance by adjusting the probability output to focus more on hard negative samples. Nevertheless, large information from simple positive samples. On the contrary, small makes the gradient decay slowly so that easy positive samples can be sufficiently learned up to high confidence. An appropriate B can mining more discriminative features on the whole. Similarly, large 8 only keeps the samples at the same level of confidence and cannot extract more meaningful features from challenging samples. and B improved the mining capability of Softmax two different dimensions. The exploration of optimizing the effects of both hyperparameters improve model calibration represents promising approach.
only smooth the output of all categories and cannot mine more
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Motivation and two technical contributions articulated in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please find limitations and future works in Appendx A.3
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please find Train setting in Section f and Algorithm T
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
The code available in https//githbb.com/HHHi/PP _GD Please find
Abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
ssiicyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please find Train setting Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The descriptions of the figures are in the caption. All metrics are defined, such as (I ( or added relevant references.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please find Appendix A.1
A.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper is foundational research and does not deal with applications.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "m9itaKNLrW", "reviewer": "Reviewer_iW8b", "summary": "The authors propose a method for improving the calibration of neural networks, which are known to be overconfident in their predicitions. Their method is based on modifying the softmax function to include a tunable hyperparameter- which they call the gradient decay coefficient- which is controlled throughout the optimization by assessing the level of calibration of the model on a validation set and tuning the gradient decay coefficient using a PID controller. They also propose an adaptive learning rate scheduler to ensure that the changing of the gradient decay coefficient doesn't result in vanishing gradients. The authors conduct empirical experiments to validate the effectiveness of their PID controller based approach compared to other calibration methods.", "questions": "Did the authors try different optimizers for the model itself, other than SGD (e.g Adam)? Perhaps using a different optimizer would make the learning rate scheduler unnecessary?\n\"However, post-processing calibration methods rely on an optimized independent output-probability mapping, which doesn’t alter the optimization process of the original model itself. Consequently, these methods can solely refine the probability distribution of the model output.\"- Why is this a disadvantage? \nWhat are the computational requirements of the proposed method compared to other calibration methods- in particular does the tuning of the gradient decay coefficient greatly slow down optimization?", "limitations": "The authors have adequately addressed the limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The issue of calibration and the mitigation of overconfident predictions by neural networks is clearly important, and the authors have proposed what strikes me as a sensible method to tackle it. PID controllers are widely used in industry, but remain underutilized in ML applications, so their use by the authors is refreshing. The empirical experiments conducted by the authors seem adequate to demonstrate their claims.", "weaknesses": "The definition of the gradient decay coefficient is not well motivated- it is not clear to me why this particular modification of the softmax is to be preferred over others. It is likewise not clear to me what motivates the specific form of the learning rate scheduler in formula (15). The authors ackowledge that their method does not currently have theoretical justification, which is understandable given the general lack of solid theory for deep learning, but given this situation it is hard to assess whether the proposed method is likely to succeed in more general settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Answers to Questions**\n* We appreciate the reviewer's technical comment. We had previously considered using the Adam optimizer instead of SGD to achieve a stable gradient. In response to your question, we show additional ablation studies to evaluate the optimization performance when Adam replaces SGD. Our experimental results indicate that Adam can indeed provide a more stable gradient and calibration performance, particularly in conjunction with our PID controller approach for model calibration. However, it is notable that Adam results in reduced accuracy, achieving only 63.5% on CIFAR-100 with ResNet35, significantly lower than the baseline accuracy of 73.8%.\n\n| SGD       | Adam      | PID Controller Approach | Gradient Compensation | Accuracy | ECE   | AdaECE |\n| --------- | --------- | ----------------------- | --------------------- | --------| ----- | ------ |\n| ✓         | -         | -                       | -                     | 73.8%   | 0.172 | 0.172  |\n| ✓         | -          | ✓                       | -                     | 72.5%   | 0.022 | 0.023  |\n| -         | ✓         | ✓                       | -                     | 63.5%   | 0.023 | 0.024  |\n| ✓         | -         | ✓                       | ✓                     | 74.7%   | 0.012 | 0.013  |\n\nA key difference arises in the baseline case handled by Adam. In our proposed PID controller method, which adjusts the hyperparameter $\\beta$ during model calibration, the loss function is dynamic. While Adam retains previous gradient information, this can conflict with the current gradient vector direction because the optimization objective is dynamic. In contrast, our compensation method only modifies the learning rate and retains gradient vector direction pertinent to the current loss function. This may explain why the Adam optimizer does not yield better results.\n\n* Post-processing calibration methods necessitate the creation of an additional output-probability mapping $z \\to p$ . Although this approach does not alter the decisions made in the classification task, it does increase the overall complexity of the model  $x \\to z$. The input-probability mapping  $x \\to p$ in post-processing calibration approaches is more complex than that in training-based calibration methods.\n\n* Our proposed method does not require significantly more computational resources compared to the baseline optimization strategy. It only involves adjusting the $\\beta$ parameter in the loss function during the optimization process, and this adjustment is controlled by the PID of the RCE based on the validation set. Besides, through extensive empirical experiments, we find that the dynamic tuning mechanism of the gradient decay coefficient does not significantly impact the optimization of accuracy in the model. In other words, the dynamic gradient decay rate through PID control method for model calibration does not adversely affect model convergence regarding accuracy with proposed gradient compensation. Moreover, the PID parameter settings are not sensitive to the results, as shown in Figure 5.\n\n**Motivation for gradient decay coefficient**\n\nWe would like to provide an additional explanation for reviewer to clarify the motivation of the gradient decay coefficient we have proposed.\n\nThe probability-dependent gradient decay coefficient indicates the rate at which a sample's gradient magnitude decreases as the confidence of the softmax function increases (see Figure 1). A large gradient decay coefficient means that as the model's confidence increases, its gradient decreases more rapidly. Conversely, a small gradient decay coefficient implies that the gradient's magnitude decreases more slowly as confidence increases. This allows the sample to reach higher confidence levels at smaller gradient decay rates. As shown in Figure 2, Figures 6-9 in the Appendix, and Tables 4-6, the gradient decay rate $\\beta$ exhibits a negative correlation with the confidence level for pairs of samples passing through the model. Higher gradient decay rates correlate with lower average confidence levels, motivating the use of a PID control method to manage sample confidence levels.\n\nAdditionally, Equation (5) shows that the hyperparameter $\\beta$ can be regarded as soft margin in the cross-entropy function with Softmax, allowing optimization of the samples to reach a soft confidence threshold. This approximation of the equation (5) on the max function demonstrates how this threshold compares across different class output. There is a compelling relationship between this hyperparameter $\\beta$ and the model's confidence distribution. While this connection cannot be proven theoretically, it is supported by empirical experiments and logical reasoning. To the best of our knowledge, this is the first discussion of the probability-dependent gradient decay rate in relation to model calibration.\n\nBased on the above results, we adopt this probability-dependent gradient decay rate as the controlled variable of the controller for model calibration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NA5KFkSIV5", "reviewer": "Reviewer_dGQ5", "summary": "The paper presents an approach to ensure consistent optimisation of both model accuracy and calibration. The authors used a PID-based controller for the task. The PID-based controller adjusts the gradient decay rate, which ultimately optimises the neural network by gradient descent. Further ablation studies have shown that the PID controller was effective in controlling the accuracy and ECE of the model.", "questions": "1. How sensitive is the proposed method to the choice of PID controller parameters? Could you provide a sensitivity analysis or guidelines for selecting these parameters?\n2. Why were CIFAR-10/100 and Tiny-ImageNet chosen for the experiments? Have you considered testing on other datasets, especially those from different domains, to demonstrate the generalizability of your method?", "limitations": "1. The study does not address the scalability of the proposed method to larger and more complex datasets or models. The computational overhead introduced by the PID controller and adaptive learning rate adjustments is not discussed.\n2. The paper suggests that the proposed method can prevent overconfidence, but it does not adequately address how it handles overfitting to the training data. A more detailed analysis of overfitting prevention mechanisms is needed.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is written in a very good format, covering every aspect.\n2. Proper equations and graphs are provided for a better understanding of the paper.\n3. The method is compared against other post-processing calibration methods, and it shows better results than them.\n4. The ablation study further provides deeper understanding of how the variation of the parameters of the controller effects the performance.", "weaknesses": "1. The combination of PID controllers with gradient decay for model calibration appears to be an incremental improvement rather than a groundbreaking innovation. The paper does not sufficiently differentiate its contribution from existing methods using PID controllers in optimization tasks.\n2. The paper claims to address both over-confidence and under-confidence in model predictions. However, the analysis of how well the proposed method balances these two aspects is not thoroughly presented. The paper should include more detailed experiments and discussions on this balance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Answers to Questions**\n\n* In the experiments detailed in Sections 4.1 and 4.2, the hyperparameters of the PID controllers were determined through trial-and-error. In Section 4.3, we present ablation experiments that explore various P/I/D hyperparameters in the PID controller. As shown in Figure 5, the results from these ablation experiments demonstrate that the calibration results and model accuracy are insensitive to hyperparameter settings. Different hyperparameter settings in the PID controller yield better model calibration results and produce different outcomes compared to models without PID controller. In summary, our method is robust to variations in P/I/D settings as shown in Section 4.3.\n\n* In the original submission, we validated our approach on image classification tasks using CIFAR-10, CIFAR-100, SVHN, 102 Flowers, and the Tiny-ImageNet. The above datasets have been widely used in experiments and their results are representative. As suggested by the reviewer, we applied our proposed PID Controller Approach to the object detection tasks in VisDrone and COCO, where YOLOv3 serves as the object detection model. The confidence of the object classification was calibrated, and the experimental results validated the effectiveness of our approach. \n\n|Dataset|Model|Metric|Uncalibrated|Hist. Bin.|Temp. Scaling|TS-AvUC|Ours|\n|:----------|:------:|:------:|-------------:|-----------:|--------------:|--------:|------:|\n|VisDrone|YOLOv3|ECE|0.101|0.084|0.086|0.075|0.043|\n|||MCE|0.231 | 0.365 |0.180|0.165|0.142|\n|||AdaECE |0.100|-|0.089|0.079|0.046|\n|COCO|YOLOv3|ECE|0.121|0.101|0.093|0.091|0.081|\n|||MCE|0.236|0.169|0.165|0.154|0.184|\n|||AdaECE|0.126|-|0.096|0.092|0.082|\n\n**Rebuttal to Weakness**\n\n**_The motivation and novelty of our work:_** Consider the reviewer's comment in the \"Weakness\" section. We would like to provide further explanation about the motivation and novelty of our approach for the reviewer and clarify how it differs from other PID-based optimization study.\n\nMost previous PID methods are applied to model optimization, such as gradient optimization. Moreover, many optimization techniques, like momentum and Adam, share similarities with PID control concepts. However, the problem we aim to address is not only related to optimization or expediting hyperparameter tuning but rather to ensuring model calibration during the optimization. To the best of our knowledge, no existing work has employed a PID controller approach for model calibration, making our work distinct from prior PID-based optimization strategies. We are tackling a different problem by focusing on model calibration.\n\nIn addition to utilizing a PID control method, the primary novelty of our work is the introduction of a probability-dependent gradient decay coefficient. In this paper, we have verified the relationship between this coefficient and model calibration both deductively and empirically, as demonstrated in Figures 1-2, Equations (4-9), and the experiments detailed in the Appendix. This approach provides a new perspective on model calibration, shedding light on the calibration issues prevalent in modern neural networks.\n\nOverall, this work introduces a novel approach to calibration that significantly differs from previous methods. As discussed in the paper, the connection between the gradient decay rate and model calibration offers a new explanation for the overconfidence of modern models. To the best of our knowledge, this is the first discussion of the probability-dependent gradient decay rate in relation to model calibration.\n\n**_How is calibration implemented using a PID controller approach?_**  We introduce a probability-dependent gradient decay coefficient into the loss function. This coefficient regulates the rate at which the gradient of a sample decays as the confidence level increases, as illustrated in Figure 1 and Equations (4-9). Figures 2, 6-9 in the Appendix, and Tables 4-6 demonstrate that the rate of gradient decay negatively correlates with the confidence distribution of sample pairs passing through the model. The experiments described above clarify the different phenomena of overconfidence and underconfidence exhibited by model confidence when various rates of gradient decay are chosen.  Please refer to Figure 2 and Figures 6-9. This observation motivated the use of a PID control method to manage the confidence levels of samples. \n\nDuring model optimization, we monitor the average confidence level of the dataset in real-time, thereby ensuring confidence calibration through PID control. Our work calibrates model by managing a single probability-dependent gradient decay rate and presents an innovative approach to this problem.\n\n**Rebuttal to Limitation** \n* **_Computational overhead of our approach:_** In our approach, there is just a single hyperparameter that needs to be tuned. This adjustment comes from the computation of the relative calibration error with respect to the validation set. Therefore, compared to the baseline optimization strategy, the additional computation is very small, which is negligible for the whole optimization process.\n\n* **_The scalability of the proposed method:_** In our original experiments, we evaluated our method on CIFAR-10, CIFAR-100, SVHN, FLOWER102, and Tiny-ImageNet datasets across four models and nine different methods. The publicly available datasets described above are representative. In response to the reviewer's suggestions and concerns, we applied our method to object detection tasks and demonstrated its effectiveness from different domains. See \"Answers to Questions.\"\n\n* **_Overfitting to the training data:_** Our paper focuses on the model calibration problem. The problem of overfitting for the training set is not in our consideration, but we will consider the impact of our proposed method of dynamically adjusting the gradient decay rate on overfitting in future work. Thanks to the reviewer for the suggestion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lcWbM4VURM", "reviewer": "Reviewer_3YTY", "summary": "The submitted paper proposes a PID-based controller to ensure the consistent optimisation of model accuracy and model calibration. The controller with proposed Relative Calibration Error (RCE) dynamically adjust gradient decay rate to \"control\" model confidence. By applying a learning rate compensation mechanism, the side effects of the dynamic gradient decay rate, such as fluctuations in gradient amplitude, can be mitigated.", "questions": "1, Have you thoroughly investigated any related research about PID on model calibration since no related work section in this paper? PID controller is a classical and historical method. There are several researches related to PID and model training but I'm not sure whether there are similar researches of this work. Additionally, in \"$\\textit{Line 68-69}$ [29]\", I guess you may want to highlight the success of PID controller in the machine leaning filed (Interdisciplinary filed) to enhance the feasibility of this work.\\\n2,  $\\textit{Line 102}$ : The temperature coefficient $\\tau$ is introduced as an hyper-parameter. However, it seems this hyper-parameter is fixed to 1 ($\\textit{Line 115}$) without any further discussion. Have you have tried any different value or any methods to optimise this hyper-parameter?\\\n3, Have you tried any different optimisers especially any adaptive optimisers like Adam since you want to mitigate the gradient fluctuation ,or only SGD with the proposed adaptive learning rate strategy is applied as you mentioned in $\\textit{Algorithm 1}$ (Page 7)?\\\n4, What is the strategy for selecting/tuning the value of P/I/D terms?\\\n5, Could you please explain more details about the statement (threshold related) in $\\textit{Line 178-179}$\\\n6, How did you fine-tune the comparative methods where you mentioned in $Table$ 1? Have you chosen the best result of other methods in their experiments to compare yours?\\\n$\\textbf{(Possible) Minor issues:}$\\\n1, $\\textit{Abstract}$ : you mentioned \"During model optimization, the expected calibration error tends to overfit earlier than classification accuracy, indicating distinct optimization objectives for classification error and calibration error\" which points the accuracy and calibration are two distinct but related definitions. Maybe you can briefly describe the difference between them in Introduction section for more general audience.\\\n2, $\\textit{Line 50}$ : If I understand correctly, \"These methods\" represents the \"Training-based model calibration methods\". If it is, maybe you can reaffirm the subject instead of pronoun since you use singular from in previous sentences (Line 48 and 49).\\\n3, $\\textit{Line 96}$ : \"$D_b$ contains all sample with $\\hat{p} \\in [\\frac{b}{M},\\frac{b+1}{M})$\" but you also mentioned the $b$ starts from $1$ to $M$. It seems the first bin $[0,\\frac{1}{M}]$ and the right boundary of the last bin $[..., 1]$ are missed. One simple way is to change it to $\\hat{p} \\in [\\frac{b-1}{M},\\frac{b}{M})$ and additionally make the last bin compact if I understand it correctly.\\\n4, $\\textit{Line 98}$ : \"ECE and MCE\". Better to use the full term when you first mention it in the main text. For example, \"Expected Calibration Error (ECE) and Maximum Calibration Error (MCE)\".\\\n5, Changing the label of y-axis \"\\% of samples\" in $Figure$ 2 to \"$\\times100$% of samples\" may more precise?\\\n6, The label of x-axis in $Figure$ 5 is missed. Is that \"Epoch\"?\\", "limitations": "Please see the weaknesses part.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1, The proposed Relative Calibration Error (RCE) is a simple but highly efficient way to exhibit over-confidence and under-confidence, which can be exploited by PID-based controller. \\\n2, A very clear illustration (Figure 3) is used to connect PID controller and model optimisation.\\\n3, Empirical experiments show good results for the proposed method.\\\n4, The potential application (dynamic calibration) on online machine learning.", "weaknesses": "1, This work lacks theoretical analysis as authors mentioned in the appendix.\\\n2, Please refer to the \"Questions\" part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Answers to Questions**\n* To the best of my knowledge, no existing work addresses model calibration using PID control. Most prior approaches apply PID concepts to optimization problems rather than model calibration. Our work, however, establishes a connection between model calibration and gradient decay rate by introducing a probability-dependent gradient decay coefficient. We then implement a model calibration strategy using the PID method by controlling this gradient decay rate. This novel approach introduces a probabilistically related gradient decay rate, which is controlled to achieve effective model calibration. To the best of our knowledge, this is the first discussion of the probability-dependent gradient decay rate in relation to model calibration.\n\n* The temperature coefficient and gradient decay rate have distinct effects on model optimization. As the temperature coefficient increases, the variability in confidence levels among different samples decreases, causing their confidence levels to converge. Conversely, a smaller temperature coefficient results in greater variation in confidence levels among samples. Temperature has been intensively studied in areas such as model calibration and knowledge distillation aspects, and there are many well-established optimization methods. Our method does not address the optimization of the temperature coefficient. We appreciate the reviewer for highlighting this important issue. We will jointly consider the effects of temperature and probability-dependent on model calibration in future work.\n\n* We thank the reviewer for the insightful question. We have tested the Adam optimizer to assess its ability to provide a relatively stable gradient for model parameter optimization. Below is a demo experiment. Our experiments were conducted on CIFAR-10 and CIFAR-100 using ResNet and VGG networks. The results indicate that, while Adam offers a stable gradient, its accuracy is lower compared to the SGD optimizer with dynamic gradient decay coefficients and our proposed gradient compensation approach. For example, Adam achieved only 63.5% accuracy on CIFAR-100 with ResNet35, which is significantly lower than the baseline accuracy of 73.8%.\n\n| SGD       | Adam      | PID Controller Approach | Gradient Compensation | Accuracy | ECE   | AdaECE |\n| --------- | --------- | ----------------------- | --------------------- | --------| ----- | ------ |\n| ✓ | - | -  | -| 73.8%   | 0.172 | 0.172  |\n| ✓ |- | ✓| - | 72.5%   | 0.022 | 0.023  |\n| - | ✓| ✓| - | 63.5%   | 0.023 | 0.024  |\n| ✓  | - | ✓| ✓| 74.7%   | 0.012 | 0.013  |\n\nA key difference arises in the baseline case handled by Adam. In our proposed PID controller method, which adjusts the hyperparameter $\\beta$ during model calibration, the loss function is dynamic. While Adam retains previous gradient information, this can conflict with the current optimization direction. In contrast, our compensation method only modifies the learning rate and retains gradient information pertinent to the current loss function. This may explain why the Adam optimizer does not yield better results.\n\n* The current PID settings are based on a trial-and-error approach. However, it is important to note that varying the PID setting does not significantly affect model accuracy or calibration, as demonstrated by the ablation experiments with different P/I/D settings shown in Figure 5. Different PID settings provide effective calibration results and maintain model accuracy. In summary, our method is robust to variations in P/I/D settings.\n\n* The hyperparameter $\\beta$ is designed to correlate with the overall confidence level of the samples, as illustrated in Figure 2 and Figures 6-9 in the Appendix, as well as Tables 4-6. Additionally, Equation (5) shows that the cross-entropy loss function in conjunction with Softmax can be approximated using a max function, where $\\beta$ serves as a threshold. This is further illustrated by Equation (5).\n\n**Confidence Distribution of Samples with Different Gradient Decay of Three-Layer FCNN on MNIST**\n The `#` indicates the number of samples that belong to the confidence interval.\n\n| Gradient Decay Factor |1|0.5|0.1|0.01|0.001|\n|-----------------------|-----|-----|-----|------|-------|\n| #${p_c}\\le 0.2$|903|828|1105|1325|2375|\n| #$0.2<{p_c}\\le0.4$|454|206|119|91|142|\n| #$0.4<{p_c}\\le0.6$|528|245|132|92|116 |\n| #$0.6<{p_c}\\le0.8$|1291|484|191|100|193|\n| #$0.8<{p_c}\\le1$|56824|58237|58453|58362|57147|\n\nWe will also give an example to illustrate this point. We give the confidence distribution of the MINST dataset in a fully connected network for the training set samples under varying $\\beta$. Our experiments reveal that a smaller $\\beta$ results in a higher overall confidence level in the distribution, such $\\beta=0.5$. However, when $\\beta$ exceeds a certain threshold $\\beta>=0.1$, the number of high-confidence samples decreases. This phenomenon occurs because a small decay rate creates a curriculum learning sequence, where the confidence in low-confidence samples only increases once the confidence in high-confidence samples surpasses a soft threshold. If this threshold is too high, the confidence of high-confidence samples continues to increase, while low-confidence samples fail to achieve a higher confidence level. \n\nWe also give another chart in the added PDF response, which can also give more details about the statement.\n\n* We fine-tuned the hyperparameters for all the compared methods, including the learning rate for post-processing techniques, and selected the best results from these experiments.\n\n* We thank the reviewer for the careful review and for these minor questions and suggestions! We will address all minor issues raised and revise the presentation accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Some supplementary Figures and Tables.

Meta-review: This work proposes the use of a proportional-integral-derivative (PID) controller to address overconfidence in deep neural networks. Reviews were generally positive, commenting on the excellent presentation, the importance of addressing overconfidence and minimizing calibration error, and the surprising lack of PID in the literature given its practical success in industry. Initial concerns regarding novelty relative to competing methods, scalability, and sensitivity to minor alterations to methodology, were all addressed in the discussion phase. The authors included several additional experimental results in their rebuttal that are expected to greatly improve the work. I would also suggest that the authors increase the size of axis labels to ensure plots are easily visible in the final version. Subject to these inclusions, I recommend acceptance.
Decision: Accept (poster)