Sharpness-Aware Minimization Activates the Interactive Teaching's Understanding and Optimization
Authors: Mingwei Xu
Xiaofeng Cao
Ivor Tsang
Venue: NeurIPS 2024 poster (2024)
Keywords: Interactive Teaching
Sharpness-Aware Minimization
Co-teaching
Expectation-Maximization
Optimization
Primary area: optimization_for_deep_networks

Abstract: Teaching is a potentially effective approach for understanding interactions among multiple intelligences. Previous explorations have convincingly shown that teaching presents additional opportunities for observation and demonstration within the learning model, such as data distillation and selection. However, the underlying optimization principles and convergence of interactive teaching lack theoretical analysis, and in this regard co-teaching serves as a notable prototype. In this paper, we discuss its role as a reduction of the larger loss landscape derived from Sharpness-Aware Minimization (SAM). Then, we classify it as an iterative parameter estimation process using Expectation-Maximization. The convergence of this typical interactive teaching is achieved by continuously optimizing a variational lower bound on the log marginal likelihood. This lower bound represents the expected value of the log posterior distribution of the latent variables under a scaled, factorized variational distribution. To further enhance interactive teaching's performance, we incorporate SAM's strong generalization information into interactive teaching, referred as Sharpness Reduction Interactive Teaching (SRIT). This integration can be viewed as a novel sequential optimization process. Finally, we validate the performance of our approach through multiple experiments.

Sharpness-Aware Minimization Activates the Interactive Teaching's Understanding and Optimization
Mingwei Xu +1 Xiaofeng Cao 11, Ivor W. Tsang 2
School of Artificial Intelligence, Jilin University, China CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore College of Computing and Data Science, Nanyang Technological University, Singapore xumr23@mails, jlu. edu. cn, saofengcaa@jju edu. cn, ivor tsang&cfar .a-aaaaa s-sa edu, ag
Abstract
Teaching potentially effective approach for understanding interactions among multiple intelligences. Previous explorations have convincingly shown that teach- ing presents additional opportunities for observation and demonstration within the learning model, such as data distillation and selection. However, the underlying optimization principles and convergence of interactive teaching lack theoretical analysis, and in this regard co-teaching serves as a notable prototype. In this paper, we discuss its role reduction the larger loss landscape derived from Sharpness-Awwe Minimization (SAM). Then, we classify it as an iterative pa- rameter estimation process using weteettttioo-aaimizitttee this typical interactive teaching is achieved by continuously optimizing varia- tional lower bound on the log marginal likelihood. This lower bound represents the expected value of the log posterior distribution of the latent variables under a scaled, factorized variational distribution. To further enhance interactive teaching 's performance, incorporate SAM's strong generalization information into interac- tive teaching, referred as Sharpness Reduction Interactive Teaching (SRIT). This integration can be viewed novel sequential optimization process. Finally, we validate the performance of our approach through multiple experiments.
The convergence of
1 Introduction
Backgrounds. Teaching recognized as pervasive mechanism for disseminating knowledge within human society, has found extensive application in contemporary deep learning methodologies. It serves as cornerstone for various techniques such as knowledge distillation [17
32 15]. data
distillation [43] model compression (34 77. and machine teaching, facilitating optimal training con- trol [46] 47.. Recent investigations into pedagogy have illuminated the integration large language models and multi-agent systems into educational frameworks. This novel approach emphasizes the significance of understanding interactions among multiple clients and agents. Multi-agent systems, anchored in language models, orchestrate the accomplishment of intricate tasks by assigning specific roles and preseribing behavioral norms to individual agents 18 37]. Furthermore, they exhibit the capacity transcend individual intelligence barriers through collaborative endeavors 24 6 competitive dynamics, deliberative discourses (66 5], and other strategic modalities.
Question: However, the optimization and generalization mechanisms concerning interactive teaching strategies remain an incipient stage of exploration (38). Consequently, interactive teaching-basee
"Equal contribution.
Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

strategies hold promise in furnishing substantial inductive biases for further advancement in Al agent research. Especially for bidirectional interactive teaching, which lacks sufficient attention and theoretical exploration.
In the interactive teaching paradigm, the learning algorithm represented primarily by co-teaching [16 44] has demonstrated notable efficacy in achieving successful learning outcomes in the context of noisy data. This pedagogical approach can be delineated as an interactive teaching prototype wherein two networks with identical architectures serve as peer entities, engaging in interactions aimed at selecting samples with minimal loss values for parameter refinement. Despite the practical efficacy exhibited by interactive teaching methodologies like co-teaching, a notable lacuna persists in terms of theoretical comprehension and requisite convergence analyses. In light of this, we consider co-teaching as an incipient prototype for interactive learning, warranting further scholarly exploration. more profound understanding of this interactive teaching paradigm holds the potential to yield significant insights into the dynamics governing interactions among intelligent agents [5). For instance, amid projections of diminishing data availability for training large-scale language models in the foreseeable future and the attendant challenges posed by synthesized data, ongoing research endeavors are directed towards facilitating collaborative engagements between disparate Al models to collectively generate data of higher fidelity and reliability 35 14.
The exploration of the loss landscape is paramount in elucidating the dynamics of interactive teaching. driving advancements in model generalization. Dinh et al. (10) introduce an important view that favors flat minima over sharp minima in terms of generalization. However, applying this proposition directly is hindered by overlooking the loss landscape geometric properties inherent in commonly used deep architectures. Foret et al. [13] introduce a pioneering methodology termed Sharpness- Aware Minimization (SAM), which entails delineating a perturbation neighborhood in parameter space, identifying the perturbation point that maximizes the loss value, and subsequently optimizing it via gradient descent. This formulation lends itself to a min-max optimization problem, effectively solvable through gradient descent techniques. The conception of SAM inspires contemplation on the interaction and update mechanisms of two networks within the context of interactive, within the framework of the loss landscape. While SAM operates on a single network, it necessitates the computation of two separate gradients at distinct locations. In this case, the initial step involves identifying the sharpest points within specified range of data proportions within the loss landscape. While diverging from SAM's approach, interactive teaching omits the utilization of these sharp points and instead updates parameters based on the minima of the remaining loss points.
Our contributions. In this paper. we initially ascertain that interactive teaching effectively dimin- ishes the loss landscape by strategically discarding specific subset of high-loss data points. This process serves to optimize the training procedure, particularly beneficial in scenarios characterized by noisy data, ultimately resulting in a transition to low-loss landscape during each iterative interaction. Such an approach may be construed as prior induction of bias, strategically acknowledging the presence of noise within the training dataset. Subsequently, we advance the conceptualization of interactive teaching as an EM-iterative parameter estimation technique, drawing upon seminal work by Dempster et al. [9]. The method's 's convergence is predicated upon the continual refinement of the lower bound of maximum likelihood estimation. This refinement targets the expectation of the noise posterior distribution pertaining to latent variables, enacted via a relaxation of inequalities. Moreover, to mitigate the inherent challenge of local optima within the interactive framework, we add a level of sharpness knowledge exchange that includes gradient information, which we refer to as Sharpness Reduction Interactive Teaching (SRIT). This amalgamation delineates a novel dual-level sequential optimization paradigm. Finally, empirical validation of our proposed methodology is undertaken across diverse datasets, serving to substantiate its efficacy in augmenting model generalization capabilities. In summation, this manuscript underscores the following key contributions:
From perspective centered on loss sharpness, interactive teaching methodologies, such as co-teaching, serve to facilitate parameter adjustments directed at alleviating elevated loss values within the optimization landscape. This mechanism exhibits parallels with SAM optimization, notably in its emphasis on reducing sharpness.
Our analysis establishes that interactive teaching can be delineated as a probabilistic model, with the incorporation of noisy data as latent variables shedding light on its operational intricacies. This elucidation presents robust framework for the optimization of interactive teaching methodologies.
2

---

Theoretically, our research confirms that the interactive teaching paradigm and the EM algorithm share certain underlying principles. Combined with the SAM method, it can effectively alleviate the issue of local optima. Therefore, this integration promotes better convergence towards reducing global sharpness in the optimization of loss landscape.
2 Related Work
Interactive teaching In teaching community, the use of two networks for interactive teaching has gained prominence, Blum and Mitchell [ divide examples into two views and trained separate algorithms on each, using their predictions to expand the other's training set. To address noisy labels, Malach and Shalev-Shwartz [29] propose a meta-algorithm with two identical predictors that update parameters based prediction disagreements. Jiang et al. [19] introduce MentorNet, neural network that guides a deep network (StudentNect to focus on likely correct samples, reducing overfitting to corrupted labels. Co-teaching [16] employs two networks to combat noisy labels, with each network teaching its peer using small-loss instances. Variants like co-teaching+ [44), JoCoR (36), and CNLCU 40 have emerged. Co-teaching+ selects data with inconsistent predictions, JoCoR promotes prediction consistency and applies constraints, and CNLCU uses interval estimation to account for loss uncertainty. Based on these investigations, this paper adopts co-teaching as the prototype for interactive teaching research. In contrast to the exchange of loss information in a single interaction, our proposed algorithm introduces an additional level of sharpness knowledge exchange that includes gradient information. This can be viewed as a form of dual-level interactive learning.
Loss landscape Several studies have explored the relationship between loss landscape flatness and optimization. Li et al. (25 investigate how network structures impact the loss landscape, finding that shorteut connections in ResNet lead to convergence towards better minima, while deeper models have sharper landscapes, and wider models tend to be flatter with better performance. Yao al. 42 suggest that increasing batch size increases the spectrum of the Hessian matrix, resulting in convergence towards sharper solutions and higher error rates for deeper local minima. Baldassi et al. 2 demonstrate that the error loss function exhibits few extremely wide flat minima and propose entropy-driven algorithms for searching these regions. Bisla et al. 3 derive an optimization algorithm using low-pass filters to actively search for flat regions in the deep learning optimization landscape, similar SGD.
Sharpness-Awee Minimization (SAM) SAM aims to improve the generalization performance of deep neural networks by seeking minima with flatter loss landscapes. Researchers have investigated various aspects related to the weight perturbation radius. Adaptive SAM (22) demonstrates that fixed- adaptive sharpness. Surrogate Gap Minimization [50] defines an casily computable surrogate gap, which is equivalent to the dominant eigenvalue of the Hessian matrix. Du et al. [11] propose Efficient SAM, which incorporates two training strategies: Stochastic Weight Perturbation and Sharpness-Sensinnee Sensitive Data Selection. To alleviate high complexity, Liu et al. [27] propose novel algorithm called Look- SAM that only periodically calculates the inner gradient ascent. Jiang et al. [20] design an adaptive policy based on the geometric structure of the loss function to enable random or periodic switching between SAM updates and ERM updates. Sparse SAM 31 accelerates training by introducing sparse perturbations through binary mask. Other perspectives on SAM include Andriushchenko and Flammarion [T]. who suggest that a smaller number of data points within each batch can result in better implicit bias of gradient descent for commonly used neural network architectures, and Zhang et al. [48] who propose first-order flatness to bound the maximal eigenvalue of the Hessian at local minima. Additionally, Dai al. X point out that normalization in SAM helps stabilize the algorithm and makes it less sensitive to the choice of the hyperparameter p.
radius sharpness is sensitive parameter rescaling, therefore incorporating
Preliminaries
Non- Mon-Convex Optimization for Loss Function For a training dataset ID ((x, [ where Jt; E X' represents the input and yi )' represents the outputs or targets. We use fo(x)) to denote a commonly used neural network with representing the weight parameters of the network. l 0 x R2 represents per-data-point loss function, the training of neural network is
3

---

typically a non-convex optimization problem, the empirical risk minimization is shown as Equation
i = arg min Lo (fa) where LD (fo)
1
Σ
D
(DL.
we make the assumption that the function LE (fo) is both continuous and differentiable. During each iteration, the optimizers randomly select mini-batch B, from the set D, using fixed batch size.
Sharpness-Aware Minimization (SAM) SAM 13 expects the training process to unfold in flatter region, resulting in smaller training losses around the neighborhood of the converged parameter i and improving the model's s generalization performance. The sharpness measure term is defined as the maximum change between the loss caused by parameter perturbation within a neighborhood and the previous loss, expressed as: m|xe:lellasp Lu(fo+e) LD(fe). SAM optimizes a min-max problem as depicted in Equation 2
arg min max S.AM (fo+c) 2l101k 9 clkd2SS
where o represents a pre-defined constant that limits the radius of the neighborhood, while denotes the weight perturbation vector responsible for maximizing the training loss within the neighborhood constrained by p. Å denotes hyperparameter that dominates a Le regularization term about weights. é is obtained by approximating the first-order Taylor expansion of Ln (fe+e) at 0 and solving it as classical dual norm problem:
VOLD (fo)
é arg max Ca(fe+) = e:leelasp
IV (fo)
after obtaining é, the outer optimization ming in SAM follows the usual gradient descent update for 0: 0,41 02 (0 + E), where 7 is the learning rate. The difference lies in the computation of
g(0 + +
where the gradient is evaluated A &
consists of two models, f and g, along with their corresponding weights O and 0g. The algorithm starts by shuffling the training set D, which represents a noisy dataset. In each iteration, the algorithm fetches mini-batch D from the shuffled dataset. Next, it selects a subset D, and D. of DD by choosing the instances with the smallest losses based on the models f and g respectively. This selection is done by randomly sampling a fraction R(T) of the instances with the smallest losses:
Co-teaching The architecture of co-teaching 16
Dj = arg min (((P)2)()))))))
C(f.D'); D3 arg min Dlg,D') D'i PT(P)
where R(t) is a parameter that determines the proportion of the smallest values to retain. [ represents the number of samples in the dataset D. Then, the algorithm updates the weights 0 and 8g by applying a gradient descent step using the selected subsets D,, and D, respectively:
0, = O, VVVL Dg); 0g = 0g- v/V Dy).
(5)
The algorithm updates the value of R(T)) E (0, 1] based on the current epoch T and predetermined parameter Tg. The update rule given by R(T) min To T:
4 Theoretical Analysis and Solution
In this section, we put forward three main points of view:
1. Interactive teaching methods like co-teaching update parameters by reducing high loss values in the landscape. By actively involving two teachers, models in interactive framework learn from each other's strengths through a collaborative filtering mechanism and focus on minimizing loss examples.
2. Our core assumption is that the cleanliness of the data distribution serves as a latent variable, as remains unknown within the training dataset. Based on this assumption, the interactive teaching process can be effectively exemplified as unique type of parameter iteration within the EM framework. This perspective provides a probabilistic modeling- based explanation for the iteration and convergence of interactive teaching, such co-teaching.

---

Oc+1 -nPP(linee
B711 = 0, -nnDL(bb+e
D1
L(go
algow
D
cdoon)
E(gom
Interactive teaching, Sharpness Reduction Interactive Teaching (SRIT) the plane in the figure represents the loss landscape, which gradually becomes flat during the iterative optimization process due to the receipt of flat gradient information cues from each other.
Figure I:
3. Concerning the local convergence in EM, we observe that a flatter loss landscape facilitates the optimization process in escaping local optima. Under such conditions, we incorporate the SAM to flatten the loss landscape and promote more global convergence within the interactive teaching paradigm. Building upon the exchange of loss information in a single interaction, an additional level of sharpness knowledge exchange containing gradient in- formation has been introduced, which can be regarded learning. This interactive teaching process, augmented by SAM, enables the acceptance of flatter probabiibabylity regions from the peer network, thereby enhancing both the predictive performance and generalization capability of the model, as illustrated in Figure]
form of dual-level interactive
The essence of the SAM method lies in a more refined exploration of the gradient space, implicitly utilizing second-order information about the parameter space in the loss landscape. In experiments, significantly improves generalization performance but incurs some unavoidable additional com- putations. The computational power consumption of the current algorithm complexity can be kept within a reasonable range and does not exponentially increase with the scale of data and models. This increase in complexity, compared to computational resources, is considered acceptable.
4.1 Analysis of parameter update mechanism
The critical first step While both co-teaching and SAM update network weights in two steps per iteration, they differ fundamentally in how they compute the first step. Specifically, in co-teaching, during the selection pro- cess of data points with small losses, the network is pre- trained and kept fixed. In SAM, the training data source domain remains unchanged, but perturbations are applied to the network weights. Within set distance neighbor- hood, SAM seeks the direction of maximum offset that induces the greatest change in loss values compared to the original loss, i.e., arg maxi|ellSo, Lu (fo+e) LD (fee) SAM not only seeks out points within a defined domain where the loss function undergoes the greatest change, ex- hibiting high curvature characteristics geometrically, but also aims to minimize the value of the loss function re- gions of high curvature (sharp regions). The computation V CDD(f) (fa)] implicitly depends on the Hessian proper- ties of (fe)), as the perturbation value é is determined by the gradient (L)
Loss Landscape
Cutting Hanc
Figure 2: Loss landscape, we can broadly assume that in each iteration, a fixed cut- ting plane is used to remove the peaks with high loss values.
Loss landscape perspective In the non-convex optimization of neural networks, the relationship between the magnitude of the loss value and its gradient not clearly causal. Let's 's consider network fe and set of samples (z4, y1) D. and i 1,.,,N N. The sum of the gradients of the loss function

---

can be represented by the following Equation 6 The network 9 undergoes the same process:
Ve D) E (Li(fo(z)).))) VZ(((())))))) + VL(Jo(XN): UN);
(6)
after receiving K data points D,, = arg min [(g,, D) from network gu. (zj E Dz. = 1, b..,KK K, have:
VC(s, D2) VZ(((())))))) - + VL((a(1)))))) + V (7)
further, according to Equation 6 and
we have:
VC(fo, Dy)) VC(fo, Dn) VV((o(TK+))) + (CC
(8)
((C)
VC(fo. Dz) VL Dz)
(9)
(1 (L-R(O)++(C
where D. Dg' D. (ERK+1,0K+L] (N {N]}, }} the symbol #(C) represents the quantity of data point loss values. Therefore, for the update at time + 1, gradients are computed on examples with low loss values:
0441 = 0f v/VC(Se, Dg),
(10)
Dy) C(So, = 0 - (VL(So, VC)o, (.)) = of - C (g))
(11)
As illustrated by the gradient decomposition discussed above, can be observed that co-teaching segments the loss landscape by reducing high loss values and only optimizes parameters the low loss landscape. This concept is visually depicted in the schematic diagram presented in Figure 2
4.2
EM iterative process in typical interactive teaching
The analysis in Section 4.1 reveals that the interactive teaching utilizes low-loss samples from peer network as prior knowledge, subsequently refining model parameters. This approach mirrors the iterative parameter estimation of the EM algorithm, commonly employed for probabilistic models with hidden variables. To elucidate the iterative steps and convergence, we adopt the EM framework, grounded in Maximum Likelihood Estimation. In typical interactive teaching, specifically teaching, we treat the cleanliness of training data as hidden variables, with 0y and fy representing the parameters be estimated for two structurally identical neural networks. The iterative convergence of the interactive teaching process within the EM framework is then characterized by the following proposition.
Given the training dataset D = -X;= H which contains noisy samples, and assuming the samples are independent, we define the hidden variable Z. that the c-th sample is cleaner sample, meaning it has lower loss value compared to noisy data. Z o Z {Z represents the set of hidden variables for all samples, and the corresponding latent distribution is denoted as q(Z). The joint probability of pXX,, , Zel0y, 0g) obtained by simultaneously updating neural networks I and g for X, and Z The logarithm likelihood of the observed data D has the following lower bound T.:
Proposition 4.1.
indicate
K
((zc) log PXx, Zcl0s, 0g)
L(0,,0g:g) ΣΣ>
(12)
g(Zc)
the EM algorithm approximates the maximization of log pl DD ,,0000,)) 0g) by maximizing this lower bound
L(0,, og q). Specifically, the EM iteration process for the interactive teaching paradigm is as follows:
E-step:
Q(0,,0(0) EzIDD ,, [lg [log 010,, (0 Q(0 0,) E E2g 05 log 2Z log 9,)l
(13)
(14)

---

The subscript 1D f: 00g (or 2g(D, , 0) of the expectation represents the corresponding set of low-loss samples selected by network (or g) and is used to update network g (or f).
M-step:
o(+11 arg max Q(0)) ,0),
(15)
8,
g(+1) = arg max Q( 8g
(16)
9
Please refer to Appendix the deduction process of the proposition. Remark 4.2. (1) In the E-step, we first calculate the distribution of the hidden variable DD of" 0g) using the current fixed parameters 0f) and the dataset D. In the context of interac- tive teaching, this involves selecting a specific proportion of clean samples to provide to the peer net- work. We then utilize this posterior distribution compute the expectation of log p / D D 0!").
f ,8y) optimizes the parameters of the f network to maximize the logarithm likelihood function on the subset of samples Zg selected by the g network. The network g follows the same process. And as know, maximum likelihood estimation is equivalent to minimizing the empirical loss.
(2) the M-step, of(+1) = arg maxe, Q(1)))
(3) The key distinction of interactive teaching from the conventional EM algorithm lies in one party receiving data information on the latent variable distribution from the other party. It not only allows for priori determination of which important information needs to be preserved but also introduces distribution from the counterpart, thereby increasing randomness and diversity to prevent overfitting. From a probabilistic modeling perspective, one party receives the nigh-probabilig region of clean samples from the other party, which effectively enhances the overall performance.
Algorithm 1: Sharpness Reduction Interactive Teaching (SRIT)
Input: Initial network parameters f.o 8g.. learning rate 7). fixed parameter т, iteration counts Ti and T max. maximum iteration count Nmax: pre-defined constant Output: Updated network parameters 0, and 0g.
for = to Tmax do
Shuffle the training set (noisy dataset);
for N 1 to Nmax do
Sample mini-batch D from D:
Dual-level optimization: The first level of loss information exchange.
Compute the loss of network f on D and obtain Dy:
Dj arg [ ((T)) L(f, D); I/sample R(T) [D] small-loss instances;
Compute the loss of network on D and obtain Dg:
D,, arg min p1) R(T) G(g. D) //sample R(T) |D] small-loss instances;
Dual-level optimization The second level of sharpness element exchange. Volo, (fa)
VoLo, (90)
Update ē(0,) P VoLo, [F)] and é(O,,) (oooo
Compute the approximate gradient for network f: G Vo, C(fo. Dy,) a +E(
10
11
Compute the approximate gradient for network g: G Va, L(go. D, 10, +i)))
12
Update the network parameters of f using gradient descent: 0, 0 G f:
13
Update the network parameters of g using gradient descent: 0g 0 G
end
Compute R(T) min { 12 T
15
end
4.3 Sharpness Reduction Interactive Teaching (SRIT)
Since the convergence of the EM algorithm guarantees only local optima, while SAM can flatten the loss landscape and effectively alleviate local optima, favoring global optima, incorporate SAM into
7

---

the interactive teaching process, which referred as Sharpness Reduction Interactive Teaching (SRIT) enhance performance and generalization. We conclude the proposed method novel dual-Ievel sequential optimization process. For the first level, we start by screening the required low-loss dataset ming L(fo, For the second level, we transfer the loss information the counterpart model for SAM optimization: l arg ming max; L(gote: D). Among the second level, we have two crucial steps: 1).The first step estimates the direction of the change in the network weights é(0y) and é(0,) based on the gradient information of the loss function, and then computes the new approximate gradients G and Gg.
= = L(fo. Do)lotte(1))- G. 9== Vo, Voge(go +
(17)
V V, (fo)
Vel. 125,,(0) (ne)
where E(Oy) / b, (1w)
2). The second step involves updating the parameters of networks and g based on the estimated gradients. The detailed algorithm procedure is shown in Algorithm
((0g)
P Vo (em
In summary, involves levels of interaction: the first level is the exchange of loss information, and the second level is the exchange of sharpness knowledge containing gradient information. The first level filters data that are harmful to the model. while the second level flattens the optimized loss landscape, making it less prone to local optima, thereby enhancing optimization and generalization performance.
5
Experiments
this section, we will conduct experiments on two core baselines, co-teaching [T6] and CNLCU 40)
Co-teaching has served as foundation for the development of various optimization techniques within this framework, and CNLCU a cutting-edge research achievement. In the experiment, we use four NVIDIA RTX 6000 GPUs with 24GB of memory each.
Table 1: Test accuracy (%) five datasets. The best results are highlighted in bold.
Noise type
Symmetric. 20%
Pairflip.
Tridiagonal.
Instance.
Noise ratio
20%
40%
20%
40%
20%
40%
MNIST
97.50
94.96.
95.49
91.54
96.61
92.76
95.90
91.23
Co-teaching
#0.06
10.07
#0..1
=50.15
$0.09
:0.05
10.18
99.42
99.19
99.35
98.14
99.47
98.75
99.43
98.03
SRIT
:0.03
10.03
=00022
-1:007
+0.03
1:0.05
1:0002
10010
CIFARIO
82.15
77.38
82.32
75.37
82.77
76.41
81.86
73.61.
Co-teaching
+0.09
450.15
+0008
+
+-007
+
+0.11
+
85.64
79.83
85.10
76.95
85.39
78.90
84.77
74.07
SRIT
10.15
10.12
10.20
:10.17
10.18
-10.12
11..19
-00.25
CIFARTOO
50.21
42.40
48.27
34.74
50.32
38.78
49.74
38.57
Co-teaching
+0.23
16..16
+0.11
+30.13
19
+:0.16
18
59.66
50.57
57.16
35.82
59.07
42.27
59.66
40.36
SRIT
10.6
+0.21
+0.00
1-0.16
+0.15
+.0.2
10.16
1:018
FMNIST
91.13
87.99
89.83
85.44
90.42
86.09
90.27
85.63
Co-teaching
10.09
10.09
10.10
Lo.12
10.07
10.099
Lo.12
L0.13
92.68
88.77
92.76
89.25
91.44
89.97
91.44
86.02
SRIT
+0.10
+0.13
$0.09
+00.11
+0.11
+:000
t0000
+0021
SVHN
91.83
88,72
91.49
85.09
92.16
87,51
91.26
86,33
Co-teaching
+0.08
+0.10
+0.10
+:0.15
+0.10
1:0.13
+:0.18
+:0.3
94.95
93.06
94.34
89.37
94.66
91.56
94.45
90.50
SRIT
+.0.0
+t0.05
+.0.08
+-0020
+.0.0
+-0.12
++0.08
+-0.10

---

5.1 Experimental Settings
Datasets and type of noise Based on previous research 16
44 40
we conduct experiments on
five widely used datasets to effectively demonstrate the efficacy of the co-teaching algorithm. These datasets include MNIST [23], FMNIST [41], CIFARIA 21, SVHN
[33, and CIFARTO0 [21]. In
co-teaching, we do not use validation dataset in research [16]. However, to maintain consistency with CNLCU, we use 90% of the training data and 10% as the validation set in CNLCU. In CNLCU, conduct experiments utilize various types of noise commonly used in multiple studies (28 44 39 49 40), including symmetric noise, tridiagonal noise, pairfipp noise, and instance noise. To facilitate comparison with previous research [40], we set the noise rates in the datasets to 20% and 40% respectively. On the test dataset, consider the average test accuracy of the last ten epochs as the final test accuracy, accompanied by a 95% confidence interval.
three representative datasets: MNIST, CIFARIO. and CIFARI00. We
Models and hyper-parameters For datasets, we utilize a 9-layer CNN architecture [66 with dropout and batch normalization for the classification task. In co-teaching, for all datasets, we use the Adam optimizer with a momentum 0.9, an initial learning rate of 0.001, and trained for 200 epochs. For R(T)
where T3 is set to 10 by default [16). In SAM related optimization, such as SRIT and SRCNLCU, we use an SGD optimizer with an initial learning rate of 0.1, momentum of 0.9, weight decay of 0.0001, epochs of 200, and set p 0.05 [3] It has
min Tz Ts
been pointed out by Andriushchenko and Flammarion [[] that the generalization performance of model is influenced by the number of data points within a batch. Insufficient data quantity in a batch leads to inefficient utilization of GPU accelerators, while excessively large data quantity can result in suboptimal generalization. Therefore, taking reference from []. we empirically set the batch size to 128 as an optimal choice.
Table 2: Test accuracy (%) on CEW.CCCCAARO,CIFARIIII The best results are highlighted in bold.
Noise type
Symmetric.
Pairflip.
Tridiagonal.
Instance.
Noise ratio
20%
40%
20%
40%
20%
40%
20%
40%
MNIST
98.70
98.24
98.44
97.37
98.89
97.92
98.74
97.42
CNLCU-H
10.06
10.06
10..19
10.32
10.15
10.05
10.16
10.39
99.16
98.81
99.01
98.38
99.04
98.42
98.88
97.84
SRCNLCU-H
+0.02
+0005
:00003
+0000
+0003
1.005
+0000
0.04
CIFARIO
83.03
78.33
83.39
73.40
82.52
74.79
81.93
73.58
CNLCU-H
+:0.77
+0050
+:068
11.53
+0071
+11.13
+.025
11.39
85.43
80.88
84.89
75.19
85.35
78.94
83.87
75.49
SRCNLCU-H
+.0.11
10.16
+-0.12
+.0.27
+0.14
+
+
0-0.1
CIFARI00
46.27
42.05
43.25
30.79
45.02
35.24
45.02
36.17
CNLCU-H
+0.38
10.87
=00.55
10.86
1.06
1:0.3
1:007
11.5
55,84
44.72
53.33
33.03
54.28
38.81
54.98
37.88
SRCNLCU--
10.24
10.43
1.0.22
10.27
10.19
10042
10.15
100.7
5.2 Experimental results on SRIT, Co-teaching, SRCNLCU and CNLCU
In this part, experimental results are saved in Table T and 2 In Figure 3 we showcase the test performance on co- co-teaching and SRIT, more specific details are presented in Appendix B Under all dataset and noise conditions, SRIT and SRCNLCU with SAM both consistently achieve significantly higher test accuracy compared to using co-teaching and CNLCU alone. This advantage evident across different types of noise and noise ratios. Additionally, it is apparent from the figures presented in Appendix B that incorporating SAM into the training process demonstrates remarkable generalization capabilities, effectively mitigating overfitting.
Performance on Different Datasets SRIT achieves exceptional performance on the MNIST dataset. For the CIFAR1O dataset, SRIT also performs well. This indicates that even on the more complex CIFAR LO datasct, employing SAM in interactive teaching (co-tcaching) can significantly enhance

---

Tead EPMGT Hola Molyy
licting Parterraa va Swiet Nolin Type Moder Dada
Rachar
wutter
Codry
Tosting Performance of MNIST by Noise Type under 20% Noley Data
Testing Performance of CHEARLO by Noise Type Under 20% Noisy Data
1.00
80-
70
60-
te
40-
SRITiOur))
20
SRIT/Ouss
Co-tecching
10- 50
0
50
73
100
123
150
175
200
25
75
100
125
150
175
200
Epochs
Ecochs
Figure 3: Testings of five datasets by noise type, and the noise ratio is 20%.
the model's generalization ability and accuracy. Similarly, on the FMNIST, SVHN, and CIFAR100 datasets, SRIT outperforms co-teaching in all noise scenarios. In SRCNLCU, nearly all experiments demonstrate a sigmificant advantage. should be noted that we selected the experimental results of CNLCU-H directly for comparison, without any bias.
From a macro-level perspective, co-teaching and CNLCU with SAM consistently outperformed co- teaching alone across all tested datasets and noise conditions. Whether it the simple MNIST dataset or the complex CIFARIO and CIFARIO0 datasets, SRIT demonstrate strong robustness and high accuracy. The introduction of the sharpness reduction strategy effectively improves the performance of interactive teaching methods such co-teaching and CNLCU, particularly when faced with high noise ratios and complex types of noise, resulting in even more significant enhancements.
6 Conclusions
In this paper, we first analyze how the low-loss selection of noisy data for interactive teaching reduces high-loss regions. Then, we introduce the EM framework to explore the interactive teaching mechanism, using the co-teaching algorithm as an example, which is a typical algorithm within the interactive teaching paradigm. We demonstrate that the iteration process of the typical interactive teaching algorithm follows the EM algorithm. ensuring its convergence. Since SAM makes the loss landscape flatter, it helps interactive teaching to escape local optima. Finally, based on sharpness reduction, we propose a dual-level interactive strategy to further enhance performance and general- ization, validating its effectiveness through experiments. In the future, will further investigate the strategy design interactive teaching in intelligent agents and consider how to reduce the complexity of the SAM algorithm in the context of interactive teaching.
Acknowledgments and Disclosure of Funding
This work was supported by National Natural Science Foundation of China, Grant Number: 62476109, 62206108 108, and the Natural Science Foundation of Jilin Province, Grant Number: 20240101373JC. and Jilin Province Budgetary Capital Construction Fund Plan, Grant Number: 2024C008-5, and Research Project of Jilin Provincial Education Department, Grant Number: JJ820044285KKJ
10

---

References
[1] Maksym Andriushchenko and Nicolas Flammarion. Towards understanding sharpness-aware minimization. In International Conference Machine Learning, pages 339-666. PMLR, 2022.
[2] Carlo Baldassi, Fabrizio Pittorino, and Riccardo Zecchina. Shaping the learning landscape in neural networks around wide flat minima. Proceedings of the National Academy of Sciences, 117(1):161-170, 2020.
[3] Devansh Bisla, Jing Wang, and Anna Choromanska. Low-pass -w--pass filtering sgd for recovering flat optima in the deep learning optimization landscape. In International Conference on Artificial Intelligence and Statistics, AISTATS 2022. 28-30 March 2022, Virtual Event, volume 151 of Proceedings of Machine Learning Research, pages 8299-839388 PMLR, 2022.
[4] Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training. In Pruceedings of the eleventh annual conference on Computational learning theory, pages 92-100, 1998.
[5] Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better Ilm-based evaluators through multi multi-agent debate. arXiv preprint arXi::2308.07200 2023.
[6] Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation. arXiv preprint arXiv:2309. 2023.
[7] Tejalal Choudhary, Vipul Mishra, Anurag Goswami, and Jagannathan Sarangapani. A com- prehensive survey on model compression and acceleration. Artificial Intelligence Review, 53: 5113-5155 2020.
[8] Yan Dai, Kwangjun Ahn, and Suvrit Sra. The crucial role of normalization in sharpness-aware minimization. In Advances in Neural Information Processing Systems, 2023.
[9] Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39 (1):1-22, 1977.
[10] Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In International Conference on Machine Learning, pages 1019-1028. PMLR, 2017.
[11] Jiawei Du, Hanshu Yan, Jiashi Feng, Joey Tianyi Zhou, Liangli Zhen, Rick Siow Mong Goh, and Vincent Tan. Efficient sharpness- aware minimization for improved training of neural networks. In International Conference on Learning Representations, 2022.
[12] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improv- ing factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305. 14325, 2023.
[13] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware min- imization for efficiently improving generalization. In International Conference Learning Representations, 2021.
[14] Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight, John Hughes, Tomasz Korbak, Rajashree Agrawal, Dhruv Pai, Andrey Gromov, al. Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data. arXiv preprint arXiv:2404.011 004401.,011,..... 2024.
[15] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: survey. International Journal of Computer Vision, 129(6): 1819, 2021.
[16] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018
11

---

[17] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: 11.5503..23,,2200 2015.
[18) Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprini 200.02230..0035,, 2023.
[19] Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentorne: Learning data-driven curricllum for very deep neural networks on corrupted labels. In International conference machine learning, pages 2304-231 2313. PMLR, 2018.
[20] Weisen Jiang, Hansi Yang. Yu Zhang, and James Kwok. An adaptive policy to employ sharpness- aware minimization. In International Conference on Learning Representations, 2022.
[21] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[22] Jungmin Kwon, Jeongseop Kim, Hyunseo Park, and In Kwon Choi. Asam: Adaptive sharpness- aware minimization for scale-invariant learning of deep neural networks. In International Conference on Machine Learning, pages 5905-5914. PMLR, 2021.
[23] Yann LeCun, Corinna Cortes, and Christopher J.C. Burges. The MNIST database of handwritten digits. http://yann.lecum
cww/exlb/mmiss// http://sann.cccm..cmeeexb///////
1998.
[24] Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: Communicative agents for "mind" exploration of large language model society. Advances in Neural Information Processing Systems, 2023.
[25] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 31, 2018.
[26] Tian Liang. Zhiwei Hc, Wenxiang Jiao, Xing Wang. Yan Wang, Rui Wang, Yujiu Yang. Zhaopeng Tu, and Shuming Shi. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305. 19118, 2023.
[27] Yong Liu, Siqi Mai, Xiangning Chen, Cho-Jui Hsieh, and Yang You. Towards efficient and scalable charpness-aware minimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12360-12370, 2022.
[28] Xingjun Ma, Yisen Wang. Michael E Houle, Shuo Zhou, Sarah Erfani, Shutao Xia, Sudan- thi Wijewickrema, and James Bailey. Dimensionality-drvee learning with noisy labels. In International Conference on Machine Learning, pages 3355-3364. PMLR, 2018
[29] Eran Malach and Shai Shalev-Shwartz. Decoupling" when to update" from" how to update". Advances in neural information processing systems, 30, 2017.
[30] Geoffrey McLachlan and Thriyambakam Krishnan. The EM algorithun and extensions. John Wiley & Sons, 2007.
[31] Peng Mi, Li Shen, Tianhe Ren, Yiyi Zhou, Xiaoshuai Sun, Rongrong Ji, and Dacheng Tao. Make sharpness-aware minimization stronger: A sparsified perturbation approach. Advances in Neural Information Processing Systems, 35:30950-30966 2022.
[32] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI conference on artificial intelligence, pages 5191-5198, 2020.
[33] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, number 5, page 7, 2011.
[34] Bharat Bhusan Sau and Vineeth N Balasubramanian. Deep model compression: Distilling knowledge from noisy teachers. arXiv preprint arXiv:1610.09650, 2016.
12

---

[35] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal. Nicolas Papernot, and Ross Ander- son. The eurse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305. / /743,, 2023.
[36] Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In Proceedings of the IEEE/CVF conference on computer vision and pultern recognition, pages 13726-13735, 2020.
[37] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework. arXiv preprini arXiv:2308.08/35 2023.
[38] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. arXiv preprint 200900309,,7864, 2023.
[39] Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang. Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. Advances in Neural Information Processing Systems, 33:7597-7610, 2020.
[40] Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. Sample selection with uncertainty losses for learning with noisy labels. In International Conference on Learning Representations, 2022.
[41] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 10::17000 1708.07747, 2017.
[42] Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, and Michael W Mahoney. Hessian- -bsed analysis of large batch training and robustness to adversaries. Advances in Neural Information Processing Systems, 31, 2018.
[43] Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K Jha, and Jan Kautz. Dreaming distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8715-8724, 2020.
[44] Xingrui Yu, Bo Han, Jiangehao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In International conference on machine learning, pages 7164-7173, PMLR, 2019.
[45] Ye Yuan, Can (Sam) Chen, Zixuan Liu, Willie Neiswanger, and Xue (Steve) Liu. Importance- aware co-teaching for offline model-based optimization. In A. Oh, T. Naumano, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors. Advances Neural Information Processing Systems, volume 36, pages 35718-55735. 2023.
[46] Chen Zhang, Xiaofeng Cao, Weiyang Liu, Ivor Tsang, and James Kwok. Nonparametric teach- ing for multiple learners. In Advances in Neural Information Processing Systems, volume 36, pages 7756-7786, 2023.
[47] Chen Zhang, Xiaofeng Cao, Weiyang Liu, Ivor Tsang, and James Kwok. Nonparametric iterative machine teaching. International Conference on Machine Learning, pages 40850-40870. PMLR, 2023.
[48] Xingxuan Zhang, Renzhe Xu, Han Yu, Hao Zou, and Peng Cui. Gradient norm aware minimiza- tion seeks first-order flatness and improves generalization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20247-20251 2023.
[49] Yivan Zhang, Gang Niu, and Masashi Sugiyama. Learning noise transition matrix from only noisy labels via total variation regularization. In International Conference on Machine Learning, pages 12501-12512. PMLR, 2021.
[50] Juntang Zhuang, Boqing Gong, Liangzhe Yuan, Yin Cui, Hartwig Adam, Nicha C Dvomek, sekhar tatikonda, James Duncan, and Ting Liu. Surrogate gap minimization improves sharpness-awe aware training. In International Conference on Learning Representations, 2022.
13

---

A Appendix A
Given independent observed data D, hidden variable 7, and the probability model p(D, 2,0,, 0,) with parameters f and lg. according to MLE, the optimal point estimate of O is obtained when the likelihood of the model is maximized: 0 arg maxe, p 0 0g). Suppose the training dataset is D = [X, = (Iz, )=) where 2; represents the input, yi represents the label, and noise is present. We define hidden variable Ze E (0. } to indicate whether the e-th sample is a noisy sample, where Ze 0 denotes a noisy sample and Ze = 1 denotes a clean sample. Z = EL U z = { JKK represents the set of all sample hidden variables Ze. The joint probability of p(D, Ze 1..00.000 0g) is obtained by simultaneously updating f and for X; and Ze. Considering the hidden variables, the likelihood of the discrete variable model can be expanded as follows:
==
p(D 0g. = Σ (D. (D. 0g), Z = (21 3x}
(18)
The hidden variable Z represents any unobservable random variable in the probability model, and p (D, Ze 0g,0g) is referred to as the joint likelihood of 'D and Ze- Using the general approach of MLE and considering the independence of the observed data, we can take the logarithm of the equation above to obtain:
N
K
log P(P (/ 0g. log II p (x, 0g) Σ log (X, 0g)
Σ log Σ
(X,, Zo 0z 0g)
Introducing a probability distribution 2(2) related to the hidden variables, known as the latent distribution, which can be seen as the posterior of the hidden variables given the observed data. By applying Jensen's 's inequality (og(E( 2 E[log(X)) (X)). The logarithm likelihood of the observed data 'D can be related as follows:
q (Ze)
p(D 00,0,)
log
Σ
P (X,, Ze 0g)
(19)
q (2.)
K
p (X,, Ze 8g)
9 (Zo) log
L(0,, (g.g)
(20)
(Zo)
c=1
where 0(0 9) is lower bound for log ,,, 0g). The EM algorithm 9 approximates the maximization of log p(D) 0,) by maximizing this lower bound. When 0y, og q maximize the right-hand side of the incquality, the obtained 0,, 0, at least yield local maximum for the left-hand side of the inequality. Therefore, expressing the right-hand side as L(O,, fg q), the EM algorithm aims to solve the following optimization problem:
θ 0g = arg max L(0,, 8g. 0y
(21)
where L(O,, 0g.0g q) is a lower bound for the MLE optimization problem, and the EM algorithm approximates [30] the maximum likelihood by maximizing surrogate function. Specifically, the EM iteration process for the co-teaching algorithm can be expressed using the following proposition:
Proposition A.1. Given the training dataset D = {X; (2;, y:) which contains noisy samples, and assuming the samples are independent, we define the hidden variable Z, 1 to indicate that the e-th sample is cleaner sample, meaning it has lower loss value compared to noisy data. Z = !! Z = [Za) represents the set of hidden variables for all samples, and the corresponding latent distribution is denoted g(Z). The joint probubility of v(.X., Zel0,, 0g) is obtained by simultaneously updating neural networks f and g for X, and Ze. The logarithm likelihood of the observed data D has the following lower bound L:
K
((x)) log p(X,, f,10
L(0,)0g.g)
Σ
(22)
g(Zc))
the EM algorithm approximates the maximization of log pl (D10,, 0g) by maximizing this lower bound L(0,, (g.g). Specifically, the EM iteration process for the interactive teaching paradigm is as follows:
14

---

E-step:
Q(0,,05)) 22 8, [logp(Z D 03))
(23)
Q10)) 0,) - -Ezei,,,, . logg( D 0,)l,
(24)
The subscript D 4 0g (or Z DD 0 of the expectation represents the corresponding set of low-loss samples selected by network f (or g) and is used to update network g (or f).
M-step:
0ft11 = arg max 9,
(25)
0)+1) arg max 20,
(26)
9
6g
Proof. The EM algorithm in the interactive teaching process is a set of iterative computations, consisting of two steps: the Etstep and the M-step. In the E-step, denoted as E, the previous iteration's
values of g (or o(") are fixed, and the posterior latent distribution of qf (1+1) (or 49 ((+11) with respect to L(ff, 0g. 9) is calculated. In the M-step, denoted as M, the weights of the network parameters
g((+1) (or o((++)) are updated using qg ((-1) (or 9f ((++1) to maximize lower bound L(O 8g. q). The interactive teaching starts iterating after initializing the model parameters, with the E-step and M-step alternating during each iteration. This aligns with the consistent update process of general EM algorithm. The following outlines the derivation of the E-step and M-step of the EM algorithm about the interactive teaching update process.
1. E-step (Expectation-step)
According to the objective of the EM algorithm, the E-step involves computing the latent variable dis- tribution q that maximizes the lower bound, given the fixed network parameters Oy (or 0g computed in the previous step, the lower bound is as follows L:
K Σ
q (Z,)log log (X,,, Ze 0g,0,)
q arg max L(0g, 9) arg max
Σ
(27)
q (Z.)
Taking into account the previous Inequality 20
log PD 00,0,y I(Og, 0g 9) as follows:
log P(D
f 0g) L(0,, 0g. 4)
N
K
N -
p (X,, Ze Oz, 0g)
log
(X,, 2e (0,
9 (Z.) log
(Zo)
i=1
i=1 c=1
K Σ 9 (Z.) Σ
(X,, Ze 00,00g
log (Xi 10,, 0g)
q (Z.) log
q (Zc)
c=1
N
K
(X2, Ze 1,, 0g)
a (Zc)
logp (X; 100,0 - -log"
q (Zzc)
N
K
(X, 0,, 0,) (Ew)
ạ (Zo) log
p (X,, 7e 0g,0g)
N K
4 (2x)
(Z.) log
KL (Z K1,0,,0,)1
(28)
pp (Ze Xx, (,,0s,,0)) 0g)
#=1
= 00: 9) loog ((D 0,)
Σ ((Z))lpp
(29)
where KT. represents the Kullback-Leeible divergence. Based on the properties of KI. divergence, its minimum value is achieved when the two probability distributions are equal. Therefore, when
15

---

0(++1) arg max E p fg), the lower bound L(d,, 8g. is maximized. For the t-th iteration, the E-step computed as follows, taking into account that 2om1 q (Ze) by applying Bayes' theorem, the above equation can be transformed into:
max T(0gg:00 = min Σ
KL [g( 1p (Z 0z.05,0))]
(30)
9
=p (++1) =p
z D. f.. 0g)
1++1) p z D, fy. 8(()
(31)
At this point, we have completed the calculation of the posterior distribution of latent variables the E-step of the EM algorithm, as described in Equation 31
M-step (Maximization step)
Building upon the E-step, the M-step involves solving for model parameters that maxi- mize L(0 q). For network f. the necessary condition for this extremum problem is OL(0,, Bg 9)/ f 0:
max L(O,, Og 9) + [ 0g 4) = 0, 0f
(32)
;0,
q (Ze) log (X,, Ze (..10100g)
0,
(33)
,
log p PDD, Z 6 0g)1 =0
(34)
00,
where E. represents the mathematic expectation of the joint likelihood p(D, Z 00, 0,) with respect to the hidden distribution ((). Based on this, the computation for the M-step in interactive teaching obtained follows:
0 0(++1) arg max B,
[log((Z D 16 0g)],
(35)
%
1o D 0,, )]
(36)
of
For network g. the derivation follows the same process as described above. Maximum likelihood estimation is equivalent to minimizing empirical risk in machine learning.
B Appendix B
B.1 Figures on datasets
We present the test accuracy figures for different noise types across MNIST, FMNIST.CIFARI 10. SVHN, CIFAR 100, with a noise level of 20%. It is evident from the figures that Sharpness Reduc- tion Interactive Teaching (SRIT) exhibits superior generalization performance and is less prone to overfitting.
Noise Type: Polrfiip
Noise Type: Symmatric
Noise Type:
Noise Type: Instance
300
100
100
100
40
40
40
metttechhnng 50m10uu))
teaching SAm(Uu)))
teaching SAm(Curs)
SRIT[Ouss
Figure 4: Plotting charts based on the type of noise, with noise rate of 20%. The charts compare the test performance of co-teaching and Sharpness Reduction Interactive Teaching (SRIT) on the MNIST dataset.
16

---

Noise Type: Pairfiip
Type: Symmetric
Type:
Noise Type: Instance
80
B0
80
80-
60-
Accurscy
00-
60
60
no-Seachinn SRITIOurst
no-teaching SAmOurs)
SamOurs)
tiltccccinng
20-
to-teaching
100 Epochs
150
200
100
150
200
1,00
150
200
100
150
200
Eooths
Epochs
Epechs
Figure 5: Figures based on the type of noise, with noise rate of 20%. The charts compare the test performance of co-teaching and Sharpness Reduction Interactive Teaching (SRIT) on the FMNIST dataset.
Noise Type: Pairflip
Noise Type: Symmetric
Noise Type: Trid
Noise Type: Instance
80-
D0
60
80
60
60-
40-
40-
Co-teaching
Co-teachhing SAmours)
Ga-teacting
20-
Ca-teaching SAM/Duur))
SAmOurs)
1000
100
200
Epochs
Eoochs
Epochs
Epochs
Figure 6: Figures based on the type of noise, with noise rate of 20%. The charts compare the test performance of co-teaching and Sharpness Reduction Interactive Teaching (SRIT) on the CIFARIO dataset.
Noise Type: Pairflip
Noise Type: Symmetric
Noise Type: Trid
Noise Type: Instance
80
BC
80-
80-
60-
80-
80
80
20
Co-teaching SRIT|Ours!
20-
Co-teaching SRIM(Ouss
20-
Co Ninttecching SRM(Ours
Co hi-beachhngg SAIT(Durs)
0
50
100 Epochs
150
200
0
50
100
150
200
0
50
100
150
200
0
50
100
150
200
Eoochs
Epochs
Epochs
Figure 7: Figures based on the type of noise, with noise rate of 20%. The charts compare the test performance of co-teaching and Sharpness Reduction Interactive Teaching (SRIT) on the SVHN dataset.
Noise Type: Pairflip
Noise Symmetric
Noise Type: Thidd
Noise Instance
60
60
50-
40
40-
30-
20
20-
Co-texching SkITTOuss
Co-texching SAITrurrs
Co te-tteeching SAITiOurs
Op Co-teaccing SAITiOurs)
0
50
100
150
200
0
50
100
150
200
0
50
100
150
200
0
50
100
150
200
Epochs
Eoocha
Epochs
Epochs
Figure 8: Figures based on the type of noise, with a noise rate of 20%. The charts compare the test performance of co-teaching and Sharpness Reduction Interactive Teaching (SRIT) the CIFAR 100 dataset.
17

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims put forward are consistent with theoretical and experimental results, and can be generalized under similar assumptions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The assumption of this article is that the training samples are subject to noisy conditions, and assumed that noisy data difficult to observe. We conduct experiments multiple widely-used datasets and metrics.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
18

---

Answer: [Yes]
Justification: In the paper, we make necessary assumptions regarding the theoretical results, which outlined in the main text and elaborately proven in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed description of the algorithm implementation process in the paper, along with the key experimental hyperparameters required to reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
19

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: The paper will make the code openly accessible for reproducing the experi- mental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips. wwtpcc/pccppcpcceeeeeeooocp gwwcc//gu/es///ee /oaeeeeeeoodeeumiiicccee ceabbbmmsssonni ssiniioiiooyyyyy Policyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper specifies all the training and testing details in the experimental section, including data splits, hyperparameters, selection methods, optimizer types, and more.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The article calculates confidence intervals for the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We provide relevant information about the computational resources to serve as reference for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics ftctps://neurrps.coom http:///wwrgp..ccpppa.c.aaaataaa
Question:
Answer: [Yes]
Justification: The research conducted in the paper complies with NeurlPS* ethical guidelines in all aspects.
Guidelines:
The answer NA that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses the positive societal impacts of the work performed, without addressing any negative impacts. addresses efficient interaction teaching strategies for intelligent agents, which has positive impact.
21

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not address data or models with a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have provided proper attribution and citation for the assets used in the paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
22

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not involve any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve these things.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve any potential risks for study participants.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
23

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "OsiOKvRGav", "reviewer": "Reviewer_N9mR", "summary": "This work mainly proposes an understanding of the interactive teaching algorithm update mechanism led by co-teaching, as well as further optimization using the sharpness technique. The article proceeds from two aspects: one is to summarize that the co-teaching update process can be explained by the EM method, and the second is to enhance generalization with the help of the SAM algorithm.", "questions": "See above", "limitations": "See above", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clear and easy to read, with a complete and logical argumentation. The authors attempt to understand the co-teaching training process using the principles of the EM method. Due to the limitations of the EM method, they use the SAM method to improve the experimental results.", "weaknesses": "1. The authors' arguments and experiments are only on the interactive teaching of two neural networks. If conducted on three or even more networks, it is unclear whether the insights provided in the paper would still hold. The generalization capability on three or more intelligent agents remains unknown.\n2.In terms of generalization, the authors argue that SAM can flatten the loss landscape of co-teaching, making it less prone to local optima and thus improving generalization. They provide empirical evidence in the paper, but it should be noted that this approach increases the complexity of the algorithm. SAM requires computing parameters twice for a single network update, while co-teaching already involves interaction between two networks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful question. We provide the following explanation in response to your question.\n\n**Q1**: The authors' arguments and experiments are only on the interactive teaching of two neural networks. If conducted on three or even more networks, it is unclear whether the insights provided in the paper would still hold. The generalization capability on three or more intelligent agents remains unknown.\n\n**A**: In the current trend of language model-based agents learning, we have witnessed the extensive research value of collaborative interactive learning. In the existing research on Interactive teaching, such as Co-training[1], MentorNet[2], and Decoupling[3], typically only involve knowledge exchange between two networks. Multiple networks interactions are often built upon the foundation of interactions between two networks. When training interactions involve three or more neural networks, it may entail more complex tasks, requiring strategies designed according to practical contexts. We believe that in interactions involving three or more networks, complex tasks often need to be decomposed into pairwise interactions, adopting a divide-and-conquer strategy, and then judiciously merging them. In summary, we reasonably infer that the generalization of multiple networks interactions still maintains the properties observed with two networks.\n\n**Q2**:  In terms of generalization, the authors argue that SAM can flatten the loss landscape of co-teaching, making it less prone to local optima and thus improving generalization. They provide empirical evidence in the paper, but it should be noted that this approach increases the complexity of the algorithm. SAM requires computing parameters twice for a single network update, while co-teaching already involves interaction between two networks.\n\n**A**:  Due to the inherent property of requiring two gradient computations, using SAM involves higher complexity compared to conventional interactive teaching methods. Experimental results demonstrate a significant enhancement in generalization performance on noisy data with the use of SAM. This is a reasonable cost paid to improve accuracy computation, as it only incurs a linear increase in time complexity. \n\nWhen dealing with multiple networks computations, it does not exponentially increase the computational load, but rather linearly adds complexity. However, it is important to objectively point out that in interactive teaching, multiple networks and multiple intelligent agents often need to ultimately resort to pairwise computations. In special environments with limited computational resources, such as edge computing, computational complexity cannot be ignored. In such cases, sacrificing a portion of computational accuracy and considering complexity-reducing methods like LookSAM [4] becomes necessary, but this is beyond the scope of this paper's discussion.\n\n[1] Blum A, Mitchell T. Combining labeled and unlabeled data with co-training. InProceedings of the eleventh annual conference on Computational learning theory 1998 Jul 24 (pp. 92-100).\n\n[2] Jiang L, Zhou Z, Leung T, Li LJ, Fei-Fei L. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. InInternational conference on machine learning 2018 Jul 3 (pp. 2304-2313). PMLR.\n\n[3] Malach E, Shalev-Shwartz S. Decoupling\" when to update\" from\" how to update\". Advances in neural information processing systems. 2017;30.\n\n[4] Liu Y, Mai S, Chen X, Hsieh CJ, You Y. Towards efficient and scalable sharpness-aware minimization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 (pp. 12360-12370).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c1EBIFQd9j", "reviewer": "Reviewer_hGPy", "summary": "In this paper, the author focus on the research of teaching method, choosing the well-known Co-teaching as a prototype to make deeper analysis. Co-teaching simultaneously optimizes dual networks and both networks select small-loss samples for the other larger loss landscape reduction. On the other hand, Sharpness-Aware Minimization (SAM) learns the model on a flatter loss landscape, which also provides with robustness and stability, and can be well-combined with Co-teaching. Besides, the author further explain the learning process of Co-teaching in an EM aspect. Sufficient experiments are completed to demonstrate the superior performance of the proposed method.", "questions": "Please refer to the weaknesses.", "limitations": "The proposed method is aimed to combat training data with noise. However, the author didn't offer a section claiming limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The strengths of the paper are listed as follows,\n\n1. The paper is quite well-written and easy to follow, the motivation from teaching aspect introduced in Intro is interesting.\n2. The chosen teaching prototype Co-teaching select small-loss samples for reducing larger loss landscape, while SAM provides flatter loss landscape. Such combination is moderate and well-explained in Fig.2.\n3. The experiments are done sufficiently on five datasets and different data noises to prove the performance.", "weaknesses": "The weakness of this paper are listed as following,\n\n1. From my perspective, Co-teaching and SAM can be well combined from loss landscape aspect, which has been explained in Section 4.1. On the other side, the EM algorithm seems to be an elucidation of Co-teaching method, which seems dissected with the former sections. I wonder if the author can give more explanation, or the section of EM might be optimized and replaced by sections making further optimization to Co-teaching.\n2. About the proposed method, what I most concern is the computational cost. Co-teaching cooperates dual networks to select cleaner data samples for training, which has demanded for doubled computing time. Besides, SAM calculates losses before and after perturbation, which also requires additional compute cost. I wonder the author can provides with time cost ablation on training data, comparing to Co-teaching itself and other NLL methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive question. In our response below, we further explain the motivation behind the article and provide additional experiments regarding the time complexity.\n\n**Q1**:  From my perspective, Co-teaching and SAM can be well combined from loss landscape aspect, which has been explained in Section 4.1. On the other side, the EM algorithm seems to be an elucidation of Co-teaching method, which seems dissected with the former sections. I wonder if the author can give more explanation, or the section of EM might be optimized and replaced by sections making further optimization to Co-teaching.\n\n**A**:  Regarding the description in the fourth section, there may be some confusing points that require further clarification and elaboration.\n\n- The title of the paper is \"Sharpness-Aware Minimization Activated Interactive Teaching Understanding and Optimization\". Taking co-teaching as an example, inspired by SAM, we consider the actual optimization process from the perspective of the loss landscape. The preliminary conclusion we arrive at is that co-teaching eliminates data points with large loss values, which is reflected in the loss landscape as a reduction in the surfaces with high loss values.\n\n- However, in practical experiments, co-teaching tends to overfit in the later stages of training. At this point, we find that the co-teaching process can be modeled probabilistically using a variant of the EM algorithm. The goal of co-teaching is to select clean data, while EM models clean data as latent variables. Additionally, EM easily falls into local optima, which is a primary cause of overfitting. On the other hand, SAM method demonstrates good generalization by optimizing sharp points on surfaces, reducing surface curvature, and enhancing performance.\n\n- Therefore, introducing SAM into the optimization of co-teaching helps in obtaining flatter loss surfaces, making it easier to escape local optima. These correspond to the sections in the paper as follows: Section 4.1 introduces the optimization process of co-teaching's loss landscape, Section 4.2 describes the EM parameter update process of co-teaching, and Section 4.3 details the specific algorithmic process of integrating SAM into co-teaching.\n\n**Q2**:  About the proposed method, what I most concern is the computational cost. Co-teaching cooperates dual networks to select cleaner data samples for training, which has demanded for doubled computing time. Besides, SAM calculates losses before and after perturbation, which also requires additional compute cost. I wonder the author can provides with time cost ablation on training data, comparing to Co-teaching itself and other NLL methods.\n\n**A**:  Due to the inherent property of requiring two gradient computations, using SAM results in higher complexity compared to general interactive teaching methods such as co-teaching. Experimental results from co-teaching and CNLCU show that employing the SAM method significantly enhances generalization performance on noisy data. While introducing SAM does indeed increase computational time, this is a reasonable cost paid for achieving better generalization performance. For example, on MNIST with a Noise Ratio of 0.4, the accuracy increased from [91%, 94%] to [98%, 99%], on CIFAR100 with a Noise Ratio of 0.2, the accuracy increased from [48%, 50%] to [57%, 59%]. We believe that this increase in computational cost is justified as it notably improves the stability and reliability of the model in noisy environments. The increase in computational time complexity is linear, and when dealing with larger amounts of data, it does not exponentially increase computational load.\n\nIn response to the experimental proposals from the reviewers, we have conducted disintegration experiments regarding the time cost of training data and will provide additional information in the subsequent appendix.\n\nTime expenditure (S) on $\\text{MNIST,CIFAR10,CIFAR100}$\n| Noise type | Symmetric. | Pairflip. | Tridiagonal. | Instance. |\n| :---: | :---: | :---: | :---: | :---: |\n| MNIST, Noise ratio $=0.2$  |\n| Co-teaching | 5991.20 | 5845.71 | 5758.94 | 5835.25 |\n| Co-teaching + SAM | 12447.52 | 12364.43 | 125569.80 | 12751.16|\n| CIFAR10, Noise ratio $=0.4$ |\n| Co-teaching | 6292.33 | 6416.06 | 6356.10 | 6436.37 |\n| Co-teaching + SAM | 131391.94 | 13860.54 | 13873.92 |13945.70| \n| CIFAR 100 , Noise ratio $=0.2$|\n| Co-teaching | 6524.65 | 6356.50 | 6381.92 | 6321.79 |\n| Co-teaching + SAM | 13903.19 | 13949.09  | 13727.83 | 13825.99 |\n\nIn the presence of NVIDIA RTX 6000 GPUs with 24GB of memory, we record the computational times of certain experiments. Each GPU runs a subprocess independently. In the computations for co-teaching, we accounted for the additional computational time introduced by incorporating the SAM method. By randomly selecting two different noise ratios under the MNIST, CIFAR10, and CIFAR100 datasets, along with four noise types, and calculating algorithm running times, upon analyzing the table \"Time expenditure (S) on MNIST,CIFAR10,CIFAR100\". We observe that for each dataset and noise ratio combination, the computational time with SAM added typically doubles compared to the co-teaching method without SAM. This aligns with our analysis that SAM requires an extra gradient calculation. Furthermore, the computational time on MNIST is slightly lower than that on CIFAR10 and CIFAR100. The computation times for CIFAR10 and CIFAR100 are comparable.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZxHAuNfwvi", "reviewer": "Reviewer_4Qn5", "summary": "This paper delves into the understanding and optimization of teaching strategies between artificial intelligence agents. Taking co-teaching as an example, the authors propose an understanding of co-teaching based on the EM algorithm within a probabilistic framework. Building on this, they point out that co-teaching can get stuck in local optima during the optimization update process. To address this, they propose the use of Sharpness Aware Minimization (SAM) to make the loss surface smoother and improve generalization performance.", "questions": "see weakness.", "limitations": "The limiations are discussed in the paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper provides a clear description of the explored problem. The authors offer a probabilistic understanding of heuristic interactive teaching strategies. Leveraging SAM, they propose a solution and optimization method, achieving performance improvements in their experiments.", "weaknesses": "1 The paper explores the understanding of interactive teaching from a probabilistic perspective, using co-teaching as an example. To introduce the EM framework, the authors assume the cleanliness of the data distribution is unknown and treat it as a latent variable. However, as co-teaching progresses and reduces the noise in the data, the distribution of this latent variable is likely to change. It remains unclear whether the EM process would still hold under such circumstances.\n2 The description of Proposition 4.1 in the paper remains questionable and lacks clarity. A concise explanation of the formula in the M step needs to be provided in the proposition.\n3 The authors use Proposition 4.1 with a probabilistic model to explain coteaching, making strong assumptions about certain concepts, such as the jointprobability $p(X_i, Z_c | \\theta_f, \\theta_g)$.\n4 The complexity of Algorithm 1 still needs to be optimized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough comments on this paper. Based on the comments provided, we will offer specific explanations.\n\n**Q1**: The paper explores the understanding of interactive teaching from a probabilistic perspective, using co-teaching as an example. To introduce the EM framework, the authors assume the cleanliness of the data distribution is unknown and treat it as a latent variable. However, as co-teaching progresses and reduces the noise in the data, the distribution of this latent variable is likely to change. It remains unclear whether the EM process would still hold under such circumstances. \n\n**A**: In traditional EM algorithm processes, the distribution of latent variables remains unchanged. However, in co-teaching, as the training progresses, the selected data points tend to have fewer clean data points to reduce the adverse effects of noisy data on the model in the later stages of training. An intuitive analysis reveals that the data distribution of latent variables changes. Initially, a larger number of data points with small loss values are selected, and when the model performance is not sufficiently robust, the data distribution of latent variables contains more noisy data. Subsequently, more clean samples are included, reducing the diversity of samples and potentially leading to overfitting.\n\nThis change in data distribution does not affect the subsequent update process. In extreme cases where only clean samples are selected, the EM process updated by co-teaching will operate on the same latent variable distribution rather than on different distributions as seen earlier.\n\n**Q2**: The description of Proposition 4.1 in the paper remains questionable and lacks clarity. A concise explanation of the formula in the M step needs to be provided in the proposition. \n\n**A**: In the E-step, for example, the right side of Equation (13) indicates the use of the $f$ network to select training data for the training process of the $g$ network, where $\\theta_g^{(t)}$ represents the involvement of the $g$ network in computations. On the left side of Equation (13), $Q(\\theta_f, \\theta_g^{(t)})$ corresponds to Equation (16), signifying the parameter update of the $g$ network under the existing data selected by the $f$ network. Reviewers can also refer to Remark 4.2 and the appendix section, where the parameter update process is conducted iteratively.\n\n**Q3**: The authors use Proposition 4.1 with a probabilistic model to explain coteaching, making strong assumptions about certain concepts, such as the joint probability $p(X_i,Z_c|\\theta_f,\\theta_g)$.\n\n**A**: We have indeed made a strong assumption, which is typically assumed in probability theory that there is always a certain gap between the data distribution and the actual data distribution. However, we believe there is merit to this assumption. In the joint probability $p(X_i,Z_c|\\theta_f,\\theta_g)$, we consider that in the presence of both networks $f$ and $g$, the training data distribution $X_i$ and the latent variable data distribution $Z_c$ occur simultaneously. There is a certain overlap in the data distributions between $X_i$ and $Z_c$.\n\n**Q4**: The complexity of Algorithm 1 still needs to be optimized.\n\n**A**:  As pointed out by other reviewers, the complexity of Algorithm 1 is indeed high due to involving computations of two networks and two gradient calculations in the SAM method.  However, we want to emphasize that the key aspects of our approach are: \n\n1) Experimental results show that compared to the original method, using the SAM method significantly enhances generalization performance on noisy data, for example, on MNIST with a Noise Ratio of 0.4, the accuracy increased from [91%, 94%] to [98%, 99%], which is a reasonable cost for accuracy improvement calculations.\n\n2) It has a linear increase in time complexity, increasing linearly only with the input size. \n\nIn special environments with limited computational resources such as embedded systems, and edge computing, sacrificing some computational accuracy may be necessary. In such cases, methods like LookSAM, which reduce complexity, should be considered. We will consider optimizing the computational complexity of SAM method in interactive teaching scenarios like co-teaching in the future.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S9kw1aecjs", "reviewer": "Reviewer_d2nj", "summary": "The authors investigate the parameter update mechanism of interactive teaching, represented by co-teaching, in their article. They provide an algorithmic understanding from the perspective of Expectation-Maximization (EM) and utilize SAM (Sharpness-Aware Minimization) for interactive teaching optimization from the loss landscape perspective. The authors demonstrate the effectiveness of their approach through experiments.", "questions": "see weakness.", "limitations": "The limitations are briefly discussed in the paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors present a cohesive logical framework with a clear structure in their article. The accomplished work provides insightful implications for further exploration in the domain of interactive teaching tasks.", "weaknesses": "1. The description of the experimental section in the article is somewhat weak, and the authors need to present the relationship between the baselines in the experiment. As mentioned in the paper, there are many methods with the theme of interactive teaching, such as co-teaching+, JoCoR, CNLCU. Why were experiments only conducted on co-teaching and CNLCU?\n2. The authors' choice of using co-teaching as the backbone for studying interactive teaching raises questions about its representativeness and persuasiveness as a technique.\n3. The author's description of the E-step update in Proposition 4.1 is not clear enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and for your interest in the experimental section. We will explain the rationale behind the experiments and provide details on updating certain formulas.\n\n**Q1**: The description of the experimental section in the article is somewhat weak, and the authors need to present the relationship between the baselines in the experiment. As mentioned in the paper, there are many methods with the theme of interactive teaching, such as co-teaching+, JoCoR, CNLCU. Why were experiments only conducted on co-teaching and CNLCU?\n\n**A**: Co-teaching, co-teaching+[1], JoCoR, and CNLCU were developed sequentially over time. Therefore, during the experimental process, we chose to conduct experiments on the most classic version of co-teaching and the recent CNLCU. Additionally, methods used in MentorNet[2] differ significantly from the co-teaching series. MentorNet trains an additional network to filter out noisy data, thereby providing sample weighting for the StudentNet. This approach differs from the analysis of the co-teaching algorithm in this paper from the perspectives of the loss landscape and EM. Hence, we opted to conduct multiple experiments on representative versions of co-teaching and CNLCU. \n\n**Q2**: The authors' choice of using co-teaching as the backbone for studying interactive teaching raises questions about its representativeness and persuasiveness as a technique.\n\n**A**: Strictly speaking, co-teaching cannot fully represent interactive teaching. Due to the diversity of research, various teaching strategies have their own advantages and characteristics, making it difficult to find an algorithm that can comprehensively cover the vast topic of interactive teaching. We acknowledge that co-teaching is just one of many interactive teaching methods, albeit the most representative one. Apart from co-teaching and its direct extensions, there have been numerous research efforts inspired by the co-teaching framework, such as JoCoR, co-learning[3], and CNLCU. Therefore, a deeper understanding of interactive teaching algorithms and research on optimizing generalization still require further exploration based on the specific settings of each algorithm.\n\n**Q3**: The author's description of the E-step update in Proposition 4.1 is not clear enough.\n\n**A**: In the E-step of Proposition 4.1, taking Equation (13) as an example, the left side of the equation $Q(\\theta_f, \\theta_g^{(t)})$ represents the $g$ network with fixed parameters of network $f$, where $\\theta_f$ can be considered a constant. These parameters are then inputted into Equation (16) for updating the parameters of network $g$. On the right side of Equation (13), it represents an expected log-likelihood. The latent variable data distribution $Z^{f}_c$ pertains to the data distribution selected from network $f$. The logarithmic term indicates the probability of network $g$ occurring under the data distribution $Z^{f}_c$. \n \nReference:\n\n[1] Yu X, Han B, Yao J, Niu G, Tsang I, Sugiyama M. How does disagreement help generalization against label corruption?. InInternational conference on machine learning 2019 May 24 (pp. 7164-7173). PMLR.\n\n[2] Jiang L, Zhou Z, Leung T, Li LJ, Fei-Fei L. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. InInternational conference on machine learning 2018 Jul 3 (pp. 2304-2313). PMLR.\n\n[3] Tan C, Xia J, Wu L, Li SZ. Co-learning: Learning from noisy labels with self-supervision. InProceedings of the 29th ACM International Conference on Multimedia 2021 Oct 17 (pp. 1405-1413).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their detailed questions regarding this paper. We acknowledge that this work still requires additional clarifications and explanations, broadly in three areas: **core logic**, **experiments and complexity**, and **explanations of certain principles**. We provide the following general comments and feedback for all reviewers and the area chair. 

- Firstly, regarding the core logic of the paper, it can be understood from three perspectives. 

  (1). **Motivation.**  From the loss landscape perspective, we believe that the co-teaching strategy, which involves interactive teaching through the selection of small-loss data points, reduces the high-loss plane. This is inspired by the SAM (Sharpness-Aware Minimization) method, which optimizes the sharp points of the loss plane to make the loss landscape flatter. 

  (2). **Insights.**  We offer an understanding of the co-teaching update steps as an EM (Expectation-Maximization) parameter update process. This is based on a probabilistic view, assuming noisy data as latent variables and iteratively updating parameters through mutual interaction. However, the EM algorithm has inherent drawbacks, such as easily getting stuck in local optima.

  (3). **Solutions.**  Since co-teaching does not alter the curvature of the loss landscape, incorporating the SAM method makes the loss plane flatter, enabling parameter updates to more easily escape local minima. Therefore, SAM can further optimize the co-teaching and other teaching methods.

- Secondly, in terms of experiments, we additionally provide representative experimental results to indicate that there is an increase in computation time after using SAM. This is an intrinsic property of the SAM method, which involves computing gradients twice.

   (1). **Experiment.**  Our experimental results show that the use of SAM significantly improves the performance of the co-teaching method on noisy data. For example, on MNIST with a Noise Ratio of 0.4, the accuracy increased from [91%, 94%] to [98%, 99%], on CIFAR100 with a Noise Ratio of 0.2, the accuracy increased from [48%, 50%] to [57%, 59%]. 

   (2). **Analysis.**  These results indicate that the SAM method has significant advantages in enhancing model generalization. While the introduction of the SAM method does increase computational time, this is a reasonable cost paid for achieving better generalization performance. It has a linear increase in time complexity, and the additional gradient computations do not exponentially rise with the increase in input size; they only linearly increase in complexity in terms of gradient calculations. We believe that this increase in computational cost is worthwhile as it significantly improves the stability and generalization of the model in noisy environments.

 - Finally, in the explanation provided for the formula updating process in Proposition 4.1, we elucidate the differences of the co-teaching cross-iterative parameter process from conventional gradient updating methods. Therefore, when describing it using the EM framework, we highlight the details of interactive iterative parameters in Proposition 4.1, which distinguishes it from the classical EM method.

Meta-review: This paper presents a probabilistic perspective on the co-teaching method, where the cleanliness of training data constitutes the hidden variables of the problem. Expectation Maximization (EM) can be naturally applied to this setting, but the authors argue that from an optimization viewpoint, EM may fail to produce high-quality optima for this problem. This motivates their adoption of the SAM optimizer instead of the EM algorithm to build their co-teaching algorithm. Their empirical results indeed confirm that utilizing SAM in co-teaching can consistently lead to better test accuracy across different levels and types of noise in the data.

The submission received 4 reviews, which raised some questions and concerns such as computational cost, asking for clarifications in the EM section, and how to extend the idea when co-teaching more than two networks. Authors provided a detailed response to these questions, as a result of which 3 reviewers increased and 1 reviewer decreased their initial rating. The final ratings are all in the accept zone, and overall assessment of reviewers indicates that they find the contributions of the paper non-trivial and interesting. In concordance with them, I recommend accept.
Decision: Accept (poster)