Robust Automatic Modulation Classification with Fuzzy Regularization
Authors: Xinyan Liang
Ruijie Sang
Yuhua Qian
Qian Guo
Feijiang Li
Liang Du
Venue: ICML 2025 spotlightposter (2025)
Keywords: Robustness
Fuzzy Regularization
Automatic Modulation Classification
Primary area: general_machine_learning->supervised_learning

Abstract: Automatic Modulation Classification (AMC) serves as a foundational pillar for cognitive radio systems, enabling critical functionalities including dynamic spectrum allocation, non-cooperative signal surveillance, and adaptive waveform optimization. However, practical deployment of AMC faces a fundamental challenge: prediction ambiguity arising from intrinsic similarity among modulation schemes and exacerbated under low signal-to-noise ratio (SNR) conditions. This phenomenon manifests as near-identical probability distributions across confusable modulation types, significantly degrading classification reliability. To address this, we propose Fuzzy Regularization-enhanced AMC (FR-AMC), a novel framework that integrates uncertainty quantification into the classification pipeline. The proposed FR has three features: (1) Explicitly model prediction ambiguity during backpropagation, (2) dynamic sample reweighting through adaptive loss scaling, (3) encourage margin maximization between confusable modulation clusters. Experimental results on benchmark datasets demonstrate that the FR achieves superior classification accuracy and robustness compared to compared methods, making it a promising solution for real-world spectrum management and communication applications.

Robust Automatic Modulation Classification with Fuzzy Regularization
Xinyan Liang' Ruijie Sang'
Yuhua Qian
Qian Guo
Feijiang Li"
Liang Du'
Abstract
Expectation distributionn
Automatic modulation classification (AMC) serves as foundational pillar for cognitive radio systems, enabling critical functionalities includ- ing dynamic spectrum allocation, non-cooperative signal surveillance, and adaptive waveform op- timization. However, practical deployment of AMC faces a fundamental challenge: prediction ambiguity arising from intrinsic similarity among modulation schemes and exacerbated under low signal-to-noise ratio (SNR) conditions. This phe- nomenon manifests near-identical probability distributions across confusable modulation types, significantly degrading classification reliability. address this, we propose Fuzzy Regularization- enhanced AMC (FR-AMC), novel framework that integrates uncertainty quantification into the classification pipeline. The proposed FR has three features: (1) Explicitly model prediction ambigu- ity during backpropagation, (2) dynamic sample reweighting through adaptive loss scaling, (3) en- courage margin maximization between confus- able modulation clusters. Experimental results benchmark datasets demonstrate that the FR achieves superior classification accuracy and ro- bustness compared to compared methods, making it promising solution for real-world spectrum management and communication applications.
091
Fuzzy distribution
0.42
0.1
0L045
(8)
10"2
640A
ASS
ADW
GMSK
0P8K
QPSK
0QPSK
DALLSTM
Data2016a Data2018
30.3%
FEAT
MCLDWN
24.6%
Resser
67.6%
ThreeStram
29.3%
22.22%
29.0%
20.9%
(b)
23.7%
0%
10% 20%
30%
40%
50%
Figure f. The blue line in (a) illustrates the prediction ambiguity, and the red line is the ideal case. (b) The sample proportion of prediction ambiguity in different models under different data calculated, indicating that prediction ambiguity is not an accidental phenomenon.
unmanned aerial vehicle regulation (Guo et al., 2024), and biomedical signal processing (Gao et al., 2023). The goal of signal classification is to identify the target category of digital signals collected by physical sensing devices. Al- though data acquisition technology has advanced rapidly, the unique characteristies of signals make them highly sus- ceptible interference from various noise sources during the collection process. Consequently, effectively recogniz- ing noisy data has become a prominent and widely discussed challenge today.
1. Introduction
Automatic modulation classification (AMC) plays a critical role in various fields, including electromagnetic spectrum management (Peng et al., 2021), radar systems (Li, 2020),
'Institute of Big Data Science and Industry, Key Labora- tory Evolutionary Science Intelligence Shanxi Province, Shanxi University, Taiyuan, China "Shanxi Key Laboratory of Big Data Analysis and Parallel Computing, School of Com- puter Science and Technology, Taiyuan University of Science and Technology, Taiyuan, China. Correspondence to: Yuhua Qian <jinchengqyhhe 126.com>.
Deep learning has achieved remarkable success in various fields such as natural language processing (Zini & Awad, 2022), image processing (Qian & Fouhey, 2023) and multi- modal learning (Liang al., 2025). In recent years, it has also been introduced into signal classification tasks, significantly improving classification accuracy (Yuan et al., 2025; Zeng et al., 2024). However, with the continuous evolution of signal modulation methods, some modulation methods the signal modulation category are only different
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Robust Automatic Modulation Classification with Fuzzy Regularization
in the code bits. We plot the IQ of some modulation classes (see A.I) and find that there is little difference between these modulation modes with only different coding bits, such as 8PSK and QPSK. Models often exhibit low accuracy when recognizing such categories.
We identify the prediction ambiguity phenomenon in automatic modulation classification and demonstrate that the model has the potential to achieve more optimal parameters by reducing ambiguity in its predictions.
2) We quantify the degree of prediction ambiguity and pro- posed Fuzzy Regularization (FR) with an adaptive gra- dient update mechanism. By penalizing model's 's pre- diction ambiguity, the model can be effectively guided toward learning more robust and optimal parameters.
To investigate the classification challenges of these fine- grained modulation types, we analyze the Softmax values of the model's final layer, and observe an interesting phe- nomenon: the model's predictions for fine-grained samples are uncertain. For instance, in a nine-class classification task, when one model takes signal samples as input, and it outputs nine values representing the probabilities of the sam- ple belonging to each of the nine categories. Among these probabilities, the difference between the top two rankings is very small. For example, the highest predicted probability is 0.45, while the second-highed is 0.42, as shown by the blue curve in Fig. (a)) This phenomenon is referred to in this paper as the model prediction ambiguity phenomenon. We believe that the essential reason for this phenomenon is that the model has not learned effective features between similar classes. So, the model is easily confused when predicting such samples, and the prediction results are uncertain. Ide- ally, the distribution should resemble the red curve in Fig- (a)) where category predictions are more definitive.
3) We evaluate FR on the RADIOML 2016.10a RA- DIOML 10b, and RADIOML 2018.01A datasets. The results demonstrate that Fuzzy Regularization not only enhances the model's robustmess but also im- proves its convergence speed to certain extent.
2. Related Work
2.1. Automatic Modulation Classification
Automatic modulation recognition, as a key technology for managing the electromagnetic spectrum, has attracted increasing attention in recent years. Compared to previ- ous methods based on hypothesis testing (Panagiotou et al., 2000; Wang & Wang, 2010; Xu et al., 2010) or feature en- gineering (Boutte & Santhanam, 2009; Su, 2013; Wu et al., 2008), deep learning- based automatic modulation methods show superior performance and can automatically construct effective discriminative features. Li et al. proposed a deep convolutional neural network based on AN-SF-CNN for very high frequency (VHF) radio signals (Li et al., 2018). 8). Zhang et al. (Zhang et al., 2020) proposed a CNN-LSTM dual-stream architecture that effectively explores the inter- action between time and spatial features. Xiao et al. (Xiao et al., 2023) designed a complex-
Unfortunately, prediction ambiguity phenomenon com- monly exists in AMC. To illustrate the fact, we conducted classification tasks on the Data2016a and Data2018 datasets using five methods. These samples are characterized with IQ features with lengths of 128 and 1024, respectively. Dur- ing the experiments, we counted the number of samples in the final training batch where the difference between the highest and second-highess probabilities was less than 0.3. Such samples were considered to exhibit prediction ambi- guity. We also calculated the proportion of these samples relative to the entire training set. From the bar chart Fig. (b). it evident that approximately 25% of samples in each of the ten tasks exhibited prediction ambiguity, indicating that prediction ambiguity phenomenon is not a coincidence.
network that di- rectly models the complex relationships between complex- valued features. Zhang al. (Zhang al., 2023) intro- duced an automatic modulation classification method based on a multispectral attention mechanism from a frequency perspective. Peng et al. (Peng et al., 2018) transformed signal data into image representations for modulation clas- sification. In summary, existing methods mainly focus on model architecture and model input. In contrast, we propose Fuzzy Regularization-a universally applicable regulariza- tion strategy that operates in the prediction space of auto- matic modulation recognition (AMR). This paradigm shift from archittettur//nnput-enniic to prediction-centric regu- larization establishes a new framework for building robust AMR systems in open-world scenarios.
In this paper, we conducted a modeling analysis of this phenomenon, quantified the degree of prediction ambigu- ity in the model, and examined the relationship between prediction ambiguity and regularization gradients during training. When prediction ambiguity is pronounced, the regularization should return larger gradient update to sup- press the phenomenon. As prediction ambiguity diminishes, the regularization gradient should decrease correspondingly to ensure stable model training. Ultimately, we proposed a Fuzzy Regularization with an adaptive gradient mechanism to mitigate prediction ambiguity and guide the model toward learning more robust parameters in noisy environments.
2.2. Regularization
Regularization technique is frequently employed to better guide parameter optimization, ultimately reducing valida- tion errors. Many strategies have been proposed typically
In summary, ours main contributions are as follows:
2

---

Robust Automatic Modulation Classification with Fuzzy Regularization
targeting domain-specific challenges. In the field of image recognition, defend against adversarial attacks, Lee et al. (Lee et al., 2022) proposed gradient diversity regu- larization to constrain and reduce gradient concentration, thereby building robust neural networks. Zhao et al. (Zhao et al., 2024) enhanced model performance by imposing penalties that encourage deterministic overall likelihood pre- dictions. In multi-modal tasks, Ghahremani et al. (Ghahre- mani Boozandani & Wachinger, 2024) used the Frobenius norm to build regularization--ased batch normalization technique to reduce confounding effects and achieve in- dependence between different modalities. For the open- set problem in automatic modulation recognition, Li et al. (Li et al., 2023) designed a margin prototype constraint to reduce the open space size by restricting the sample dis- tribution range, thereby reducing the risk of open space. Additionally, Szegedy et al. (Szegedy et al., 2016) proposed a label smoothing regularization to alleviate the overfitting problem of the model. Compared with the aforementioned regularization methods, the fuzzy regularization proposed in this paper considers the impact of the regularization gradient during the model training process and design an adaptive gradient update mechanism.
with target label Y {V), Among them, &/2 samples are labeled as y1 and labels are labeled as y2. In addition during training the true labels are one-hot vectors and the model outputs are [o(" _b""), with v56 denoting the prob- ability that the ith training sample belongs to class j. j.The corresponding cross-entropy loss (Loatch) is:
2
Lbatch Σ L
(
log log(U,)) (0)
1=1
i=1
j=1
log(U1
log(0 (0)))))
+
4 = k log
k
1
log(1 - 01 )( (1)
Define pt is an independent and identically distributed random variable, represents the highest probability value predicted by the model for each category of the ith sam- ple (p: = Max(i) (") .322 D). For a batch of k samples in which m samples are misclassified, the corresponding cross- entropy loss (C((nd))) can be expressed as:
3. The Proposed Method
m
k
Sbatch (pi)
Σ
log(1 pt) + Σ log(p;)). (2)
This section primarily introduces the construction of the Fuzzy Regularization (FR) mechanism Subsection 3.1 presents a modeling analysis of the prediction ambiguity phenomenon, demonstrating that the model has the opportu- nity learn better parameters by suppressing the ambiguity. Subsection 3.2 introduces how entropy and variance func- tions can be used to quantitatively measure the prediction ambiguity. Subsection 3.3 explains the implementation of the adaptive gradient mechanism and final definition of FR.
i=m+1
If the probability of the model classifying a single sample correctly is a, then the probability of classification error is In binary classification task involving k samples, the probability that the model misclassifies m samples is:
p(m)
batch (a) CF (1 a ()) (a)--
(3)
Combining Eqs. (2) and (3), we construct the following expected loss model:
3.1. Problem Modeling
E(Lbateh) )
=
plrac (a) ((m) Clmud (p:).
(4)
As mentioned in the introduction, when prediction ambi- guity occurs in signal classification tasks, it indicates that the model has the potential to learn more robust parameters. This is because, for the cross-entropy loss function, when the model classifies correctly, the larger the maximum pre- dicted probability for the correct class and the smaller the predicted probabilities for other classes, the lower the cross- entropy loss. So why does not the model further optimize this part of the loss?
m=0
Due to the model's poor discriminative ability between these two classes, the classification accuracy a can be assumed to fluctuate around 0.5. According to the law of large numbers, as the number of trials increases, the sample mean of the random variable will converge to its theoretical expectation, thus a = 0.5. Moreover, during the same training batch, the model's performance is unlikely to change significantly. This implies that the highest value of the predicted probabil- ity tend to stabilize and can be approximated by a constant u (0 < < 1). Consequently, it can further be assumed that p, Based on these assumptions. Eq. (4) can be simplified as follows:
We attempt to model and analyze this issue from a prob- abilistic statistics perspective. To simplify the modeling, assume this is a binary classification problem with single target. For training batch X {x1, 22..2....... ,2...., rx)
'The code is available at https: httpss:/githhui /gtph/gttthh//////// com/
Σ (m) 1 S/md (w),
rui FFoessann//R/AAeee
B(Lhatch)
(5)
m m=0
3

---

Robust Automatic Modulation Classification with Fuzzy Regularization
where Clare (u) = P(mn (t) (5) CT.
| x (og(1 - u) + (k- - -m)) log(u)),
Let M denote the prediction fuzzy loss for c-class classifi- cation problem, where higher values of M indicate greater ambiguity in the model's predictions. We can quantify it using either the entropy function or the L2 norm. The two formulations are given as follows:
By reorganizing and combining Eq. (5), Eq. (4) can be further expanded as:
k = Σ batch (g ()) () (u)
E(Lbatch)
M = - Σ
Dg log(ūs);
m=
(6)
1=1
= (log(u) + -logg(1 - u]))
M = Σ
(ū 1=1
1 2
(8)
Q² (log(u- -1²)),,
where --= -ー= Ck - i))-
where új represents the model's prediction probability value that the sample may belong to each class.
During the training process, the model adjusts its parameters by optimizing the objective function arg min,, E(Chatch). We observe that when u = 0.5, the expected loss E(Cbutch) reaches its minimum. We believe this result occurs because the model's discriminative ability is weak when dealing with similar classes. Even after multiple iterations, the model fails to effectively extract the critical discriminative features of these similar classes. As result, the model exhibits "lazy" behavior, tending to smooth the predicted probability distribution, which leads to prediction fuzziness. As seen in the above formula, by fuzzifying the prediction distribution, the model can effectively control the growth of the loss. To alleviate this issue, we propose Fuzzy Regularization (FR). When the model exhibits weak discriminative power for sim- ilar classes and shows prediction fuzziness, FR introduces an additional penalty term. This term forces the model to learn the relationships between the top k predicted classes, thereby encouraging the model to learn better parameters and improve its robustness.
3.3. Adaptive Adjustment of the Update Gradient
After constructing the quantitative representation of the pre- diction ambiguity, we attempt to directly add Eq. (7) or Eq. (8) from Section 3.2 as regularization term to the loss function, aiming to impose an additional penalty when prediction ambiguity occurs, in order to suppress this phe- nomenon. However, the experimental results were not as expected. Upon analysis, we found that this issue might be due to the gradient update strength of the entropy function or L2 norm. See A.2 for detailed analysis. The final definition of fuzzy regularization (FR) is as follows:
When the prediction is ambiguous, the loss function should return larger value; when the prediction ambi- guity is not significant, the loss function should return a smaller value.
When the prediction is ambiguous, the gradient of the loss function should correspond to a larger value; when the prediction ambiguity is not significant, the gradient of loss function should correspond to a smaller value.
3.2. Quantifying Ambiguity Degree of Predict Results
To optimize the prediction fuzziness issue, we first need to quantify and define the degree of fuzziness in the model's predictions. The degree of fuzziness essentially reflects the level of disorder in the model's predictive distribu- tion, which represents the degree of informational disorder. Therefore, one can naturally think of entropy functions and 1.2 norms, both of which describe the degree of disorder in the information. Thus, we can use entropy functions and L2 norms to quantify the degree of prediction fuzziness.
Under the binary classification problem, we assume that the probability value of the first category is p. then the probabil- ity value of the second category is - p, and the coordinate system is established with the prediction probability value of the first category p as the horizontal coordinate and the degree of prediction ambiguity as the vertical coordinate. The red dot indicates the most serious prediction ambiguity.
Specifically, for the entropy function, when the prediction is fuzzy, meaning the information is more disordered, it returns larger value. When the prediction distribution is more certain, meaning the information is more definite, the entropy function returns smaller value. For the L2 norm, when the prediction is fuzzy, meaning the predicted probability distribution is more spread out, the L2 norm returns a smaller value. When the prediction fuzziness is not significant, meaning the predicted probability values are more concentrated, the L.2 norm returns a larger value.
That is, we expect the gradient of the regularization term to have a shape similar to y = - as shown Fig. 2(a). By integrating y = -1 we obtain y = log(l=| 0.5). To reduce the computational complexity, we use the L2 norm as the quantitative measure of model prediction ambiguity. Meanwhile, to eliminate the influence of the model's pre- dicted distribution on M (for example, when the two largest prediction probabilities are [0.3, 0.3] or [0.4, 0.4), both indi- cating significant prediction ambiguity. but the resulting M values are different), we focus solely on the distribution state
4

---

Robust Automatic Modulation Classification with Fuzzy Regularization
Table I. Performance comparison with best-performing methods.
Datasets
Methods
F1-Score
ACC
HLACC
DAELSTM
0.7766
78.55%
80.84%
FEAT
0.7544
76.67%
78.31
Data2016a
MCLDNN
0.6772
69.66%
70.98%
ThreeStream
0.7683
78.11%
80.35%
Resnet
0.7347
74.48%
77.29%
(a) V
(b) Fuzzy regularization (ours)
Ours 0.8091
81.68%
84.90%
DAELSTM
0.9139
91.63%
93.02%%
Figure 2. Ideal curve and its derivative are shown in black and red, respectively, in (a); FR curve and its derivative in (b). x and y axises denote the category probability value and fuzziness degrec.
FEAT
0.8186
83.10%
83.66%
Data2016b
MCLDNN
0,9065
91.16%
92.02%
ThreeStream
0,9035
90.76%
92.34%
Resnet
0.8919
89.529
90.69%
Ours 0.9210
92.62% %
93.58%
and normalize M. Based on this, the prediction ambiguity loss for a single sample is given as follows:
DAELSTM
0.7535
75.81%
89.33%
FEAT
0.7129
72.33%
79.52%
Data2018
MCLDNN
0.7895
80.28%
91.62%
Σj= 03 1: x
ThreeStream
0.8197
82.08% %
93.74%
Loss log(
(9)
Resnet
0.7267
73.67%
86.46%
(2)=1 0² k
Ours
0.8666
86.75%
96.62%
Here, fiy represents the model's predicted probability for each class, -5_10, represents the L2 norm of the sample,
kx x مين 2222 represents the minimum L2 norm when con- sidering the top & values. For instance, when k=3, if the predicted distribution is
As illustrated in Fig. 2(b), the black curve represents the Fuzzy Regularization (FR) function, while its derivative is shown in red. The derivative of FR does not fully satisfy the second part of the definition, but it does not contradict our intention. Because the prediction ambiguity will gradually weaken under the supervision of FR, we only hope that the gradient of FR will decrease with the weakening of the fuzziness in the that the prediction ambiguity is not very serious (the red shaded part).
k
E)=1 03 Σi=1 úJ Ei B1 the pre- y diction ambiguity is most severe. (Σ²=1 úz represents the maximum 1.2 norm when considering the top l values. For example, when k=3, if the predicted distribution is - 0, 0), there is no prediction ambiguity. (og(*) ensures the adaptive adjustment of the regularization gradient.
3
In the training process, batch training is typically used, where each batch consists of multiple samples, and a sin- gle loss value is returned for the batch. The loss function deseribed above only reflects the prediction distribution of an individual sample, without fully utilizing the informa- tion between samples to guide the model. To address this, we integrate both individual sample information and the overall information of the entire batch using a log-normal distribution function. Our final FR is defined as follows:
4. Experiments
In this section, we show the advantages of the Fuzzy Regu- larization (FR) from effectiveness, generalizability, robust- ness, training behaviour, and parameter sensitivity.
4.1. Experiments Settings
Our datasets are primarily derived from publicly available wireless modulation type recognition datasets. We evalu- ated the effectiveness of Fuzzy Regularization (FR) in sup- pressing prediction ambiguity using five models?: ResNet (O'Shea et al., 2018), DAELSTM (Ke & Vikalo, 2021), MCLDNN (Xu al., 2020), Three-Stream (Liang al., 2021), and FEAT (Chen et al., 2023) the RadioML 2016.10a (Data2016a) 6a) (O'shea & West, 2016), RadioML 2016. (Data2016b) (A.3), and RadioMI. 2018.01A A (Data2018) (O'Sher Shea et al., 2018) signal datasets. The eval- uation was based on three metrics: FI-Score, ACC, and H-ACC. Detailed descriptions of datasets (A.3), the meth- ods (A.4) and evaluation metrics (A.5) can be found in the appendix.
2² exp
F(O) =
log(T)² (T)
(10)
TV
TV2x
2m
2
N
Σ
0. log
2i=1
(og(7)
(1)
2 6
s.t.
(11)
T=
E) "ー))² )/0))~
Σt= (1)
where 9 denotes the output of the model, (6) denotes the probability that the ith sample belongs to class j. N denotes the sample size of the batch, 7 denotes the 7 classification task and & denotes the selection of the first l values.
Thhe code can be found https://github.. DTMB- Mian-Dtl/Transrrouple....
5

---

Robust Automatic Modulation Classification with Fuzzy Regularization
Table 2. Comparison of methods across datasets.
Data2018
Data2016a
Data2016b
Methods
FI-Score
ACC
H_ACC
FI-Score
ACC
H_ACC
F1-Score
ACC
HLACC
DAELSTM
0.7535
75.81%
89.33%
0,7766
78.55%
80.84%
0.9139
91.63%
93.02%
DAELSTM+FR
0.7684
78.08%
92.02%
0.8091
81.68%
84.90%
0.9157
92.03%
93.22%
A%
+1.49%
+2.27%
+2.69%
+3.25%
+3.13%
+4.06%
+0.18%
+0.40%
+0.20%
FEAT
0.7129
72.33%
79.52%
0.7544
76.67%
78.31%
0.8186
83.10%
83.66%
FEAT+FR
0.7275
74.36%
81.89
0.7713
78.25%
79.62%
0.8295
83.69
84.45%
A%
+1.46%
+2.03%
+2.37%
+1.69%
+1.58%
+1.31%
+1.09%
+0.59%
+0.79%
MCLDNN
0.7895
80.28%
91.62%
0,6772
69.66%
70.98%
0.9065
91.16%
92.02%
MCLDNN+FR
0,8026
81.48%
92.29%
0.7237
73.79%
75.32%
0,9210
92.62%
93.58%
A%
+1.31%
+1.20%
+0.67%
+4.65%
+4.13%
+4.34%
+1.45%
+1.46%
+1.56%
StreaStream
0.8197
82.08%
93.74%
0.7683
78.11%
80.35%
0.9035
90.76%
92.34%
StraeStream+FR
0,8666
86.75%
96.62%
0.7938
80.37 %
83.04%
0.9119
91.67%
92.65 %
A%
+4.69%
+4.67%
+2.88%
+2.55%
+2.26%
+2.69%
+0,84%
+0.91%
+0.31%
Resnet
0.7267
73.67%
86.46%
0.7347
74.48%
77.29%
0.8919
89.52%
90.69%
Resnet+FR
0.7442
74.98%
87.39%
0.7500
75.81 %
77.99%
0.8997
90.43%
91.44%
A%
+1.75%
+1.31%
+0.93%
+1.53%
+1.33%
+0.70%
+0.78%
+0.91%
+0.75%
4.2. Comparison with Other Methods
consistent before and after applying FR.
In this section, we respectively selected the sess-oerroning models Data2016a, Data2016b and Data2018 datasets, namely DAELSTM, DAELSTM and ThreeStream. Ver- ify whether it is possible to further enhance the perfor- mance of the best-performing models by adding FR. As shown in Table 1, our method outperforms the current best-performing models across all three datasets. On the Data2016a, Data2016b, 6b, and Data2018 datasets, our method shows 1%-4% performance improvement over the best- performing models in both ACC and H-ACC.
As demonstrated in Table 2, the methods enhanced with FR consistently outperform their counterparts across all datasets and performance metrics. Specifically, for the same task (i.c., the same model and dataset), the maximum im- provement in accuracy (ACC) is observed from 69.66% to 73.79%, while the harmonic accuracy (H-ACC) increases from 70.98% to 75.32%, both achieving an approximate 4% enhancement. Conversely, the smallest improvement is noted for ACC, which rises from 11.63% to 92.03%, and H-ACC, which increases from 93.02% to 93.22%, with an increment of only 0.3%.
It is also found that the performance improvement brought by the FR regularization varies across different datasets. This phenomenon might be attributed to the fact that the existing SOTA methods already achieve high performance on the Data2016h 6b dataset, with a peak average accuracy of 91.63%. In contrast, the highest average accuracy on the Data2016a 6a and Data2018 datasets is only about 80%. This indicates that the SOTA models on the Data201 6b dataset have already identified relatively optimal classification plane, resulting in smaller corrective effect when the FR regularization is applied.
To further analyze the variability in performance improve- ments across tasks, we investigated the correlation between the effectiveness of FR and the proportion of samples ex- hibiting prediction ambiguity in tasks without FR. No- tably, for the task with the highest performance improve- ment, 35.2% of the samples exhibited prediction ambiguity, whereas for the task with the lowest improvement, this pro- portion was only 1.%. This finding suggests that FR provides greater performance gains for tasks with more se- vere prediction ambiguity. highlighting its ability to address such challenges effectively.
Across all three datasets, our method outperforms the com- pared models in both average accuracy and peak accuracy, demonstrating that the FR term effectively refine the classification plane, thereby improving model performance.
In summary, the experimental results comprehensively vali- date the general applicability and effectiveness of FR across diverse datasets, particularly in scenarios where prediction ambiguity is prevalent.
4.3. Generalizability of the Fuzzy Regularization (FR)
4.4. Robustness of the Fuzzy Regularization (FR)
This section aims to validate the generalizability of the pro- posed FR by integrating it into five state-of-the--rt (SOTA) methods. To ensure the faimess and reliability of the experi- ments, all controllable parameters- inraaeters-iicluuding the learning rate, random seeds, and model initialization- weee weer kept
Post-deployment automatic modulation recognition models inevitably encounter interference from the different envi- ronment from the training data. Therefore, enhancing the model's robustness becomes a critical consideration. Theo-
6

---

Robust Automatic Modulation Classification with Fuzzy Regularization
Table 3. Performance comparison data with different noise factors.
Noisy Factor 20%
Noisy Factor 40 %
Noisy Factor 600 %
Datasets
Methods
F1-Score
ACC
H ACC
Fi-Score
ACC
H ACC
F1-Score
ACC
H ACC
DAELSTM
0.7662
77.18%
79.66%
0.6918
71.41%
74.17%
0.5733
60.04%
63.500%
DAELSTM-FR
0.7754
79.61%
82.82%
0.7220
73.14%
75.64%
0.6322
63.51
66.45
FEAT
0.7558
76.66%
78.08%
0.5648
58.80%
59.90%
0.5398
56.38%
58.08%
FEAT+FR
0,7868
79.80%
80.87 %
0.5675
59.27%
59.79%
0.5487
56.79%
58.08%
MCLDNN
0.6615
68.62%
70.09%
0.5518
57.85%
59.28%
0.5294
56.03%
58.20%
Noise2016a
MCLDNN+FR
0.7027
71.44%
72.98%
0.5679
58.90%
60.41%
0.5408
56.67%
59.13%
ThreeStream
0.6749
70.76%
73.13%
0.6436
65.63%
69.08%
0.5315
53.13%
58.25%
ThreeStream++RR
0.7789
78.90%
81.36 %
0.7140
72.12%
75.76%
0.6017
61.06% %
64.80%
Resnet
0.7282
73.75%
76.28%
0.6838
68.94%
72.01%
0.5971
60.21%
63.28%
Re
0.7393
74.70%
76.59 %
0.6992
70.68 %
72.60%
0.6326
64.24%
66.96%
DARL STM
0,8899
89.36%
90.95%
0.8147
81.81%
83.73%
0.6660
68.55%
70.92%
DAELSTM=FR
0.8932
89.86 %
91.13%
0.8196
82.75%
84.56%
0.6890
69.79%
72.29
FEAT
0.8177
82.65%
83.68%
0.7656
78.81
79.79%
0.6839
71.72%
72.99%
FEAT+FR
0,8166
82.84%
83.75% %
0.7941
80.36%
81.52%
0.7246
73.97%
75.49%
MCLDNN
0.8406
85.26%
86.47%
0.7164
76.21%
77.52%
0.5410
59.02%
60.95%
Noise2016b
MCLDNN
0,9028
90.73%
91.71% %
0.8115
81.91%
83.30%
0.7245
74.31 %
75.91
ThreeStream
0,8500
85.91%
87.25%
0.7149
75.86%
78.06%
0.5270
57.13%
60.14%
ThreeStream+FR
0,8719
87.66
88.78% %
0.7605
78.70%
80.57%
0.6834
69.95%
72.20%
Resnet
0.8654
87.02%
88.17%
0.7887
80.15%
81.29%
0.6790
70.02%
71.98%
Resnet+FR
0.8665
87.24%
88.33 %
0.8036
81.09
82.57%
0.7038
72.09%
73.91
DAELSTM
0.6846
70.1
81.47%
0.5563
58.04%
66.50%
0.4407
46.98%
54.24%
DAELSTM=FR
0.7049
71.03%
81.75%
0.5826
59.40%
68.22%
0.4673
48.27%
55.71% %
FEAT
0.6205
64.20%
70.48%
0,5341
56.53%
62.20%
0.4766
50.18%
55.85%
FEAT+FR
0.6369
65.99%
73.30 %
0,5742
59.19%
65.32%
0.4847
51.03%
56.40 %
MCLDNN
0.6826
70.12%
78.82%
0.4829
50.02%
56.14%
0.2193
27.25%
30.23%
Noise2018
MCLDNN+FR
0.7241
73.33%
82.71 %
0.5043
53.43%
60.82%
0.2417
29.93%
33.06% %
ThreeStream
0.7387
74.40%
84.12%
0,4475
46.28%
49.82%
0.0804
12.93%
14.46%
Thre
0,7600
76.38%
85.58
0.4429
46.52%
50.24%
0.1613
19.90%
22.74%
Resnet
0.6330
64.75%
74.06%
0.5199
54.06%
60.49%
0.4222
44.16%
49.43%
Resnet+FR
0.6370
64.866
74.30 %
0.5359
54.89
61.25%
0.4314
45.75%
51.42%
retically, FR improves robustness by suppressing ambiguity through reducing intra-class distances and enlarging inter- class distances, thereby enhancing the model's performance.
tively identifies classification planes with smaller intra-class distances and larger inter-class distances, thereby enhancing robustness against noisy data.
To empirically validate whether FR enhances model robust- ness, we conducted a series of experiments. When data are perturbed, smaller intra-class distances and larger inter-class distances ensure that the projections perturbed samples on the classification plane remain within controllable range, thereby reducing the likelihood of misclassification. To this end, we designed experiments using the Data201 6a, Data2016b, and Data2018 datasets, generating correspond- ing noisy datasets named Noise2016a, Noise2016b, and N01se2018. The detailed noise generation process is de- scribed in Appendix A.4.
supervision
FR supervision (Ours)
Factor 0.2
Noise
Factor
As shown in Table 3, the robustness of models equipped with FR modules is significantly improved across different tasks. For instance, MCLDNN-FR achieves performance improvement of up to 15% (from 59.02% to 74.31 %) on Noise2016b under a 60% noisy factor, increasing accuracy. For other tasks, performance gains of 1%-3% are consis- tently observed. These results demonstrate that FR effec-
Figure 3. T-SNE intuitively shows that two similar modulation classes can be effectively distinguished by adding FR regulariza- tion. (Red dots represent 8PSK and blue dots represent QPSK)
Furthermore, to visually illustrate the impact of FR on the
7

---

Robust Automatic Modulation Classification with Fuzzy Regularization
0.8
0.8
0.8
Classificatioon Acc
0.6
0.6
0.6
0.4
(FR,
0.4
TA
Train Acc (NF TA]
NF TA
0.2
Valid Acc (FR, VA
0.2
VA
0.2
FR. VA
NF_WJ VA
200
50
100
150
200
50
100
150
Trainn Loss (FR. TL)
3.0
FR. TL
3.0
Train Loss (NF TL)
(FR
2.5
NF TL
NF TL
9 2.0
8 2.5
Valid Loss (NF VL
2.0
NF VL
1.5
1.5
1.5
MMl Mptttt ViN
1.0
1.0
1.0
0.5
0.5
Epochs
Epochs
Epochs
(a) 6oise2016a.n5 20% 20%
(b) Noisc2016a.4000 40%%
(c) Noise2016a.
Figure 4. It presents the accuracy curves and loss curves on different noise datasets when there FR and when there is NO FR(NF).
model's classification hyperplane, we selected two simi- lar modulation classes, QPSK and 8PSK, as discussed in Section The same baseline model was trained with and without FR regularization, and the final layer outputs of the two classes were visualized using t-SNE. As shown in Fig. 3, the classification hyperplane under FR supervision exhibits clearer separation between classes compared to the one without FR regularization, confirming its effectiveness in encouraging margin maximization between confusable modulation clusters.
cus more on those incorrectly predicted samples, allowing for quicker correction. Thus, under FR supervision, the model converges faster. It can also be observed that the red curve represents smoother overall training.
90%
85%
80%
In summary, both quantitative metrics and visualization results validate that FR significantly enhances the robustness of automatic modulation recognition models for noisy data.
10-8
10-7
10
10
A.k
A.k
A.km5
A_k=6
A
B.k-4 6.k
4.5. Training Behaviour Analysis
This section elucidates the mechanisms behind the acceler- ated convergence and stabilized training trajectories induced by Fuzzy Regularization (FR).
The horizontal coordinate is 7 which represents the scaling factor before FR, represents the selection parameter i under the 6a dataset, and B. represents the selection of parameter à under Data2016b dataset.
Figure 5.
As shown in Fig. 4, the Fe-regularized model (red curve) achieves both faster convergence and smoother training tra- jectories compared to the baseline (blue curve). We attribute this dual improvement to FR's ambiguity-aware curriculum learning mechanism: (1) Early-Star Prediction Sharpen- ing. FR induces rapid entropy reduction for high-confidence predictions during initial training phases. Sharpening the probability distribution of correctly classified samples leads to their prediction distribution becoming more certain early, reducing further optimization for those samples. This aligns with curriculum learning principles (Bengio et al., 2009), where easier patterns are mastered before complex ones. (2) Dynamic Hard Sample Emphasis. Sharpening the pre- diction distribution for incorrect samples makes the errors more pronounced, causing the cross cross-entropy function to fo-
4.6. Parameter Sensitivity Analysis
In this section, we will discuss how to quickly determine the satisfying values for two hyperparameters when using FR across different tasks. Fig. 5 illustrates the model accuracy achieved by the ThreeStream+FR with various parameter combinations on the Data2016a and Data20161 datasets. Through comprehensive analysis, is can be seen that the optimal value of 7 is related the magnitude of the ini- tial FR compared when the initial FR is approximately two orders of magni- tude smaller than the cross-entropy loss, the model tends to achieve peak performance. Regarding the parameter k our analysis reveals that the parameter is intrinsically linked to the number of semantically similar classes associated
the cross-entropy loss. Specifically,
8

---

Robust Automatic Modulation Classification with Fuzzy Regularization
with each category in the dataset. For instance, if a class has m semantically related classes, the output activations corresponding to these m + 1 classes tend to be the most proximate. Consequently, setting k to the cardinality of the semantically similar class set generally yields optimal performance. These insights provide valuable guidance for parameter tuning in FR applications.
Processing Letters, 16(5):402-405, 2009.
Chen, Y., Dong, B., Liu, C., Xiong, W., and Li, S. Abandon locality: Frame-wise embedding aided transformer for au- tomatic modulation recognition. IEEE Communications Letters, pp. 327-331, Jan 2023.
Gao, Q. Omran, A. H., Baghersad, Y., Mohammadi, O., Alkhafaji, M. A., Al-Azzawi, A. K. J., Al-Khafaji, S. H., Emami, N., Toghraie, D., and Golkar, M. J. Electroen- cephalogram signal classification based on fourier trans- form and pattern recognition network for epilepsy diag- nosis. Engineering Applications of Artificial Intelligence, 123:106479, 2023.
5. Conclusion
In this paper, we start from the prediction ambiguity phe- nomenon that arises in deep learning- baaseng-based automatic mod- ulation recognition tasks, providing a theoretical derivation for this phenomenon. We then design the FR method to suppress the occurrence of prediction ambiguity, thereby guiding the model to find better and more robust decision boundary. When designing the FR regularization, we also consider the impact of the gradient of the regularization term on model training, so we introduce an adaptive gra- dient mechanism. As the prediction ambiguity gradually diminishes, the absolute value of the update gradient of the FR decreases accordingly, ensuring that the model gradually converges during training. Furthermore, the experimental section demonstrates the effectiveness and generality of our method, and verifies that the model under FR supervision exhibits improved robustness when confronted with noisy data. In the future, we will focus on exploring the more effective fuzzy regularization forms.
Ghahremani Boozandani, M. and Wachinger, C. Regbn: Batch normalization of multimodal data with regulariza- tion. Advances in Neural Information Processing Systems, 36, 2024.
Guo, L., Wang, Y., Liu, Y., Lin, Y., Zhao, H., and Gui, G. Ultralight convolutional neural network for automatic modulation classification in internet of unmanned aerial vehicles. IEEE Internet of Things Journal, 11(11):20831- 20839, 2024.
Ke, Z. and Vikalo, H. Real-time radio modulation classifi- cation with an Istm auto-encoder. In ICASSP 2021 2021 I IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021.
Acknowledgements
Lee, S., Kim, H., and Lee, J. Graddiv: Adversarial robust- ness of randomized neural networks via gradient diversity regularization. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(2):2645-2651, 2022.
This work supported by National Natural Science Foun- dation of China (Nos. T2495251, 62406218, 62306171, T2495253, 62476160, 62376146), the Science and Technol- and
ogy Major Project of Shanxi (No. Fundamental Research Program of Shanxi Province (No. 002203011222183).
Li, P. Research on radar signal recognition based on auto- matic machine learning. Neural Computing and Applica- tions, 32(7):1959-199 1969, 2020.
Impact Statement
Li, R., Li, L., Yang, S., and Li, S. Robust automated vhf modulation recognition based on deep convolutional neu- ral networks. IEEE Communications Letters, 22(5):946- 949, 2018.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Li, T. Wen, Z., Long, Y., Hong, Z.,, Zheng, S., Yu, L.,, Chen, B., Yang, X., and Shao, L. The importance of expert knowledge for automatic modulation open set recogni- tion. IEEE Transactions Pattern Analysis and Machine 2023.
References
Bengio, Y., Louradour, J., Collobert, R., and Weston, Curriculum learning. In Proc. of the 26th International Conference on Machine Learning (ICML), ICML. '09. pp- 41-48, 2009.
Intelligence, 45(11):
Liang, R., Yang, L.,, Wu, S., Li, H., and Jiang, C. A three- stream cnn-lstm network for automatic modulation classi- fication. In 2021 13th International Conference on Wire- less Communications and Signal Processing (WCSP). pp. 1-5, Oct 2021.
Boutte, D. and Santhanam, B. A hybrid ica-svm approach to continuous phase modulation recognition. IEEE Signal
9

---

Robust Automatic Modulation Classification with Fuzzy Regularization
Liang, X., Fu, P., Qian, Y., Guo, Q.. and Liu, G. Trusted multi-view classification via evolutionary multi-view fu- sion. In Proceedings of the 13th International Conference on Learning Representations, pp. 1-14, 2025.
modulation classification. IEEE Transactions on Instru- mentation and Measurement, 72:1-10, 2023.
Xu, J., Luo, C., Parr. G., and Luo, Y. A spatiotemporal multi-channel learning framework for automatic modu- lation recognition. IEEE Wireless Communications Let- lers, (110):622-16322 2020. doi: 10. 109/ 2999453.
O shea, T. J. and West, N. Radio machine learning dataset generation with gnu radio. In Proceedings of the GNU radio conference, volume 1, 2016.
O' S'Saa T. J., Roy, T., and Clancy, T. C. Over-the-air deep learning based radio signal classification. IEEE Journal of Selected Topics in Signal Processing, 12(1):168-179, 2018.
Xu, I.., Zhou, M., and Su, W. Discrete likelihood ratio test for intelligent signal recognition in software defined radio. In The 19th Annual Wireless and Optical Com- munications Conference (WOCC 2010), pp. 1-6. IEEE, 2010.
Panagiotou, P., Anastasopoulos, A., and Polydoros, A. Like- lihood ratio tests for modulation classification. MIL.- COM 2000 Proceedings. 21st Century Military Communi- cations. Architectures and Technologies for Information Superiority (Cat. No. 00CH37155), volume 2, pp. 670- 674. IEEE, 2000.
Yuan, Z., Qian, Y., Liang, X., Kou, Y.. Hou, C., and Hu, Q. Fasten: Fuzzy neural support vector machine for classification. IEEE Transactions Fuzzy Systems, pp. 1-13, 2025. doi: 10. 10.119/7FU7722222535555555
Zeng. R., Lu, Z., Zhang, X., Wang, J.. and Wang, Convo- lutional neural network assisted transformer for automatic modulation recognition under large cfos and sros. IEEE Signal Processing Letters, 31 31:741-7455 2024.
Peng. S., Jiang, H., Wang, H., Alwageed, H., Zhou, Y., Sebdani, M. M., and Yao, Y.-D. Modulation classification based on signal constellation diagrams and deep learning. IEEE Transactions on Neural Networks and Learning Systems, 30(3)771--777, 2018.
Zhang, D., Lu, Y., Li, Y., Ding, W., Zhang, B., and Xiao, J. Frequency learning attention networks based on deep learning for automatic modulation classification in wire- less communication. Pattern Recognition, 137:109345, 2023.
Peng, S., Sun, S., and Yao, Y.-D. A survey of modulation classification using deep learning: Signal representation and data preprocessing. IEEE Transactions on Neural Net- works and Learning Systems, 33(12):7020-7038, 2021.
Zhang, Z., Luo, H., Wang, C., Gan, C., and Xiang, Y. Au- tomatic modulation classification using cnn-lstm based dual-stream structure. IEEE Transactions Vehicular Technology, 69(1 (0(1))13521-1333 2020.
Qian, S. and Fouhey, D. F. Understanding 3d object in- teraction from single image. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 21753-21763, 2023.
Zhao, Q., Wang, Y., Dou, S., Gong, C., Wang, Y., and Zhao, C. Adaptive discriminative regularization for visual classification. International Journal of Computer Vision, 132(10):4327 4342, 2024.
Su, W. Feature space analysis of modulation classification using very high-order statisties. IEEE Communications Letters, 179):1688-1691, 2013.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 2818-2826, 2016.
Zini, E. and Awad, M. On the explainability of natural lan- guage processing deep models. ACM Computing Surveys, 55(5):1-31, 2022.
Wang, and Wang, X. Fast and robust modulation classifi- cation via kolmogorov-smmiroy test. IEEE Transactiony on Communications, 58(8) 2010.
Wu, H.-C., Saquib, M., and Yun, Z. Novel automatic modu- lation classification using cumulant features for commu- nications via multipath channels. IEEE Transactions on Wireless Communications, 708):398-31055 2008
Xiao, C., Yang, S., and Feng, Z. Complex-valus depthwise separable convolutional neural network for automatic
10

---

Robust Automatic Modulation Classification with Fuzzy Regularization
A. Appendix
the supplemental material:
A.1. Signal Visualization.
A.2. Regular Gradient Problem.
A.3. Datasets.
A.4. Compared Methods.
A.5. Evaluation Metrics.
A.1. Signal Visualization
303B
1681
8PSK
QPSK
AM-DSB-WC
AM-SSB-WC WC
8ASK
2008
16dB
AM-SSB-SC
OQPSK
GMSK
640AM
Figure An IQ map partial automatic modulation recognition under 30dB, with 8PSK and QPSK the dashed line representing two more similar modulation classes.
A.2. Regular Gradient Problem
0.0
0.1
0.2
0.3
0,4
0
0.6
0.7
0.8 0.9 1.0
k=
1=1.2
1=4.00
0.0 0.1 0.2
0.3
0.4
5
0.6
0.7
0.8
0.9
1.0
k=2.07
(a) Entropy Function
(b) Variance Function
Figure The horizontal coordinate represents the prediction probability value that the model considers the sample to belong to the first category under the binary classification task. and the vertical coordinate represents the degree of prediction ambiguity. (a) means that entropy function is used to measure the degree of prediction ambiguity; (b) means that variance function is used to measure the degree of prediction ambiguity.

---

Robust Automatic Modulation Classification with Fuzzy Regularization
As shown in the Fig.7, when the prediction ambiguity is not obvious, that is, when the model is more confident in the prediction results, the update gradient of the entropy function variance function still corresponds to large value, which leads to a large degree of change in the parameters of the model when the distribution of the predicted probability value of the model reaches our ideal distribution, resulting in unstable model training and difficulty in finding the optimal solution.
A.3. Datasets
Our datasets mainly come from publicly available wireless modulation recognition datasets. We evaluated the effectiveness of FR in suppressing prediction ambiguity on six signal datasets: RadioML2016.1 10a, RadioML2016.1000 RADIOML 2018.01 A and their corresponding noise versions including Noise2016a, Noise2016b and Noise2018.
RADIOML 2016. This dataset contains 220,000 signal samples, consisting of 11 types of wireless modulation (eight digital modulations and three analog modulations). The eight digital modulations are BPSK, QPSK, 8PSK, 16QAM, 64QAM, BFSK, CPFSK, and PAM4. The three analog modulations are WB-FM, AM-SSB, and AM-DSB. The signal-to-noise ratio (SNR) ranges from -20dB to 18dB, with 20 different SNR levels. For each SNR and modulation type, there are 1,000 signal samples, each with a sample length of 128.
RADIOML 20b6 This dataset contains 1,200,000 signal samples, consisting of 10 wireless modulations. The modulation classes are 8PSK, AM-DSB, BPSK, CPFSK, GFSK, PAM4, QAMI6, QAM64, QPSK and WBFM. The signal-to-noise ratio ranges from -20db to 18db, with 20 different signal-to-noist ratios. There are 6000 signal samples with the same SNR and the same modulation category respectively, and the sampling length of each signal sample is 128. The dataset can be accessed for download from the following URL: http::/wwwww.deeessggooodtatttt
RADIOMI 2018.01 (O'Shea et al., 2018): This dataset consists of 2,555,900 signal samples, encompassing types of wireless modulation. The modulation types include OOK, 4ASK, 8ASK,, BPSK, QPSK, 8PSK, 16PSK, 32APSK. 64APSK, 128APSK, 16QAM, 32QAM, 64QAM, 128QAM, 256QAM, AM-SSB --CSSS--C,, AM-SSB-SC, AM-DSB WC, AM-DSB SC, FM, GMSK, and OQPSK. The signal-to-noise ratio (SNR) ranges from -20BB to 30dB, with 26 different SNR levels. For each SNR and modulation type, there are 4,096 signal samples, each with sample length of 1,024.
To validate the impact of FR on model robustness under noisy data conditions, we generated signal data with different noise intensities based on the three datasets. Specifically, we first designed an adjustable scaling factor to control the noise intensity. Then, we calculated the standard deviation of each sample in the datasets and multiplied by the scaling factor to determine the intensity of the white noise for each sample. Subsequently, white noise was generated for each sample based the calculated intensity, Finally, the generated white noise was added to the original sample to obtain the final noisy data.
Noise2016a, 6b,, Noise2018:
It is noted that adding noise low-SNR data would significantly degrade data quality, making the model's recognition performance on such low-SNR noisy data extremely poor and thus lacking analytical significance. Therefore, in our experiments, we only used data with an SNR of or higher from the aforementioned three datasets.
A.4. Compared Methods
For the baseline models in our experiments, we selected five state-of-the-art models commonly used in deep learning and signal classification tasks to evaluate the effectiveness of FR regularization. These models include ResNet, DAELSTM, MCLDNN, Three-Stream, and FEAT. These five models encompass several critical neural network architectures, such as convolutional neural networks, recurrent neural networks, multi-stream networks, and Transformer-bast networks, ensuring diversity across the selected models. Below a brief introduction to each network:
ResNet (O'Shea et al., 2018); The ResNet model is convolutional neural network with multiple stacked residual blocks. It efficiently extracts important features from time-series data through skip connections and convolution operations, demonstrating strong representational capabilities.
DAELSTM (Ke& & Vikalo, 2021): DAELSTM a model that combines an LSTM autoencoder and deep fully connected layers, suitable for feature learning and classification of time-series data. It extracts temporal features using the LSTM network, performs data reconstruction with the autoencoder, and classifies using the fully connected layers.
12

---

Robust Automatic Modulation Classification with Fuzzy Regularization
MCLDNN (Xu al., 2020): The MCLDNN model integrates Convolutional Neural Networks (CNN) and Long Short- Term Memory networks (LSTM). extracts local features through convolution layers, captures temporal dependencies with LSTM layers, and classifies using fully connected layers.
"hree-Stream (Liang et al., 2021): The Three-Stream model is designed to handle signal data with multiple input channels. It consists of three main sub-networks (one for each input channel), each processing different input features independently, and finally merging the extracted features for classification.
FEAT (Chen et al., 2023): FEAT model effectively extracts information and classifies electromagnetic signals by combining time-series data processing, feature extraction, and multi-head attention mechanisms.
A.5. Evaluation Metrics
We use three metrics to evaluate the model's performance: F1-Score (F1), average model classification accuracy across all signal-to-noise ratios (ACC), and highest model classification accuracy at single signal-to-noise ratio (H-ACC), F1-score is a commonly used metric in classification tasks as it balances precision and recall, providing comprehensive assessment of model's overall performance across different classes. Since model's classification performance varies across different signal-to-me -o-iise ratios, we use ACC and H-ACC to explore the average effect of FR across all signal-to-noise ratios and its effect single signal-to-noise ratio, respectively. Assume there are C signal-to-noise ratios in K-class classification task, where T Tij represents the true positives (TP) for class at signal-to-noise ratio i, the number of samples where the actual class is j and it is predicted as j: FP.j represents the false positives (FP) for class at signal-to-non number of samples where the actual class is not j but predicted as j: FNg represents the false negatives (FN) for class j at sigmal-to-non toise ratio i, the number of samples where the actual class is j but predicted as different class. The formulas for the three metries are as follows:
ratio i, the
F1 & DE ACC Ei= ΣØ= b
K 2x
exb
a+5
K
(12)
ACC maxieo 12 b}.
where P.,
TPg b
TPs F
13


Reviews:
[{"review_id": "MR55E5DJIh", "reviewer": "Reviewer_GYDk", "summary": "This article primarily introduces a fuzzy regularization method applicable to the field of automatic modulation. The authors first measure the prediction ambiguity by an entropy function or a regular function, and then gradually introduced adaptive gradient and exponential normal distribution to further optimize the metrics to design FR. The authors' main contribution is that FR mitigates ambiguities and improves the classification accuracy in the signal classification task, and the method is more effective at low signal-to-noise ratios.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for professional comments. We have tried our best to address your questions and revised our paper by following suggestions from all reviewers.  \n\n**W1: Section 4.5 lacks experimental validation regarding the time taken for a single training round. Fast convergence in terms of training epochs does not necessarily equate to rapid convergence in actual time. It is hoped that the authors can provide more detailed experiments in this aspect.**  \n\nRE: Thanks for your professional suggestion. The reviewer's concern: although FR converges faster in terms of training epochs; However, if FR brings too much time consumption in a single round, then FR cannot be considered as effective in speeding up the experimental convergence. According to this suggestion, we conduct addtional experiment and record the shortest training time, the longest training time, and the average training time per round for all rounds in the training process. During the experiments we ensured the consistency of the controllable parameters. The experimental results are as follows:  \n\n|||Noise2016a_20%|||Noise2016a_40%|||Noise2016a_60%||\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n||MIN|MAX|AVG|MIN|MAX|AVG| MIN|MAX|AVG|\n|FR|0.75s|2.14s|0.90s|0.76s|1.60s|0.91s|0.75s|1.92s|0.91s|\n|Without_FR|0.71s|1.91s|0.89s|0.77s|1.50s|0.92s|0.70s|1.67s|0.80s|  \n\nFrom the experimental result, it can be seen that there is no significant change in the shortest training time, longest training time, and average training time per round whether the  FR is added or not. Combined with the fact that FR can speed up the convergence rounds during training as mentioned in the manuscript, it can be shown that FR can speed up the convergence in actual time.\n\n**W2: As dataset sizes continue to expand, could this regularization potentially reduce the training efficiency of the model?**  \n\nRE: Thanks for the valuable question. The calculation of FR focuses on $\\sigma$ and $T(\\hat{y}_{i},\\tau)$ and their values are obtained by matrix operations. Therefore, this process does not consume too much time. The experiment further proved that FR does not lead to a slowdown in training efficiency. We conducted experimental analyses on three different sized datasets, and recorded the shortest training time, the longest training time, and the average training time per round to the three datasets of different sizes. The specific experimental results are as follows:  \n\n|||2016a(611.23MB)|||2016b(3.26GB)|||2018(19.98GB)||\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n||MIN|MAX|AVG|MIN|MAX|AVG| MIN|MAX|AVG|\n|FR|0.73s|1.57s|0.85s|3.24s|4.64s|3.59s|93.15s|94.04s|93.42s|\n|Without_FR|0.65s|1.57s|0.77s|2.83s|3.98s|3.15s|92.37s|93.19s|92.55s|  \n\nThe above experimental results show that the average rounds of training time increases by 0.1s and 0.4s on the smaller datasets Data2016a and 2016b, respectively. It is noted that the average time increase is not more than 1s on Data2018a with  20GB, and  the increase of the training time for the 200 rounds  is not more than three minutes.\nWhile the performance is improved by three percentage points. This is acceptable in the field of signal recognition.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "The methods and evaluation criteria proposed in the paper are effective for addressing the intended problems.", "theoretical_claims": "The paper does not involve theoretical claims.", "experimental_designs_or_analyses": "I have checked all the experiments in the experimental section.", "relation_to_broader_scientific_literature": "The core contribution of this paper is proposing a novel and effective regularization method for signal classification tasks. The authors elucidate the negative impact of the ambiguity phenomenon on the task and design a corresponding regularization constraint to mitigate this phenomenon, thereby enhancing the performance of the classification task.", "essential_references_not_discussed": "There are no additional relevant literatures that need to be supplemented.", "other_strengths_and_weaknesses": "Strengths:\n\n1. This paper focuses on discussing an important issue.\n2. The methodology presented in this paper is simple, novel, yet effective.\n3. The authors have validated the effectiveness of the method through experiments from multiple perspectives.\n\nWeaknesses:\n\n1. Section 4.5 lacks experimental validation regarding the time taken for a single training round. Fast convergence in terms of training epochs does not necessarily equate to rapid convergence in actual time. It is hoped that the authors can provide more detailed experiments in this aspect.\n2.  As dataset sizes continue to expand, could this regularization potentially reduce the training efficiency of the model?", "other_comments_or_suggestions": "See Weaknesses", "questions_for_authors": "1. Section 4.5 lacks experimental validation regarding the time taken for a single training round. Fast convergence in terms of training epochs does not necessarily equate to rapid convergence in actual time. It is hoped that the authors can provide more detailed experiments in this aspect.\n2.  As dataset sizes continue to expand, could this regularization potentially reduce the training efficiency of the model?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MA7IL75hR8", "reviewer": "Reviewer_F7As", "summary": "This paper proposes a method to improve the reliability of signal classification models by means of fuzzy regularization. Starting from the prediction fuzzy phenomenon, the authors first experimentally prove that the prediction fuzzy phenomenon is a common phenomenon in automatic modulation recognition, and then deeply discuss the impact of this phenomenon on the model performance and propose a corresponding solution, i.e., fuzzy regularization. Finally, the effectiveness and generalization of the method are verified through experiments.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for professional comments. We have tried our best to address your questions and revised our paper by following suggestions from all reviewers.  \n\n**W1: Does FR sharpen misclassified samples?**  \n\nRE: Thanks for your valuable comment. We can suppress this phenomenon by adjusting the hyperparameters $\\gamma$. In the training process, the model is trained with the guidence of both the FR and the cross-entropy function. When the FR originally misclassifies the sample, its corresponding sample' prediction probability value becomes sharp. Although this leads to the decrease of the FR value, but at the same time, it will also lead to the increase of the value of the cross-entropy function. Hence, we can inhibit the occurrence of this phenomenon by selecting an appropriate $\\gamma$ value. That is to say, the decrease of the FR loss is smaller than the increase of the cross-entropy loss  when the wrong prediction becomes sharp. \n\nFor the appropriate value, we have pointed out in Section 4.6 of the paper that the proposed model usually performs better when the FR canonical value is two orders of magnitude different from the value of the cross-entropy function.  \n\n**W2: Has the author considered that FR regularization might lead to overconfidence issues?**  \n\nRE: We thank the reviewer for the insightful comments. The work [1,2] shows that overconfidence mainly targets the following two types of samples. The one is that the model's predicted probability value for the class is very high, but the classification result is wrong. And the other is that the model's predicted probability value for the class is very high, but the actual accuracy is much smaller than the predicted probability value.  \n\nFirstly, for the first class of samples, we have explained in W.1 that the generation of this class is suppressed by choosing an appropriate hyperparameter $\\gamma$. For the second type of samples, we have introduced an adaptive gradient mechanism in the design of FR. This mechanism ensures that the gradient returned by the FR decreases as the value of the predicted probability increases. As a result, the model's predicted probability value for the sample does not always keep increasing. In summary, FR does not lead to overconfidence issues.  \n\n[1] A. Roitberg, et al. Is My Driver Observation Model Overconfident? Input-Guided Calibration Networks for Reliable and Interpretable Confidence Estimates, in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 25271-25286  \n[2] D Wu, et al. The overconfident and ambiguity-averse newsvendor problem in perishable product decision[J]. Computers & Industrial Engineering, 2020, 148: 106689.  \n\n**W3: The parameter k in the regularization proposed by the authors is quite significant, but how should I go about selecting this parameter k?**  \n\nRE: For the choice of the value of $k$, it is suggested to set the maximal number of confusion categories. This is because if a deterministic class has $m$ similar classes, then the model's output value for these $m+1$ classes is closest. This is helpful for FR to accurately measure the degree of predictive ambiguity of a sample.\n\nIn this paper, $k$ is set the number of coding categories of the modulation with the most coding types in the dataset. For example, in a quadrature dataset, the categories are: 'QAM16', 'QAM64', 'QPSK' and 'WBFM'. Of these, QAM has two coding types, while the other modulations each have only one coding type. Therefore, the value of $k$ is initially chosen to be 2.\n\n**Other Comments Or Suggestions**  \n\nC represents the number of samples included in a single round of training for the model.  \nWe cite the source of the dataset in Text 4.1Experiments Settings and indicate the download link for the dataset in Appendix A.3.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "I have meticulously reviewed the experimental section in the fourth chapter of the article, and the overall experiments are well-conceived. I have also noted that the authors have ensured consistency across various controllable parameters in the experiments, such as random seeds, which greatly contributes to the fairness of the experiments.", "relation_to_broader_scientific_literature": "The authors have made pivotal contributions to the field of signal classification. They have conducted an in-depth analysis of the ambiguity phenomena in signal classification tasks and proposed an effective solution to address these issues.", "essential_references_not_discussed": "No significant omissions of relevant literature were detected.", "other_strengths_and_weaknesses": "Strengths:\n1.The overall content of this paper is complete and clear. The author elaborates on the process of identifying problems, exploring issues, solving problems, and verifying the validity of the methodology.\n2. The design of the fuzzy regularity proposed in this paper is not complicated, but the overall design is more skillful and at the same time more comprehensively considered. For example, when designing the fuzzy regularity in Section 3.3, the authors consider eliminating the influence of the model predictive distribution on the predicted fuzzy values through normalization.\n3.The validity and generalizability of the method were verified by experiments.\n\nWeaknesses:\n1.Does FR sharpen misclassified samples?\n2. Has the author considered that FR regularization might lead to overconfidence issues?\n3.The parameter k in the regularization proposed by the authors is quite significant, but how should I go about selecting this parameter k?", "other_comments_or_suggestions": "C in equation (8) does not find a specific meaning in the context.\nIt is best to cite reference sources for the dataset.", "questions_for_authors": "Please See Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3l6RYHA5Eq", "reviewer": "Reviewer_X5bz", "summary": "This paper proposes a regularization method aimed at enhancing classification performance in signal classification tasks, particularly for those with low signal-to-noise ratios. It achieves this by constraining the model's predictive ambiguity for samples during the task, thereby increasing the inter-class distance between different categories and reducing the intra-class distance within the same category, which in turn improves classification performance. The authors have validated the effectiveness of this regularization on both synthetic and benchmark datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for professional comments. We have tried our best to address your questions and revised our paper by following suggestions from all reviewers.   \n\n**W1: The work appears to bear some resemblance to curriculum learning. Could the authors provide a detailed explanation of the similarities and differences between these two methods?**  \n\nRE: This is very good question. There are two differences between FR and curriculum learning.  Firstly, the training strategy is different. Curriculum learningis a two-stage learning strategy: learning the easy samples then the difficult samples [1]. FR did not have this strategy and focused on marginal samples(which can be regarded as the difficult samples in course learning) from the beginning.  The second and biggest difference is that the both of them utilize different information. Since FR can measure the degree of predictive ambiguity of the samples, it utilizes more information about the predictive distribution of the samples than course learning. \n\n[1] Liu Y, Wang J, Xiao L, et al. Foregroundness-aware task disentanglement and self-paced curriculum learning for domain adaptive object detection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023.\n\n**W2: The backward derivation of this regular derivative is not seen in the text, but it is important for the subsequent optimization of the model, and it is hoped that the author can add this part of the content.**  \n\nRE: In order to facilitate the detailed explanation of the subsequent derivation process, we provide a more detailed additional description of Eqs. (7)(8), which is supplemented as follows:  \n$$\\mathrm{F}(\\hat{y}\\_{j})=\\frac{\\sigma}{T(\\hat{y}\\_{j},\\tau)\\sqrt{2\\pi}}exp\\left[-\\frac{\\sigma^2}{2}log\\left(T\\left(\\hat{y}\\_{j},\\tau\\right)\\right)^2\\right]$$\n$$\\mathrm{s.t.}\\quad\\begin{cases}\\sigma=-\\frac{1}C\\sum\\_{j=1}^c\\frac{\\sum\\_{i=1}^{k}\\widehat{y}\\_{ji}log(\\widehat{y}\\_{ji})}{log(\\tau)}, \\\\\\T(\\widehat{y}\\_{j},\\tau)=\\frac{\\sum\\_{i=1}^{k}\\widehat{y}\\_{ji}^{2}-k\\left(\\frac{\\sum\\_{i=1}^{k}\\widehat{y}\\_{ji}}{k}\\right)^2}{\\left(\\sum\\_{i=1}^{k}\\widehat{y}\\_{ji}\\right)^2-k\\left(\\frac{\\sum\\_{i=1}^{k}\\widehat{y}\\_{ji}}{k}\\right)^2}.&&&\\end{cases}$$   \n\nwhere $\\hat{y}\\_{j}$ denotes the jth sample, $\\hat{y}\\_{ji}$ denotes the probability that the sample belongs to class i, C denotes the sample size of the batch, $\\tau$ denotes the $\\tau$ classification task and k denotes the selection of the first k values.\n\n1. Calculating derivatives of intermediate variables  \n\n1.1 The derivative of $\\sigma$ with respect to $\\widehat{y}\\_{ji}$  \n\n$\\frac{\\partial\\sigma}{\\partial\\hat{y}\\_{ji}}=-\\frac{1}{\\mathrm{Clog}\\tau}\\cdot\\frac{\\partial}{\\partial\\hat{y}\\_{ji}}(\\hat{y}\\_{ji}\\mathrm{log}\\hat{y}\\_{ji})=-\\frac{\\log\\hat{y}\\_{ji}+1}{C\\log\\tau}$.  \n\n1.2 The derivative of $T(\\widehat{y}\\_j,\\tau)$ with respect to $\\widehat{y}\\_{ji}$  \n\nLet $S\\_1=\\sum\\_{i=1}^k\\hat{y}\\_{ji}, S\\_2=\\sum\\_{i=1}^k\\hat{y}\\_{ji}^2$.  \nThen $T=\\frac{S\\_2-\\frac{S\\_1^2}{k}}{S\\_1^2-\\frac{S\\_1^2}{k}}=\\frac{kS\\_2-S\\_1^2}{(k-1)S\\_1^2}$, we let the numerator in $T$ be N and the denominator D.  \n\nSo $\\frac{\\partial T}{\\partial\\hat{y}\\_{ji}}=\\frac{\\partial N/D}{\\partial\\hat{y}\\_{ji}}=\\frac{\\partial N}{\\partial\\hat{y}\\_{ji}}\\cdot\\frac{1}{D}-\\frac{N}{D^2}\\cdot\\frac{\\partial D}{\\partial\\hat{y}\\_{ji}}$.\n\n1.2.1 The derivative of the molecule N  \n\nDue $\\frac{\\partial S\\_2}{\\partial\\hat{y}\\_{ji}}=2\\hat{y}\\_{ji}, \\frac{\\partial S\\_1}{\\partial\\hat{y}\\_{ji}}=1$. So $\\frac{\\partial N}{\\partial\\hat{y}\\_{ji}}=k\\cdot\\frac{\\partial S\\_2}{\\partial\\hat{y}\\_{ji}}-2S\\_1\\cdot\\frac{\\partial S\\_1}{\\partial\\hat{y}\\_{ji}}=2k\\hat{y}\\_{ji}-2S\\_1$.\n\n1.2.2 The derivative of the denominator D  \n\n$\\frac{\\partial D}{\\partial\\hat{y}\\_{ji}}=(k-1)\\cdot2S\\_1\\cdot\\frac{\\partial S\\_1}{\\partial\\hat{y}\\_{ji}}=2(k-1)S\\_1$.\n\n1.2.3 The final derivative of $T(\\widehat{y}\\_{j},\\tau)$ with respect to $\\widehat{y}\\_{ji}$  \n\n$\\frac{\\partial T}{\\partial\\hat{y}\\_{ji}}=\\frac{2k\\hat{y}\\_{ji}-2S\\_1}{(k-1)S\\_1^2}-\\frac{(kS\\_2-S\\_1^2)\\cdot2(k-1)S\\_1}{(k-1)^2S\\_1^4}=\\frac{2k(\\hat{y}\\_{ji}S\\_1-S\\_2)}{(k-1)S\\_1^3}$.  \n\n2. Chain rule for derivation  \n\nDue $\\frac{\\partial\\ln F}{\\partial\\hat{y}\\_{ji}}=\\frac{1}{\\sigma}\\frac{\\partial\\sigma}{\\partial\\hat{y}\\_{ji}}-\\frac{1}{T}\\frac{\\partial T}{\\partial\\hat{y}\\_{ji}}-\\sigma(\\log T)^2\\frac{\\partial\\sigma}{\\partial\\hat{y}\\_{ji}}-\\frac{\\sigma^2\\log T}{T}\\frac{\\partial T}{\\partial\\hat{y}\\_{ji}}$.  \n\nSo $\\frac{\\partial F}{\\partial\\hat{y}\\_{ji}}=F\\cdot\\frac{\\partial\\ln F}{\\partial\\hat{y}\\_{ji}}=F(\\hat{y}\\_j)\\cdot\\left[\\frac{\\partial\\sigma}{\\partial\\hat{y}\\_{ji}}\\left(\\frac{1}{\\sigma}-\\sigma(\\log T)^2\\right)+\\frac{\\partial T}{\\partial\\hat{y}\\_{ji}}\\left(-\\frac{1}{T}-\\frac{\\sigma^2\\log T}{T}\\right)\\right]$.  \n\n**Other Comments Or Suggestions**  \n\nWe will correct the numbering problem in the appendix.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes. The method proposed in this paper is effective in enhancing the performance of classification tasks. The evaluation criteria selected in this paper can effectively distinguish the performance differences between the proposed method and the baseline.", "theoretical_claims": "There are no theoretical claims in the paper.", "experimental_designs_or_analyses": "I have reviewed the experimental section, including Experiments Settings (4.1),\nComparison with Other Methods(4.2), Generalizability of the Fuzzy Regularization (FR)(4.3), Robustness of the Fuzzy Regularization (FR)(4.4), Training Behaviour Analysis(4.5), Parameter Sensitivity Analysis(4.6).", "relation_to_broader_scientific_literature": "It contributes to the field of signal classification by introducing a regularization technique that enhances the reliability of classification tasks. This regularization technique is proposed to address the phenomenon of ambiguity, and its effectiveness has been validated through multi-dimensional experiments.", "essential_references_not_discussed": "This paper includes key relevant literature to help readers understand the research background and significance of the issue.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper elaborates on the research motivation in detail, with a well-structured and fluently written narrative.\n2. The paper discusses the existing issues in automatic modulation classification tasks and provides corresponding solutions.\n3. The experimental results consistently demonstrate that models incorporating FR regularization outperform the baseline, highlighting the effectiveness and versatility of this method in signal classification tasks.\n\nWeaknesses:\n1. The work appears to bear some resemblance to curriculum learning. Could the authors provide a detailed explanation of the similarities and differences between these two methods?\n2. The backward derivation of this regular derivative is not seen in the text, but it is important for the subsequent optimization of the model, and it is hoped that the author can add this part of the content.", "other_comments_or_suggestions": "In the appendix, the numbering for Datasets should be A.3 instead of A.4.", "questions_for_authors": "See weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CPYdMKyngO", "reviewer": "Reviewer_NJBZ", "summary": "The paper introduces Fuzzy Regularization (FR) as a novel solution to mitigate prediction ambiguity in Automatic Modulation Classification (AMC). This ambiguity is caused by similar characteristics between modulation schemes, especially under noisy conditions. The FR approach is characterized by three key features: modeling prediction ambiguity, dynamic sample reweighting through adaptive loss scaling, and promoting margin maximization between similar modulation classes. The experimental results show that FR significantly improves the robustness and classification accuracy across various datasets and noise levels.\n\n## update after rebuttal\n\nI will keep my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for professional comments. We have tried our best to address your questions and revised our paper by following suggestions from all reviewers.  \n\n**Q1: Can you provide a more formal and detailed explanation of the adaptive gradient mechanism used in Fuzzy Regularization? This would help clarify how it dynamically adjusts during training.**  \n\nRE:Thank you for your question, it helps to articulate our work more clearly. We have explained the establishment of Eq. (6) in more detail and hope that this will help you to understand the automatic gradient mechanism in FR. First choose the first k predicted values of each sample to calculate the degree of fuzziness ($\\mathrm{M}$), and the theoretical maximum value $\\mathrm{M}\\_{\\mathrm{max}}=\\left(\\sum\\_{i=1}^{k}\\hat{y}\\_{i}\\right)^{2}-{k\\mu}^2$, minimum value $\\mathrm{M}\\_{\\mathrm{min}}=k\\times\\left(\\frac{\\sum\\_{i=1}^{k}\\hat{y}\\_{i}}{k}\\right)^{2}-{k\\mu}^2$, and mean value $\\mu=\\frac{\\sum\\_{i=1}^{k}\\hat{y}\\_{i}}{k}$ for a single sample are known. Then  $\\mathrm{M}$ can be expressed as: $\\mathrm{M}=\\sum\\_{i=1}^\\mathrm{k}\\left(\\hat{y}\\_{i}-\\mu\\right)^2=\\sum\\_{i=1}^\\mathrm{k}\\hat{y}\\_{i}^2-{k\\mu}^2$; we focus only on the distribution state, so by normalizing the formula we get $\\mathrm{M\\_{norm}=\\frac{M-M\\_{min}}{M\\_{max-M\\_{min}}}=\\frac{\\sum\\_{i=1}^{k}\\hat{y\\_{i}}^{2}-k\\times(\\frac{\\sum\\_{i=1}^{k}\\hat{y\\_{i}}}{k})^{2}}{(\\sum\\_{i=1}^{k}\\hat{y\\_{i}})^{2}-k\\times(\\frac{\\sum\\_{i=1}^{k}\\hat{y\\_{i}}}{k})^{2}}}$.The automatic gradient mechanism is realized by the log(\\*) function to correct the gradient of $M\\_{norm}$. The absolute value of the gradient of the log(\\*) function decays symmetrically when it deviates from the central axis, which can satisfy the second definition of the design of FR. The final loss of a single sample is: $\\mathrm{Loss}=\\log(\\mathrm{M}\\_{\\mathrm{norm}})=\\log\\left(\\frac{\\sum\\_{i=1}^k\\hat{y}\\_i^2-k\\times\\left(\\frac{\\sum\\_{i=1}^k\\hat{y}\\_i}{k}\\right)^2}{\\left(\\sum\\_{i=1}^k\\hat{y}\\_i\\right)^2-k\\times\\left(\\frac{\\sum\\_{i=1}^k\\hat{y}\\_i}{k}\\right)^2}\\right)$.  \n\n**Q2:How does the FR mechanism compare to label smoothing or other entropy-based regularization methods in terms of performance and robustness? Would it be beneficial to combine these techniques?**  \n\nRE:We would like to thank the reviewers for their valuable comments. First we define Ent to denote the entropy-based method, LS to denote the label smoothing method, and FRLS to denote the joint training of the label smoothing and FR methods. NF_* indicates that the noise factor of the dataset is *.For the sake of intuition and brevity, the data recorded is the performance difference between the two methods, e.g. Ent_FR is the value obtained by subtracting the entropy-based model accuracy from the FR model accuracy in the original article.  \n\nThrough Table I we find that the performance difference of entropy-based methods compared to WF methods is both good and bad. We analyze that the reason for the deterioration of performance may be due to the gradient update of the entropy function and other problems, which is one of the difficulties solved in this paper. Although the performance of the entropy-based method can be close to the FR method in some tasks, the performance is weaker than the FR method in general tasks. This also shows that the FR method is better than the entropy-based method.  \n\nTable 1\n\n||NF_0|NF_0|NF_20%|NF_20%|NF_40%|NF_40%|NF_60%|NF_60%|\n|-|-|-|-|-|-|-|-|-|\n||Ent_FR|Ent_WF|Ent_FR|Ent_WF|Ent_FR| Ent_WF| Ent_FR| Ent_WF|\n|DAE|-4.18%|-1.05%|-0.83%|+1.6%|-0.83%|+0.9%|-4.77%|-1.3%|\n|FEA|-0.42%|+0.03%|-2.84%|+0.3%|-0.07%|+0.4%|-0.17%|+0.24%|\n|MCL|-3.56%|+1.52%|-0.82%|+2%|-0.4%|+0.65%|-0.14%|+0.5%|\n|Res|-1.51%|-0.18%|-1.05%| -0.1%|-2.34%|-0.6%|-4.13%|-0.1%|\n|Thr|-6.97%|-4.71%|-5.84%|+2.3%|-8.59%|-2.1%|-9.03%|-1.1%| \n\nWith Table 2 we find that the FRLS models generally outperform the models trained with label smoothing loss, which further demonstrates the FR validity. Meanwhile, we find that the model trained by FRLS joint loss does outperform the model trained by FR supervision on some tasks. This may be due to the fact that LS optimizes the target labels, which make the values of non-target classes non-zero.This helps the model to learn more information from other classes. FR mainly focuses on the model's predictive distribution information. The two focus on different information, so the performance of the model can be further optimized when trained jointly. We will continue to study the joint training strategy in depth.  \n\nTable 2\n  \n||NF_0|NF_0|NF_20%|NF_20%|NF_40%|NF_40%|NF_60%|NF_60%|\n|-|-|-|-|-|-|-|-|-|\n||FRLS_FR|FRLS_LS|FRLS_FR|FRLS_LS|FRLS_FR|FRLS_LS|FRLS_FR|FRLS_LS|\n|DAE|-0.1%|+1%|+1.9%|+1.3%|+0.9%|+1.7%|+1.9%|+0.7%|\n|FEA|+0.9%|+1.9%|+0.8%|+0.6%|+0.6%|-0.5%|-0.8%|+1.2%|\n|MCL|+1.4%|+0.7%|+1.7%|+2.1%|+1.8%|+0.4%|+1%|+0.2%|\n|Res|-0.8%|+0.3%|+0.8%|+0.6%|+1.2%|+0.7%|-1.9%|+0.4%|\n|Thr|+0.6%|+0.1%|+0.9%|+1.2%|+2.2%|+3.1%|+2%|+2%|", "claims_and_evidence": "The claims about FR improving model performance and robustness are supported by extensive experimental evidence, including results across multiple datasets (RadioML 2016.10a, 2016.10b, and 2018.01A). The improvements in accuracy and robustness are clearly documented. However, the theoretical analysis and description of the FR mechanism could be clearer and more rigorously formalized, particularly concerning the adaptive gradient mechanism.", "methods_and_evaluation_criteria": "The proposed method, Fuzzy Regularization (FR), is appropriate for addressing the problem of prediction ambiguity in AMC tasks, particularly under noisy conditions. The use of standard benchmark datasets and performance metrics like F1-Score, ACC, and H-ACC is suitable for evaluating the method's effectiveness. The experimental design, which compares FR with existing methods, helps establish the validity of FR as an enhancement to AMC models.", "theoretical_claims": "The theoretical claims around prediction ambiguity and the need for a regularization mechanism to address it are well-grounded. However, the mathematical formulation of the FR mechanism could benefit from more clarity. The paper lacks a formal definition of how the adaptive gradient mechanism works mathematically, which would strengthen the theoretical claims.", "experimental_designs_or_analyses": "The experimental designs are sound, with clear benchmarks and comparisons with other state-of-the-art (SOTA) methods. The paper includes a comprehensive evaluation across different noise levels, which validates the robustness of the FR method. The use of multiple datasets and the careful selection of evaluation metrics (F1-Score, ACC, H-ACC) further strengthens the experimental analysis.", "relation_to_broader_scientific_literature": "The paper positions itself within the broader context of automatic modulation classification, comparing FR to other regularization techniques and deep learning models. It highlights the gap in the literature regarding the explicit handling of prediction ambiguity and offers a new approach with FR. However, the paper could better compare FR with other regularization strategies specifically designed for handling ambiguity in classification tasks, such as label smoothing or other entropy-based methods.", "essential_references_not_discussed": "The paper cites key references, but it could benefit from more discussion on the connection to related regularization techniques, particularly those addressing prediction uncertainty or ambiguity, such as label smoothing, entropy regularization, and adversarial robustness techniques.", "other_strengths_and_weaknesses": "The paper is strong in its originality, proposing a novel regularization mechanism that specifically targets prediction ambiguity, a critical issue in AMC. The experimental results are robust and demonstrate the method's efficacy across various noise conditions. However, the explanation of the FR mechanism lacks some clarity, particularly in its formalization. The paper would benefit from a deeper theoretical exploration of the method.", "other_comments_or_suggestions": "Some sections could benefit from clearer writing and better organization, especially the presentation of the mathematical formulas and experimental setup. It would also be helpful to provide a more detailed comparison with other ambiguity-handling techniques.", "questions_for_authors": "- Can you provide a more formal and detailed explanation of the adaptive gradient mechanism used in Fuzzy Regularization? This would help clarify how it dynamically adjusts during training.\n- How does the FR mechanism compare to label smoothing or other entropy-based regularization methods in terms of performance and robustness? Would it be beneficial to combine these techniques?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel framework, namely Fuzzy Regularization-enhanced AMC (FR-AMC),  which integrates uncertainty quantification into the classification pipeline. All reviewers give positive scores. The method presented is simple, novel, yet effective. Extensive experiments verify the effectiveness and generalization ability of this method.
Decision: Accept (spotlight poster)