# Robust Automatic Modulation Recognition Through Joint Contribution of Hand-Crafted and Contextual Features

Bachir Jdid, Wei Hong Lim, Iyad Dayoub, Kais Hassan, Mohd Rizon Bin Mohamed Juhari

<figure><img src="image_1.png" /></figure>

To cite this version:

Bachir Jdid, Wei Hong Lim, Iyad Dayoub, Kais Hassan, Mohd Rizon Bin Mohamed Juhari. Robust Automatic Modulation Recognition Through Joint Contribution of Hand-Crafted and Contextual Features. IEEE Access, 2021, 9, pp.104530-104546. 10.1109/ACCESS.2021.3099222. hal-03382822

HAL Id: hal-03382822

https://uphf.hal.science/hal-03382822

Submitted on 20 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Received July 9, 2021, accepted July 19, 2021, date of publication July 26, 2021, date of current version July 30, 2021.

Digital Object Identifier 10.1109/ACCESS.2021.3099222

# Robust Automatic Modulation Recognition Through Joint Contribution of Hand-Crafted and Contextual Features

BACHIR JIDID<fnref n="1" />, WEI HONG LIM<fnref n="1" />, (Senior Member, IEEE), IYAD DAYOUB<fnref n="2" />,<fnref n="3" />, (Senior Member, IEEE), KAIS HASSAN<fnref n="1" />,<fnref n="4" />, AND MOHD RIZON BIN MOHAMED JUHARI<fnref n="1" />

Corresponding authors: Mohd Rizon Bin Mohamed Juhari (mohdrizon@ucsiuniversity.edu.my) and Iyad Dayoub (iyad.dayoub@uphf.fr)

This work was supported by UCSI University through the UCSI University Research Excellence and Innovation Grant Scheme under Project REIG-FETBE-2020/025.

**ABSTRACT** Automatic modulation recognition (AMR) has become increasingly important in the field of signal processing, especially with the advancements of intelligent communication systems. Deep Learning (DL) technologies have been incorporated into the AMR field and they have shown outstanding performances against conventional AMR methods. The robustness of DL-based AMR methods under varying noise regimes is one of major concerns for the widespread utilization of this technology. Furthermore, most existing works have neglected the contributions of hand-crafted features (HCFs) in boosting the classification performances of DL-based AMR methods. In order to address the aforementioned technical challenges, a novel and robust DL-AMR method is proposed by leveraging the benefits of both contextual features (CFs) and HCFs for a specific range of signal-to-noise ratio (SNR). A novel feature selection algorithm is also proposed to search for the optimal sets of HCFs in order to reduce the dimensions of feature vectors without losing any important and relevant features. Simulation studies are performed to investigate the feasibility of proposed method in classifying 11 types of modulation schemes. Extensive performance analyses revealed the superiority of proposed method over baseline method in terms of the classification performance as well as the excellent capability of proposed feature selection algorithm in determining an optimal subset of HCFs.

**INDEX TERMS** Automatic modulation recognition (AMR), convolutional neural network (CNN), deep learning (DL), wireless signal classification.

## I. INTRODUCTION

For the communication systems widely used in both military and civilian applications, the radio signals are encoded by predefined adaptive modulation schemes with respect to the specification of transmission channel. Receiver needs to correctly identify the types of modulation scheme adopted in order to ensure successful demodulation. Most often, the receiver has no prior information about the modulation scheme of received signals in blind detection. Automatic modulation recognition (AMR) is a popular technique used

The associate editor coordinating the review of this manuscript and approving it for publication was Xiaofan He

to provide the blind recognition of the modulation scheme. In literatures, existing AMR techniques are designed and implemented based on two main approaches: (1) likelihood-based (LB) and (2) feature-based (FB) [1]. Despite being able to achieve the optimum recognition rate, most LB approaches tend to suffer with technical drawbacks such as high computational complexity and strong dependency on the prior information of received signal [1]. In contrary to LB approaches, FB approaches are considered as the more practical solutions due to their ease of implementation for real-life scenario and ability to produce sub-optimal solutions. It is notable that the performance of FB approaches relies tremendously on the hand-crafted features (HCFs) manually extracted from

<footnote n="1">¹Faculty of Engineering Technology and Built Environment, UCSI University, Kuala Lumpur 56000, Malaysia</footnote>

<footnote n="2">²Université Polytechnique Hauts-de-France, CNRS, University of Lille, ISEN, Centrale Lille, UMR 8520, Institut d'Électronique de Microélectronique et de Nanotechnologie (IEMN), Département d'Opto-Acousto-Électronique (DOAE), F-59313 Valenciennes, France</footnote>

<footnote n="3">³INSA des Hauts de France, F-59313 Valenciennes, France</footnote>

<footnote n="4">⁴Laboratoire d'Acoustique de l'Université du Mans (LAUM), UMR CNRS 6613, Le Mans University, 72085 Le Mans, France</footnote>
the received signals. Different combinations of HCFs are also found to have crucial roles in recognizing different types of modulation schemes. The incorporation of more advanced feature-learning methods for deep feature extraction is necessary to develop the more robust LB techniques in solving AMR problems.

Recently, different deep learning (DL) models such as the convolutional neural network (CNN), recurrent neural network (RNN), deep neural network (DNN) and etc. have shown their promising advantages to tackle the key emerging issues in research fields of computer vision, healthcare, robotics, internet of things, and communication systems. For instance, CNN is widely used as an image recognition method to solve multiple-object detection problems due to its promising ability to automatically extract multiple levels of features from inputs. Motivated by the successes of DL in different areas, numerous DL-based AMR techniques have been proposed based on CNN, RNN, DNN and other network architectures in recent years and they have been observed to achieve significant results [1].

Jdid et al. [1] presented a comprehensive survey on existing AMR methods developed with different machine learning (ML) and DL models. They compared these ML-based and DL-based AMR methods in terms of their network structures, training and testing hyperparameters, noise conditions, modulation pools and performances. O’Shea and West [2] employed GNU Radio to generate a synthetic dataset known as the RadioML 2016.10A that contains eleven types of digital and analog modulations. A new AMR method was developed by O’Shea and Corgan [3] by using CNN and its performance was investigated with the dataset proposed in [2]. O’Shea et al. [4] amended the tool described in [2] to generate RadioML 2018.01A and more details of this new dataset will be summarized in Section II-B. Their previous works [3], [5], [6] with adaptation of CNNs for AMR were extended to solve the new dataset, where in-depth investigation in terms of the design parameters, channel impairment and training dataset parameters were studied through various simulations including over-the-air measurement of AMR performance.

Another CNN-based AMR was proposed by Zhang et al. [7] by employing the Short-Time Fourier Transform (STFT) to convert the input signal spectrograms into image representation. Similarly, Sun et al. [8] adopted an image dataset with 10 different modulation schemes to build a CNN-based AMR from a popular deep learning model known as VGG-16 [9]. Zhang et al. [10], applied both of Wigner-Ville and Born-Jordan distributions to convert the received signals into two types of images. These converted images were then fed into two CNNs for feature extraction before fusing them with the selected HCFs in order to solve the classification problems.

The work reported by Peng et al. [11] exploited the adaptation of constellation diagrams and a popular CNN model called AlexNet [12] for AMR. Similarly, constellation diagrams were employed as the inputs of feature

extractor in [13], whereas a support vector machine (SVM) was used as the classifier for AMR. Given that the qualities of datasets are crucial to improve the performance of AMR, the pre-processing technique such as data augmentation was employed by Tang et al. [14] to ensure sufficient data were available to construct the datasets required for the training and testing of DL-based AMRs. Particularly, constellation diagrams were converted into the contour stellar images in [14] to obtain more color features than those of [11]. Several issues encountered in training process such as the divergence of generator, overfitting of discriminator and mode collapse can degrade the overall performance of traditional generative adversarial nets [15]. Numerous measures were proposed to address the poor performance issues of GANs and simulation studies revealed that the classification accuracies can be improved through the extension of datasets.

Most existing FB-AMR approaches proposed in earlier era [16] were designed based on the extraction of HCFs. Meanwhile, the DL-based AMR approaches proposed in recent literatures were found to heavily rely on the ideas of using deep neural network architecture for feature extraction [17]. Extensive amounts of previous research works in image processing have shown the advantages of concatenating the contextual features (CFs) and HCFs [18]–[20]. In particular, a proper combination between HCFs with CFs is able to achieve the performance boosting via more diverse representation of features. To the best of authors’ knowledge, limited investigations were performed to search for the optimal combinations of CFs and HCFs. This is because the quality of CFs is governed by the efficiency and abundance of the used datasets. Meanwhile, the value of each HCF may change with different signal-to-noise ratio (SNR) values. It is also noteworthy that some HCFs tend to have the same values for different types of modulations. These undesirable characteristics of HCFs tend to jeopardize the accuracies of classifiers through the distortion of their hyperplanes. In order to address these technical drawbacks, it is crucial to design an efficient feature selection algorithm that is able to build the robust feature set.

As mentioned earlier, the performances of DL-based AMR methods are governed by the qualities of datasets, hence the latter should be constructed carefully to include a variety of real-life effects. While the larger number of training examples can lead to better performance of DL-based AMR methods, the involved training processes tend to be time-consuming. This issue may be resolved by splitting the original large dataset into multiple subsets of data for the training process. The hyperplanes of classifiers also tend to be distorted when dealing with datasets that have very wide ranges of SNR. Although training a classifier with the wide range of SNR can eliminate the requirements of SNR estimation, CNNs with highly complex network structures are required to extract the relevant features. The required memory storage of classifiers could increase significantly due the tremendous amounts of training parameters involved in its feature extractor. In this paper, a novel DL-based AMR method is proposed to tackle
the aforementioned issues, where the original SNR range is
spilt into multiple ranges in order to train the good perform-
ing classifiers with lower complexity and smaller storage
requirements.

The main purpose of AMR is to configure the parame-
ters and modulation schemes of received signals in order to
achieve a successful demodulation process. Greater accuracy
of AMR process is expected to achieve through the incorpo-
ration of DL due to its excellent feature learning capability.
This paper is devoted to study the feasibility of combining the
robust set of HCFs with CFs in boosting the classification per-
formance. Furthermore, this work also aims to investigate the
performance differences of classification accuracy between
simple CNNs trained with smaller SNR ranges and complex
CNN trained with wider SNR ranges. For the former case,
a set of HCFs are first constructed and the original SNR range
is spilt into two smaller SNR ranges based on a predefined
threshold. Then, these two sets of HCFs are concatenated with
the CFs learned by the two CNNs deployed in their respective
SNR ranges.

The main contributions of this paper are summarized as
follows:

* Introduce new compositions of High-Order Cumulants (HOCs) that can be used to provide good complement for CFs in order to boost the classification performance.

* Propose a novel algorithm that is able to determine the optimal threshold and most relevant features for splitting the wide-range SNR.

* Introduce a novel criterion known as Classification Con-
fusion Index (CCI) which is the main core of the pro-
posed algorithm that is able to select an optimal subset
of HCFs to be used for solving the classification tasks.

* Propose a novel DL-based AMR method that is able to achieve high classification accuracy without compromising the desirable characteristics of low computational complexity and storage requirement.

The findings of this paper are also expected to provide
constructive insights for the following key questions:

* Is it efficient to spilt a wide-range of SNR into two smaller ranges?

* Can a narrower range of SNR reduce the complexity and depth of CNN architecture in learning more meaningful CFs?

* Is the presence of HCFs able to boost classification performance of a DL-based AMR method?

This work focuses on employing simple CNN architectures
for learning CFs and concatenating these features with a set of
HCFs in a wide-range of SNR that is split into two different
ranges. The outline of this paper is summarized as follows.
The background of AMR and specifications of RadioML
2018.01A dataset considered in current study are presented
in Section II. Other required information such as the signal
parameters and channel characteristics are also covered in
this section. Section III explains the algorithm used to extract
and select an optimal subset of HCFs, whereas the extraction

process of CFs using CNNs are elaborated in Section IV.
The overall mechanisms of proposed DL-based AMR method
are then explained in Section V and the performance com-
parisons with baseline methods are presented in Section VI.
Finally, the concluding remarks of current study are presented
in Section VII.

II. AMR BACKGROUND

AMR is an essential process used to identify the type of
modulation scheme adopted by received signals before recov-
ering these received information through the demodulation
process. In this section, a brief background of FB-AMR is
first presented. Then, we wrap up this section by providing
an overview of the most popular signals dataset, RadioML
2018.01A [4], in AMR field.

<figure><img src="image_3.png" /><figcaption>FIGURE 1. Block diagram of the Feature-based AMR method.</figcaption></figure>

A. FB-AMR BACKGROUND

Fig. 1 shows the general block diagram of conventional
FB-AMR methods that consist of three main phases:

*   Pre-processing phase: Several variables such as Carrier Frequency Offset (CFO), baud rate, Phase Offset (PO), SNR and timing offsets can be obtained from the input signals via different pre-processing techniques. The quality of received signal can be enhanced in this phase before proceeding to feature extraction phase.

* Feature extraction phase: Feature extraction is first per-
formed in this phase to obtain the HCFs or CFs. These
extracted features can also be categorized as instanta-
neous time features, wavelet features and statistical fea-
tures. Feature selection algorithm is subsequently used
to choose the most relevant features in order to reduce
the dimensional size of feature vector.

* Classification phase: This phase is responsible to decide the types of modulation scheme adopted by received signal with one or multiple classifiers that can be broadly categorized as traditional classifiers (e.g., classification tree), ML-based classifiers (e.g. SVM) and DL-based classifiers (e.g., CNN).

B. SIGNALS DATASET RadioML 2018.01A

The RadioML 2018.01A dataset proposed by
O'Shea et al. [4] is considered in this study because it is
one of the most challenging datasets for AMR problem.
<table><caption>TABLE 1. Summary of the parameters of RadioML 2018.01A Dataset.</caption><thead><tr><th>Key</th><th>Value</th></tr></thead><tbody><tr><td>Normal modulation pool (P)</td><td>{OOK, 4ASK, BPSK, QPSK, 8PSK, 16QAM, AM-SSB-SC, AM-DSB-SC, FM, GMSK, OQPSK}</td></tr><tr><td>Difficult modulation pool (P<sup>d</sup>)</td><td>{OOK, 4ASK, 8ASK, BPSK, QPSK, 8PSK, 16PSK, 32PSK, 16APSK, 32APSK, 64APSK, 128APSK, 16QAM, 32QAM, 64QAM, 128QAM, 256QAM, AM-SSB-WC, AM-SSB-SC, AM-DSB-WC, AM-DSB-SC, FM, GMSK, OQPSK}</td></tr><tr><td>SNR minimum value (S<sub>n</sub><sup>min</sup>)</td><td>S<sub>n</sub><sup>min</sup> = -20 dB</td></tr><tr><td>SNR maximum value (S<sub>n</sub><sup>max</sup>)</td><td>S<sub>n</sub><sup>max</sup> = +30 dB</td></tr><tr><td>SNR step</td><td>2 dB</td></tr><tr><td>Number of SNR values (N<sub>snr</sub>)</td><td>N<sub>snr</sub> = (S<sub>n</sub><sup>max</sup> - S<sub>n</sub><sup>min</sup>)/2 + 1 = 26</td></tr><tr><td>Signal length (l)</td><td>l = 1, 024</td></tr><tr><td>Total number of signals (N)</td><td>N = 2, 555, 904</td></tr><tr><td>Number of signals per type (N<sub>t</sub>)</td><td>N<sub>t</sub> = 106, 496</td></tr><tr><td>Number of signals per type per SNR (N<sub>t</sub><sup>s</sup>)</td><td>N<sub>t</sub><sup>s</sup> = 4, 096</td></tr></tbody></table>

This dataset consists of two different compositions of classes spread across a wide range of SNR values. The “Normal” classes of RadioML 2018.01A dataset consists of 11 modulation schemes commonly seen in the impaired environments, whereas the “Difficult” classes contain the 24 digital and analog modulation schemes. It is also notable that the RadioML 2018.01A dataset contains more than 2.5 million frames of modulation signals along with different synthetic simulated channel effects such as CFO, symbol rate offsets, delay spread and thermal noise. In this paper, the performance of proposed DL-based AMR method will be evaluated using the “Normal” classes. For the self-contained purpose, the parameters of RadioML 2018.01A dataset are presented in Table 1.

### III. HCFs EXTRACTION

Extensive literature studies revealed that most existing FB-AMR methods employ different types of HCFs such as wavelet feature [21], spectral-based features [22], time-frequency features [23], [24] [25], instantaneous features [1] and statistical features [1]. The derivations of these HCFs are further explained in the following subsections.

#### A. INSTANTANEOUS FEATURES

Several instantaneous features can be extracted by employing specific parameters such as the instantaneous amplitude $a_n$, instantaneous phase $\phi_{NL}$ and instantaneous frequency $f_N$. Table 2 presents the derivation of the five instantaneous features [26] considered in this work.

#### B. HIGH ORDER STATISTICS FEATURES

Both of the Higher-Order Moments (HOMs) and HOCs [27], [28] are reported to be the best candidates for signal recognition. Mathematically, the HOMs of a signal $x$ are

<table><caption>TABLE 2. The mathematical equations of the extracted instantaneous features.</caption><thead><tr><th>Feature</th><th>Equation</th></tr></thead><tbody><tr><td>Envelope variation</td><td>$m_A = \frac{\sigma^2}{\mu^2}$ (1)<br>where $\sigma$ and $\mu$ are the variance and mean, respectively, of $a_n$.</td></tr><tr><td>Standard deviation of the amplitude envelope</td><td>$E = \sqrt{\frac{1}{N_s - 1} \sum_{i=1}^{N_s} (a_n(i) - \frac{1}{N_s} \sum_{i=1}^{N_s} a_n(i))^2}$ (2)<br>where $N_s$ is the sample number.</td></tr><tr><td>Kurtosis of the normalized-centered instantaneous amplitude</td><td>$\mu_{42}^a = \frac{E(a_{cn}^a[n])}{(E\{a_{cn}^a[n]\})^2}$ (3)<br>where $a_{cn}$ is the normalized and centered instantaneous amplitude of the incoming signal at the time instant.<br>$a_{cn} = a_n(i) - 1, a_n(i) = \frac{a_n(i)}{m_a}, m_a = 1/N_s \sum_{i=1}^{N_s} a_n(i)$</td></tr><tr><td>Standard deviation of the absolute value of the normalized-centered instantaneous amplitude</td><td>$\sigma_{aa} = \sqrt{\frac{1}{N_i} \left( \sum_{i=1}^{N_s} a_{cn}^a(i) \right) - \left( \frac{1}{N_s} \sum_{i=1}^{N_s} |a_{cn}(i)| \right)^2}$ (4)</td></tr><tr><td>Maximum value of the power spectral density of the normalized-centered instantaneous amplitude</td><td>$\gamma_{max} = \frac{\text{MAX}(|DFT[a_{cn}(i)^2]|)}{N_s}$ (5)<br>where $DFT$ refers to the discrete Fourier transform of modulated signal.</td></tr></tbody></table>

defined as:

$$M_{km}(x) = E[x^{k-m}(x^*)^m] \quad (6)$$

where $k$ is the order of the moment. The cumulant of order $k$ of the zero-mean signal $x$ is defined as:

$$C_{km}(x) = \text{Cum}\left[\underbrace{x, \dots, x}_{(k-m) \text{ times}}, \underbrace{x^*, \dots, x^*}_{m \text{ times}}\right] \quad (7)$$

The mathematical relation between cumulants and moments can be expressed as:

$$\text{Cum}[x_1, \dots, x_n] = \sum_{\Phi} (c-1)!(\text{-1})^{c-1} \prod_{v \in \Phi} E \left( \prod_{i \in v} x_i \right) \quad (8)$$

where $\Phi \in \{1, \dots, n\}$, $v$ refers to the list member in the partition $\Phi$; $c$ is the number of elements in the partition $\Phi$. For instance, the cumulants of up to $6^{th}$ orders are defined as follows [26]:

$$C_{20} = M_{20} \quad (9)$$

$$C_{21} = M_{21} \quad (10)$$

$$C_{40} = M_{40} - 3M_{20}^2 \quad (11)$$

$$C_{41} = M_{41} - 3M_{21}M_{20} \quad (12)$$

$$C_{42} = M_{42} - |M_{20}|^2 - 2M_{21}^2 \quad (13)$$

$$C_{60} = M_{60} - 15M_{40}M_{20} + 30M_{20}^3 \quad (14)$$

$$C_{63} = M_{63} - 9M_{41}M_{21} - 6M_{21}^3 \quad (15)$$

It is evident from (9) to (15) that the estimation of moments will lead to the estimation of the cumulants as well.
However, given a signal $x$ with $N$ samples, the moments are estimated as:

$$ \hat{M}_{km}(x) = \frac{1}{N} \sum_{i=1}^{N} x^{k-m}(i)x^{m}(i) \quad (16) $$

Without loss of generality, the normalized signal $x$ is assumed to have a unity energy, i.e., $C_{21} = 1$. The normalization process is used to address the scaling issues of estimators. In practical, the self-normalized HOMs and HOCs are calculated as:

$$ \tilde{M}_{km}(x) = \hat{M}_{km}(x)/\hat{M}_{21}^{k/2}(x) \quad (17) $$

$$ \tilde{C}_{km}(x) = \hat{C}_{km}(x)/\hat{C}_{21}^{k/2}(x) \quad (18) $$

Estimating a moment of order $k$ requires only around $N$ complex addition and $k \times N$ complex multiplications. Theoretically, when $k$ is close to $N$, the complexity of cumulant estimation is of order $N^2$. Practically, the cumulant order $k$ is far smaller than $N$ and, thus, the complexity of cumulant estimation will be of order $N$ [16]. Notably, the computational cost incurred by feature calculation is of the same order with those in estimation of cumulants and moments. Therefore, the feature extraction process is considered to have very low complexity of $O(N)$ in practical scenarios [16]. In this work, we introduce a total of ten new cumulants compositions. These compositions were found by some simple experiments and they are defined as follows:

$$ P_1 = |C_{20}|/|C_{21}| \quad (19) $$

$$ P_2 = |C_{20}|/|C_{40}| \quad (20) $$

$$ P_3 = |C_{20}|/|C_{41}|^{1/2} \quad (21) $$

$$ P_4 = |C_{21}|/|C_{62}|^{1/3} \quad (22) $$

$$ P_5 = |C_{21}|/|C_{80}|^{1/4} \quad (23) $$

$$ P_6 = |C_{63}|^{1/3}/|C_{80}|^{1/4} \quad (24) $$

$$ P_7 = |C_{40}|/|C_{60}|^{1/3} \quad (25) $$

$$ P_8 = \log|C_{20}| \quad (26) $$

$$ P_9 = \log|C_{21}| \quad (27) $$

$$ P_{10} = \log|C_{41}| \quad (28) $$

To this end, a total of 15 HCFs are obtained to construct the original HCFs set denoted as $F$, where:

$$ F = \{m_A, E, \mu_{42}^a, \sigma_{aa}, \gamma_{max}, P_1, P_2, P_3, P_4, P_5, P_6, P_7, P_8, P_9, P_{10}\} \quad (29) $$

## IV. CFs EXTRACTION USING CNNs

CNN is a popular deep learning architecture used to automatically extract CFs from the input data. A typical CNN architecture consists of several convolution (Conv) and pooling (Pool) layers connected in series along with at least one fully-connected (FC) layer. The depth of CNN architecture increases with numbers of convolution, pooling and fully-connected layers.

The convolution layers of CNN play essential role in feature extraction of input data, where these features can be

learned from a set of convolutional filters constructed by a group of neurons that are arranged as the rectangular grid. When the 2-dimensional (2D) input data are forward passed within CNN, a set of 2D activation maps are produced by sliding all convolution filters across the input data. During the training of CNN, the weight values of convolution filters can be updated through the forward propagation and backpropagation of error. This enables the activation of learned filters when encountering desired types of input signals.

Pooling layers are commonly located between the single or multiple convolutional layers. Max pooling and average pooling are two popular pooling layers adopted. The former pooling takes the maximum values within the kernels, whereas the latter one considers the average values within the kernels. The pooling layers of CNN can avoid network overfitting to certain extent by reduce the dimension sizes of feature progressively. The presence of pooling layers also enables CNN to becomes invariant to small translations of input, therefore improving its network accuracy. Finally, the fully connected layers are used as the classifier part of CNN and they consist of several nodes connected to all activations received from previous layer.

An appropriate activation function needs to be defined for all CNN layers except for its input layer. Both of the rectified linear unit (ReLU) and parametric rectified linear unit (PReLU) are commonly used in CNN by executing a threshold operation on each input element. Softmax is another activation function widely used to solve the multiclassification problems by normalizing the input vectors into a probability distribution consisting of several probabilities proportional to the exponentials of the input numbers and maps them into the (0, 1) interval.

## V. THE PROPOSED DL-BASED AMR TECHNIQUE

This section is organized in a top-down manner. An overview of the proposed DL-based AMR method is first presented, followed by the explanations of technical details for each compositional sub-system in the subsequent subsections. Fig. 2 shows the block diagram of the proposed DL-based AMR method. Initially, the received signals are fed into feature extraction subsystem, where three main groups of HCFs are generated. The first group ($G_1$) is used for SNR splitter,<fnref n="1" /> while the other two groups ($G_2$ and $G_3$) are used along with CFs for modulation schemes classification. For the HCFs belonging to $G_1$ that are fed into the SNR splitter sub-system, a specific threshold (Thr) is defined to divide the overall SNR range ($R$) into two smaller ranges of $R_1$ and $R_2$, where $R = R_1 \cup R_2$. Depending on the identified SNR range, these signals are then fed into the corresponding CNN for CFs extraction. The CFs extracted from $CNN_1$ and $CNN_2$ are then concatenated with the $G_2$ and $G_3$ groups respectively. Finally, the classification of modulation schemes can be performed by one of the SVMs denoted as $SVM_1$ and $SVM_2$ that are deployed at both SNR ranges, $R_1$ and $R_2$, respectively.

<footnote n="1">¹An explanation for this will be provided later in section V-A4</footnote>
<figure><img src="image_4.png" /><figcaption>FIGURE 2. Block diagram of the proposed DL-based AMR technique.</figcaption></figure>

## A. NOVEL ALGORITHM FOR SNR WIDE-RANGE SPLITTING

In this subsection, we propose a novel algorithm to split the wide range of SNR values into two smaller ranges based on a set of HCFs and an SVM classifier. The SVM classifier that can be utilized as a SNR splitter is first presented, followed by the descriptions of novel algorithm used to select the most relevant HCFs by accurately splitting the overall SNR range based on the most effective splitting threshold. The proposed algorithm is then evaluated by using both “Normal” and “Difficult” classes. Finally, a new training method known as cross-SNR training is presented to reduce the classification errors produced by SNR splitter in order to enhance the classification accuracy of the proposed DL-based AMR method.

### 1) SNR SPLITTER

In practical, SVM classifiers can provide good computation speed and memory, and work relatively well when there is a clear margin of separation between few classes. Therefore, a fast linear SVM classifier is adopted as the SNR splitter of proposed DL-AMR method. For each signal obtained from the dataset, a vector of HCFs denoted as $G_1$ is extracted. In offline training, all HCFs vectors that are extracted from the signals with SNR values lesser than a predefined threshold (Thr) are labeled as $L_1$, whereas those with SNR values greater than Thr are labeled as $L_2$. After the SVM is trained, it will be integrated in the proposed AMR method as the SNR splitter. It will identify the range of SNR value of the received signals before HCFs and CFs extraction system. The complete pseudo-code of SNR splitter is presented using Algorithm 1. It is worth mentioning that the same criterion is used to label the modulation schemes of all signals with $L_1$ and $L_2$.

### 2) NOVEL HCFs SELECTION ALGORITHM FOR SPLITTING SNR RANGE

The total SNR range are divided into several consecutive ranges to investigate the variation of each HCF value under these ranges. This procedure aims to select the best features and threshold values that can split the overall SNR range into smaller ranges with higher splitting accuracy. The Coefficient

### Algorithm 1 Pseudo Code for SNR Splitter Function

1. **Function** SNR-Splitter
   (F<sub>p</sub>, N<sub>snr</sub>, I<sub>t</sub><sup>s</sup>, I<sub>t</sub>, T<sub>t</sub><sup>s</sup>, T<sub>t</sub>, Thr, P<sub>ct</sub>):

**Data:** Array of HCFs $F_p$ for the modulations pool P.

Testing signals per type $T_t$.

Testing signals per type per SNR $T_t^s$.

SNR threshold Thr.

Total number of signals per type per SNR $N_t^s$.

Training signals per type per SNR $I_t^s$.

Training signals per type $I_t$.

Number of SNR values $N_{snr}$.

**Result:** Vector $L_t$ of testing labels.

Vector $L_p$ of predicted labels.

The splitting accuracy $A_c$.

for i ← 1 to length of P do

2

3

4

5

for j ← 1 to length of $F_p(i)$ do

6

7

if j < Thr × $N_t^s$ then

$L_{tr}((i - 1) × I_t^s + j) \leftarrow “L_1”;$
$L_t((i - 1) × T_t^s + j) \leftarrow “L_1”;$

else

$L_{tr}((i - 1) × I_t^s + j) \leftarrow “L_2”;$
$L_t((i - 1) × T_t^s + j) \leftarrow “L_2”;$

8

Train SVM classifier using $L_{tr}$, then test it using $L_t$.

Finally, store the accuracy in $A_c$ and the predicted labels in $L_p$.

of Variation (CV) measure is used to demonstrate the variation of each HCF value under the varying SNR conditions.

CV is defined as the ratio of standard deviation $\sigma$ to mean $\mu$ and it is used to measure the dispersion of data sample around the mean value of population. CV is able to compare the degree of variations between different data series efficiently even through their mean values are drastically different from one another [29]. It is also notable that CV is sensitive to small changes in the mean if the latter has near zero value. For a group of signals denoted as $S^m$, the CV of k-th feature ($f_k$) is
<table><caption>TABLE 3. CV values for all HCFs and SNR ranges.</caption><thead><tr><th>Range</th><th>P<sub>1</sub></th><th>P<sub>2</sub></th><th>P<sub>3</sub></th><th>P<sub>4</sub></th><th>P<sub>5</sub></th><th>P<sub>6</sub></th><th>P<sub>7</sub></th><th>P<sub>8</sub></th><th>P<sub>9</sub></th><th>P<sub>10</sub></th><th>γ<sub>max</sub></th><th>σ<sub>aa</sub></th><th>μ<sup>a</sup><sub>42</sub></th><th>m<sub>A</sub></th><th>E</th></tr></thead><tbody><tr><td>S<sub>1</sub></td><td>0.522</td><td>1.859</td><td>0.704</td><td>0.268</td><td>0.202</td><td>0.349</td><td>0.590</td><td>0.189</td><td>0.756</td><td>0.227</td><td>0.190</td><td>0.029</td><td>0.084</td><td>0.082</td><td>0.021</td></tr><tr><td>S<sub>2</sub></td><td>0.523</td><td>1.727</td><td>0.713</td><td>0.266</td><td>0.202</td><td>0.350</td><td>0.588</td><td>0.188</td><td>0.758</td><td>0.228</td><td>0.190</td><td>0.029</td><td>0.084</td><td>0.082</td><td>0.021</td></tr><tr><td>S<sub>3</sub></td><td>0.521</td><td>1.712</td><td>0.712</td><td>0.267</td><td>0.204</td><td>0.353</td><td>0.588</td><td>0.188</td><td>0.756</td><td>0.229</td><td>0.190</td><td>0.029</td><td>0.085</td><td>0.082</td><td>0.021</td></tr><tr><td>S<sub>4</sub></td><td>0.521</td><td>1.972</td><td>0.707</td><td>0.267</td><td>0.203</td><td>0.352</td><td>0.586</td><td>0.187</td><td>0.753</td><td>0.229</td><td>0.190</td><td>0.029</td><td>0.085</td><td>0.082</td><td>0.021</td></tr><tr><td>S<sub>5</sub></td><td>0.522</td><td>2.271</td><td>0.702</td><td>0.265</td><td>0.202</td><td>0.351</td><td>0.588</td><td>0.857</td><td>0.753</td><td>0.228</td><td>0.189</td><td>0.029</td><td>0.084</td><td>0.083</td><td>0.021</td></tr><tr><td>S<sub>6</sub></td><td>0.524</td><td>2.122</td><td>0.705</td><td>0.264</td><td>0.202</td><td>0.352</td><td>0.587</td><td>0.193</td><td>0.760</td><td>0.228</td><td>0.191</td><td>0.029</td><td>0.084</td><td>0.083</td><td>0.021</td></tr><tr><td>S<sub>7</sub></td><td>0.575</td><td>2.102</td><td>0.748</td><td>0.264</td><td>0.202</td><td>0.352</td><td>0.585</td><td>0.218</td><td>0.851</td><td>0.229</td><td>0.191</td><td>0.029</td><td>0.084</td><td>0.085</td><td>0.023</td></tr><tr><td>S<sub>8</sub></td><td>0.749</td><td>2.078</td><td>0.887</td><td>0.263</td><td>0.200</td><td>0.350</td><td>0.592</td><td>0.309</td><td>1.11</td><td>0.235</td><td>0.218</td><td>0.029</td><td>0.083</td><td>0.097</td><td>0.032</td></tr><tr><td>S<sub>9</sub></td><td>0.891</td><td>2.167</td><td>0.826</td><td>0.282</td><td>0.198</td><td>0.367</td><td>0.605</td><td>0.804</td><td>1.365</td><td>0.323</td><td>0.694</td><td>0.032</td><td>0.081</td><td>0.15</td><td>0.059</td></tr><tr><td>S<sub>10</sub></td><td>0.935</td><td>2.089</td><td>0.729</td><td>0.373</td><td>0.242</td><td>0.335</td><td>0.689</td><td>0.694</td><td>1.515</td><td>0.545</td><td>1.462</td><td>0.052</td><td>0.081</td><td>0.317</td><td>0.118</td></tr><tr><td>S<sub>11</sub></td><td>0.933</td><td>1.453</td><td>0.669</td><td>0.481</td><td>0.372</td><td>0.345</td><td>0.877</td><td>0.788</td><td>1.609</td><td>0.668</td><td>1.831</td><td>0.101</td><td>0.085</td><td>0.673</td><td>0.216</td></tr><tr><td>S<sub>12</sub></td><td>0.930</td><td>1.887</td><td>0.673</td><td>0.530</td><td>0.530</td><td>0.494</td><td>1.235</td><td>0.732</td><td>1.704</td><td>0.635</td><td>1.935</td><td>0.179</td><td>0.121</td><td>1.148</td><td>0.337</td></tr><tr><td>S<sub>13</sub></td><td>0.931</td><td>1.944</td><td>0.687</td><td>0.562</td><td>0.664</td><td>0.645</td><td>1.761</td><td>0.701</td><td>1.812</td><td>0.661</td><td>1.97</td><td>0.278</td><td>0.175</td><td>1.496</td><td>0.449</td></tr><tr><td>S<sub>14</sub></td><td>0.934</td><td>1.808</td><td>0.694</td><td>0.604</td><td>0.768</td><td>0.764</td><td>2.279</td><td>0.701</td><td>1.914</td><td>0.728</td><td>1.983</td><td>0.375</td><td>0.215</td><td>1.659</td><td>0.533</td></tr><tr><td>S<sub>15</sub></td><td>0.935</td><td>1.973</td><td>0.702</td><td>0.651</td><td>0.853</td><td>0.860</td><td>2.667</td><td>0.717</td><td>1.995</td><td>0.792</td><td>1.988</td><td>0.448</td><td>0.239</td><td>1.727</td><td>0.589</td></tr><tr><td>S<sub>16</sub></td><td>0.936</td><td>2.263</td><td>0.708</td><td>0.693</td><td>0.927</td><td>0.942</td><td>2.925</td><td>0.734</td><td>2.051</td><td>0.838</td><td>1.991</td><td>0.495</td><td>0.254</td><td>1.754</td><td>0.625</td></tr><tr><td>S<sub>17</sub></td><td>0.936</td><td>2.092</td><td>0.710</td><td>0.728</td><td>1.002</td><td>1.022</td><td>3.122</td><td>0.749</td><td>2.084</td><td>0.865</td><td>1.993</td><td>0.523</td><td>0.264</td><td>1.767</td><td>0.648</td></tr><tr><td>S<sub>18</sub></td><td>0.936</td><td>1.967</td><td>0.710</td><td>0.753</td><td>1.075</td><td>1.10</td><td>3.261</td><td>0.759</td><td>2.10</td><td>0.879</td><td>1.996</td><td>0.539</td><td>0.291</td><td>1.772</td><td>0.661</td></tr><tr><td>S<sub>19</sub></td><td>0.937</td><td>2.018</td><td>0.713</td><td>0.766</td><td>1.15</td><td>1.178</td><td>3.353</td><td>0.764</td><td>2.108</td><td>0.884</td><td>1.999</td><td>0.548</td><td>0.500</td><td>1.773</td><td>0.669</td></tr><tr><td>S<sub>20</sub></td><td>0.937</td><td>3.683</td><td>0.714</td><td>0.775</td><td>1.224</td><td>1.255</td><td>3.629</td><td>0.766</td><td>2.111</td><td>0.887</td><td>1.998</td><td>0.552</td><td>1.192</td><td>1.775</td><td>0.672</td></tr><tr><td>S<sub>21</sub></td><td>0.937</td><td>3.701</td><td>0.711</td><td>0.783</td><td>1.283</td><td>1.317</td><td>3.834</td><td>0.768</td><td>2.113</td><td>0.889</td><td>1.999</td><td>0.554</td><td>1.984</td><td>1.774</td><td>0.675</td></tr><tr><td>S<sub>22</sub></td><td>0.937</td><td>2.142</td><td>0.711</td><td>0.789</td><td>1.327</td><td>1.362</td><td>3.688</td><td>0.769</td><td>2.113</td><td>0.889</td><td>2.001</td><td>0.555</td><td>2.414</td><td>1.774</td><td>0.676</td></tr><tr><td>S<sub>23</sub></td><td>0.937</td><td>2.126</td><td>0.711</td><td>0.794</td><td>1.356</td><td>1.392</td><td>3.49</td><td>0.770</td><td>2.113</td><td>0.889</td><td>2</td><td>0.555</td><td>2.583</td><td>1.774</td><td>0.676</td></tr><tr><td>S<sub>24</sub></td><td>0.937</td><td>2.109</td><td>0.711</td><td>0.797</td><td>1.371</td><td>1.407</td><td>3.632</td><td>0.770</td><td>2.113</td><td>0.889</td><td>2.001</td><td>0.555</td><td>2.641</td><td>1.774</td><td>0.676</td></tr><tr><td>S<sub>25</sub></td><td>0.937</td><td>3.524</td><td>0.711</td><td>0.795</td><td>1.376</td><td>1.412</td><td>3.593</td><td>0.770</td><td>2.113</td><td>0.889</td><td>2.001</td><td>0.555</td><td>2.67</td><td>1.774</td><td>0.676</td></tr></tbody></table>

calculated as follow:

$$CV_{km} = \frac{\sigma_{km}}{\mu_{km}} \qquad (30)$$

For simplicity, we denote the minimum and maximum values of SNR as $r_1$ and $r_{L+1}$, respectively. Hence, the overall range of SNR denoted as $[r_1, r_{L+1}]$ is equally split into $L$ ranges as shown in Eq. (32). The group of signals for specific SNR range of $[r_{m-1}, r_m]$ are denoted as $S^m$ as shown in Eq. (33). Hence, the signals are divided into a total of $L$ groups as:

$$R = \{[r_1, r_2], [r_2, r_3], \dots, [r_{m-1}, r_m], \dots, [r_L, r_{L+1}]\} \quad (31)$$

where $R_m$ is calculated as:

$$r_m = r_1 + \frac{(m-1)(r_{L+1} - r_1)}{L} \qquad (32)$$

$$S^i = [r_{i-1}, r_i] : 1 < i < m + 1 \qquad (33)$$

There are many signals with SNR value equal to $r_{m-1}$ are shared in any two consecutive groups of $S^{m-1}$ and $S^m$. Under this circumstance, the values of each k-th feature $f_k$ in these two groups are approximately same if both $CV_{km-1}$ and $CV_{km}$ values are approximately same. The second derivative of each $CV_k$ vector is hence calculated to determine the SNR range where the CV of an HCF starts to vary. In principle, the positive and negative values of second derivative imply that a curve starts to concave up and concave down, respectively. The SNR range can be detected when a feature value starts

to change drastically with increasing rate of variation along with the concave upward trends of curve. Table 3 shows the $CV_{km}$ values of each k-th HCF for all groups of $S^m$, where the highlighted cells indicate the $CV_k$ of particular feature is in a concave upward trend during $S^m$. Notably, two consecutive groups of $CV_{km-1}$ and $CV_{km}$ should be selected to obtain a center value of SNR as the required threshold. In order to fulfill the objectives of SNR splitting, it is also desirable to define a threshold value that is able to split the signal dataset into two subsets with similar sizes. The SNR ranges in which most of HCFs show the concave upward trend are then selected as threshold value for splitting.

Algorithm 2 presents the full pseudo code of HCFs and SNR threshold selection algorithm that considers the following inputs:

* A 3D array of HCFs denoted as $F$, where the first dimension represents the size of modulation pool ($l_p$), second dimension represents the total number of signals per type ($N_t$) and third dimension represents the HCFs count ($l_f$), respectively.

* Minimum value of SNR denoted as $S_n^{min}$.

* Maximum value of SNR denoted as $S_n^{max}$.

* Step size of SNR denoted as step.

* Percentages of signals used for testing denoted as $P_{ct}$.

The following variables are first initialized in Algorithm 2, i.e., number of SNR values ($N_{snr}$), number of testing signals per type per SNR ($N_t^s$), number of testing signals per type ($N_t$), number of training signals per type per SNR ($I_t^s$),
**Algorithm 2** The Algorithm of HCFs and SNR Threshold Selection for SNR Splitter

**Input:** F, N<sub>t</sub>, S<sub>n</sub><sup>min</sup>, S<sub>n</sub><sup>max</sup>, step, P<sub>ct</sub>

**Output:** Selected HCFs group G<sub>1</sub> for SNR range splitter.

SNR Splitter accuracy A<sub>c</sub>.

SNR splitter Threshold Thr.

1 l<sub>f</sub> ← size of 3rd dimension of F;

2 N<sub>snr</sub> ← S<sub>n</sub><sup>max</sup> - S<sub>n</sub><sup>min</sup> / 2 + 1;

3 N<sub>t</sub><sup>s</sup> ← N<sub>t</sub> / N<sub>snr</sub>;

4 T<sub>t</sub> ← floor((P<sub>ct</sub>*N<sub>t</sub>)/100);

5 T<sub>t</sub><sup>s</sup> ← floor((P<sub>ct</sub>*N<sub>L</sub><sup>s</sup>)/100);

6 I<sub>t</sub> ← N<sub>t</sub> - T<sub>t</sub>;

7 I<sub>t</sub><sup>s</sup> ← N<sub>t</sub><sup>s</sup> - T<sub>t</sub><sup>s</sup>;

8 Indices ← [1 to l<sub>f</sub>];

9 for k ← 1 to l<sub>f</sub> do

10 for m ← 1 to N<sub>snr</sub> - 1 do
11 | CV(m, k) ← σ<sub>F</sub>(m,k) / μ<sub>F</sub>(m,k)

12 Z<sub>k</sub> ← normalize(CV(:, k));

13 Calculate Z<sub>k</sub>' ← ∂Z<sub>k</sub>/∂x and Z<sub>k</sub>'' ← ∂<sup>2</sup>Z<sub>k</sub>/∂x<sup>2</sup>

14 for j ← 1 to N<sub>snr</sub> - 2 do

15 | if Z<sub>k</sub>''(j) > 0 and Z<sub>k</sub>'(j) × Z<sub>k</sub>'(j + 1) < 0 then
16 | | Remove the corresponding index of the feature f<sub>k</sub> from Indices

17 | else if Z<sub>k</sub>''(j) < 0 then
18 | | Z<sub>k</sub>''(j) ← 0;

19 | else if Z<sub>k</sub>''(j) > 0 then
20 | | Z<sub>k</sub>''(j) ← 1;

21 Q<sub>k</sub> ← Z<sub>k</sub>''(1 : end - 1) + Z<sub>k</sub>''(2 : end);

22 Remove all 1s from Q<sub>k</sub>

23 A(:, k) ← Q<sub>k</sub>;

24 Store the sum of each row in A in a vector O

25 Apply K-means clustering on O.

26 Select the cluster K with maximum values.

27 A<sub>c</sub> ← 0; for t ← 1 to length of K do

28 | T ← K(t) + 1;

29 | i ← 0;

30 | for n ← 1 to length of Indices do

31 | | idx ← Indices(n);
32 | | if Z<sub>idx</sub>''(T) is equal to 1 or Z<sub>idx</sub>''(Thr - 1) is equal to 1 then

33 | | | F<sub>indices</sub>(i++) ← idx;
34 | | | F<sub>p</sub>(:, : , n) ← F(:, : , n);

35 | [a, tl, pl] ← SNRSplitter(F<sub>p</sub>, N<sub>snr</sub>, N, Thr, P<sub>ct</sub>);
36 | if a > SA then

37 | | Thr ← T; A<sub>c</sub> ← a; L<sub>t</sub> ← tl; L<sub>p</sub> ← pl;
38 | | G<sub>1</sub> ← F<sub>indices</sub>;

the number of training signals per type (I<sub>f</sub>) and the indices vector (Indices) that contains the indices of selected features. For each HCF, the CVs of features in all signal groups S<sup>m</sup> are calculated using Eq. (30) and data normalization are subsequently performed on these CV values. The first and second derivatives of normalized CVs vector are then calculated. In order to prevent the errors in SNR splitting, the current feature needs to be excluded if it has relative minima or maxima when the CV vector is in a concave upward trend as indicated by the sign of first derivative. Each pair of consecutive signal groups (i.e., S<sup>m</sup> and S<sup>m+1</sup>) that indicate the changes of CV vector in an increasing rate of variation are then identified. Finally, the potential HCFs are fed into a SNR splitter function in order to determine the most relevant HCFs and best threshold (Thr) that can lead to the best splitting accuracy.

<table><caption>**TABLE 4.** SNR Splitter potential thresholds and the corresponding HCFs and splitting accuracies for the “Normal” classes.</caption><thead><tr><td>Potential SNR threshold (dB)</td><td>Splitting Accuracy (%)</td><td>Selected HCFs</td></tr></thead><tbody><tr><td>-4</td><td>94.94</td><td>P<sub>4</sub>, P<sub>5</sub>, P<sub>7</sub>, P<sub>9</sub>, P<sub>10</sub>, γ<sub>max</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr><tr><td>-2</td><td>97.07</td><td>P<sub>4</sub>, P<sub>5</sub>, P<sub>7</sub>, P<sub>10</sub>, γ<sub>max</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr><tr><td>0</td><td>98.84</td><td>P<sub>4</sub>, P<sub>5</sub>, P<sub>6</sub>, P<sub>7</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr><tr><td>+2</td><td>99.49</td><td>P<sub>5</sub>, P<sub>6</sub>, P<sub>7</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr></tbody></table>

<table><caption>**TABLE 5.** SNR splitter potential thresholds and the corresponding HCFs and splitting accuracies for the “Difficult” classes.</caption><thead><tr><td>Potential SNR threshold (dB)</td><td>Splitting Accuracy (%)</td><td>Selected HCFs</td></tr></thead><tbody><tr><td>-4</td><td>95.09</td><td>P<sub>1</sub>, P<sub>4</sub>, P<sub>5</sub>, P<sub>7</sub>, P<sub>9</sub>, P<sub>10</sub>, γ<sub>max</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr><tr><td>-2</td><td>96.65</td><td>P<sub>4</sub>, P<sub>5</sub>, P<sub>7</sub>, P<sub>10</sub>, γ<sub>max</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr><tr><td>0</td><td>98.16</td><td>P<sub>4</sub>, P<sub>5</sub>, P<sub>7</sub>, σ<sub>aa</sub>, m<sub>A</sub>, E</td></tr></tbody></table>

### 3) EVALUATION OF PROPOSED SNR-SPLITTING ALGORITHM
The proposed SNR-splitting algorithm is evaluated on both “Normal” and “Difficult” classes of adopted dataset. Tables 4 and 5 show some potential splitting thresholds obtained by the proposed algorithm together with their corresponding selected HCFs and splitting accuracy for both “Normal” and “Difficult” classes, respectively. For “Normal” class, Thr = +2 dB is identified as the best SNR splitting threshold that can produce the selected HCFs of G<sub>1</sub> with the highest splitting accuracy of 99.49%.

$$G_1 = \{P_5, P_6, P_7, \sigma_{aa}, m_A, E\} \quad (34)$$

Fig. 3 shows the variation of mean envelope for each selected HCF in G<sub>1</sub> along with the overall SNR range for all modulation schemes considered in the “Normal” class. Each selected HCF of modulation schemes are observed to have different values around the splitting threshold of Thr = +2 dB that divide the total SNR range into two smaller ranges. When the whole SNR ranges are considered
<figure><img src="image_5.png" /><figcaption>FIGURE 3. Values of (a) $E$, (b) $m_A$, (c) $\sigma_{aa}$, (d) $P_5$, (e) $P_6$, and (f) $P_7$ against SNR.</figcaption></figure>

in training process, the hyperplanes of classifiers tend to be distorted and more complex CNN is needed to learn the most relevant CFs. On the other hand, the strategy of splitting the total SNR range into two smaller ranges can achieve higher accuracies by training the classifiers with more robust and relevant features extracted using the CNNs with lower complexity. The feasibility of this method will be further investigated in the following sections.

### 4) CROSS-SNR TRAINING METHOD

When a cross-SNR training method is used to train both CNNs on two different SNR ranges, at least one of the SNR value can be shared between these two models. The selection of shared SNR can be done by identifying the SNR ranges of splitter that produce majority of wrong labels.

The distributions of wrong labels are subsequently presented in the histograms of Fig. 4 to identify the SNR values that produce most numbers of wrong labels. For our case, the $CNN_1$ and $CNN_2$ are trained on the SNR ranges of $R_1 \cup [+2, +4] \text{ dB}$ and $R_2 \cup \{0 \text{ dB}\}$, respectively.

## B. HCFs SELECTION FOR AMR

In this section, the characteristic of each HCF under different noise regimes is studied before designing a novel HCFs selection algorithm. The variance of each normalized HFC is first calculated for the overall SNR range. A careful study is then performed on the HCFs in the SNR ranges of $R_1$ and $R_2$ in order to identify the best combinations of HCFs that are able to boost the performance of AMR in the corresponding SNR ranges.
<figure><img src="image_6.png" /><figcaption>FIGURE 4. Histograms of the wrong labels resulted from SNR splitter for (a) "Normal" classes and (b) "Difficult" classes.</figcaption></figure>

<table><caption>TABLE 6. The variance of normalized hand-crafted features.</caption><thead><tr><th>Feature</th><th>Variance</th><th>Feature</th><th>Variance</th></tr></thead><tbody><tr><td>γ<sub>max</sub></td><td>0.4588</td><td>σ<sub>aa</sub></td><td>0.1771</td></tr><tr><td>μ<sub>12</sub><sup>a</sup></td><td>0.3631</td><td>m<sub>A</sub></td><td>0.5437</td></tr><tr><td>E</td><td>0.1430</td><td>P<sub>1</sub></td><td>0.5218</td></tr><tr><td>P<sub>2</sub></td><td>4.5219</td><td>P<sub>3</sub></td><td>0.3183</td></tr><tr><td>P<sub>4</sub></td><td>0.2865</td><td>P<sub>5</sub></td><td>0.1855</td></tr><tr><td>P<sub>6</sub></td><td>0.2567</td><td>P<sub>7</sub></td><td>0.6189</td></tr><tr><td>P<sub>8</sub></td><td>0.4413</td><td>P<sub>9</sub></td><td>0.7683</td></tr><tr><td>P<sub>10</sub></td><td>0.2829</td><td></td><td></td></tr></tbody></table>

## 1) THE VARIANCE OF THE NORMALIZED HCFs

In order to construct a more robust and relevant feature subset, the behavior of each feature stored in original feature set F are investigated under different SNR values. Denote $D_k$ as the variance of each k-th normalized HCF ($F_k$) for all modulation schemes under a specific SNR range, then:

$$D_k = \frac{1}{M \times N_t - 1} \sum_{m=1}^{M} \sum_{i=1}^{N_t} |F_k(i, m) - \mu_k(m)|^2 \quad (35)$$

where M is the size of modulation pool; $N_t$ is the number of signals per modulation for the total SNR range; $F_k(i, m)$ is the normalized feature; $\mu_k(m)$ is the mean value of normalized feature $F_k$ for m-th modulation and it is always equal to one. Both of $F_k(i, m)$ and $\mu_k(m)$ are calculated using Eqs. (36) and (37), respectively. Table 6 shows the variance of each normalized HCF stored in F.

$$F_k(i, m) = \frac{f_k(i, m)}{\frac{1}{N} \sum_{j=1}^{N} f_k(j, m)} \quad (36)$$

$$\mu_k(m) = \frac{1}{N} \sum_{i=1}^{N} F_k(i, m) = 1 \quad (37)$$

## 2) LIMITATIONS OF NORMALIZED VARIANCES FOR HCFs

Substantial amounts of literatures have revealed that it is not desirable to consider the features that can change with SNR during training process due to their high tendency of distorting the hyperplanes of classifiers and affecting their accuracies [30]. Wu et al. [30] considered the variance of

normalized features to select robust features that can be used under varying noise conditions by identifying the cluster with minimum variance value. In contrary to [30], this paper does not only consider the minimum variance criterion in selecting the relevant features that can lead to best classification performance of modulation schemes. From practical point of view, some normalized features with relatively large variances can also be considered to boost the classification accuracy of proposed method because the value of an HCF can fall into different specific ranges when different modulation schemes are applied. In this context, it is also notable that the variance of normalized features [30] did not consider the mean value. Therefore, more investigations are required to construct the robust and relevant HCFs set.

## 3) CLASSIFICATION CONFUSION INDEX (CCI)

In practical, some features can have varying values under specific modulation scheme but they remain as constant values under other schemes. If the variance of normalized feature is the only criterion to be considered in selecting HCFs, some useful features that only demonstrate large variation in certain modulation schemes could be wrongly eliminated. In order to select the features that can lead to the best modulation recognition rate, the variance of all normalized means ($\mu'_k(m)$) is calculated as follows:

$$D_{\mu_k} = \frac{1}{M-1} \sum_{m=1}^{M} |\mu'_k(m) - E(\mu'_k(m))|^2 \quad (38)$$

where $E(\mu'_k(m))$ is the mean value of the normalized mean $\mu'_k(m)$ for the k-th feature $F_k$ and under the m-th modulation scheme. Fundamentally, any features with larger values of $D_{\mu_k}$ imply for the larger distances between modulation schemes. Meanwhile, any feature with smaller variance $D_k$ tends to have better robustness throughout the given SNR range. In other words, the better classification accuracy can be achieved through larger $D_{\mu_k}$ and smaller $D_k$. For any k-th feature, the associated classification confusion index of CCI_k can be calculated as:

$$CCI_k = \frac{D_k}{D_{\mu_k}} \quad (39)$$
<table><caption>TABLE 7. The variance of normalized HFCs for $R_1$.</caption><thead><tr><td>Feature</td><td>Variance</td><td>Feature</td><td>Variance</td></tr></thead><tbody><tr><td>γ<sub>max</sub></td><td>0.2912</td><td>σ<sub>aa</sub></td><td>0.0015</td></tr><tr><td>μ<sup>a</sup><sub>42</sub></td><td>0.0079</td><td>m<sub>A</sub></td><td>0.01914</td></tr><tr><td>E</td><td>0.0025</td><td>P<sub>1</sub></td><td>0.8701</td></tr><tr><td>P<sub>2</sub></td><td>4.0764</td><td>P<sub>3</sub></td><td>0.6347</td></tr><tr><td>P<sub>4</sub></td><td>0.0838</td><td>P<sub>5</sub></td><td>0.0437</td></tr><tr><td>P<sub>6</sub></td><td>0.1528</td><td>P<sub>7</sub></td><td>0.4203</td></tr><tr><td>P<sub>8</sub></td><td>0.1170</td><td>P<sub>9</sub></td><td>1.0651</td></tr><tr><td>P<sub>10</sub></td><td>0.0923</td><td></td><td></td></tr></tbody></table>

<table><caption>TABLE 8. The variance of normalized HFCs for $R_2$.</caption><thead><tr><td>Feature</td><td>Variance</td><td>Feature</td><td>Variance</td></tr></thead><tbody><tr><td>γ<sub>max</sub></td><td>0.6005</td><td>σ<sub>aa</sub></td><td>0.2743</td></tr><tr><td>μ<sup>a</sup><sub>42</sub></td><td>0.2016</td><td>m<sub>A</sub></td><td>1.6820</td></tr><tr><td>E</td><td>0.2292</td><td>P<sub>1</sub></td><td>0.2742</td></tr><tr><td>P<sub>2</sub></td><td>1.9536</td><td>P<sub>3</sub></td><td>0.1123</td></tr><tr><td>P<sub>4</sub></td><td>0.0572</td><td>P<sub>5</sub></td><td>0.0467</td></tr><tr><td>P<sub>6</sub></td><td>0.0433</td><td>P<sub>7</sub></td><td>0.2830</td></tr><tr><td>P<sub>8</sub></td><td>0.2927</td><td>P<sub>9</sub></td><td>0.6493</td></tr><tr><td>P<sub>10</sub></td><td>0.1122</td><td></td><td></td></tr></tbody></table>

After splitting the overall SNR range of $[r_1, r_{L+1}]$ into two smaller ranges of $[r_1, r_{Thr}]$ and $[r_{Thr+1}, r_{L+1}]$, the recalculation of variance for each normalized HCF under these two new SNR ranges can produce new clusters because the behavior of each feature can change under different SNR ranges. Table 7 and Table 8 show the variance of each normalized HCF produced under the SNR ranges of $R_1 = [-20, 0] dB$ and $R_2 = [2, 30] dB$, respectively. When K-means clustering is performed on all HCFs under the SNR range of $R_1$, the following four clusters are produced:

* A1 = {$P_4, P_5, P_6, P_8, P_{10}, \sigma_{aa}, \mu_{42}^a, m_A, E$}

* B1 = {$P_7, \gamma_{max}$}

* C1 = {$P_1, P_3, P_9$}

* D1 = {$P_2$}

Meanwhile, the four clusters produced by K-means clustering in the SNR range of $R_2$ are:

* A2 = {$P_3, P_4, P_5, P_6, P_{10}$}

* B2 = {$P_1, P_7, P_8, \sigma_{aa}, \mu_{42}^a, E$}

* C2 = {$P_9, \gamma_{max}$}

* D2 = {$P_2, m_A$}

The $CCI_k$ of each k-th feature is calculated and the results of all features for SNR ranges of $R_1 = [-20, 0] dB$ and $R_2 = [2, 30] dB$ are presented in Table 10 and Table 9, respectively. The outliers that present in both SNR ranges are removed. For the SNR range of $R_1$, the HCFs of $\mu_{42}^a$ and, $P_5$ and $P_6$ are considered as the outliers and eliminated from the original feature set. K-means clustering is subsequently applied on the remaining features to produce the following four clusters:

* A3 = {$P_1, P_3, P_8, P_9, \gamma_{max}, m_A, E$}

* B3 = {$P_7, P_{10}$}

* C3 = {$P_4$}

* D3 = {$P_2, \sigma_{aa}$}

<table><caption>TABLE 9. The CCIs of HCFs for $R_1$.</caption><thead><tr><td>Feature</td><td>CCI</td><td>Feature</td><td>CCI</td></tr></thead><tbody><tr><td>γ<sub>max</sub></td><td>1.8813</td><td>σ<sub>aa</sub></td><td>42.3944</td></tr><tr><td>μ<sup>a</sup><sub>42</sub></td><td>1325.9667</td><td>m<sub>A</sub></td><td>4.6842</td></tr><tr><td>E</td><td>3.5785</td><td>P<sub>1</sub></td><td>2.6229</td></tr><tr><td>P<sub>2</sub></td><td>40.8835</td><td>P<sub>3</sub></td><td>4.3915</td></tr><tr><td>P<sub>4</sub></td><td>49.3454</td><td>P<sub>5</sub></td><td>91.6561</td></tr><tr><td>P<sub>6</sub></td><td>214.9963</td><td>P<sub>7</sub></td><td>21.6477</td></tr><tr><td>P<sub>8</sub></td><td>7.4465</td><td>P<sub>9</sub></td><td>1.1214</td></tr><tr><td>P<sub>10</sub></td><td>16.2419</td><td></td><td></td></tr></tbody></table>

<table><caption>TABLE 10. The CCIs of HCFs for $R_2$.</caption><thead><tr><td>Feature</td><td>CCI</td><td>Feature</td><td>CCI</td></tr></thead><tbody><tr><td>γ<sub>max</sub></td><td>0.1396</td><td>σ<sub>aa</sub></td><td>1.2346</td></tr><tr><td>μ<sup>a</sup><sub>42</sub></td><td>0.0650</td><td>m<sub>A</sub></td><td>0.5535</td></tr><tr><td>E</td><td>0.6293</td><td>P<sub>1</sub></td><td>0.2867</td></tr><tr><td>P<sub>2</sub></td><td>2.6910</td><td>P<sub>3</sub></td><td>0.2166</td></tr><tr><td>P<sub>4</sub></td><td>0.1320</td><td>P<sub>5</sub></td><td>0.0394</td></tr><tr><td>P<sub>6</sub></td><td>0.0339</td><td>P<sub>7</sub></td><td>0.0363</td></tr><tr><td>P<sub>8</sub></td><td>0.5489</td><td>P<sub>9</sub></td><td>0.1461</td></tr><tr><td>P<sub>10</sub></td><td>0.1685</td><td></td><td></td></tr></tbody></table>

For the SNR range of $R_2$, $P_2$, and $\sigma_{aa}$ are identified as the outliers and eliminated from the original feature sets. Four clusters are obtained from the remaining features by using K-means clustering, where:

* A4 = {$P_5, P_6, P_7, \mu_{42}^a$}

* B4 = {$P_3, P_4, P_9, P_{10}, \gamma_{max}$}

* C4 = {$P_1$}

* D4 = {$P_8, m_A, E$}

The complete pseudocode of HCFs selection based on the CCI criterion is summarized using Algorithm 3. Evidently, different clusters of HCFs can be produced under the SNR ranges of $R_1$ and $R_2$ when the criteria of CCI and normalized variance are considered. The performance of these HCFs selection algorithms will be evaluated in the next sections.

## VI. SIMULATION RESULTS

### A. SIMULATION SETTINGS

RadioML 2018.01A dataset is selected for performance evaluation. The performances of proposed HCFs selection algorithm and proposed DL-based AMR method are compared with the methods used by O'Shea et al. [4]. The parameter settings required for simulation studies are presented in Table 11.

The method introduced by Wu et al. [30] to select a robust set of HCFs based on the normalized variance is considered for performance validation of the proposed HCFs selection algorithm. In addition, a Baseline (BL) AMR method presented in [4] is also selected for AMR performance validation of the proposed AMR based on the proposed HCFs selection algorithm and SNR splitter, where the former method considers a total of 28 HCFs including HOMs, HOCs and other features. Finally, both of the CNN/VGG and RN methods introduced in [4] are also considered for performance validation of the proposed DL-based AMR method.
<table><caption>**Algorithm 3** Pseudo Code of HCFs Selection Based on the CCI Criterion</caption><thead><tr><th></th><th>Method</th><th>HCFs vector dimension on R<sub>1</sub></th><th>HCFs vector dimension on R<sub>2</sub></th></tr></thead><tbody><tr><td rowspan="4">1 Function</td><td>GetClassificationFeatures (F, Thr, N<sub>snr</sub>):</td><td></td><td></td></tr><tr><td>Data: 3D array of HCFs F(P, EF, FC) where P is the modulations pool, EF is the extracted features for all SNR values and FC is the features count.</td><td></td><td></td></tr><tr><td>SNR Threshold (Thr) where the SNR total range is split into two ranges R<sub>1</sub> and R<sub>2</sub>.</td><td></td><td></td></tr><tr><td>SNR ranges (N<sub>snr</sub>).</td><td></td><td></td></tr><tr><td rowspan="2">2</td><td>Result: Features group F<sub>1</sub> for modulation classification on range R<sub>1</sub>.</td><td></td><td></td></tr><tr><td>Features group F<sub>2</sub> for modulation classification on range R<sub>2</sub>.</td><td></td><td></td></tr><tr><td>3</td><td>Indices ← [1 to FC];</td><td></td><td></td></tr><tr><td>4</td><td>for k ← 1 to FC do</td><td></td><td></td></tr><tr><td>5</td><td>&nbsp;&nbsp;&nbsp;&nbsp;for m ← 1 to N<sub>snr</sub> do</td><td></td><td></td></tr><tr><td>6</td><td>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if m &lt; Thr then</td><td></td><td></td></tr><tr><td>7</td><td>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CCI<sub>1</sub>(k) = <sup>D<sub>k</sub></sup>&frasl;<sub>D<sub>&mu;<sub>k</sub></sub></sub>;</td><td></td><td></td></tr><tr><td>8</td><td>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;else</td><td></td><td></td></tr><tr><td>9</td><td>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CCI<sub>2</sub>(k) = <sup>D<sub>k</sub></sup>&frasl;<sub>D<sub>&mu;<sub>k</sub></sub></sub>;</td><td></td><td></td></tr><tr><td>10</td><td>Remove outliers from CCI<sub>1</sub> and CCI<sub>2</sub>.</td><td></td><td></td></tr><tr><td>11</td><td>Apply K-means clustering on both CCI<sub>1</sub> and CCI<sub>2</sub>.</td><td></td><td></td></tr><tr><td>12</td><td>Select the cluster K<sub>1</sub> with minimum values in CCI<sub>1</sub>.</td><td></td><td></td></tr><tr><td>13</td><td>Select the cluster K<sub>2</sub> with minimum values in CCI<sub>2</sub>.</td><td></td><td></td></tr><tr><td>14</td><td>F<sub>1</sub> ← corresponding indices of features in K<sub>1</sub>;<br/>F<sub>2</sub> ← corresponding indices of features in K<sub>2</sub>;</td><td></td><td></td></tr></tbody></table>

<table><caption>**TABLE 11.** Experiment parameters and setting.</caption><thead><tr><th>Parameters</th><th>Value</th></tr></thead><tbody><tr><td>Input Signals</td><td>IQ samples at each SNR value for the "Normal" classes</td></tr><tr><td>Training signals per type per SNR<br/>I<sub>t</sub><sup>s</sup></td><td>3277</td></tr><tr><td>Training signals per type I<sub>t</sub></td><td>85, 202</td></tr><tr><td>Testing signals per type per SNR<br/>T<sub>t</sub><sup>s</sup></td><td>819</td></tr><tr><td>Testing signals per type T<sub>t</sub></td><td>21, 294</td></tr></tbody></table>

## B. SIMULATION RESULTS FOR HCFs SELECTION ALGORITHM

The effectiveness of the proposed HCFs selection algorithm in identifying the most relevant features is investigated in this section. First, we make a fair comparison between the performance of AMR based on the proposed HCFs selection algorithm and that based on the normalized variance used in [30]. The proposed CCI criterion and the normalized variance [30] are used separately to select the HCFs subset on the split SNR ranges R<sub>1</sub> and R<sub>2</sub> to be then fed into SVM<sub>1</sub> and

<table><caption>**TABLE 12.** Dimension of HCFs vector of the compared methods for R<sub>1</sub> and R<sub>2</sub>.</caption><thead><tr><th>Method</th><th>HCFs vector dimension on R<sub>1</sub></th><th>HCFs vector dimension on R<sub>2</sub></th></tr></thead><tbody><tr><td>HCFs-CCI-2cl</td><td>9</td><td>9</td></tr><tr><td>HCFs-VAR-2cl</td><td>11</td><td>11</td></tr><tr><td>HCFs-CCI-1cl</td><td>7</td><td>4</td></tr><tr><td>HCFs-VAR-1cl</td><td>9</td><td>5</td></tr><tr><td>BL [4]</td><td></td><td>28</td></tr></tbody></table>

SVM<sub>2</sub> classifiers, respectively. However, we propose to use the following methods for the comparison:

* HCFs-CCI-2cl: Adopt the proposed HCFs selection algorithm with two clusters of HCFs, i.e., {A<sub>3</sub>, B<sub>3</sub>} and {A<sub>4</sub>, B<sub>4</sub>} for R<sub>1</sub> and R<sub>2</sub>, respectively.

* HCFs-VAR-2cl: Adopt the variance of normalized HCFs for HCFs selection [30] with two clusters of HCFs, i.e., {A<sub>1</sub>, B<sub>1</sub>} and {A<sub>2</sub>, B<sub>2</sub>} for R<sub>1</sub> and R<sub>2</sub>, respectively.

* HCFs-CCI-1cl: Adopt the proposed HCFs selection algorithm with one cluster of HCFs, i.e., A<sub>3</sub> and A<sub>4</sub> for R<sub>1</sub> and R<sub>2</sub>, respectively.

* HCFs-VAR-1cl: Adopt the variance of normalized HCFs for HCFs selection [30] with one cluster of HCFs, i.e., A<sub>1</sub> and A<sub>2</sub> for R<sub>1</sub> and R<sub>2</sub>, respectively.

On the other hand, the performances of the aforementioned methods are also compared with the BL method presented in [4]. Table 12 presents the dimension of HCFs vector produced by each compared HCFs selection algorithm for both SNR ranges of R<sub>1</sub> and R<sub>2</sub>, whereas the classification performances of all compared methods are illustrated in Fig. 5. It is observed that the proposed HCFs selection algorithm with two clusters of HCFs (i.e., HCFs-CCI-2cl) has achieved the best classification accuracy among all compared methods. In addition, the proposed HCFs selection algorithm is also observed to outperform the compared HCFs selection algorithms based on normalized variance [29]. For instance, the classification accuracy of HCFs-CCI-2cl is more than 81% when the SNR is equal to 2 dB but HCFs-VAR-2cl can only produce the accuracy level around 60%. It is also notable that HCFs-CCI-2cl is able to produce the maximum classification accuracy rate of 97% at the high SNR regions, whereas the best classification accuracy rates obtained by both HCFs-VAR-2cl and BL [4] are only 96.7% and 94.6%, respectively. Similar observations are made when only one cluster of HCFs is considered, where the classification accuracy rate of HCFs-CCI-1cl is around 8% better than that of HCFs-VAR-1cl throughout the SNR ranges. Referring to the simulation results presented in Table 12 and Fig. 5, it can be concluded that the proposed HCFs selection algorithm has more competitive performance for being able to select the most relevant HCFs and produce high classification accuracy with the lower dimension of HCFs vector.

## C. SIMULATION RESULTS FOR THE PROPOSED DL-BAESD AMR TECHNIQUE

In this section, simulations are conducted for comparative studies between the proposed DL-based AMR method and
<figure><img src="image_7.png" /><figcaption>FIGURE 5. Performance comparison of the proposed HCFs selection algorithm, the normalized variance, and the baseline method in [4].</figcaption></figure>

<table><caption>TABLE 13. Network trainable parameters.</caption><thead><tr><td>Network</td><td>Trainable Parameters</td></tr></thead><tbody><tr><td>CNN<sub>1</sub></td><td>145,856</td></tr><tr><td>CNN<sub>2</sub></td><td>27,328</td></tr><tr><td>CNN/VGG [4]</td><td>257,099</td></tr><tr><td>RN [4]</td><td>236,344</td></tr></tbody></table>

<table><caption>TABLE 14. Network structure for CNN<sub>1</sub> and CNN<sub>2</sub>.</caption><thead><tr><td>Layer</td><td>CNN<sub>1</sub> Output dimensions</td><td>CNN<sub>2</sub> Output dimensions</td></tr></thead><tbody><tr><td>Input</td><td>2 × 1024</td><td>2 × 1024</td></tr><tr><td>conv</td><td>32 × 1024</td><td>16 × 1024</td></tr><tr><td>BN, RELU, Max Pool</td><td>32 × 512</td><td>16 × 512</td></tr><tr><td>conv</td><td>64 × 512</td><td>16 × 512</td></tr><tr><td>BN, RELU, Max Pool</td><td>64 × 256</td><td>16 × 256</td></tr><tr><td>conv</td><td>64 × 256</td><td>16 × 256</td></tr><tr><td>BN, RELU, Max Pool</td><td>64 × 128</td><td>16 × 128</td></tr><tr><td>FC/Softmax</td><td>11</td><td>11</td></tr></tbody></table>

both of the CNN/VGG and RN methods presented in [4]. The same parameter settings summarized in Table 11 are used in this performance analysis. Table 14 present the network structures of two CNN models denoted as CNN<sub>1</sub> and CNN<sub>2</sub> used to perform the contextual features extraction from signals belong to the SNR ranges of R<sub>1</sub> and R<sub>2</sub>, respectively. Notably, both of CNN<sub>1</sub> and CNN<sub>2</sub> have fewer numbers of trainable parameters as compared to both CNN/VGG and RN employed in [4] as shown in Table 13.

CNN<sub>1</sub> is trained on 11 modulations schemes in the range of [-20, +4] dB, while CNN<sub>2</sub> is trained on those in the range of [0, +30] dB. Cross Entropy (CE) function is applied as loss

<table><caption>TABLE 15. Main hyper-parameters for CNNs training.</caption><thead><tr><td>Network</td><td>Hyper-Parameters</td></tr></thead><tbody><tr><td>mini batch size</td><td>1024</td></tr><tr><td>maximum epochs number</td><td>30</td></tr><tr><td>optimizer</td><td>SGDM</td></tr><tr><td>Loss function</td><td>cross entropy</td></tr><tr><td>Learning Rate</td><td>0.01</td></tr></tbody></table>

function, considering that AMR is essentially a multi-class classification task. The function of CE is shown as:

$$y_{CE} = - \sum_{C} y_{c} \log (\hat{y}_{c}) \quad (40)$$

where $y_c$ is the ground truth vector that can be achieved through one-hot encoding of sample label. $\hat{y}_c$ is the predicted vector. C is the number of the samples' types. Table 15 shows some key hyper-parameters for the training process of both CNN<sub>1</sub> and CNN<sub>2</sub>. The mini batch size of 1024 with 30 epochs are set for the training processes of both CNN models. In addition, a stochastic gradient descent with momentum (SGDM) optimizer is chosen to prevent the overfitting of networks. After constructing both CNN<sub>1</sub> and CNN<sub>2</sub> according to Table 14, their weights are initialized with learning rate equal to 0.01. Thereafter, CNN<sub>1</sub> and CNN<sub>2</sub> are trained on the corresponding training dataset and their weights are updated according to SGDM until validation loss is not improved in setting epochs.
<figure><img src="image_8.png" /><figcaption>FIGURE 6. Performance comparison of the proposed AMR method.</figcaption></figure>

The classification accuracies produced by the proposed DL-based AMR method and other benchmark methods are presented in Fig. 6. It is proven that the presence of HCFs can indeed contribute to the performance boosting because the networks that consider the concatenation of HCFs and CFs are able to achieve higher classification accuracy than those which only consider the CFs. The proposed method is also able to produce better accuracy than other benchmark methods regardless of the presence of HCFs. Particularly, the classification accuracy rate of proposed method is more than 87% and 95% when SNR ≥ −2 dB and SNR ≥ 0 dB, respectively. It is also notable that the proposed method is able to achieve the maximum classification accuracy rate of 100% for SNR ≥ 6 dB.

The classification performances of proposed DL-based AMR method in handling each individual modulation scheme is further illustrated in Fig. 7. Specifically, the modulation schemes with lower information rates and unique structures such as AM and FM can be classified by the proposed method with high accuracy at the low SNR regions. For instance, both of the AM and FM schemes are identified by the proposed method with accuracy levels greater than 88% and 99.8% for SNR = −4 dB and SNR ≥ −2 dB, respectively. Meanwhile, the proposed method is able to identify the low order schemes such as GMSK, 4ASK and BPSK with classification accuracy rate around 99% when SNR = −2 dB. The modulation schemes with more sophisticated structures and higher orders require higher SNR for robust performance. From Fig. 7, it is notable that the proposed method is able to classify all modulation schemes with the accuracy rate of 100% when SNR ≥ 8 dB.

<figure><img src="image_9.png" /><figcaption>FIGURE 7. Classification performance for individual modulation schemes (a) AM, FM, GMSK, and OQPSK schemes; and (b) OOK, 4ASK, BPSK, QPSK, 8PSK, and 16QAM modulation schemes.</figcaption></figure>

Six confusion matrices produced by the proposed DL-based AMR method across all 11 classes for multiple SNR values are presented in Fig. 8 in order to further analyze the classification performance of proposed method. On one
<figure><img src="image_10.png" /><figcaption>(a)</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(c)</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(e)</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>(b)</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>(d)</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>(f)</figcaption></figure>

<caption>FIGURE 8. 11-modulation confusion matrix of the proposed DL-based AMR method for (a) SNR = -4dB, (b) SNR = -2dB, (c) SNR = 0dB, (d) SNR = 2dB, (e) SNR = 6dB, and (f) SNR = 8dB.</caption>
hand, it can be observed that the majority of classification errors can be found between phase-shift keying (QPSK, 8PSK, OQPSK), especially with substantial noise. This may be due to lack of information at low SNR values [31]. Besides, they have similar symbol structure and statistical information when using features computed by using the IQ samples. On the other hand, some signals with lower information rates and unique structure such as AM and FM are more readily classified at low SNR.

## VII. CONCLUSION

In this paper, a novel DL-based AMR method that is able to perform robustly under varying noise regimes is proposed by addressing the key issues of feature extraction and selection criteria of features. Three groups of HCFs are first extracted from the received signals by using the proposed algorithms. The first group of HCFs is used to identify the SNR range of received signals, whereas the remaining two groups are concatenated with the CFs extracted by two corresponding CNNs in order to solve the modulation classification tasks. A novel algorithm is also introduced to select the best SNR threshold and HCFs that can split the total SNR range into two smaller ranges. Furthermore, an HCFs selection algorithm is proposed to select the most relevant features in order to reduce the dimension of feature vector.

Extensive simulation studies have verified the reliability and effectiveness of proposed HCFs selection algorithm in reducing the redundant features without compromising the classification performances. These desirable characteristics enables the proposed method to be implemented through simpler CNNs and verified using 11 modulation schemes. The competitive classification performance demonstrated by proposed method have proven that it is more efficient to split the total SNR range with positive and negative values into two smaller ranges. When training with the total SNR range, the hyperplanes of classifiers tend to be distorted and hence compromising their classification accuracies. Furthermore, the presence of HCFs is also proven useful to boost the classification performance of the proposed DL-based AMR method. The proposed DL-based AMR method is expected to make significant contribution in wireless communication because it is able to solve the AMR tasks with higher classification accuracy while incurring lower computation complexity due to the adoption of simpler CNN architecture. Current research work can be extended by incorporating more optimized CNN architectures into the proposed DL-based AMR method to solve larger sets of modulation schemes.

## REFERENCES

[1] B. Jdid, K. Hassan, I. Dayoub, W. H. Lim, and M. Mokayef, “Machine learning based automatic modulation recognition for wireless communications: A comprehensive survey,” *IEEE Access*, vol. 9, pp. 57851–57873, 2021, doi: 10.1109/ACCESS.2021.3071801.

[2] T. O’Shea and N. West, “Radio machine learning dataset generation with GNU radio,” in *Proc. GNU Radio Conf., vol. 1*, no. 1, 2016. [Online]. Available: https://pubs.gnuradio.org/index.php/grcon/article/view/11

[3] T. J O’Shea, J. Corgan, and T. Charles Clancy, “Convolutional radio modulation recognition networks,” 2016, arXiv:1602.04105. [Online]. Available: http://arxiv.org/abs/1602.04105

[4] T. J. O’Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning based radio signal classification,” *IEEE J. Sel. Topics Signal Process.*, vol. 12, no. 1, pp. 168–179, Feb. 2018, doi: 10.1109/JSTSP.2018.2797022.

[5] N. E. West and T. J. O’Shea, “Deep architectures for modulation recognition,” 2017, arXiv:1703.09197. [Online]. Available: https://arxiv.org/abs/1703.09197

[6] T. O’Shea and J. Hoydis, “An introduction to deep learning for the physical layer,” *IEEE Trans. Cogn. Commun. Netw.*, vol. 3, no. 4, pp. 563–575, Dec. 2017, doi: 10.1109/TCCN.2017.2758370.

[7] Q. Zhang, Z. Xu, and P. Zhang, “Modulation scheme recognition using convolutional neural network,” *J. Eng.*, vol. 2019, no. 23, pp. 9075–9078, Dec. 2019, doi: 10.1049/joe.2018.9188.

[8] Y. Sun, J. Li, F. Lin, and G. Pan, “Automatic signal modulation recognition based on deep convolutional neural network,” in *Proc. 3rd Int. Conf. Comput. Eng., Inf. Sci. Appl. Technol. (ICCIA)*, 2019, pp. 550–554, doi: 10.2991/iccia-19.2019.86.

[9] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015, arXiv:1409.1556. [Online]. Available: https://arxiv.org/abs/1409.1556

[10] Z. Zhang, C. Wang, C. Gan, S. Sun, and M. Wang, “Automatic modulation classification using convolutional neural network with features fusion of SPWVD and BJD,” *IEEE Trans. Signal Inf. Process. Netw.*, vol. 5, no. 3, pp. 469–478, Sep. 2019, doi: 10.1109/TSIPN.2019.2900201.

[11] S. Peng, H. Jiang, H. Wang, H. Alwageed, and Y.-D. Yao, “Modulation classification using convolutional neural network based deep learning model,” in *Proc. 26th Wireless Opt. Commun. Conf. (WOCC)*, Apr. 2017, pp. 1–5, doi: 10.1109/WOCC.2017.7929000.

[12] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” *Commun. ACM*, vol. 60, no. 6, pp. 84–90, May 2017, doi: 10.1145/3065386.

[13] K. Jiang, J. Zhang, H. Wu, A. Wang, and Y. Iwahori, “A novel digital modulation recognition algorithm based on deep convolutional neural network,” *Appl. Sci.*, vol. 10, no. 3, p. 1166, Feb. 2020, doi: 10.3390/app10031166.

[14] B. Tang, Y. Tu, Z. Zhang, and Y. Lin, “Digital signal modulation classification with data augmentation using generative adversarial nets in cognitive radio networks,” *IEEE Access*, vol. 6, pp. 15713–15722, 2018, doi: 10.1109/ACCESS.2018.2815741.

[15] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” 2014, arXiv:1710.07035. [Online]. Available: https://arxiv.org/abs/1710.07035

[16] K. Hassan, I. Dayoub, W. Hamouda, C. N. Nzeza, and M. Berbineau, “Blind digital modulation identification for spatially-correlated MIMO systems,” *IEEE Trans. Wireless Commun.*, vol. 11, no. 2, pp. 683–693, Feb. 2012, doi: 10.1109/TWC.2011.122211.110236.

[17] J. Nie, Y. Zhang, Z. He, S. Chen, S. Gong, and W. Zhang, “Deep hierarchical network for automatic modulation classification,” *IEEE Access*, vol. 7, pp. 94604–94613, 2019, doi: 10.1109/ACCESS.2019.2928463.

[18] I. Vernikos, E. Mathe, E. Spyrou, A. Mitsou, T. Giannakopoulos, and P. Mylonas, “Fusing handcrafted and contextual features for human activity recognition,” in *Proc. 14th Int. Workshop Semantic Social Media Adaptation Personalization (SMAP)*, Jun. 2019, pp. 1–6, doi: 10.1109/SMAP.2019.8864848.

[19] D. T. Nguyen, T. D. Pham, N. R. Baek, and K. R. Park, “Combining deep and handcrafted image features for presentation attack detection in face recognition systems using visible-light camera sensors,” *Sensors*, vol. 18, no. 3, p. 699, 2018, doi: 10.3390/s18030699.

[20] H. Wang, A. Cruz-Roa, A. Basavanhally, H. Gilmore, N. Shih, M. Feldman, J. Tomaszewski, F. González, and A. Madabhushi, “Cascaded ensemble of convolutional neural networks and handcrafted features for mitosis detection,” *Proc. SPIE*, vol. 9041, Feb. 2014, Art. no. 90410B, doi: 10.1117/12.2043902.

[21] S. Mallat and S. Zhong, “Characterization of signals from multi-scale edges,” *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 14, no. 7, pp. 710–732, Jul. 1992, doi: 10.1109/34.142909.

[22] T. V. R. O. Camara, A. D. L. Lima, B. M. M. Lima, A. I. R. Fontes, A. D. M. Martins, and L. F. Q. Silveira, “Automatic modulation classification architectures based on cyclostationary features in impulsive environments,” *IEEE Access*, vol. 7, pp. 138512–138527, 2019, doi: 10.1109/ACCESS.2019.2943300.

[23] J. Zhang, Y. Li, and J. Yin, “A novel modulation classification method for fm signals based on the time-frequency distribution and CNN,” *IET Radar, Sonar Navigat.*, vol. 12, no. 2, pp. 244–249, Oct. 2017, doi: 10.1049/iet-rsn.2017.0265.
[24] N. Daldal, Z. Cömert, and K. Polat, "Automatic determination of digital modulation types with different noises using convolutional neural network based on time-frequency information," *Appl. Soft Comput.*, vol. 86, Jan. 2020, Art. no. 105834, doi: 10.1016/j.asoc.2019.105834.

[25] D. Figueiredo, A. Furtado, and R. Oliveiray, "Modulation classification using joint time and frequency-domain data," in *Proc. IEEE 91st Veh. Technol. Conf. (VTC-Spring)*, May 2020, pp. 1-5, doi: 10.1109/VTC2020-Spring48590.2020.9128493.

[26] E. E. Azzouz and A. K. Nandi, "Automatic identification of digital modulation types," *Signal Process.*, vol. 47, no. 1, pp. 55-69, Nov. 1995. [Online]. Available: http://www.sciencedirect.com/science/article/pii/0165168495000992, doi: 10.1016/0165-1684(95)00099-2.

[27] W. Xie, S. Hu, C. Yu, P. Zhu, X. Peng, and J. Ouyang, "Deep learning in digital modulation recognition using high order cumulants," *IEEE Access*, vol. 7, pp. 63760-63766, 2019, doi: 10.1109/ACCESS.2019.2916833.

[28] Z. Zhang, Z. Hua, and Y. Liu, "Modulation classification in multipath fading channels using sixth-order cumulants and stacked convolutional auto-encoders," *IET Commun.*, vol. 11, no. 6, pp. 910-915, Apr. 2017, doi: 10.1049/iet-com.2016.0533.

[29] C. E. Brown, *Coefficient Variation*. Berlin, Germany: Springer, 1998, pp. 155-157, doi: 10.1007/978-3-642-80328-4_13.

[30] Z. Wu, S. Zhou, Z. Yin, B. Ma, and Z. Yang, "Robust automatic modulation classification under varying noise conditions," *IEEE Access*, vol. 5, pp. 19733-19741, Aug. 2017, doi: 10.1109/ACCESS.2017.2746140.

[31] S. Kharbech, I. Dayoub, M. Zwingelstein-Colin, and E. P. Simon, "Blind digital modulation identification for MIMO systems in railway environments with high-speed channels and impulsive noise," *IEEE Trans. Veh. Technol.*, vol. 67, no. 8, pp. 7370-7379, Aug. 2018, doi: 10.1109/TVT.2018.2834869.

<figure><img src="image_16.png" /></figure>

**BACHIR JIDID** received the B.Eng.Sc. degree in communications engineering from the Higher Institute for Applied Sciences and Technology, Damascus, Syria, in 2013. He is currently pursuing the M.Phil. degree with the Faculty of Engineering, Technology and Built Environment, UCSI University, Kuala Lumpur, Malaysia. His research interests include wireless communications, signal processing, image processing, embedded systems, artificial intelligence, and its application in wireless communications.

<figure><img src="image_17.png" /></figure>

**WEI HONG LIM** (Senior Member, IEEE) received the B.Eng. degree (Hons.) in mechatronic engineering and the Ph.D. degree in computational intelligence from Universiti Sains Malaysia, Penang, Malaysia, in 2011 and 2014, respectively. From 2015 to 2017, he was with the Intelligent Control Laboratory, National Taipei University of Technology, Taiwan, as a Postdoctoral Researcher, where he was a Visiting Researcher, in 2019. He is currently an Assistant Professor with the Faculty of Engineering, Technology and Built Environment, UCSI University. He is working with three national research grants awarded by the Ministry of Education, Malaysia, and five internal grant projects supported by UCSI University. He has published more than 50 research articles in research areas related to computational intelligence, optimization algorithms, energy management, and digital image processing. To date, he has been actively involved in various professional bodies and registered as the European Engineer (EUR ING) with the Fédération Européenne d'Associations Nationales d'Ingénieurs (FEANI), a Chartered Engineer (C.Eng.) with the Engineering Council UK (ECUK), and a Professional Technologist (P.Tech.) with the Malaysia Board of Technologist (MBOT). He is also an Active Reviewer for various reputable journals, such as IEEE ACCESS, Complexity, Mathematical Problems in Engineering, Computational Intelligence, and Neuroscience.

<figure><img src="image_18.png" /></figure>

**IYAD DAYOUB** (Senior Member, IEEE) received the B.Eng. degree in telecommunications and electronics, Syria, in 1993, the M.A.Sc. degree in electrical engineering from the National Polytechnic Institute of Lorraine (INPL), and the Ph.D. degree from the University of Valenciennes/Institute of Electronics, Microelectronics and Nanotechnology (IEMN), in 2001. He has worked as a System Engineer with Siemens, Middle East. He has worked as a Researcher with Alcatel Business Systems Alcatel, Colombes, Paris. From 2007 to 2014, he was a member in the area of electrical engineering, electronics, photonics, and systems of the National Council of Universities (CNU), France. From 2010 to 2014, he was an Adjunct Professor with Concordia University, Montreal. He is currently Professor of communications engineering. His current research activities at the IEMN of Université Polytechnique Hauts de France (UPHF) and INSA des Hauts de France are focused on wireless communications, high-speed communications, cognitive radio, and hybrid radio-optic technologies. He is a member of several International Conference Advisory Committees, Technical Program Committees, and Organization Committees, such as VTC, GLOBECOM, ICC, PIMRC, and WWC.

<figure><img src="image_19.png" /></figure>

**KAIS HASSAN** was born in 1975. He graduated in telecommunications engineering from the Higher Institute for Applied Science and Technology (HIAST), Syria, in 2000. He received the M.S. degree in telecommunications systems and the Ph.D. degree in telecommunications from the University of Valenciennes, France, in 2009 and 2012, respectively. From 2000 to 2008, he was a Research Engineer at HIAST. From 2013 to 2014, he was an Assistant Professor with the University of Western Brittany, France. Since September 2014, he has been an Associate Professor with Le Mans University (LAUM), France. He is with the Acoustics Laboratory, LAUM. His research interests include wireless communications, signal processing, and cognitive radio.

<figure><img src="image_20.png" /></figure>

**MOHD RIZON BIN MOHAMED JUHARI** received the bachelor's degree in engineering and the master's degree in electrical and electronics engineering from the University of Tokushima, Japan, in 1993 and 1995, respectively, and the Ph.D. degree in engineering from the Department of Computer Science and Intelligent System, Oita University, Japan, in 2002. From 1995 to 1997, he was a Software Engineer with System LSI Laboratory, Mitsubishi Electric Corporation, Itami, Japan. In 1997, he moved to the University of Malaya, Malaysia. He was the Head of the Biomedical Engineering Department, University Malaya. He was a Professor of biomedical technology with King Saud University, Saudi Arabia, in 2010. He served at MIEC, Fuzhou University, China, in 2019. He is currently a Professor with the Department of Electrical and Electronics, UCSI University, Malaysia.