## 1 Introduction  
With the recent advancements in telecommunication technologies, various bandwidth-critical applications have attracted the attention of users. This condition has required broadband data service providers to upgrade their network to fulfil the requirements of these applications. Thus, heterogeneous networks have evolved to support various types of data traffic on the basis of end user demands for supporting the existing voice and data services, and to meet the next-generation network requirements. Managing the radio and other resources in a complex heterogeneous network requires real-time information extraction and processing. With growing heterogeneity and dynamics, modulation recognition (MR) has attracted the attention of researchers in the last two decades. Radio frequency and microwave signals travelling in the air with different modulation types and frequencies fall in a huge band, and these signals should be identified and monitored for various applications. MR has been widely applied in various areas, such as mobile telephony, software and cognitive defined radio, military intelligence, communication jammers, surveillance, spectrum management, and communication reconnaissance [1]. At present, digital MR methods have witnessed a paradigm shift from manual operational systems to automatic ones because of several advantages. Manual modulation recognition (MMR) requires the measurement of parameters of intercepted signals to recognize modulation types. In MMR, four types of information are available for the search operator, namely, intermediate frequency (IF) time waveform, average and instantaneous spectrum of signal, sound, and signal and instantaneous amplitude. Manual analysis becomes problematic and inaccurate when the number of intercepted modulation types increases. In addition, this method requires experienced analyzers and does not guarantee reliable classification results. However, these shortcomings can be addressed via automatic modulation recognition (AMR). AMR is more powerful than MMR because it integrates an automatic modulation recognizer into an electronic receiver [2]. AMR refers to a process that helps in differentiating the modulation model from the received signal with unknown format and is conducted at the receiver [3]. AMR plays a significant role in cooperative and noncooperative communication. Cooperative communication is a means of transmission, wherein wireless users not only transmit their own information, but also repeat other users' information during their transmission to a common destination. Noncooperative communication is a type of classical multiple-access channel, wherein users directly send information to a common destination without repetition [4]. AMR is a challenging task, especially under noncooperative settings, because signal identification is difficult when the traits are unknown [5]. Similarly, blind AMR algorithms are considered to be complex because signal recognition is difficult due to the absence of prior knowledge of modulated signals. Thus, AMR algorithms have received considerable attention as a new and interesting research area [6]. Figure 1 illustrates the essential role of AMR in the receiver part of communication systems. The AMR block is composed of two parts: signal preprocessing and classifier modules. The preprocessing module estimates the synchronization parameters, such as the frequency offset of the received signal, timing recovery, and power. In the second part, signal disturbances (e.g., interference identification) are eliminated, resulting in enhanced performance [6]. In the literature, AMR techniques are classified into two classes, namely, likelihood-based (LB) and signal statistical feature-based (FB) approaches. In the LB approach, AMR is a multiple-composite hypothesis-testing problem.