## 1 Introduction  
As a classic pattern recognition problem, the modulation recognition of communication signals has many important research prospects and application value. In the military field, it is a precondition to carry out communication reconnaissance and jamming. Once the signal modulation of the enemy's communication system is clarified, the enemy's signals can be demodulated and communication information can be obtained. In the civil filed, signal modulation recognition can be used to signal confirmation, interference identification, spectrum management, and spectrum monitoring. Therefore, a secure and reliable feature extraction method is needed to effectively recognize the different signal modulations [1] in a complex environment. Modulation pattern recognition generally uses statistical pattern recognition methods and decision-making methods. At present, most of the applications are statistical pattern recognition methods, which can be divided into two parts: feature extraction and classifier design. Good features are a key factor to improve the classification accuracy. In recent decades, researchers have proposed many methods for extracting feature parameters such as instantaneous features [2], fractal features [3], signal constellation reconstruction [4], etc. As another key factor of signal pattern recognition, classifiers directly determine the classification results. Machine learning classifiers are widely used in signal classification, and include the decision tree classifier [5], K-nearest neighbor (KNN) Classifier [6], support vector machine (SVM) classifier [7], neural network classifier [8], etc. In addition, confusion matrix, recognition rate, and running time are widely used as evaluation methods. Reference [9] proposed a novel method called the A-test for evaluating the structural risk of a classifier both quantitatively and graphically. The structural risk of a classification method was defined as the instability of the method with the new test data. In the A-test method, the classification error percentage of the different values of \(z\) is calculated by using the balanced Z-fold verification method. In this paper, we focus mostly on fractal feature extraction. Different methods are proposed for estimating the fractal dimension. However, considering that fractal features are limited to a few fractal types (e.g., box dimension), the application of fractal methods for the purpose of extracting discriminative features has not been deeply investigated. The main goal of the present paper is thus to help fill this research gap. Therefore, many kinds of fractal dimensions (i.e., box fractal dimension, Katz fractal dimension, Higuchi fractal dimension, Petrosian fractal dimension, and Sevcik fractal dimension) are extracted in order to provide a comprehensive description. In the classifier section, Back-Propagation (BP) neural network, grey relation analysis, random forest, and K-nearest neighbor are used to classify the modulated signals. BP neural networks have powerful pattern recognition capabilities, providing better robustness and potential fault tolerance. Therefore, it is easy to obtain a higher recognition rate. A kind of elastic back propagation neural network is used as classifier in Reference [8]. In Reference [10], an improved bee colony algorithm is applied to digital modulation recognition by optimizing a BP neural network. Grey correlation can be used to analyze and describe the attributes and characteristics of the model, which is suitable for mathematical statistical analysis. In Reference [11], the basic grey relation model is proposed. Random forest has high prediction accuracy, and good tolerance to outliers and noise. It is also not prone to overfitting. In References [12,13], random forest is used for dimensionality reduction in low signal-to-noise ratio (SNR). In Reference [14], the information entropy is used as the modulation feature, and random forest is used as the classifier. A high recognition rate is obtained in low SNR.