Radar emitters classification and clustering with a scale mixture of normal distributions

IET Radar, Sonar & NavigationVolume 13, Issue 1 p. 128-138 Research ArticleFree Access Radar emitters classification and clustering with a scale mixture of normal distributions Guillaume Revillon, Corresponding Author Guillaume Revillon guillaume.revillon@l2s.centralesupelec.fr orcid.org/0000-0001-5963-0531 Laboratoire des Signaux Systèmes (L2S), Université Paris-Sud-CNRS-CentraleSupélec-Université Paris-Saclay, 3 rue Joliot Curie, 91192 Gif-sur-Yvette, France Thales Defense Mission Systems (DMS), 2 avenue Gay Lussac, 78990 Elancourt, FranceSearch for more papers by this authorAli Mohammad-Djafari, Ali Mohammad-Djafari Laboratoire des Signaux Systèmes (L2S), Université Paris-Sud-CNRS-CentraleSupélec-Université Paris-Saclay, 3 rue Joliot Curie, 91192 Gif-sur-Yvette, FranceSearch for more papers by this authorCyrille Enderli, Cyrille Enderli Thales Defense Mission Systems (DMS), 2 avenue Gay Lussac, 78990 Elancourt, FranceSearch for more papers by this author Guillaume Revillon, Corresponding Author Guillaume Revillon guillaume.revillon@l2s.centralesupelec.fr orcid.org/0000-0001-5963-0531 Laboratoire des Signaux Systèmes (L2S), Université Paris-Sud-CNRS-CentraleSupélec-Université Paris-Saclay, 3 rue Joliot Curie, 91192 Gif-sur-Yvette, France Thales Defense Mission Systems (DMS), 2 avenue Gay Lussac, 78990 Elancourt, FranceSearch for more papers by this authorAli Mohammad-Djafari, Ali Mohammad-Djafari Laboratoire des Signaux Systèmes (L2S), Université Paris-Sud-CNRS-CentraleSupélec-Université Paris-Saclay, 3 rue Joliot Curie, 91192 Gif-sur-Yvette, FranceSearch for more papers by this authorCyrille Enderli, Cyrille Enderli Thales Defense Mission Systems (DMS), 2 avenue Gay Lussac, 78990 Elancourt, FranceSearch for more papers by this author First published: 01 January 2019 https://doi.org/10.1049/iet-rsn.2018.5202Citations: 3AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract In this study, a scale mixture of normal distributions model is developed for classification and clustering of radar emitters. A radar signal is characterised by a pulse-to-pulse modulation pattern and is often partially observed. The proposed model can classify and cluster different radar emitters even in the presence of outliers and missing values. The classification method, based on a mixture model, focuses on the introduction of latent variables that give us the possibility to handle sensitivity of the model to outliers and to allow a less restrictive modelling of missing data. A Bayesian treatment is adopted for model learning, supervised classification and clustering. The inference is processed through a variation Bayesian approximation. Some numerical experiments on realistic data show that the proposed method provides more accurate results than state-of-the-art classification algorithms. 1 Introduction In electronic warfare (EW) [1], radar signals identification is a crucial component of electronic support measures (ESM) systems [2]. ESM functions allow surveillance of enemy forces such as movements of enemy planes and warning of imminent attack such as launches of rockets. By providing information about the presence of threats, classification of radar signal has a self-protection role ensuring that countermeasures against enemies are well-chosen by ESM systems [3]. Furthermore, electronic intelligence functions focus on the interception and the analysis of unknown radar signals to update and improve EW databases. Then clustering of radar signals can play a significant role by detecting unknown signal waveforms and supporting ESM functions. Through its classification and clustering aspects, identification of radar signal is a supreme asset for decision making in military tactical situations. Depending on the information available in databases, the identification process can be distinguished into source emission identification, also known as radar emitter classification (REC), which concerns the classification of types of emission sources and specific emitter identification (SEI) which focuses on recognition of copies of electromagnetic emission sources which are of the same type [4]. REC relies on statistical analysis of pulse description words (PDW) of a radar signal that gather its basic measurable parameters such as radio frequency (RF), amplitude, pulse width (PW) or pulse repetition interval (PRI). In terms of classification and clustering of emission sources from different types, many approaches based on data fusion and machine learning have been developed and traditionally proceed to feature extraction, dimensionality reduction, and classification or clustering. For example, the authors of [5-8] propose various neural classification approaches based on the PDW structure of observed signals whereas Yang et al. [9] introduce a hybrid radar emitter recognition method based on rough k-means (KM) and relevance vector machine and Chen [10] develop an efficient classification method using a weighted-xgboost model for complex radar signals in large datasets. As regards the clustering problem, He et al. [11] develop a dynamic clustering algorithm that uses designed distances and dynamic cluster centres and does not require fixing the number of classes which depends on the input data, Zhou et al. [12] also introduce a clustering framework composed of local processing and multi-sensor fusion processing and use a minimum description length criterion to update dynamically the number of clusters rather than setup in advance. In contrast, SEI aims to extract distinctive features in the process of signal processing to identify even a single copy of an emission source. For a classification purpose, Chen et al. [13] develop a fuzzy classifier, which handles unintentional jitter modulations, Dudczyk and Kawalec [14] use a graphical representation of PRI to discover PRI distortions peculiar to each copy and Shi [15] use the kernel canonical correlation analysis to achieve SEI. Kawalec and Owczarek [16] suggest extracting features from intrapulse data to find non-intentional modulations in the receiving signals and the authors of [17, 18] focus on fractal features of radar signals to identify copies. Finally, Dudczyk [4] proposes an algorithm based on hierarchical agglomerative clustering for large data sets that provide dendrograms used to analyse clustering results and to select the number of clusters. However, information required for SEI such as intrapulse data or exhaustive PDW is not always available in databases provided by operational entities. Generally, the biggest difficulties for SEI result from the lack of a precise and detailed description of a source emission model in databases [19]. Therefore, this work only focuses on REC to meet operational constraints. Most of the time, ESM systems receive mixtures of signals from different radar emitters in the electromagnetic environment. Then a radar signal, described by a pulse-to-pulse modulation pattern, is often partially observed. On the one hand, deinterleaving techniques [20] cannot manage to group all the pulses that belong to the same emitters and impute some of them to other emitters. On the other hand, even if the deinterleaving performs well, EW sensors deficiency and low signal-to-noise ratio values in sensors can also cause either missing measurements or measurement errors [21]. When measurements are known to be wrong, such as approximate PW measurements, considering them as missing measurements can be a more reliable approach than using them or discarding them. Furthermore, military databases are filled by human beings and can also be imperfect by gathering outliers and missing data. Classification and clustering problems are strictly connected with pattern recognition [22] and more general algorithms such as Random Forests [23], density-based spatial clustering of applications with noise algorithm (DBSCAN) [24] and KM algorithm [25] are also considered as state-of-the-art since they are used in various fields [26, 27]. However, these practical and general algorithms cannot handle missing data and imputation methods [28] are required to generate data to use them. Hence, the main objective of this work is to define a classification/clustering framework that handles both outliers and missing values. Here, an approach based on mixture models is preferred since mixture models provide a mathematically based, flexible and meaningful framework for a wide variety of classification and clustering requirements [29]. More precisely, a scale mixture of normal distributions [30] is updated to handle outliers and missing data issues. On the one hand, this model is robust to outliers by accounting for the uncertainties of variances and covariances since the associated marginal distributions are heavy-tailed [31]. On the other hand, dependencies between features can easily be modelled through a multivariate Gaussian distribution in order to infer on missing values by benefitting from attractive Gaussian properties. Exact inference in that Bayesian approach is unfortunately intractable, therefore a variational Bayesian (VB) inference [32] is used to find the approximate posterior distribution of parameters and to provide a lower bound (LB) on the model log evidence used as a criterion for selecting the number of clusters. The outline of the paper is as follows. An experimental protocol is developed in Section 2 to acquire unclassified realistic data. Then, the proposed model (PM) is explained in Section 3 and the inference procedure is derived in Section 4. Finally, the evaluation of the model is proposed through different experiments on realistic experimental data in Section 5. 2 Data acquisition In this section, we will describe the experimental protocol used to acquire unclassified realistic data from different radar emitters. This protocol consists of an acquisition step followed by a feature extraction step. The acquisition system is composed of two software defined radio (SDR) platforms based on Ettus USRP E312 (Emitter) and B200 (Receiver) boards, linked to a laptop to record the data. As in [33], this setup was chosen because it allows quick development and experimentation tasks on RFs from 70 MHz to 6 GHz, it is quite cheap and is available off-the-shelf. Radar waveforms, emitted by the URSP E312 board, are generated from bin files coded from a database gathering more than 40 typical radar waveforms with agile (random variation)/hopping (systematic variation) frequencies and jittered/staggered PRI. The developed system is presented in Fig. 1. To meet hardware constraints, the RF range was mapped to a 4 MHz bandwidth and patterns of Time of Arrival (TOA) and PW were slightly modified but their dynamics was preserved. Fig. 1Open in figure viewerPowerPoint Diagram of the acquisition system Then, a threshold algorithm, provided by Davies and Hollands [34], is used to detect pulses in the recorded signal s(t) of duration T and to extract its PDW. Each pulse at is characterised by a triplet , where is the difference between and and the feature is estimated with a fast Fourier transform algorithm. Fig. 2 shows the parameters measurement on real data. For a given recorded signal s gathering pulses, the following PDW matrix is obtained (1) where is the index of pulses in the recording. Fig. 2Open in figure viewerPowerPoint Acquired pulses from a radar emitter where the three features (PRI, PW, RF) are shown in the figure SDR platforms are imperfect [35] and their defects can introduce outliers due to measurement errors. Hardware imperfections are visible in Fig. 3 where the third pulse is cut into two pulses which leads to the formation of PRI and PW outliers. Furthermore, since experiments took place in real outside conditions, other signals and reflections can disturb the acquisition [34]. Fig. 3Open in figure viewerPowerPoint Outliers formation during primary parameters measurement on real data (a), (c), (e) Are respectively exact TOA, PW and PRI, (b), (d), (f) Are outliers for TOA, PW and PRI Finally, for each signal gathering pulses of the J recorded signals , a matrix is created from (1) and an observation vector is defined according to (2) such that (2) where is the average value of RF, is the average value of PW and is the average value of PRI defined as (3) (4) (5) Once all observation vectors have been constructed, they are normalised to meet constraints of machine learning algorithms. Fig. 4 shows the distribution of 150 normalised observation vectors of a radar emitter, where three outliers are visible. Fig. 4Open in figure viewerPowerPoint Presence of outliers in observations of a radar emitter 3 Model In this section, the standard Gaussian mixture model (GMM) is briefly presented as a hierarchical latent variable model before introducing missing values and outliers modelling. Finally, the PM [36] is developed and approaches for classification and clustering with a mixture model are explained. 3.1 Latent variable model A GMM [37] is a natural framework for classification and clustering. It can be formalised as (6) where is an observation, is a finite and known set of clusters and , with , stands for parameters. Moreover, and are, respectively, the mean and the covariance matrix of the component distribution with a weight where and . The GMM can be formalised as a latent model since the component label associated with each data point is unobserved. To this end, a categorical variable can be considered to describe the index of the component distribution generating the observation variable . Then, the mixture distribution (6) is expressed as (7) where (8) (9) and denotes the Kronecker symbol which is 1 if and 0 otherwise. 3.2 Missing values Missing values can be handled by decomposing the features vector into observed features and missing features modelled by a latent variable such that and . Reminding that conditionally to its index cluster the features vector is Gaussian distributed the latent variable can be expressed as a Gaussian distributed variable conditionally to z such that (10) where (11) Then, the joint distribution of is derived from (10) such that with 3.3 Outliers Outliers in a GMM can be handled by introducing a latent variable u to scale each mixture component covariance matrix . That family of mixture models is known as scale mixtures of normal distributions [30]. Introducing the latent positive variable u into (8), the following scale component distribution is obtained (12) and the joint distribution of is derived from (12) such that (13) where is the prior distribution of u conditionally to . For the sake of keeping conjugacy between prior and posterior distributions of u, a Gamma distribution with shape and rate parameters is chosen for . Integrating (13) out u, the resulting marginal is a heavy-tailed distribution known as the Student-t distribution [38]. 3.4 Proposed model Combining (8), (10) and (12), the following joint latent representation is obtained where are the latent variables. Finally, assuming a dataset of i.i.d observations and independent latent data , the complete likelihood function can be expressed as where , , , is the discrete variable introduced to indicate which cluster the data belongs to and is the scale variable associated with . Finally, the Bayesian framework imposes to specify a prior distribution for the parameters . The resulting conjugate prior is with where follows a Dirichlet distribution, a normal-inverse-Wishart distribution and is defined below. To avoid a non-closed-form posterior distribution for , the following conditional prior is introduced: (14) where and where (15) The normalisation constant is intractable and a Laplace approximation method [39] is derived to estimate it. The directed acyclic graph of the PM is shown in Fig. 5. Fig. 5Open in figure viewerPowerPoint Graphical representation of the PM. The arrows represent conditional dependencies between the random variables. The K-plate represents the K mixture components and the J-plate the independent identically distributed observations , the scale variables and the indicator variables 3.5 Classification and clustering According to the degree of supervision, three problems can be distinguished: supervised classification, semi-supervised classification and unsupervised classification known as clustering. The supervised classification problem is decomposed into a training step and a prediction step. The training step consists of estimating parameters given the number of classes K and a set of training data with known labels . Then, the prediction step results in associating label of a new sample to its class chosen as the maximum a posteriori (MAP) solution given the previously estimated parameters . In the semi-supervised classification, only the number of classes K is known and both labels of the dataset and parameters have to be determined. As for the prediction step, the MAP criterion is retained for affecting observations to classes such that Given a set of data , the clustering problem aims to determine the number of clusters , labels of data and parameters . Selecting the appropriate seems like a model selection issue and is usually based on a maximised likelihood criterion given by (16) where (17) Unfortunately, (17) is intractable and many penalised likelihood criteria such as Akaike Information Criterion (AIC) [40], Bayesian Information Criterion (BIC) [41] and Integrated Completed Likelihood (ICL) [29] have been proposed. In this study, an LB for (17) is found in Section 4 and is preferred to other criteria since it does not depend on asymptotical assumptions and does not require maximum-likelihood estimates. Then, according to , a priori range of numbers of clusters, the semi-supervised classification is performed for each and both and are estimated. Then, is chosen as the maximiser of the LB introduced in the next section. After determining , only and are kept as estimated labels and parameters. 4 Inference In this section, a brief introduction to variational Bayes is proposed before detailing variational posterior distributions related to latent variables H and parameters and elements of the variational LB. 4.1 Introduction to variational Bayes VB can be viewed as a Bayesian generalisation of the expectation–maximisation algorithm [42] combined with a mean field approach [43]. It consists of approximating the intractable posterior distribution by a tractable one whose parameters are chosen through a variational principle to minimise the Kullback–Leibler (KL) divergence Noting that , the KL divergence can be written as with (18) where denotes the expectation with respect to q. As an LB for the log evidence, [31] can be used as a validation criterion to choose the number of classes in (16) such that (19) Then, minimising the KL divergence is equivalent to maximising . Assuming that can be factorised over the latent variables and the parameters , a free-form maximisation with respect to and leads to the following update rules: The expectations and are, respectively, taken with respect to the variational posteriors and . Thereafter, the algorithm iteratively updates the variational posteriors by increasing the bound . Running the algorithm steps, each posterior distribution is obtained in the following subsection. 4.2 Variational posterior distributions Posterior distributions are similarly obtained from classical posteriors related in [31, 44, 45]. However, in this study, missing values are incorporated as latent variables in posterior calculations and a posterior distribution for missing data is proposed. Then, noting that the following factorised forms are similarly chosen for where due to conjugacy properties (20) Update rules for hyper-parameters defined in (20) are detailed below. 4.3 Variational posterior distributions for latent variables The Variational Bayes Expectation (VBE) step can be computed by developing the expectation (21) where (22) Conditionally to and a given , follows a Gaussian distribution with mean and covariance matrix since Therefore, the distribution for in (20) is deduced from (10) and (11) where (23) (24) Auxiliary variables and are introduced for the Variational Bayes Maximisation (VBM) step such that Marginalising over , (22) becomes (25) where is the dimension of and Then the conditional posterior for in (20) is deduced from (25) such that where (26) (27) Expectations of are derived from the Gamma distribution properties such that where is the digamma function. Noting that , called responsibility is obtained as follows: (28) The expectation of is deduced from the categorical distribution property such that 4.4 Variational posterior distributions for parameters The VBM step can be computed by developing the expectation and gathering common terms to obtain hyper-parameters defined in (20) such that (29) where auxiliary variables are obtained as follows: Using the properties of the Dirichlet and the Inverse Wishart distribution, the following expectations are defined 4.5 LB elements and expectations The LB (18) is proven to increase at each VB iteration and its difference between two iterations can be used as a stop criterion. The introduction of slightly modifies the LB since the prior distribution (14) as well as the posterior distributions (20) have to be taken into account. LB elements are presented below. where and are the normalisation constants of the Dirichlet and Inverse Wishart distributions. Posterior expectations of are derived from the posterior Gamma distribution (20) properties and can easily be computed conditionally to However, expectations depending on are intractable (30) (31) (32) Since LB calculation is required as a stop criterion, expectations (30), (31) and (32) have to be approximated. As for the normalising constant defined in (15), a Laplace approximation is performed for each expectation. 5 Experiments In this section, the proposed method is performed on the set of acquired data. For comparison, a standard neural network (NN), the K-nearest neighbours (KNN) algorithm, random forests (RdF) the KM algorithm and the DBSCAN are also evaluated. Two experiments are carried out to evaluate classification and clustering performances with respect to a range of percentages of missing values. First, characteristics for realistic data acquisition and imputation methods for missing data are detailed. Then, both experiments are described with their error measure and performances are shown to exhibit the effectiveness of the PM. For each experiment, hyper-parameters are initialised as in Table 1 and 100 simulations are performed to take into account randomness of data deletion. Table 1. Initialisation of hyper-parameter values 0.5 1 1 1 1 1 5.1 Data Real data are acquired from the system detailed in Section 2. For each recording, the sampling frequency and the observation time T are, respectively, chosen as 4.17 MHz and 20 ms. The database exactly gathers 42 different radars waveforms and 150 observations are recorded for each waveform. Outliers and missing values are naturally embedded in observations due to material defects and real conditions detailed in Section 2. The dataset is shown in Fig. 6. However, extra missing values are added to evaluate the limits of the proposed approach. Missing information is introduced by randomly deleting coordinates of for each of the 42 radar emitters. Percentages of deletion range from 5 to 40%. Nevertheless, comparison algorithms do not handle datasets including missing values. Discarding observations that contain missing values can be a restrictive solution, therefore imputation methods have been developed [46]. In this study, two classical imputation methods, based on statistical analysis and machine learning, are performed. First, the mean imputation consists of filling a missing component of an observation by the average of observed values of that component. This method has the obvious disadvantage that it underrepresents the variability and also ignores correlations between observations [47]. Then, imputation can be processed through a KNN method [48] in order to replace missing values of an observation with a weighted mean of the K nearest completed observations where the weights are inversely proportional to the distances from the neighbours. Since replacements are influenced only by the most similar cases, the KNN method is more robust with respect to the amount and type of missing data [28]. These imputation methods are compared with the proposed approach in terms of classification, clustering and reconstruction performances. For the comparison of reconstruction performances, mean-squared errors between original data and previous imputation methods are compared with the mean-squared error between original data and the variational posterior marginal mean of missing data given by (33) Fig. 6Open in figure viewerPowerPoint Dataset gathering 6300 observations from 42 radar emitters. Some clusters are completely separable whereas some others share features and cannot be linearly separated 5.2 Classification experiment The classification experiment evaluates the ability of each algorithm to assign unlabelled data to one of the K classes trained by a set of labelled data. As developed in Section 3.5, the classification task is decomposed into a training step and a prediction step defined in procedures 1 and 2 (see Figs. 7 and 8). The training step consists of estimating variational parameters of defined in (20) given a set of training data with known labels. As for the prediction step, it results in associating new data to the class that maximises their posterior probabilities. Since comparison algorithms do not handle datasets including missing values, a complete dataset is used to enable their training. During the prediction step, incomplete observations are either discarded and gathered in a reject class or completed thanks to the mean and KNN imputation methods. Standard configurations provided by Matlab are chosen for the RnF, the NN and the KNN algorithm. The PM and comparisons algorithms are trained on 70% of the initial database without extra missing values and tested on the remaining 30% of the database whose elements are randomly deleted according to different proportions of missing values. The RnF gathers 50 trees. The NN is composed of one hidden layer of 70 neurons and a softmax output layer and is trained with a cross-entropy loss. An accurate metric is chosen for the classification experiment and observations belonging to the reject class are considered as misclassification errors. Fig. 7Open in figure viewerPowerPoint Procedure 1: classification: training step Fig. 8Open in figure viewerPowerPoint Procedure 2: classification: prediction step For the classification experiment, results are shown in Fig. 9. Without missing data, both algorithms perfectly classify the 42 radar emitters. When the proportion of missing values increases, the PM outperforms comparisons, algorithms and achieves an accuracy of 85% for 40% of deleted values whereas the accuracy of NN and KNN is <50% with or without missing data imputation. As for the RnF, it outperforms both NN and KNN by achieving accuracies of 67 and 72% with standard imputation methods for 40% of deleted values. This higher performance of the PM reveals that the proposed method embeds a more efficient inference method than other imputation methods. That result is confirmed in Fig. 9 where comparison algorithms are applied to the data reconstructed by the PM. Indeed when the proposed inference is chosen, performances of NN and KNN increase up to 80% for 40% of deleted values and the RnF has almost the same performances than the PM. Fig. 9 also reveals that the proposed approach is more robust to missing data since it has a lower variance than other algorithms and imputation methods. Finally, this efficiency is shown in Fig. 10 where the PM exhibits a lower mean-squared error for missing data imputation than the mean and KNN imputation methods. The effectiveness of the PM can be explained by the fact that missing data imputation methods can create outliers that deteriorate performances of classification algorithms whereas the inference on missing data and labels prediction are jointly estimated in the PM. Indeed, embedding the inference procedure into the model framework allows p