# Automatic Wireless Signal Classification: A Neural-Induced Support Vector Machine-Based Approach
## 2 Signal Model and Problem Statement
### Support Vector Machine  
The basic idea of the SVM classifier is to transform the input space into a higher dimensional space by non-linear transformation, then use the risk minimization criteria to construct an optimal separating hyperplane in the new space. SVM optimizes the following optimization problem:  
\[\begin{array}{l}\mathcal{P}1:\quad\underset{w,\ b,\ \xi}{\min}\frac{1}{ \xi}w^{2}+C\sum\limits_{i=1}^{n}\xi_{i}\\ s.t.\quad y_{i}\Big{(}w^{T}\mathbf{\cdot\Phi}(x_{i})+b\Big{)}\geq 1-\ \xi_{i}\\ \xi_{i}\geq 0,\ i=1,\ \ldots,\ n\end{array} \tag{16}\]  
where, \(((x_{1},\ y_{1}),\ \ldots,\ (x_{M},\ y_{M})(x_{1},\ y_{1}),\ \ldots,\ (x_{M},\ y_{M}))\) are \(M\) samples for \(x_{i}\in\mathbb{R}\), in which the class labels \(y\in(+1,-1)^{I}\) are either positive or negative, which can be separated by the hyper-plane \((\mathbf{w}^{T}\mathbf{\cdot\Phi}(x_{i})+b)\), \(\mathbf{\Phi}:\mathbb{R}^{N}\rightarrow\mathbb{R}^{F}\) is the mapping function of the input data into some higher dimensional Hilbert space \(\mathcal{H}\), where \(\mathcal{W}\in\ \mathcal{H}\) is orthogonal to the separating hyper-plane in that space, C is a positive number and \(\xi_{i}\) is the error in the soft margin. \(\xi_{i}\geq 0\) are the non-zero slack variables introduced to enable the learning algorithm to deal with data that could not be precisely separated, such as data with noise. \(b\in\ \mathbb{R}\) is the bias and \(C\in\ \mathbb{R}\) is a penalty factor used to establish a trade-off between maximizing the margin and minimizing the classification error. By taking the Lagrangian of \(\mathcal{P}1\), we have:  
\[\begin{array}{l}\mathcal{L}(w,b,\alpha,\xi)=\\ \frac{1}{2}w^{2}+C\sum\limits_{i=1}^{n}\xi_{i}-\sum\limits_{i=1}^{l}\alpha_{i} \Big{[}y_{i}\Big{(}w^{T}\mathbf{\cdot\Phi}(x_{i})+b\Big{)}-1+\xi_{i}\Big{]} \end{array} \tag{17}\]  
Minimizing \(\mathcal{L}(w,b,\alpha,\xi)\) w.r.t to \(w,b,\alpha,\xi\) respectively, we get the dual form of \(\mathcal{P}1\) given as:  
\[\begin{array}{l}\mathfrak{D}1:\ arg\underset{\alpha_{i}\geq 0}{\max}\sum \limits_{i=1}^{M}\alpha_{i}-\frac{1}{2}\sum\limits_{i=1}^{M}\lambda_{i}\alpha_ {j}y_{i}y_{j}\mathbf{\Phi}^{T}(x_{i}),\ \mathbf{\Phi}\Big{(}x_{j}\Big{)}\\ =\ arg\underset{\alpha_{i}\geq 0}{\max}\sum\limits_{i=1}^{M}\alpha_{i}-\frac{1}{2} \sum\limits_{i=1}^{M}\lambda_{i}\alpha_{j}y_{i}y_{j}\mathbf{\xi}\Big{(}x_{i}, \ x_{j}\Big{)}\\ s.t\ 0\geq\alpha_{i}\geq C\ \forall i,\ \sum\limits_{i=1}^{M}\alpha_{i}y_{i}=0 \end{array} \tag{18}\]where, \(\kappa(.)\) is a positive defined kernel, representing the dot product \(x(x_{i},\ x_{j})=\ \Phi(x_{i}),\ \Phi\big{(}x_{j}\big{)}\) of the data point in \(\mathcal{H}\). The strategy of a kernel is to map the training data into the feature space \(x_{i}\rightarrow\Phi(x_{i})\) to separate them from the origin with maximum margin, which map the features to higher space, a Gaussian kernel is being employed in this paper. The Gram matrix of the Gaussian kernel is given as:  
\[G=\big{(}y_{i}y_{j}x\big{(}x_{i},x_{j}\big{)}\big{)}_{i,\ y=1}^{l} \tag{19}\]  
Which is a positive semi-definite that ensures that the maximal margin optimization has a unique solution that could be found efficiently. This effectively rules out the problem of being stuck in local minima that has been mostly encountered while training neural networks. Because of the optimality relations between the primal and dual problem, the hyperplane that separates the data in the high dimensional space, determined by the normal \(w^{*}\) and the intersection with the axis, \(b^{*}\), satisfies the following:  
\[\begin{array}{l}w^{*}=\ \sum\limits_{i=1}^{M}\alpha^{*}y_{i}\Phi\big{(}x^{i} \big{)}\\ b^{*}=1-\max_{\{y_{j}=1\}}(w^{*})^{\dagger}\Phi\big{(}x^{i}\big{)}\end{array} \tag{20}\]  
A SVM separating the hyperplane for binary classification is shown in Figure 3. The function used to classify a new point X can be written as:  
\[\mathrm{f}(\mathrm{X})=\mathrm{sign}\Bigg{(}\sum\limits_{i=1}^{M}\alpha^{*}_{ i}y_{i}k\big{(}x^{i}\big{)}+b^{*}\Bigg{)} \tag{21}\]