## 2 Methods
### 2.5 Addictive Margin (AM)-Softmax Function  
Softmax function is the most popular output function in classification task. Softmax function is suitable in optimizing inter-class difference. However, If we can reduce intra-class distance, we will also expand inter-class distance, and this will help to increase classification accuracy. Cosine-similarity-based softmax aims to compute the angle between the input vector and the center vector of modulation category. The process of narrowing the angles within a class is also the process of widening the distance between classes. Therefore, we choose cosine-similarity-based function for our modulation classification. The softmax function can be written as  
\[ \begin{split}\mathcal{L}_{S}&=-\frac{1}{n}\sum_{i= 1}^{n}\log\frac{e^{W^{T}_{y_{i}}f_{i}}}{\sum_{j=1}^{c}e^{W^{T}_{j}}f_{i}},\\ &=-\frac{1}{n}\sum_{i=1}^{n}\log\frac{e^{\|W_{y_{i}}\|\|f_{i}\| \cos(\theta_{y_{i}})}}{\sum_{j=1}^{c}e^{\|W_{j}\|f_{i}\|\cos(\theta_{j})}}, \end{split}  \tag{17}\]  
where \(\mathbf{f}_{i}\) is the \(i\)-th element of the output feature, \(W_{j}\) is the \(j\)-th column of the output. The \(W^{T}_{y_{i}}f_{i}\) denotes the target score of the \(i\)-th sample. As is the equation shown, the softmax function can be written as the product of the magnitudes of two vectors and the cosine of their angular distance. As a result, the cos-similarity softmax can be implemented by normalizing the weight vectors and the features. In this way, \(\|W_{j}\|\) and \(\|\mathbf{f}_{i}\|\) are both normalized to 1, and the data at the exponential position only leave the cosine of theangular distance. Addictive Margin Softmax (AM-Softmax) is proposed to increase a cosine margin in cos-similarity softmax, which is denoted as  
\[ \begin{split}\mathcal{L}_{AMS}&=-\frac{1}{n}\sum_{i=1 }^{n}\log\frac{e^{s\cdot(\cos\theta_{y_{i}}-m)}}{e^{s\cdot(\cos\theta_{y_{i}}-m )}+\sum_{i=1,j\
eq y_{i}}^{c}e^{s\cdot\cos\theta_{j}}}\\ &=-\frac{1}{n}\sum_{i=1}^{n}\log\frac{e^{s\cdot(W_{y_{i}}^{T}f_{i }-m)}}{e^{s\cdot(W_{y_{i}}^{T}f_{i}-m)}+\sum_{j=1,j\
eq y_{i}}^{c}e^{sW_{j}^{T}f _{i}}},\\ &\textit{where}\|W_{j}\|=\|f_{i}\|=1,\end{split}  \tag{18}\]  
\(s\) and \(m\) are both preset hyper parameters. \(m\) denotes the designed cos similarity margin between classes. The difference between conventional softmax decision boundary and AM-softmax decision boundary is shown in Figure 5.  
Where the red arrows \(W_{1}\) and \(W_{2}\) denote the center vectors of two modulation categories respectively. The decision boundary of conventional softmax function is the green line \(P_{0}\), where  
\[cos(\theta_{W_{1},P_{0}})=W_{1}^{T}P_{0}=W_{2}^{T}P_{0}=cos(\theta_{W_{2},P_{ 0}}). \tag{19}\]  
We must declare that the conventional softmax function in Figure 5 is not similar to the softmax function applied in the existing literature. This is because \(W_{1}\), \(W_{2}\) and \(P_{0}\) are all normalized to 1 while the softmax function applied in existing literature do not normalize parameters and features. Thus the conventional softmax function in Figure 5 is a type of cosine similarity metrics. The decision boundary of softmax function bisects the angular of two center vectors. For AM-softmax function, the decision boundary of the category \(W_{1}\) is at \(P_{1}\). Similarly, the decision boundary of the category \(W_{2}\) is at \(P_{2}\), the margin \(m\) can be written as:  
\[m=cos(\theta_{W_{1},P_{1}})-cos(\theta_{W_{1},P_{2}})=W_{1}^{T}P_{1}-W_{1}^{T} P_{2} \tag{20}\]  
The greater the value of \(m\) is, the larger the cosine similarity distance is. If \(m\) is set to 0, that means the AM-softmax function degrades into a cosine-similarity metrics. \(s\) is the scale parameter to accelerate convergence. If \(s\) is set too small, the convergence will be too slow. However, if \(s\) is set too large, the convergence will be too fast to find better local optimal value. Therefore, no matter the setting of \(m\) or the choice of \(s\), the trainer must be careful.