## 3 Method of SBL-EM-QHLRT Modulation Identification
### Modulation Identification Method of EM-QHLRT  
The performance of likelihood based identification method will be greatly affected because of the error of channel estimation and the unknown original constellation. Therefore, EM is used to compensate the error in channel estimation and the phase. Signal after channel equalization is modelledas GMM. We assume the number of Gaussian models is M and the probability density function of GMM can be expressed as [32]:  
\[f(r)=\sum_{m=1}^{M}\frac{1}{M}CN(r|s_{m},h,\sigma) \tag{17}\]  
where \(h=ae^{j\varphi}\) is channel coefficients, \(\alpha\) is amplitude attenuation, \(\varphi\) is frequency offset, \(CN(r|s_{m},h,\sigma)\) is the probability density function of the \(mth\) Gaussian model, \(s_{m}\) is the symbol of the original constellation. Step Expectation aims to estimate the likelihood value of symbols regarding to every Gaussian model. Setting the hidden variable \(z_{n,m}\) to represent that symbol \(r_{n}\) comes from the \(mth\) Gaussian model. \[z_{n,m^{\prime}}=\left\{ \begin{array}{ll}1&\quad\text{if}m^{\prime}=\underset {m}{\text{argmin}}\ |r_{n}-hs_{m}|^{2}\\ 0&\quad\text{\emph{other}}\end{array} \right. \tag{18}\]  
Equation (17) can be expressed as:  
\[f\left(r,z\ |s,h,\sigma\right) =\prod_{n=1}^{N}\sum_{m=1}^{M}\frac{1}{M}N(r_{n}\ |s_{m},h,\sigma)z_{n,m} \tag{19}\] \[=\frac{1}{M^{N}}\prod_{n=1}^{N}\prod_{m=1}^{M}N(r_{n}\ |s_{m},h,\sigma)\]  
Taking logarithm for Equation (19):  
\[\ln f\left(r,z\ |s,h,\sigma\right)=-N\ln M+\sum_{n=1}^{N}\sum_{m=1}^{M}z_{n,m} \ln N(r_{n}\ |s_{m},h,\sigma) \tag{20}\]  
We define Q function as:  
\[Q(h,\sigma,h^{i},\sigma^{i}) =\text{E}_{z}\left[\ln f\left(r,z\ |s,h,\sigma\right)\ \Big{|}\text{r},h^{i},\sigma^{i}\right] \tag{21}\] \[=-N\ln M+\sum_{n=1}^{N}\sum_{m=1}^{M}\text{E}(z_{n,m}\ \Big{|}\text{r},h^{i},\sigma^{i})\] \[\times\left[\ln(\pi\sigma)+\left|r_{n}-hs_{m}\right|^{2}/\sigma\right]\]  
where  
\[ \begin{split}\text{E}(z_{n,m}\ \big{|}r_{n},h^{i},\sigma^{i})& =f(z_{n,m}=1\ \big{|}r_{n},h^{i},\sigma^{i})\\ =\exp(-\frac{\big{|}r_{n}-h^{i}s_{m}\big{|}^{2}}{\sigma^{i}})/ \sum_{k=1}^{M}\exp(-\frac{\big{|}r_{n}-h^{i}s_{k}\big{|}^{2}}{\sigma^{i}})\end{split}  \tag{22}\]  
where \(h^{i}\) and \(\sigma^{i}\) represents the channel coefficients and noise power of the previous iteration, assuming the value of Equation (20) is \(\gamma_{n,m}\). We replace \(h\) with \(\text{Re}\{h\}+j\text{Im}\{h\}\) and Q function can be expressed as:  
\[Q(h,\sigma,h^{i},\sigma^{i}) =-N\ln M+\sum_{n=1}^{N}\sum_{m=1}^{M}\gamma_{n,m} \tag{23}\] \[\times\left[\ln(\pi\sigma)+\frac{\big{|}r_{n}-(\text{Re}\{h\}+j \text{Im}\{h\})s_{m}\big{|}^{2}}{\sigma}\right]\]  
Step Maximization aims to maximize the Q function. The derivatives of \(Q(h,\sigma,h^{i},\sigma^{i})\) with respect to the \(\text{Re}\{h\}\), \(\text{Im}\{h\}\) and \(\sigma\), while setting the derivatives to zero, we get the new estimates, as shown as:\[\text{Re}\{\hat{h}\}=\frac{\sum\limits_{n=1}^{N}\sum\limits_{m=1}^{M}\gamma_{n,m} \left[\text{Re}\{r_{n,m}\}\text{Re}\{s_{m}\}+\text{Im}\{r_{n,m}\}\text{Im}\{s_{ m}\}\right]}{\sum\limits_{n=1}^{N}\sum\limits_{m=1}^{M}\gamma_{n,m}|s_{m}|^{2}} \tag{24}\]  
\[\text{Im}\{\hat{h}\}=\frac{\sum\limits_{n=1}^{N}\sum\limits_{m=1}^{M}\gamma_{n,m}\left[\text{Im}\{r_{n,m}\}\text{Re}\{s_{m}\}+\text{Re}\{r_{n,m}\}\text{Im} \{s_{m}\}\right]}{\sum\limits_{n=1}^{N}\sum\limits_{m=1}^{M}\gamma_{n,m}|s_{m} |^{2}} \tag{25}\]  
\[\hat{\sigma}=\left(\sum\limits_{n=1}^{N}\sum\limits_{m=1}^{M}\gamma_{n,m}|r_{n }-hs_{m}|^{2}\right)/\left(\sum\limits_{n=1}^{N}\sum\limits_{m=1}^{M}\gamma_{n,m}\right) \tag{26}\]  
Repeat Equations (22)-(26) until the end of the iteration.