## 6 Results
### PMI Performance Evaluations  
To define the covertness performance, we conducted modulation identification evaluations with the CNN-based modulation identification algorithm. The purpose of the evaluation was to check whether a third party could identify what modulation scheme was being used. The CNN algorithm was constructed and trained with BPSK, QPSK, 8 PSK, 16 PSK, 32 PSK, 64 PSK, 8 QAM, 16 QAM, 32 QAM, and 64 QAM modulation scheme data with a 0-30-dB SNR channel and using Matlab. With the trained CNN algorithm, we could obtain PMI as a performance metric. PMI is an indicator showing the probability of recognizing the modulation scheme, where a higher value indicates a higher probability to be recognized.  
From the modulation identification accuracy experiment, we evaluated the prediction accuracy of the CNN algorithm for trained modulations. We generated 20,000 signal data per each modulation scheme, and each instance of signal data had 100,000 data symbols. The CNN algorithm was trained with 10 different modulation schemes. Within 20,000 signal sets, 80% of them were used for training, 10% for validation, and 10% for testing, while 2000 signal data were used to evaluate the CNN algorithm performance. As shown in Figure 15, in the case of QAMs, the probabilities were at least 80%, even for the higher modulation orders. However, in the case of PSKs, the probabilities were significantly lowered for the modulation order of 32 or higher. Therefore, the modulation order of up to 16 PSK for PSK and any QAM could be accurately detected. Therefore, the PMI obtained from the CNN was trustable.  
Using the developed CNN algorithm, we evaluated which modulation was recognized from the eavesdropper point of view when the ring-shaped signaling system was applied to a conventional QPSK system. For this, we varied the magnitude- and phase-modifying factors. Both factors of the ring-shaped signaling system, the change in magnitude and phase, were fixed.  
We conducted an experiment of \(l_{p}=1\) with a varying magnitude-modifying factor. As shown in Figure 16, the CNN algorithm properly detected the modulation scheme as QPSK for a high magnitude-modifying factor. However, as the magnitude-modifying factor decreased, the CNN algorithm recognized the system as 64 QAM and further confused which modulation scheme was used. From this experiment, we observed that the modulation scheme could be effectively hidden when \(I_{m}\) was less than about 0.3. Additionally, we  
Figure 14: Performance evaluations with varying (**a**) magnitude-modifying factors (\(l_{m}\)) and (**b**) phase-modifying factors (\(l_{p}\)).  
could conclude that the ring-shaped signaling system had LPI communication functionality due to the ability to hide the modulation scheme from the eavesdropper.  
We conduct another similar experiment but with \(I_{p}=4\). As shown in Figure 17, the CNN algorithm could not be used to infer QPSK for any scenario. Moreover, when the magnitude-modifying factor was decreased, the algorithm was recognized as QAM signals. In addition, even when the magnitude increased, the algorithm recognized the signals as 16 PSK rather than QPSK signals, the original modulation scheme. Therefore, we confirmed that the modulation scheme could not be inferred when the ring-shaped signaling system was applied, achieving LPE communication functionality. Furthermore, we are sure that if the phase modifying factor was large enough to scatter the I/Q constellation points as white noise, modulation scheme leakage would not occur even for the high magnitude-modifying factor, and it would be possible to achieve both LPI and LPE communication functionality at the same time.  
Figure 16: PMI evaluation with varying \(I_{m}\) and \(I_{p}=1\).  
Figure 15: PMI accuracy table of constructed CNN algorithm.