  Given the rapid changes in telecommunication systems and their higher
dependence on artificial intelligence, it is increasingly important to have
models that can perform well under different, possibly adverse, conditions.
Deep Neural Networks (DNNs) using convolutional layers are state-of-the-art in
many tasks in communications. However, in other domains, like image
classification, DNNs have been shown to be vulnerable to adversarial
perturbations, which consist of imperceptible crafted noise that when added to
the data fools the model into misclassification. This puts into question the
security of DNNs in communication tasks, and in particular in modulation
recognition. We propose a novel framework to test the robustness of current
state-of-the-art models where the adversarial perturbation strength is
dependent on the signal strength and measured with the "signal to perturbation
ratio" (SPR). We show that current state-of-the-art models are susceptible to
these perturbations. In contrast to current research on the topic of image
classification, modulation recognition allows us to have easily accessible
insights on the usefulness of the features learned by DNNs by looking at the
constellation space. When analyzing these vulnerable models we found that
adversarial perturbations do not shift the symbols towards the nearest classes
in constellation space. This shows that DNNs do not base their decisions on
signal statistics that are important for the Bayes-optimal modulation
recognition model, but spurious correlations in the training data. Our feature
analysis and proposed framework can help in the task of finding better models
for communication systems.
