## 2 Background and Prior Art
### State-of-the-Art CNNs  
CNNs are great for image-based applications because their convolution kernels are able to learn and differentiate shapes, colors, hues, etc. found in the training images. The unique characteristics associated with each class are learned during training and then used later for identifying classes during inference. The deep learning revolution was accelerated with the groundbreaking results achieved by the LeNet architecture developed by LeCun [15]. The AlexNet [16] architecture later smashed previous records on the ImageNet [17] data set by combining convolution layers, fully connected layers, rectified linear units (ReLU), and dropout layers. ImageNet is a benchmark set of images including 1000 different classes and is frequently used to compare architecture performance. Since AlexNet, a number of significant improvements have been made in CNN architectures, layers, and optimizers. Adam, for example, is an optimizer that provides adaptive learning rates [18]. As training progresses, learning rates are automatically adjusted up or down so weights do not train too slowly or diverge, because learning rates are too high. ResNet created another revolution in the CNN architecture field by introducing the concept of special processing blocks surrounded by identity connections [19]. ResNet is able to address the problem of diminishing gradients by allowing vital image information to be made available, through the identity connections, to deeper layers of the network. ResNet is able to bypass the accuracy inflection point of the VGG architecture, where adding layers caused decreases in performance [20]. ResNet is able to continue improving performance by stacking much deeper layers, beyond which VGG performance degraded. ResNeXt is an extension of the ResNet architecture [21]. They postulated that gains could be made through widening the architecture. They introduced the idea of cardinality, where \(N\) branches were introduced and each branch included a small number of kernels. DenseNet provides a complimentary approach to ResNet [22]. Rather than using identity connections, DenseNet is able to feed forward feature maps from each processing block (which is composed of convolution layers and other operations). Each processing block is provided feature maps, at its input, from all previous processing blocks. SqueezeNet is an architecture designed for small memory footprint applications [23]. It is composed of 'Fire modules' which include squeeze (1 \(\times\) 1 convolutions) and expand (1 \(\times\) 1 and 3 \(\times\) 3 convolution) layers. It was found to perform comparable to AlexNet on the ImageNet set, but with 50\(\times\) fewer parameters. MobileNet is a series of streamlined architectures that use depth-wise separable convolutions [24]. The series was designed to be lightweight so as to be appropriate for applications such as self-driving cars, robotics, etc. ShuffleNet is an architecture designed to be memory efficient, so as to be deployed in robotics and mobile devices [25]. While its accuracies are not competitive with architectures such as ResNet, when trained and tested against ImageNet, it has a much lighter memory footprint and can fit in small devices where the larger architectures will not. It was shown to have better accuracies on the ImageNet set than MobileNet. SqueezeNet, MobileNet, and ShuffleNet are small, efficient architectures where AlexNet, DenseNet, ResNet, ResNeXt, and WideResNet are large but produce better results on the ImageNet benchmark data set. These architectures will be used and contrasted with each other in this work to determine which provides the best performance in the OAM communication domain. Ref. [26] provides an interesting insight that some of the state-of-the-art CNNs struggle in their ability to generalize and are in fact 'brittle'. This warrants a careful consideration in applying the state-of-the-art to new domains. The best architecture in ImageNet classification does not necessarily mean optimal performance in another domain.