# Moving Deep Learning to the Edge
## 4 Deep Learning Inference on Edge Devices
### Edge-Oriented Deep Neural Networks
The new versions tune the previous one using two algorithms to explore the model space: the Neural Architecture Search (NAS) algorithm [100], which constructs the whole model iteratively by adding sub-modules, and the NetAdapt algorithm [101] to determine the number of filters in each layer. The difference between the large and the small versions is that the first has four more bottleneck modules. The large version is 25% faster than MobileNetV2 with similar accuracy, but larger. The small version is comparable in size and delay to MobileNetV2, but 6.6% more accurate. SqueezeNet [102] is a network with approximately the same accuracy as AlexNet, but with \(50\times\) fewer parameters. Several strategies were followed to achieve this high reduction of parameters: (1) replace \(3\times 3\) filters by \(1\times 1\) filters; (2) reduce the number of inputs of \(3\times 3\) filters; and (3) downsample with pooling layers late in the network. The inputs to larger filters are first squeezed with \(1\times 1\) filters. Then, they are expanded with a mix of parallel convolutional layers of size \(1\times 1\) and \(3\times 3\). Together, these operations form a block designated fire module. SqueezeNet has an initial convolutional layer followed by eight fire modules and a final global average pool. SqueezeNext [103] proposes a different block to replace the fire module of SqueezeNet. It uses a two-stage bottleneck module to reduce the number of input channels entering the \(3\times 3\) convolution. This \(3\times 3\) convolution is further replaced by separable convolutions followed by a \(1\times 1\) expansion module. The final model has \(112\times\) fewer parameters than AlexNet and better accuracy. A novel model optimization method proposed in [104] applies different convolutions to different parts of the input maps, instead of applying each convolution to the whole image. This reduces the number of parameters and operations of the model. Since different convolutions extract different features of the image, the outputs of the convolutions are then shuffled. The ShuffleNet unit replaces \(1\times 1\) convolutions of the bottleneck module by group convolutions and places a channel shuffle operation after the first convolution. A second version of the ShuffleNet unit implements the \(3\times 3\) convolution with a stride of two, adds an average pooling layer in the shortcut path, and replaces the element-wise addition with a channel concatenation. The complete model has an initial convolutional and pooling layers followed by three stages of ShuffleNet units. A hyperparameter, \(s\), is considered to scale the number of channels. The model is therefore designated as ShuffleNet \(s\times\). The network improves by about 3% the accuracy of MobileNetV1 with similar computational complexity. Grouped convolutions have proven to be useful to improve accuracy. CondenseNet [105] also applies grouped convolutions, but instead of randomly shuffling the channels, it learns which channels should be grouped during training. Before training, the output maps are split into \(G\) groups of the same size. The method also integrates pruning with another hyperparameter, \(C\). At the end of each condensing stage, \(1/C\) parameters are pruned. An extra optimization stage is applied after the condensing stage that removes less important feature maps. CondenseNet achieves an accuracy similar to ShuffleNet, but with about 50% of the parameters. Instead of training a predefined network, NASNet [106] starts with an overall architecture with a sequence of blocks that are not predefined. The structure of the blocks is determined with a reinforcement learning search method. There are two types of blocks: normal block (whose convolutional layers keep the size of feature maps) and reduction block (whose convolutional layers reduce by 4\(\times\) the size of feature maps). Three different versions were generated and identified as NASNet-A, -B, and -C. The models improve MobileNet by about 3% with a similar number of parameters.