# EffShuffNet: An Efficient Neural Architecture for Adopting a Multi-Model
## 2 Related Work  
This section is divided into five paragraphs. The first paragraph introduces the foundational models for image classification, which sets the context for subsequent discussion on recent trends and advancements in the field. The second paragraph highlights current trends in image classification, which is a rapidly evolving field with new developmentsfrequently emerging. The third paragraph discusses previous research aimed at achieving lightweight, efficient models, which is an important area of focus for practical deployment in edge devices. The fourth and fifth paragraphs briefly present the potential of light-weight models in practical applications, underscoring the importance of developing models that are both accurate and efficient. Finally, the sixth paragraph provides a brief overview of the proposed architecture and the purpose for introducing it, positioning it as a novel contribution that addresses the limitations of existing lightweight models. In the field of deep learning-based image classification, convolutional neural network (CNN) models have demonstrated remarkable performance. AlexNet [2], which is considered the origin of CNN, established the standard of CNN models in image classification by incorporating diverse-sized convolutional filters, Rectified Linear Unit (ReLU) activation function, and max pooling in its design, drawing inspiration from LeNet [15]. Subsequently, the VGGNet [3] model was introduced with the unique characteristic of having varying receptive fields from the input, achieved through the use of 3 \(\times\) 3 convolution filters. GoogleNet [4], on the other hand, was designed with an Inception block that concatenates the outputs of convolutional filters of different sizes and was constructed with a complex model using the bottleneck technique and actively incorporating 1 \(\times\) 1 convolutional filters. However, building deep layers remained a challenge. ResNet [5] resolved this challenge by introducing residual learning that added the input value as is after performing convolution operation through a short-cut connection, enabling the construction of deep CNN models. Since then, various studies have been conducted based on ResNet, among which DenseNet [6] adopted a structure that concatenated all previous inputs through dense connection, instead of simply adding them as is. However, these CNN models that enhance image classification performance have the disadvantage of increased model size with increased classification accuracy  
In the realm of deep learning-based image classification models, the attention mechanism of the Transformer [7; 13], a novel approach in the field of natural language processing, has been incorporated. The Residual Attention Network [8] proposed a structure that utilized an independent attention module to incorporate both the spatial and channel information of previous feature maps, while the SE Net [9] incorporated an attention mechanism based on channel information of previous feature maps. In the BAM [11] model, both spatial and channel information of previous feature maps were considered as attention in parallel, and the CBAM [12] model proposed a method of applying spatial attention after channel attention. The Non-local Network employed self-attention generation and was applied based on spatial information, as in the Transformer method. Although models incorporating the attention mechanism have demonstrated improved classification accuracy compared to conventional convolutional neural network-based models, they entail additional computation. ViT and DeiT [13; 14], which directly apply the Transformer to image classification, exhibit promising performance, but have a significant time and space complexity. On the other hand, as research on image classification models based on convolutional neural networks has been around for a long time, there are studies on models focused on lightweight so that they can work well in limited hardware, such as smartphones or embedded boards.