# EffShuffNet: An Efficient Neural Architecture for Adopting a Multi-Model
## 2 Related Work
In the field of deep learning-based image classification, Xception [16] introduced a simplified version of the Inception block by implementing independent 3 \(\times\) 3 convolutional filters for each channel of the output feature map from a 1 \(\times\) 1 convolutional filter, reducing computational cost while preserving high performance. SqueezeNet [17] reduced the model size by 50 times compared to Alexnet through the use of a bottleneck 1 \(\times\) 1 filter and a fire block of 1 \(\times\) 1 filter for half and 3 \(\times\) 3 filter for the rest. MobileNet [18] introduced the concept of 3 \(\times\) 3 depthwise convolution and 1 \(\times\) 1 pointwise convolution by combining 3 \(\times\) 3 convolutional filters from Xception with 1 \(\times\) 1 convolutional filters. ShuffleNet [19] proposed a lighter model by grouping and processing the MobileNet structure based on channels and later concatenating the results. MobileNetV2 [20] further reduced its size through the implementation of Bottleneck Residual Block, and ShuffleNetV2 [21] also achieved lighter weight through a thorough analysis of lightweight models. In recent years, there has been a trend in deep learning towards training generable big models with unsupervised pre-text tasks on redundant datasets, followed by fine-tuning on a supervised downstream task with a smaller model. For instance, the Contrastive Captioner (CoCa) [22] pretrain image-text encoder-decoder achieves state-of-the-art accuracy of 91.0% on ImageNet, while Contrastive Language-Image Pre-training (CLIP) [23] enables zero-shot transfer for generalization to unseen object categories with 76.2% accuracy on ImageNet. Model distillation [24; 25; 26], where the predictions of large, complex teacher models are distilled into smaller models, is the most widely known form of distillation. Interestingly, Noisy Student Training [27] is one of a counter example of light-weight distillation, which combines self-training with distillation using equal-or-larger student models and adding noise during learning. When applied to labeled and pseudo-labeled images with noisy data and stochastic depth, Noisy Student Training using the Efficient Net student model achieved 88.4% top-1 accuracy on ImageNet. Practitioners seeking to implement lightweight neural models in practical applications can refer to prior research. For instance, ref [28] presents various shallow neural networks for fall detection in the elderly, integrated with wearable devices that capture accelerometer data as input signals. Moreover, ref [29] introduces WearNet, a lightweight convolutional neural network that detects scratches resulting from cylinder flat sliding. These models represent only a small selection of the numerous endeavors aimed at incorporating lightweight neural network models into low-computational products. Despite their lightweight structure, these models have limitations in terms of low classification accuracy compared to general convolutional neural network models and Transformer series models. Therefore, to effectively address multi-classification problems, such as the classification of both age and gender from a single input, it is crucial to evaluate the strengths and limitations of various image classification models. Merely optimizing for classification performance in a single model for multiple tasks may result in a model that is too complex, and a focus on minimizing model size could negatively impact classification accuracy. In light of this, we propose EffShuff and EffShuff-Dense models, which leverage the attributes of each convolutional output feature map to enhance classification performance while still being relatively lightweight. These models will be incorporated into a multi-model approach aimed at solving the multi-classification problem of age and gender, and the efficacy of this approach will be demonstrated.