# EffShuffNet: An Efficient Neural Architecture for Adopting a Multi-Model
## 3 Methodology
### EffShuffDenseNet  
The EffShuffNet successfully achieves both lightweight and performance, leveraging its ability to retain the low-level features generated from the stem area to the classifier. Building on this, the EffShuffDenseNet model is proposed, incorporating the concept of Dense connection from DenseNet, which optimizes the preservation of low-level features. The structure of the EffShuff-Dense block is depicted in Figure 5. The input feature map is divided into two parts in the channel direction, with one part remaining unchanged and the other part undergoing lightened convolution operations consisting of \(1\times 1\) convolution, \(3\times 3\) depthwise convolution, and \(1\times 1\) pointwise convolution. The outputs of these two parts are then concatenated. Unlike the previously introduced EffShuff block, the concatenated output from EffShuff-Dense block is not mixed, but instead utilized as input for the next EffShuff-Dense block. Such a connection is necessary since the EffShuff-Dense transition can be utilized as is to reduce both the horizontal and vertical size, while doubling the number of channels. The concept of EffShuff-Dense block is illustrated in Figure 6. The channel of the input feature map utilized in EffShuff-Dense block is represented by the variable \(C\), which is divided into two parts, \(C_{front}\) and \(C_{back}\). \(C_{front}\) is utilized as is, while \(C_{back}\) undergoes a lightweight convolution process that involves a \(1\times 1\) convolution, followed by a \(3\times 3\) depthwise convolution and finally, a \(1\times 1\) pointwise convolution. This results in an increase in the channel size, denoted by K, referred to as the growth rate, with a larger K representing a larger model as a hyperparameter. The concatenated output results in an increase in the channel size to \(C_{front}+C_{back}+K=C+K\). As the EffShuff-Dense block is repeated, the outputs accumulate as much as \(K/2\), calculated for the previous feature map \(C_{front}\) and each block. The use of EffShuff transition in pooling helps to suppress overfitting of low-level features by combining the entire feature map. The EffShuffDenseNet is lightweight while still considering all features calculated from the stem area to the previous classifier stage. As shown in Table 2, the overall EffShuffDenseNet model receives inputs of \(224\times 224\times 3\) and utilizes the stem area and Y-Transition a total of three times for the extraction of low-level features through convolution and max pooling. The structure employs EffShuff-Dense block three, seven, and three times between transitions, with a growth rate of 16 for EffShuff-Dense block. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Layers** & **Stage** & **Output Size** & **Output Filters** & **Repeat** \\ \hline Input & & \(224\times 224\) & 3 & 1 \\ \hline Convolution & General Stem & \(112\times 112\) & 24 & 1 \\ \hline MaxPool & & \(56\times 56\) & 24 & 1 \\ \hline EffShuff transition & & \(28\times 28\) & 116 & 1 \\ \hline EffShuff block & Stage 1 & \(28\times 28\) & 116 & 1 \\ \hline EffShuff block & & \(28\times 28\) & 116 & 2 \\ \hline EffShuff transition & & \(14\times 14\) & 232 & 1 \\ \hline EffShuff block & Stage 2 & \(14\times 14\) & 232 & 1 \\ \hline EffShuff block & & \(14\times 14\) & 232 & 6 \\ \hline EffShuff transition & & \(7\times 7\) & 464 & 1 \\ \hline EffShuff block & Stage 3 & \(7\times 7\) & 464 & 1 \\ \hline EffShuff block & & \(7\times 7\) & 464 & 2 \\ \hline GAP & Classifier & \(1\times 1\) & 464 & 1 \\ \hline Softmax & & \(1\times 1\) & No of classes & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: EffShuffNet architecture. The primary difference between the EffShuff unit and the ShuffleNet-v2 unit is the replacement of the \"depth-wise convolution\" with \"average pooling\" during the transition, which results in a lighter version of ShuffleNet. Empirical findings demonstrate that the accuracy of EffShuffNet is comparable to that of ShuffleNet-v2 across all tasks.