# GhostNeXt: Rethinking Module Configurations for Efficient Model Design
## 3 The Proposed Method
### The Proposed Method: GhostNeXt  
Our goal is to design a better lightweight block by adjusting the module configuration inside the Ghost block to reduce unnecessary computational costs:  
\[\textbf{B}^{*}=\arg\max_{\textbf{B}_{1},\ldots,\textbf{B}_{\textbf {j}}}\text{Acc}(N(\textbf{B}_{1},\ldots,\textbf{B}_{\textbf{j}}))\] \[\text{s.t.~{}Params}(N)\leq P,\quad\text{FLOPs}(N)\leq F, \tag{2}\]  
where \(k_{i},s_{i},c_{i},r_{i}\in\textbf{B}_{\textbf{i}}\), and Acc and \(\textbf{B}_{\textbf{i}}\) denote the network validation accuracy and the \(i\)-th block configuration of the network \(N\). \(k_{i}\), \(s_{i}\), \(c_{i}\), and \(r_{i}\) denote the kernel size of the convolution filter, stride, and the number of channel and linear transformation of the \(i\)-th block, respectively. \(P\) and \(F\) denote the number of target parameters and FLOPs, respectively. Since most parameters and FLOPs of the network are determined by \(c_{i}\), we design the module configuration by introducing a controller to adjust \(c_{i}\) dynamically. We design the block by considering only \(c_{i}\) while \(k_{i}\), \(s_{i}\), and \(r_{i}\) are held fixed. In particular, we focus on adjusting \(d^{\textit{mid}}_{1}\), which is the number of channels of the first module \(M_{1}\). To optimize a block with two modules in a light-weight manner, we reduce the computational complexity by controlling \(d_{1}^{mid}\), a major component that increases the complexity of the blocks. We first define the set of channel reduction ratios for each block that serves as a criterion for dynamically adjusting \(c_{i}\). Then, the controller selects one of the channel reduction ratios to design the dynamic module configuration. We define the set of channel reduction ratios as follows:  
\[T_{i}=\left\{t\left|\frac{d_{1}^{mid}}{t}\geq d_{2}^{mid},\quad t\in\mathbb{Z}^ {+}\right.\right\}, \tag{3}\]  
where \(T_{i}\) denotes the set of channel reduction ratios of the \(i\)-th block. \(t\) denotes the channel reduction ratios for the \(i\)-th block. We define the set of channel reduction ratios for light-weight blocks based on the design guide of the network [11]. Ref. [11] empirically proved that drastic channel expansion disrupts the expressive power of an expansion layer. Therefore, we allow the channel reduction ratio to reduce \(d_{1}^{mid}\). In order to keep the block in a form of the inverted residual block [7], \(d_{1}^{mid}/t\) is always greater than or equal to \(d_{2}^{mid}\). Finally, the module configuration for each block is constructed by the controller selecting one of the channel reduction ratios and determining the channel operation based on it, as shown in Figure 1. We define the controller \(C_{i}\) as:  
\[C_{i}(t)=\left\{ \begin{aligned} & R,\quad\text{if}\ d_{1}^{in}>\frac{d_{1}^{mid}}{t}\\ & E,\quad\text{otherwise},\end{aligned} \right. \tag{4}\]  
where \(d_{1}^{in}\) denotes the input channel dimension of \(M_{1}\). \(R\) and \(E\) are the channel reduction and expansion operations, respectively. If channel operation is \(E\) for all blocks, the network architecture is equivalent to GhostNet. Note that the largest network in our module configuration is GhostNet. The computational cost of the \(i\)-th block decreases whenever the channel operation of \(E\) is not selected. If \(T_{i}\) satisfies the conditions described in Equation (3), it signifies that the stage has been reached where it is possible to manually examine all possible combinations for each block. In other words, it implies that the number of combinations is not large. Thus, the controller manually selects the channel reduction ratio to explore the most suitable module configuration, considering validation accuracy. This process is reminiscent of the approach taken in a previous study [32], where an empirical distribution function (EDF) was employed to narrow the search space. Figure 1: _Cont._  
The presented dynamic module configuration has three positive effects over GhostNet.