## 2 Methods
### Data Training with Compound Model Scaling
#### 2.3.2 Compound Model Scaling  
This study tries to use the CNN with compound model scaling, which means to use three scaling factors at the same time, and then evaluate the effect of compound model scaling on classification. The CNN layer \(i\) can be expressed as a function: \(Y_{i}=F_{i}(X_{i})\), where \(F_{i}\) is the operator, \(Y_{i}\) is output tensor, \(X_{i}\) is input tensor, with tensor shape \((H_{i},W_{i},C_{i})^{1}\), where \(H_{i}\) and \(W_{i}\) are spatial dimensions and \(C_{i}\) is the channel dimension [27,28]. A CNN \(N\) can be represented by a list of composed layers: \(N=F_{k}\odot\cdots\odot F_{2}\odot F_{1}(X_{1})=\odot_{j=1\cdot\cot k}F_{j}^{X_ {1}}\), where \(\odot\) means the connection between consecutive layers. CNN layers are often partitioned into multiple stages and all layers in each stage share the same architecture. Thus, the CNN can be defined as [27]:  
\[N=\odot_{i=1\cdots s}F_{i}^{L_{i}}(X_{(H_{i},W_{i}C_{i})}), \tag{1}\]  
Figure 2: A location of the 9 recording loci on the surface of the tongue. T, tip; Tr and TI, tip right and tip left; Er and El, edge right and edge left; Dr and DI, dorsal right and dorsal left; Pr and PI, fungiform papillae just anterior to the circumvallate papillae.  
where \(F_{i}^{L_{i}}\) denotes \(F_{i}\) of layer i is repeated \(L_{i}\) times in stage \(i\), \(\langle H_{i},W_{i},C_{i}\rangle\) denotes the shape of input tensor \(X\) of layer \(i\).  
When it is possible that all layers should be scaled with constant ratio, the problem for model scaling can be an optimization problem, which is to maximize the model accuracy for any given resource constraints. This is achieved by the following algorithm:  
\[\begin{split}\underset{d,w,r}{\text{maximize}}& Accuracy(N(d,w,r))\\ \text{subject to}& N(d,w,r)=\odot_{i=1\cdots s}\hat{F}_{i}^{d \cdot\hat{L}_{i}}(X_{(r\cdot\hat{H}_{i},r\cdot\hat{W}_{i},w\cdot C_{i})})\\ & Memory(N)\leq target-memory\\ & FLOPS(N)\leq target-flops\end{split} \tag{2}\]  
where \(w\), \(d\), \(r\) are coefficients for scaling network width, depth, and resolution, and \(\hat{F}_{i}\), \(\hat{L}_{i}\), \(\hat{H}_{i}\), \(\hat{W}_{i}\), \(\hat{C}_{i}\) are predefined parameters in baseline network.  
The compound scaling method uses a compound coefficient \(\phi\) to scale network width, depth, and resolution:  
\[\begin{split}\text{depth:}& d=\alpha^{\phi}\\ \text{width:}& w=\beta^{\phi}\\ \text{resolution:}& r=\gamma^{\phi}\\ \text{subject to}&\alpha\cdot\beta^{2}\cdot\gamma^{2} \approx 2\\ &\alpha\geq 1,\beta\geq 1,\gamma\geq 1\end{split} \tag{3}\]  
where \(\alpha\), \(\beta\), \(\gamma\) are constants that can be determined by a small grid search.  
From the start with the baseline EfficientNet-B0, the compound scaling method is applied to scale it up with two steps. At first, it performs the search for \(\alpha\), \(\beta\), \(\gamma\) through Equations (2) and (3) after fix \(\phi=1\). The values for EfficientNet-B0 are \(\alpha=1.2\), \(\beta=1.1\), and \(\gamma=1.15\) under the condition of \(\alpha\cdot\beta^{2}\cdot\gamma^{2}\approx 2\). Then, scale up baseline network with different \(\phi\) through Equation (3) after fix \(\alpha\), \(\beta\), \(\gamma\), to gain EfficientNet-B1 to B7.