# Ensembling EfficientNets for the Classification and Interpretation of Histopathology Images
## 2 Materials and Methods
### Deep Learning Methods
#### 2.1.1 EfficientNets  
EfficientNets are a group of deep convolutional networks that achieve and surpass state-of-the-art accuracy in different classification tasks with up to ten times better efficiency, thus the name (smaller and faster). Their main novelty lies on the latest achievement of AutoML, and, specifically, on the intelligent and controlled expansion of the three dimensions (width, depth, resolution) of a neural network by the utilization of a compound coefficient. Throughout years of research, the basic concern has been the growth of a neural network's dimensions in such a way that accuracy is improved with the minimum of operations given certain resources' constraints. Even when the minimum of operations is not a basic goal, increasing the dimensions of a neural network in a greedy manner does not have the expected results due to the vanishing gradients' phenomenon. Efficient Nets address this issue by exploring the relation of the increase in each dimension and applying a grid search under a fixed resources constraint instead of arbitrarily changing these dimensions. The compound scaling method is summarized in the set of Equation (1):  
\[\begin{array}{l}\text{d}=\alpha^{\varphi}\\ \text{w}=\beta^{\varphi}\\ \text{r}=\gamma^{\varphi}\\ \alpha\cdot\beta^{2}\cdot\gamma^{2}\approx 2\\ \alpha\geq 1,\,\beta\geq 1,\,\gamma\geq 1\end{array} \tag{1}\]  
where \(\varphi\) is a global scaling factor that controls how many resources are available and \(\alpha\), \(\beta\), \(\gamma\) determine how to allocate these resources to network depth, width, and resolution, respectively. By assigning \(\varphi=1\) and applying grid search, \(\alpha\), \(\beta\) and \(\gamma\) can be determined for a given convolutional architecture to achieve better accuracy. Once concluding with the definition of \(\alpha\), \(\beta\) and \(\gamma\), \(\varphi\) can be gradually increased to augment the dimensions of the network towards better accuracy. The scaling method is applicable to any convolutional architecture that consists of a repeated pattern of layers. However, the authors of EfficientNets paper proposed a specific architecture where the main building block is the mobile inverted bottleneck convolution (MB Conv), shown in its three basic configurations in Figure 1. The base model of the EfficientNets group is Efficient Net B0 and its architecture is shown in Table 1, consisting mainly of MBConv1 and MBConv6. By utilizing MBConv blocks and increasing the value \(\varphi\), Efficient Net group reaches its most complicated form B7. In the heart of these building blocks, two important innovations have found grounds to act: the depthwise separable convolution [34] that performs the functionality of a normal convolution with less resources and the squeeze and excitation unit that enables the network to perform dynamic channelwise feature recalibration [35]. Concerning depthwise separable convolution, the convolution operation is divided into two parts. First, the convolution is conducted depthwise, meaning that the convolution kernel is applied to each channel individually in order to learn channel dependent features and second, pointwise, meaning that a 1 \(\times\) 1 kernel is applied to each point in order to combine the channel dependent learned features. In reference to the squeeze and excitation unit, the unit consists of two parts. Starting the squeeze part, global average pooling is applied to each channel leading to the formation of an 1