# Entropy-Based Ensemble of Convolutional Neural Networks for Clothes Texture Pattern Recognition
## 3 The Proposed Method
### Ensemble CNN Model  
We model \(f\) as an ensemble of CNN models and use entropy voting-based fusion i.e.,  
\[f(x;\theta)=\varepsilon(f_{1}(x;\theta_{1}),f_{2}(x;\theta_{2})...f_{n}(x; \theta_{n})) \tag{2}\]  
where \(f_{i},i=1,2,...n\) are base learners, \(\theta_{i},i=1,2,...n\) are their learnable parameters, and \(\varepsilon\) is the fusion function. There are two possibilities to design the base learners. One approach is to learn diverse base learners using the same model but different training sets drawn with, e.g., bootstrap sampling. Another approach is to use diverse models but train with the same dataset. We follow the second approach and employ CNNs to model \(f\)_is_. An overview of the ensemble is shown in Figure 2.  
A convolutional neural network (CNN) is a deep end-to-end learning model that shows attractive performance in the computer vision field. It is typically composed of three types of layers: convolution, pooling, and fully connected layers. As opposed to conventional algorithms, CNNs do not require a prior human-crafted feature extraction. Instead, hierarchical features are learned adaptively through the network, where low-level information is involved in the shallow layers and high-level information is embedded in deep layers.  
The transfer learning approach allows one to exploit and leverage the knowledge a pre-trained model learned from a specific task to a related target task and dataset. As the pre-trained models are trained on massive datasets, e.g., ImageNet, applying them to tasks similar to the task at hand has been proven to result in improved performance with reduced computational cost in terms of time and dataset size. In this work, we employ pre-trainedCNN models as base learners. Transfer learning is utilized to fine-tune pre-trained models where the task-dependent fully connected layer is replaced by another one whose neurons are equal to the number of the classes of the current problem. More specifically, some of the final layers are replaced by new trainable layers while the early layers' learned weights are kept as initial parameters. This procedure is motivated by the observation that the early layers of a model seem to involve general features that can fit other tasks different than the original one while the features in the final layers are problem-specific. As the model is modified, it is retrained using the clothes pattern dataset where the early layers are frozen and the newly added trainable layers are learned. Fine-tuning is performed by unfreezing layers of the original model and training the model by continuing back-propagation using the new dataset. This retraining should be at a very low learning rate to prevent significant updates to the gradient and thus avoid overfitting while improving the performance.  
ResNet50 is used as the backbone model in the transfer learning paradigm. The ResNet model [5] introduces a residual block where every few stacked convolution layers are skipped, and \"shortcut connections\" are initiated to bypass the layers in between. It addresses the problem of degrading accuracy with increased depth by reformulating the original unreferenced mapping into residual mapping. That is, the original mapping is reformulated into \(F(x)+x\) as optimizing the residual function \(F(x)\) is easier and capable of enhancing the performance even with increased depth. To perform the element-wise addition in the operation \(F(x)+x\), \(x\) can be an identity mapping if it has the same dimension as \(F(x)\) or can be linearly projected using a \(1\times 1\) convolution layer.