# How Deeply to Fine-Tune a Convolutional Neural Network: A Case Study Using a Histopathology Dataset
## 3 Methodology
### Transfer Learning  
The source dataset is the dataset being used to train the CNN weights to be used for another target dataset; usually, the source dataset contains millions of images with thousands of classes, like the ImageNet dataset [32]. The following four techniques for transfer learning were introduced in the literature:  
* The first technique is to freeze the source CNN's weights (like ImageNet's weights) and then remove the original fully connected layers and add another classifier, either a new fully connected layer or any machine learning classifier, like support vector machine (SVM), that is, to use the original weights for feature extraction. * The second technique is to fine-tune the top layers of the source CNN with a very small learning rate and freeze the bottom layers, under the assumption that the bottom layers are very generic and can be used for any kind of image dataset [16]. * The third technique is to fine-tune the entire network's weights using a very small learning rate to avoid losing the source weights, then remove the last fully connected layers, and add another layer to suit the target dataset. * The fourth technique is to use the CNN's original architecture without importing any weights, that is, to initialize the weights from scratch. The point of this technique is using a well-known architecture that has been experimented with challenging datasets and proven to be good. Different transfer learning techniques are shown in Figure 4.  
Figure 4: _Cont._According to [PERSON] and [PERSON] [33], transfer learning in the image classification domain can be defined given the following parameters: \(S:\)_Source_, \(\mathrm{T}:\)_Target_, \(n:\)_Source dataset size_, \(m:\)_Target dataset size_, \(\phi(. ):\) objective function, \(\mathcal{D}:\)_Domain_, \(\mathcal{F}:\)_feature space_, \(P(X):\)_Marginal probability distribution_, and _learning samples_: \(X\ =\ \{x_{1},x_{2},\ \ldots,x_{n}\}\in\mathcal{F}\), \(\Upsilon:\)_Label space_, \(\text{task}:\ \mathrm{K}\ =\ \left\{\Upsilon,\phi(. )\right\}\), \(n\gg m\). An image domain, \(\mathcal{D}\), is defined as having two components: a feature space, \(\mathcal{F}\), and a probability distribution, \(P(X)\):  
\[\mathcal{D}\ =\ \left\{\mathcal{F},P(X)\right\} \tag{5}\]  
For a source domain, \(S\), the dataset is \(X_{S}\ =\ \left\{x_{S_{1}},x_{S_{2}},\ \ldots,x_{S_{n}}\right\}\in\mathcal{F}_{S}\). The source domain data can be denoted by:  
\[\mathcal{D}_{S}\ =\ \left\{\left(x_{S_{1}},y_{S_{1}}\right),\left(x_{S_{2}},y_{S _{2}}\right),\ldots,\left(x_{S_{n}},y_{S_{n}}\right)\right\} \tag{6}\]  
where \(x_{S_{i}}\in\mathcal{F}_{S}\) is the data instance, and \(y_{S_{i}}\in\Upsilon_{S}\) is the corresponding class label. For the target domain, \(\mathrm{T}\), the target domain can be denoted by:  
\[\mathcal{D}_{\mathrm{T}}\ =\ \left\{\left(x_{\mathrm{T}_{m}},y_{\mathrm{T}_{m}} \right),\left(x_{\mathrm{T}_{m}},y_{\mathrm{T}_{m}}\right),\ldots,\left(x_{ \mathrm{T}_{m}},y_{\mathrm{T}_{m}}\right)\right\} \tag{7}\]  
where \(x_{\mathrm{T}_{i}}\in\mathcal{F}_{\mathrm{T}}\) is the data instance, and \(y_{\mathrm{T}_{i}}\in\Upsilon_{\mathrm{T}}\) is the corresponding class label. A task, \(\mathrm{K}\), consists of two components: a label space, \(\Upsilon\), and an objective function, \(\phi(. )\):  
\[\mathrm{K}\ =\ \left\{\Upsilon,\phi(.)\right\} \tag{8}\]  
Figure 4: This figure shows different transfer learning techniques: (**a**) a generic CNN trained on ImageNet dataset, (**b**) the first technique, in which the source weights are fixed and the original classifier layers will be replaced by new layers to suit the target dataset, (**c**) the second technique, in which the top layers will be fine-tuned, the weights of the bottom layer will be fixed, and the last fully connected layers will be replaced, (**d**) the third technique, in which the original classifier layers will be replaced and the entire network will be fine-tuned, and (**e**) the fourth technique, in which the original architecture will be used without any weights.