# Double-Shot Transfer Learning for Breast Cancer Classification from X-Ray Images
## 2 Materials and Methods
### Double-Shot Transfer Learning  
Transfer learning is a powerful technique that allows knowledge to be transferred across various tasks of neural networks. In transfer learning, a pre-trained network that has already learned informative features from a certain image classification task can be used as a starting point to learn a new task using a smaller number of training samples. Knowledge transferring can be done by fine-tuning some layers in the pre-trained network, such as input layer, fully-connected layer, classification layer, and train the pre-trained network on a new dataset. Fine-tuning a pre-trained network usually produces better accuracy, and it is faster than training a new network from scratch [25]. It has been shown in a previous work that transfer learning is very effective when the source and \begin{table}
\begin{tabular}{c c} \hline \hline
**Training Options** & **Configuration** \\ \hline Optimizer & SGDM a \\ Mini Batch Size & 40 \\ Momentum Value & 0.9 \\ Maximum Epochs & 100 \\ Initial Learning Rate & 0.001 \\ Execution Environment & GPU \\ Learning Rate Schedule & Constant \\ \hline \hline \end{tabular}  
* SGDM: Stochastic Gradient Descent with Momentum.  
\end{table} 
Table 6: Training options for all pre-trained models. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Depth** & **Size** & **Parameters (Millions)** & **Image Input Size** \\ \hline AlexNet & 8 & 227 MB & 61 & 227 \(\times\) 227 \\ GoogLeNet & 22 & 27 MB & 7 & 224 \(\times\) 224 \\ VGG-16 & 16 & 515 MB & 138 & 224 \(\times\) 224 \\ VGG-19 & 19 & 535 MB & 144 & 224 \(\times\) 224 \\ MobileNet-v2 & 53 & 13 MB & 3.5 & 224 \(\times\) 224 \\ ResNet-50 & 50 & 96 MB & 25.6 & 224 \(\times\) 224 \\ ResNet-101 & 101 & 167 MB & 44.6 & 224 \(\times\) 224 \\ ShuffleNet & 50 & 6.3 MB & 1.4 & 224 \(\times\) 224 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Pre-trained networks properties. target domains/tasks are similar. In the previous studies, instead of learning from scratch, SSTL takes advantage of knowledge that comes from previously learned datasets, especially when the training samples in the target domain are scarce. Unfortunately, SSTL has been applied without taking into account that these pre-trained models had been trained on ImageNet, which has different feature space and distribution from our target datasets. In other words, the previous works did not take into account the relationship between source and target domain when SSTL is applied. A domain can be represented as \(\mathcal{D}=\{\mathcal{X},P(X)\}\), where \(\mathcal{X}\) is the feature space, \(P(X)\) is the probability distribution function, and \(X=\{x_{1},\ldots,x_{n}\}\in\mathcal{X}\). A task can be represented by \(\mathcal{T}=\{\mathcal{Y},f(. )\}\), where \(\mathcal{Y}\) is the label space and \(f(. )\) is the objective predictive function. \(\mathcal{T}\) can be learned from the training data, which consists of pairs \(\{x_{i},\ldots,y_{i}\}\), where \(x_{i}\in X\) and \(y_{i}\in\mathcal{Y}\). The function \(f(. )\) can be used to predict the corresponding label, \(f(x)\), of a new instance \(x\). \(f(x)\) can also be considered as a conditional probability function \(P(y|x)\)[26]. In SSTL, given a domain source \(\mathcal{D}_{s}\) for a learned task \(\mathcal{T}_{s}\) can help to learn a target task \(\mathcal{T}_{t}\) of the domain \(\mathcal{D}_{t}\). In most of the cases, \(\mathcal{D}_{s}\
eq\mathcal{D}_{t}\) and/or \(\mathcal{T}_{s}\
eq\mathcal{T}_{t}\). However, the DSTL aims to bring the marginal probability distributions of both domains \(\mathcal{D}_{s}\) and \(\mathcal{D}_{t}\) similar to each other, \(\mathcal{D}_{s}\simeq\mathcal{D}_{t}\), by providing \(\mathcal{D}_{s}\) with a large number of instances that are similar to \(\mathcal{D}_{t}\), especially when \(\mathcal{D}_{t}\) has insufficient training samples. Hence, the performance of the prediction function \(f_{T}(x)\) for learning task \(\mathcal{T}_{t}\) can be improved. In most cases, \(\mathcal{D}_{s}\) data are larger than \(\mathcal{D}_{t}\) data.