## 3 Method
### Tmulti: Transfer Learning with Multi-Task Learning  
When doctors diagnosis a lesion or classify cancer, they diagnose it by looking at the RoI identified, the type of lesion, and the occurrence of cancer at the same time. Therefore, inspired by doctors' diagnoses, this study assumed that performance can be improved by performing RoI segmentation and classification simultaneously. Thus, multi-task learning was conducted to simultaneously perform segmentation and classification with the same image using the baseline model of each application. For the classification branch, global averaged pooling and fully connected layers were added from the final visual feature of the baseline model. For example, as seen in Figure 5, pooling and fully connected layers were added from the final feature extracted from the U-Net encoder for the classification branch. For the segmentation branch, a decoder using the up-sampling layer was added to the baseline model of the application and transformed into a U-Net model, as shown in Figure 5. The approach is referred to as \(T_{multi}\), and the performance was measured for the three applications. In this approach, the loss weight is given using the following equation as a two-stream loss calculation inspired by [90]. \(f^{W}(x)\) is the output of a neural network with weights \(W\) on input \(x\). Equation (3) is the multi-task likelihood function with segmentation output \(y_{1}\) and classification output \(y_{2}\). We designed the multi-task model with two branches of classification and segmentation and used the encoder's weight for transfer learning. In addition, the encoder was designed with a single-task output through the fully connected layer and used to evaluate the task performance.  
\[p\Big{(}y_{1}=c_{1},y_{2}=c_{2}\Big{|}f^{W}(x)\Big{)}=p\Big{(}y_{1}=c_{1}\Big{|} f^{W}(x)\Big{)}\,p\Big{(}y_{2}=c_{2}\Big{|}f^{W}(x)\Big{)} \tag{3}\]  
Equation (3) leads to Equation (4) being minimized with temperatures \(\sigma_{1}\) and \(\sigma_{2}\) (the input is scaled by each sigma according to the output type):  
\[L(W,\sigma_{1},\sigma_{2})=-logp\Big{(}y_{1}=c_{1},y_{2}=c_{2}\Big{|}f^{W}(x) \Big{)} \tag{4}\]  
In Equation (5), we adapt segmentation and classification likelihood to squash a scaled version of the model output through a sigmoid function and softmax function with positive scalars \(\sigma_{1}\) and \(\sigma_{2}\). The sigmoid function is used for the segmentation task because the task is to classify pixels into 0 or 1, and the softmax function is used to classify various types for the classification task (e.g., 7 types for skin lesion classification). The scalars relate to the model's uncertainty, as measured in the entropy. Equation (5) can be written as Equations (6) and (7):  
\[=-log\Big{(}Sigmoid\Big{(}y_{1}=c_{1};f^{W}(x),\sigma_{1}\Big{)} Softmax\Big{(}y_{2}=c_{2};f^{W}(x),\sigma_{2}\Big{)}\Big{)} \tag{5}\]  
\[=-log\Big{(}Sigmoid\Big{(}y_{1}=c_{1};f^{W}(x),\sigma_{1}\Big{)}\Big{)}\,\,- \,\,log\Big{(}Softmax\Big{(}y_{2}=c_{2};f^{W}(x),\sigma_{2}\Big{)}\Big{)} \tag{6}\]  
\[=-log\Big{(}Sigmoid\big{(}\frac{1}{\sigma_{1}}f^{W}(x)\big{)}\Big{)}\,\,-\, \,log\Big{(}Softmax(\frac{1}{\sigma_{2}}f^{W}(x))\Big{)} \tag{7}\]  
The final joint loss is given in Equation (8):  
\[\approx\frac{1}{\sigma_{1}^{2}}L_{1}(W)+\frac{1}{\sigma_{1}^{2}}L_{2}(W)+log \sigma_{1}+log\sigma_{2} \tag{8}\]  
where \(L_{1}(W)=-log(Sigmoid\big{(}y_{1},\,f^{W}(x)\big{)})\) and \(L_{2}(W)=-log(Softmax\big{(}y_{2},\,f^{W}(x)\big{)})\). We minimize Equation (8) with respect to W, \(\sigma_{1}\), and \(\sigma_{2}\).  
In this study, after measuring the performance of the four approaches for each application, we propose an effective transfer learning approach and a high-performance ensemble approach.