# Identifying Malignant Breast Ultrasound Images Using ViT-Patch
## 3 Experiments
### Experimental Results  
We validate the performance of the ViT-Patch for whole-image-based malignant identification and patch-based tumor localization, based on the breast ultrasound dataset. The \begin{table}
\begin{tabular}{c c c} \hline \hline
**Characteristics** & **Benign** & **Malignant** \\ \hline Shape & Regular & Irregular \\ Margins & Smooth & irregular \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Morphological characteristics of breast ultrasound images. Figure 4: Breast ultrasound image images and their masks. experimental results for malignant identification are compared with the standard ViT, and some traditional models which have been widely used for comparison. The learning curves of the standard ViT and ViT-Patch models are compared, to validate the improvement of the proposed architecture for model training. In addition, the differences between the deep token features learned by the standard ViT and the ViT-Patch models are analyzed to show the optimization of the proposed architecture for self-attention-based feature learning. _Malignant identification:_ The primary task (malignant identification) is evaluated using Accuracy (\(ACC\)) and Sensitivity (\(SEN\)),  
\[ACC=\frac{TP+TN}{TP+TN+FN+FP} \tag{11}\] \[SEN=\frac{TP}{TP+FN} \tag{12}\]  
where \(TP\) indicates the number of True Positive samples, \(FP\) indicates False Positive, \(FN\) indicates False Negative, and \(TN\) indicates True Negative. Table 2 gives the ACC and SEN evaluation of the involved models for the primary task. The top two results (bolded in the table) are all from the ViT-Patch architecture. Compared to the standard ViT counterparts, the ViT-Patch models have advantages at all patch sizes, suggesting that adding a suitable secondary task to the patch tokens can improve the classification performance on the class token. The performance of the ViT-based models decrease significantly for larger patch sizes (\(S=64,128\)), which is lower than that of the CNN-based baselines. The reason for the drop may be that the patch embedding leads to significant feature loss when the patch size is large, and it is difficult for the self-attention mechanism to take advantage of long-range feature extraction, based on the barren embedded vectors. In terms of SEN alone, the ViT-based models have some advantages over the CNN-based models. In Table 2, the ACCs of the ViT-based models with large patch are lower than those of the CNN-based models, but their SEN still remains comparable. _Lesion localization using ViT-Patch:_ We also evaluated the tumor localization ability of the ViT-Patch, i.e., the performance for the secondary task, which is a _free gift_ provided by the proposed architecture. Figure 5 shows the localization results of several testing images predicted by the ViT-Patch models of various patch sizes. In the predicted mosaic maps (from the third to the last columns), the yellow blocks denote TP, the green blocks denote FP, the blue blocks denote FN, and the purple blocks denote TN. By comparing the predicated mosaics with the labeled segmentation masks (the second column in the figure), it can be seen that the ViT-Patches with different patch sizes can provide relatively reliable patch-based tumor localization, which can be used for subsequent segmentation tasks, e.g., optimizing the RPN or ROI modules of the Faster-RCNN model [33]. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **ACC (\%)** & **SEN (\%)** & **Params** \\ \hline ResNet50 [29] & 83.1 & 48.5 & - \\ InceptionV3 [30] & 86.4 & 54.5 & - \\ VGG16 [31] & 88.1 & 66.7 & - \\ Swin-B [23] & 86.4 & 60.6 & 88 M \\ TNT-B [32] & 83.1 & 63.6 & 65.6 M \\ T2T-ViT-24 [22] & 84.7 & 51.5 & 64.1 M \\ PVT-Large [11] & 85.6 & 60.6 & 61.4 M \\ ViT/**ViT-Patch (16)** & 85.6/**89.0** & 66.7/**69.7** & 88.46 M/90.82 M \\ ViT/**ViT-Patch (32)** & 84.7/**89.8** & 60.6/**72.7** & 89.89 M/92.26 M \\ ViT/ViT-Patch (64) & 80.5/85.6 & 57.6/54.5 & 96.89 M/99.25 M \\ ViT/ViT-Patch (128) & 79.7/78.8 & 54.5/54.5 & 125.18 M/127.54 M \\ \hline \hline \end{tabular}
\end{table}