###### Abstract  
We propose a method for effectively utilizing weakly annotated image data in an object detection tasks of breast ultrasound images. Given the problem setting where a small, strongly annotated dataset and a large, weakly annotated dataset with no bounding box information are available, training an object detection model becomes a non-trivial problem. We suggest a controlled weight for handling the effect of weakly annotated images in a two stage object detection model. We also present a subsequent active learning scheme for safely assigning weakly annotated images a strong annotation using the trained model. Experimental results showed a 24% point increase in correct localization (CorLoc) measure, which is the ratio of correctly localized and classified images, by assigning the properly controlled weight. Performing active learning after a model is trained showed an additional increase in CorLoc. We tested the proposed method on the Stanford Dog datasets to assure that it can be applied to general cases, where strong annotations are insufficient to obtain resembling results. The presented method showed that higher performance is achievable with lesser annotation effort. active learning; breast ultrasound; convolutional neural networks; mass classification; object detection; weakly supervised learning +
Footnote †: journal:  
unless special superimposing among all aspect angles will be performed later [7]. The loss of palpation information and the diversity of image aspects makes it challenging for a CAD to improve. Conventional methods for BUS image classification that does not use a neural network framework are based on preprocessing and feature extraction of BUS images following region detection or segmentation with those features. Selected features after region detection are classified with different methods [1]. Most of these works focus on feature extraction of the image when the aim is to classify an image to benign or malignant. Other works that aim to localize the lesion use rule based approaches, such as the deformable parts model [8]. Recent deep learning based frameworks conduct both classification and region detection as annotated data became more available. Semantic segmentation is performed with BUS images in [9] by replacing the last three fully connected layers of AlexNet to fully convolutional networks that perform pixel-wise classification. The work utilizes mask labels that have labels for every pixel in the image as ground truth for all of the images. Mask annotations require more labor from a clinician and, therefore, are harder to obtain. [PERSON] et al. [10] proposed a method for object localization and classification using a Faster-RCNN model. While using bounding box annotations as ground truth for the localization task, it makes use of weakly supervised data only comprised of image level label to aid the classification model. We present a method for sequentially localizing and classifying BUS images based on the Faster-RCNN model presented in [11]. We train a convolutional neural network (CNN) for bounding box regression and mass classification. A fully connected network (FCN) that classifies bounding boxes as either benign, malignant, or background is trained concurrently with the earlier network. The ground truth information are bounding box coordinates and classification labels for each mass. However, BUS data consisting of only the classification labels for each image are more accessible while bounding box annotations require additional expert effort. As BUS image classification still remains a difficult problem, a large dataset size will be beneficial to enhance the performance. Weakly supervised learning is a technique for machine learning with noisy, sparse annotations. A customized alteration, depending on the degree of the annotations, is needed in order to use data with different levels of supervision. Methods for utilizing image level annotations for segmentation are proposed in [12]. An initial segmentation model is trained using a few strongly annotated images.