# Surgical Tool Detection in Open Surgery Videos
## 1 Introduction  
Automated analysis of surgery videos has been indispensable for collecting information about surgery. Combined with computer vision and machine learning, it can improve performance and contribute to enhancing surgical safety and efficiency. A great deal of attention to surgical video analysis has sparked active research in the medical computer vision community. Most research on surgery video analysis has been conducted in the area of minimally invasive surgery (MIS) and cataract surgery. MIS has especially gained great attention, and many tasks have been proposed to analyze surgery, such as workflow analysis [1], phase recognition [2], type recognition [3], video segmentation [4], and video summarization [5]. Here, we focus on surgical tool detection, which can provide an estimation of the identification and position of each surgical tool that appears in a frame (Figure 1). This task is fundamental for recognizing the surgical scene. It can be used for a lot of downstream applications, such as tool tracking [6; 7; 8], tool pose estimation [9; 10; 11], prediction of the remaining surgery duration [12], and surgical technical skill assessment [13]. Surgical tool detection has also been well investigated in MIS [13; 14; 15; 16] and cataract surgery [17] as there are various datasets available in public. However, tool detection in the open surgery field has been limited because of the dataset collection and privacy-preserving costs. In MIS and cataract surgery, the surgeon sees the surgery through an endoscope camera, and videos can be easily recorded. The recordings only include surgical fields and tools and do not contain information that may identify individuals. This enables large and well-annotated datasets in MIS [18; 19; 20] and cataract surgery [17]. On the other hand, in open surgery, the surgeon sees the surgery with their own eyes. Thus, head-mounted cameras or cameras in the operating room are needed for recording videos. The recordings may include surgeon and patient faces andinformation that may identify individuals. Thus, the datasets annotated for open surgery are small and often collected in simulations [21; 22; 23], which is difficult to be transferred for use in real-world surgery. Moreover, the computer vision task in open surgery has challenging inherent properties: varied number and roles of people (e.g., surgeons, anesthesiologists, perfusionists, nurses), the complex interaction of tools and hands, various operative environments, and lighting conditions. Hence, a diverse and large-scale dataset is indispensable. In this paper, we introduce a large-scale egocentric open surgery dataset that contains densely annotated bounding boxes and their categories of tools. Compared with the conventional open surgery dataset for tool detection [22; 23; 24; 25], our dataset contains a tremendous number of annotated frames and is captured at an actual surgery scene. The exampled images in the dataset are shown in Figure 1. Our data were collected by six surgeons in real-world open surgery at our university's school of medicine. We recorded 15 videos of seven different types of surgery. Different from MIS and cataract surgery, there are several choices of the recording viewpoint (e.g., view from the head-mounted camera, the camera attached to the light [26], the surveillance camera). We choose a first-person view as the viewpoint of recording because the data from an egocentric viewpoint are suitable for capturing the details during the surgery. The dataset has 15 h of recording from a camera attached to the surgeon's head, densely annotated bounding boxes, and the corresponding category of the surgical tool. In total, there are 19,560 frames with the bounding boxes of the 31 categories, which is relatively large-scale compared to conventional open surgery datasets. We provide an extensive evaluation of the conventional object detection methods that have been trained with our proposed dataset.