# Instrument Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset
## 2 Materials and Methods
### Gesture Annotations
#### 2.3.1. ROSMAG40 Annotations  
ROSMAG40 is a subset of ROSMA dataset that includes gesture annotations for 40 videos (72,843 frames). The distribution of the annotated videos according to the user, the task, and the dominant hand of each trial is described in Table 3. ROSMAG40 includes videos from five different users: users _X01_, _X02_, and _X07_ are dexterous, while users _X06_, and _X08_ are ambidextrous. For each user, we have annotated eight video, four for the _pea on a peg_ task, and four for _post and sleeve_. For each task, we have annotated two trials performed using PSM1 as the dominant tool and other two trials using PSM2 as the dominant tool (for _pea on a peg_; the dominant tool is the one in charge of picking and placing the peas, while for _post and sleeve_, the dominant tool is considered the one that picks the sleeves).  
For the annotations, we have followed the previous idea of defining the gestures to facilitate their generalization to other tasks. In this sense, we define a descriptive gesture as a gesture or action characterized by an action adjective. For example, we can define a descriptive gesture as a _precision action_, regardless of whether we are inserting the needle in a particular point, putting a staple, or dissecting a duct. This work defines two classes of descriptive gestures: maneuver descriptors (MD) and fine-grain descriptors (FGD). Ma-never descriptors represent high-level actions that are common to all surgical tasks, such as precision or collaborative actions. On the other hand, fine-grain descriptors represent low-level actions that are specifically performed in the ROSMA training tasks, such as picking or placing, but which are not generalizable for other types of tasks. ROSMAG40 provides manual annotations for these two classes of gestures. This dataset is available for download at the Zenodo website [34], and its directory structure is as follows:  
* FGDlabels: this folder contains text files with the FGD annotations. Each row, which corresponds with a video frame, contains two items: the gesture label for PSM1 and the gesture label for PSM2.
* MDlabels: this folder contains text files with the MD annotations with the same structure as FGDlabels files.
* Kinematics: this folder contains text files with the 34 kinematic features. The first row of these files contains the name of the kinematic feature corresponding with each column.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**User ID** & **Task** & **Dominant Tool** & **Annotated Videos** \\ \hline \multirow{4}{*}{X01} & \multirow{4}{*}{Pea on a Peg} & PSM1 & 2 \\  & & PSM2 & 2 \\  & & PSM1 & 2 \\  & Post and sleeve & PSM2 & 2 \\ \hline \multirow{4}{*}{X02} & \multirow{4}{*}{Pea on a Peg} & PSM1 & 2 \\  & & PSM2 & 2 \\  & & PSM1 & 2 \\  & Post and sleeve & PSM2 & 2 \\ \hline \multirow{4}{*}{X06} & \multirow{4}{*}{Pea on a Peg} & PSM1 & 2 \\  & & PSM2 & 2 \\  & Post and sleeve & PSM1 & 2 \\  & & PSM2 & 2 \\ \hline \multirow{4}{*}{X07} & \multirow{4}{*}{Pea on a Peg} & PSM1 & 2 \\  & & PSM2 & 2 \\ \cline{1-1}  & & PSM1 & 2 \\ \cline{1-1}  & & PSM2 & 2 \\ \hline \multirow{4}{*}{X08} & \multirow{4}{*}{Pea on a Peg} & PSM1 & 2 \\ \cline{1-1}  & & PSM2 & 2 \\ \cline{1-1}  & & PSM1 & 2 \\ \cline{1-1}  & & PSM2 & 2 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Description of the ROSMAG40 dataset distribution.  
* Video: this folder contains the video files in mp4 format for the 40 trials of the tasks.