# Implementation Details for Controlling Contactless 3D Virtual Endoscopy
## 4 Results
### Visual Feedback  
Feedback on visual interaction provides important information for the contactless hand gesture control system. In order to enable all the features, it is necessary that the surgeon understands how to control our application. Several requirements/program features were implemented in our user interface, as shown in Figure 2, to enable fast feedback and accurate and precise interaction with the system:  
* UI that considers human ergonomics: We created ribbon-based UI that can be controlled contactless during the surgery and in the surgery planning.
* Fast and informative visual feedback: Within 100 ms we provide the info on what happened and the surgeon can see what are the next options.
* Intuitive and clear visual designs and text feedback.
* Visual feedback for the optimal user distance from the camera (\"Optimal distance\", \"Move Closer\", \"Move Back\").
* User viewport: A small screen where the surgeon can see what the camera detects.  
Figure 1: Motions recognized by our solution.  
Control Ribbons 1 and 2 shown in Figure 3 can control different parameters. Control Ribbon 1 sets spatial and temporal filtering and threshold for depth dimension. For temporal and spatial filtering, we used medium smoothing of Alpha and Delta channels to preserve edge and corner detection of the hand. Values of these filters depend also on the background light. For darker backgrounds, these filters should be adjusted to higher values. The Filter Smooth Alpha channel for indicating the movements should be adjusted on values above 0.7 when there is less than 300 lux in the room. Similarly, the Filter Smooth Delta channel for edge-preserving in the same conditions should be above 30. The adjustments are usually made only for the initialization of the software in the runtime environment since the system once installed stays in the OR.  
Control Ribbon 2 serves for the Gesture settings and defines the region of interest where the hand will be tracked (this is the purple rectangle shown in Ribbon 3). Adjusting the size of the region of interest is important to focus on the hand of the surgeon in control, removing all unnecessary background noise and other background movements. Ribbon 3  
Figure 3: Control Ribbon.  
Figure 2: User Interface.  
shows the view of the user and how the camera interacts with the user. Ribbon 4 represents the visualization of the medical data. This ribbon could also be put in full-screen mode, once all the initial adjustments are performed.  
While the hand moves on motion Five, the system is engaged, and moving the hand will move the data visualized in Ribbon 4. Different options for visualization could be engaged with initializing the motion Two to choose between the options for visualization, as shown in Figure 4. Options for the visualization of medical data include changing the center of visualization and changing the size of the window, volume opacity, plane opacity, camera pitch, and setting the 3D slices coordinates. The main mode of control is visualization in virtual endoscopy where the current visualization dataset could be fully moved and controlled.  
Ribbon 5 shows what the system can read and the current gesture state and parameters for setting the 2D centers for the planes and overall 3D center. Ribbon 6 gives an option to record the visualization sequence for later surgical planning, while Ribbon 7 represents feedback on the current position, zoom options, center of the planes, and opacity modes.