# The importance of image viewing conditions for timing and precision in minimally invasive surgery

Birgitta Dresp

► To cite this version:

Birgitta Dresp. The importance of image viewing conditions for timing and precision in minimally invasive surgery. 2024, 10.32907/RO-140-6297620026. hal-04877899

HAL Id: hal-04877899

https://hal.science/hal-04877899v1

Submitted on 9 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<table><tr><td>Category</td><td>Health & Medicine</td></tr><tr><td>Headline</td><td>The importance of image viewing conditions for timing and precision in minimally invasive surgery<br>Birgitta Dresp-Langley, CNRS UMR 7357, Strasbourg, FRANCE</td></tr><tr><td>Abstract</td><td>Minimally invasive medical procedures such as laparoscopic surgery are challenging because they involve navigating the instruments in use via image views from tiny cameras attached to them. Surgeons need to interpret visual information correctly to guide their instruments with precision. Understanding how the brain adapts to these procedures will help improve fine surgical skills. Dr Birgitta Dresp-Langley, Research Director at the French National Centre for Scientific Research (CNRS) has designed well-tailored experiments with to-the-single-pixel precise measurement of hand-tool movement trajectories in task time at the millisecond level, highlighting the drawbacks of virtual reality headset viewing, and revealing critical factors that affect the timing and precision of image-guided tasks.</td></tr></table>

<table><tr><td>Research Objectives</td><td>Dr Birgitta Dresp-Langley and her team investigated viewing factors affecting hand-tool movement timing and precision in image-guided tasks</td></tr><tr><td>Funding</td><td><ul><li>Initiative D'EXcellence (IDEX) Programme Strasbourg University 2015-2021</li><li>Mission Interdisciplinarité du CNRS</li></ul></td></tr><tr><td>Collaborators</td><td>Ufuk Batmaz, former graduate student, currently Associate Professor at Concordia University Montreal, Canada<br>Michel de Mathelin, Professor of Robotics, Strasbourg University, France, co-supervisor of the thesis</td></tr><tr><td>Author's biosketch</td><td>Dr Birgitta Dresp-Langley has been a tenured research scientist with CNRS since 1993 and a research director since 2018. She received her PhD in cognitive science from Paris Descartes University in 1991, and defended her Habilitation in behavioural and computational neuroscience at Strasbourg University in 1998.</td></tr><tr><td>Contact</td><td>E: birgitta.dresp@cnrs.fr<br>W: cv.hal.science/birgitta-dresp</td></tr><tr><td>Competing interest statement</td><td>NONE</td></tr></table>
**Introduction**

Image-guided processes have revolutionised surgical procedures and are crucial in minimally invasive surgery, i.e. laparoscopy. The surgeon uses specifically designed tools equipped with tiny cameras to look inside your body via a small incision in the abdomen or pelvis. Laparoscopy is used to diagnose or treat a range of conditions, including appendicitis, endometriosis, and certain types of cancer.

Performing laparoscopy can be challenging because humans are accustomed to interacting with objects at the real 3D scale rather than responding to virtual images of the environment. Dr Birgitta Dresp-Langley, research director at the French National Centre for Scientific Research (CNRS), has run studies [1, 2, 3] to identify the factors that determine the timing and precision of hand-tool movements and their trajectories in image-guided tasks. Her findings can help us understand how surgeons may improve their performance in critical medical procedures such as minimally invasive surgery.

**Virtual navigation challenges**

In virtual navigation, the image representations need to be processed and interpreted by the brain to ensure the correct positioning of hands and tools while surgeons are looking at a two-dimensional (2D) camera image or a three-dimensional (3D) reconstruction of the camera image by means of software piloting a virtual reality (VR) headset. Experienced surgeons become proficient with training and practice and, finally, adapt to the constraints of virtual navigation. The cognitive mechanisms of this adaptation are still not well understood. A major problem in virtual navigation is that depth information needs to be reconstructed by the brain because reliable information about depth and relative distances in the task space the surgeon has to act upon (a patient's organ) is missing from the image representation. Time-to-contact anticipation, movement planning, and eye-hand coordination are challenged compared with direct "natural" viewing conditions. Also, the image views change in computational time as the tool moves through the body in real time. Research by Dresp-Langley has investigated performance characteristics of healthy volunteers with no specific previous experience in such procedures.

**Task-user system design**

An equal number of men and women took part in the study. The task described in [1] involved executing a series of moves in a particular order to place a small object as swiftly and precisely as possible on the indicated spots on a task board. The participants were tested under different conditions, either using their hands or a surgical tool. They were given an unconstrained direct view of their hand-tool movements, were guided by 2D fisheye camera images displayed on a monitor, positioned either straight ahead or sideways, or a reconstructed 3D view from a virtual reality (VR) headset. Before starting the trial, the participants had the chance to look at a printout of the trajectory steps they had to follow during the task. Each participant completed two rounds of every experimental condition described above. For a round to be counted as complete, they had to execute the series of moves in the correct order and without dropping the object. Specific software allowed measuring both the precision (in image pixels) and time (in milliseconds) of each participant in a fully complete task set. Individual completion time corresponded to the duration from the moment the object was picked up to the time it was placed back to its initial position on the task board.

**2D versus virtual 3D image guidance**

Image conditions directly affect task performance. Direct "natural" viewing of the task space generates the fastest and most precise hand-tool movement trajectories compared with any of the 2D image
viewing conditions. The reconstructed 3D view of the task space conveyed by the VR headset produced the worst performance in terms of precision and completion times. This drawback is further highlighted by hand movement trajectories recovered by processing of the video images of task execution by the different participants [2], displayed in the trajectory reconstructions here below.

<figure><img src="image_2.png" /></figure>

A direct "natural" view of the task space generates the fastest and most precise movement trajectories (top). A 3D view reconstructed from the 2D camera view of the task space by software piloting a virtual reality (VR) headset produced the worst performance (middle) by comparison with a direct view and the a monitor view of the camera image (bottom). Different trajectories in a given plot represent task performance data of different participants.

## Speed versus precision

Participants consistently performed faster in the second session compared to the first, suggesting there was an immediate training effect on task time. However, no training effect observed for movement or trajectory precision, which did not improve in the second round. Compared to an image-guided setting, subjects performed faster and more precisely when they had a direct view of the task-space and their hand movements and did not have to use the tool to move the object across the task board. Perception of the so-called near-body space and the task space within arm's reach directly determines performance depending on the viewing condition. An extra effort is needed to further adjust hand movements to the constraints of the tool. The researchers also noticed that performance was significantly affected by the monitor position in the 2D viewing conditions. Although subjects were faster when the monitor was positioned straight ahead, they were more precise when the monitor was placed sideways. This finding suggests effects of viewing angle and or attention on the recovery of efficient image cues for movement planning. When looking straight ahead, there is less strain on the neck and participants feel more comfortable, which results in faster moves. Sideways viewing may generate more effective cues to relative distances in the task field and, although the participants are in a less comfortable position, this also prompts them to pay more attention, which contributes to more precise moves. Another advantage
of sideways viewing may be better control of hand and tool movements due to changes in lighting conditions, allowing for better discrimination of objects and their trajectories in the sideways viewing condition. Trade-off between speed and precision is always a critical aspect of training for image-guided tasks and depends on the specific task conditions.

## The road ahead

Safe and effective performance of tasks guided by virtual information, such as minimally invasive surgery, is of paramount importance in modern medicine. However, processes that involve image-guided tasks entail serious challenges due to the risk of misperception of relative distances and time-to-contact of moving objects. Dresp-Langley's work highlights the need for precise measurement criteria to establish objective gold-standard benchmarks for training novices in image-guided precision tasks [3]. This will help ensure safety and precision in virtually guided surgery in the future.

## Author's personal response

This research was inspired by the author's recognition of the need for a transdisciplinary research effort towards understanding perceptual and cognitive constraints of novel technologies such as virtually guided precision tasks. How such effort can lead to objective performance benchmark criteria in the design of training programmes for practicing surgeons to improve their skills and performance in image-guided surgery is discussed in further work by the authors.

## References

Batmaz, AU, de Mathelin, M, Dresp-Langley, B, (2017), Seeing virtual while acting real: Visual display and strategy effects on the time and precision of eye-hand coordination, *PLoS ONE*, 12(8), e0183789.
https://doi.org/10.1371/journal.pone.0183789

Batmaz, AU, de Mathelin, M, Dresp-Langley, B, (2018), Effects of 2D and 3D image views on hand movement trajectories in the surgeon's peri-personal space in a computer controlled simulator environment, *Cogent Medicine*, 5:1. https://doi.org/10.1080/2331205X.2018.1426232

Liu, R., Wandeto, J, Nageotte, F, Zanne, P, de Mathelin, M, Dresp-Langley, B. (2023). Spatiotemporal Modeling of Grip Forces Captures Proficiency in Manual Robot Control. *Bioengineering*, 10(1), 59.
https://doi.org/10.3390/bioengineering10010059

Dresp-Langley, B, (2018), Towards Expert-Based Speed-Precision Control in Early Simulator Training for Novice Surgeons, *Information*, 9(12), e316. https://doi.org/10.3390/info9120316

Dresp-Langley, B, (2022), Grip force as a functional window to somatosensory cognition. Front. Psychol.
[CrossRef] [PubMed]

Liu, R, Dresp-Langley, B, (2021), Making Sense of Complex Sensor Data Streams. *Electronics*, 10(12), e1391. https://doi.org/10.3390/electronics10121391