Abstract-In order to realize many of the potential benefits associated with robotically assisted minimally invasive surgery, the robot must be more than a remote controlled device. Currently, using a surgical robot can be challenging, fatiguing, and time-consuming. Teaching the robot to actively assist surgical tasks, such as suturing, has the potential to vastly improve both the patient's outlook and the surgeon's efficiency. One obstacle to completing surgical sutures autonomously is the difficulty in tracking surgical suture threads. This paper presents novel stereo image processing algorithms for the detection, initialization, and tracking of a surgical suture thread. A nonuniform rational Bspline (NURBS) curve is used to model a thin, deformable, and dynamic length thread. The NURBS model is initialized and grown from a single selected point located on the thread. The NURBS curve is optimized by minimizing the image matching energy between the projected stereo NURBS image and the segmented thread image. The algorithms are evaluated using suture threads, a calibrated test pattern, and a simulated thread image. In addition, the accuracies of the algorithms presented are validated as they track a suture thread undergoing translation, deformation, and apparent length changes. All of the tracking is in real time. Note to Practitioners-The problem of tracking a surgical suture thread was addressed in this paper. Since the suture thread is highly deformable, any tracking algorithm must be robust to intersections, occlusions, knot tying, and length changes. The detection algorithm introduced in this paper is capable of distinguishing different threads when they intersect. The tracking algorithm presented here demonstrates that it is possible, using polynomial curves, to track a suture thread as it deforms, becomes occluded, changes length, and even ties a knot in Manuscript received February 12, 2017; accepted July 5, 2017 This paper has supplementary downloadable multimedia material available at http://ieeexplore.ieee.org provided by the authors. The Supplementary Material contains a video that demonstrates the successful initialization and tracking of a surgical suture thread. The first video segment shows how the thread is initialized even when it overlaps a different thread. While both threads are segmented, only the thread of interest is tracked. The second segment demonstrates how the thread tracking is robust against changing thread lengths as well as tightening of a suture knot. The third segment demonstrates the tracking of a printed calibration pattern. The fourth segment demonstrates tracking a simulated suture thread with a surgical background under occlusion. The last segment validates that tracking the thread allows for robotic manipulation. This is completed by successful grasping and pulling a surgical suture thread. This material is 8.16 MB in size. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.
Algorithms that use a stereo image for detection, initialization, and tracking of a suture thread through fitting a nonuniform rational B-spline (NURBS) curve allowed real time tracking a suture thread undergoing translations and deformation REF .
Real-Time Visual Tracking of Dynamic Surgical Suture Threads
IEEE Transactions on Automation Science and Engineering
IEEE Transactions on Automation Science and Engineering
Computer Science Medicine