Source: https://patents.google.com/patent/US20070293734
Timestamp: 2018-04-23 23:11:08
Document Index: 361814396

Matched Legal Cases: ['Application No. 60', 'art 300', 'art 300', 'art 300', 'Application No. 60', 'Application No. 60']

US20070293734A1 - Methods and apparatus for surgical planning - Google Patents
US20070293734A1
US20070293734A1 US11677747 US67774707A US2007293734A1 US 20070293734 A1 US20070293734 A1 US 20070293734A1 US 11677747 US11677747 US 11677747 US 67774707 A US67774707 A US 67774707A US 2007293734 A1 US2007293734 A1 US 2007293734A1
US11677747
US8170716B2 (en )
This non-provisional application claims the benefit of priority from U.S. Provisional Patent Application No. 60/296808, filed Jun. 7, 2001, the full disclosure of which is incorporated herein by reference.
Traditional forms of minimally invasive surgery typically include endoscopy, which is visual examination of a hollow space with a viewing instrument called an endoscope. Minimally invasive surgery with endoscopy may be used in many different areas in the human body for many different procedures, such as in laparoscopy, which is visual examination and/or treatment of the abdominal cavity, or in minimally invasive heart surgery, such as coronary artery bypass grafting. In traditional laparoscopic surgery, for example, a patient's abdominal cavity is insuflated with gas and cannula sleeves (or “entry ports”) are passed through small incisions in the musculature of the patient's abdomen to provide entry ports through which laparoscopic surgical instruments can be passed in a sealed fashion.
Such incisions are typically about ½ inch (about 12 mm) in length.
FIG. 7 d is a perspective view of a computer validation of a surgical procedure as shown in FIG. 7 b,
Master control station 200, assistant controller 200A, cart 300, auxiliary cart 300A, and assistant display 14 (or subsets of these components) may allow complex surgeries to be performed by selectively handing off control of one or more robotic arms between operator O and one or more assistants. Alternatively, operator O may actively control two surgical tools while a third remains at a fixed position. For example, to stabilize and/or retract tissues, with the operator selectively operating the retracting or stabilizer only at designated times. h still further alternatives, a surgeon and an assistant can cooperate to conduct an operation without either passing control of instruments or being able to pass control of instruments with both instead manipulating his or her own instruments during the surgery.
Finally, FIG. 2 shows a processor 400 coupled with master control station 200 and cart 300 and a tangible medium 410 embodying machine readable code, or software. The software typically includes instructions which enable various embodiments of the methods of the present invention. The tangible medium 410 may be coupled with the processor 400 for use. Generally, the software may be used with any suitable hardware, such as a personal computer work station with graphics capabilities, such as but not limited to a PENTIUM Ill: or equivalent processor with a GEFORCE2® graphics card. Other hardware which may be used with software of the present invention includes a display monitor, such as a 17″ monitor, a processor with 256 Mbytes of RAM and a 10 Gigabytes hard disk. Input devices will typically include a mouse and may also include a 3D mouse or a PHANTOM® arm.
Although in some embodiments, as just described, hardware will include a stand-alone PC workstation or similar stand-along hardware, other embodiments will be integrated with an existing system. For example, hardware may be embedded in a dedicated apparatus such as a robotic surgical system. In one embodiment, hardware is embedded in a part of DAVINCI™ robotic system (Intuitive Surgical, nc., Sunnyvale, Calif.) such as the master control station 200.
The robotic manipulator arms will move and articulate the surgical tools in response to motions of the input devices at the workstation, so that the surgeon can direct surgical procedures at internal surgical sites through minimally invasive surgical apertures. The workstation 200 is typically used within an operating room with the cart, but can be positioned remote from the cart, even miles away. An exemplary master control input device for manipulation by the surgeon is more fully described in co-pending U.S. patent application Ser. No. 09/398,507, entitled “Master Having Redundant Degrees of Freedom,” as filed on Sep. 17, 1999, the full disclosure of which is incorporated herein by reference. Exemplary manipulator arms are more fully described in co-pending U.S. patent application Ser. No. 09/368,309 as filed on Aug. 3, 1999, for a “Mancipulator Positioning Linkage for Robotic Surgery,” (the full disclosure of which is also incorporated herein by reference), which also describes manually positionable linkages supporting the manipulators. It should be noted that a number of alternative robotic manipulator arms might be used, including those described in U.S. Pat. No. 5,855,583, the full disclosure of which is also incorporated herein by reference.
Preliminary data processing 110 generally includes processing imaging data, such as radiological data from computed tomography (CT) and/or magnetic resonance imaging (MIR) scans. Such processing may include segmentation, 3D reconstruction, robot modeling and/or the like. Planning 120 generally includes choosing locations for two or more entry ports into a defined volumetric space, such as a patient, for allowing entry of surgical tools, robotic tools or arms, one or more endoscopes, retractors, and/or the like. Typically, planning 120 involves combining data in an optimization algorithm where mathematical criteria have been integrated. The criteria translate features such as collision avoidance between the manipulator arms and reachability of targeted organs. Validation 130 refers to a process of testing the feasibility of the operation by reproducing the expected movements of the surgeon and looking for collisions or other problems, such as an out of reach condition. Finally, simulation 140 allows a surgeon or other user to use the chosen entry ports and robot position to perform a practice operation. In many embodiments, if the surgeon judges the proposed ports and/or robot position less than optimal, the surgeon may reject the chosen locations and new ones may be chosen by the system.
Each of the steps or processes described above may involve various components or steps in various embodiments. For a more detailed discussion of each step, see the master's thesis of Louai Adhami, attached as Exhibit C to U.S. Provisional Patent Application No. 60/296808, previously incorporated by reference. For example, with reference to FIG. 4, some embodiments include multiple stages or steps at the preliminary data acquisition 110 phase. In one embodiment, for example, steps include data acquisition 112, segmentation 114, reconstruction 116 and robot modeling 118. Again, in various embodiments these steps may be carried out in any suitable order and/or steps may be added, eliminated, and/or carried out simultaneously.
Segmentation 114 generally first involves separating out different anatomical entities within the defined volume of the operation, such as various anatomical organs and tissues within a patient. For a TECAB procedure, for example, bones (such as ribs), heart, and left inferior mammary artery (LIMA) are typically segmented. Segmentation of bones from surrounding soft tissues is automatically performed, based on the significantly higher density of the bones, by the “extractcontour” computer software. (“Extractcontour” is software developed by JNRA Sophia Antipolis, and is in the public domain and available from INRIA.) Heart and LIMA segmentation are generally performed manually, such as by a radiologist or other suitable technician. The LIMA is approximated by a fixed-size circle on each CT slice, in an area specified manually. The heart is approximated by splines built around a set of points that are manually drawn. Typically, this process is invariant from one patient to another, meaning that it does not require adjustments by a radiologist or other radiology technician between patients.
Another part of the segmentation step 114 is to define admissible points for entry into the defined volume, as well as admissible directions for entry. In other words a list is compiled of possible entry points and directions, Admissible points of entry are sites on a surface of the volume that allow the introduction of robot arms, an endoscope, and/or any other tools to be used for the operation. In a TECAB operation, for example, admissible points may include any points within the intercostal spaces (spaces between the ribs) of a patient. Points which would cause a tool to pass through bone, such as a rib, are typically eliminated as not being admissible. Admissible directions are directions generally pointing outward and perpendicular to the skin, which replicate directions of orientation that robotic arms, endoscopes and the like will have during the operation.
Another component in preliminary data processing 110 is reconstruction 16. Reconstruction 116 generally refers to formation of acquired, segmented data into a 3-dimensional model of the defined volume which will be operated upon. Generally, such 3D models are constructed using computer software, such as the nuages software, described in Bernhard. Geiger, “Three Dimensional Modeling of Human Organs and its Application to Diagnosis and Surgical Planning,” Technical Report 2105, IA-Sophia, 1993, the entire contents of which is hereby incorporated by reference. A public version of nuages software is available at ftp:/ftp-sop.inria.fr/prisme/NUAGES/Nuages. Again, this software may be run on conventional, off-the-shelf hardware, such as a PENTIM III® processor. The underlying algorithm used for reconstruction 116 via nuages software is based on projected {circumflex over (V)}oronoï diagrams, where the input is a set of closed non-intersecting contours, and the output is a mesh of triangles representing the reconstructed surface in 3D. This algorithm has the advantages of outputting a relatively low, manageable number of triangles and of not being prone to distortive effects such as the staircase effect observed in marching cubes algorithms.
Another aspect of preliminary data processing 111, in some embodiments, includes robot modeling 118. Generally, robot modeling 118 involves combining a geometric model of a robot with the acquired radiological data from the patient or other defined volume in an interactive interface. In the preliminary phase, for example, Denavit-Hartenberg (DH) models may be used, along with a generic C++ library, where OPENGL™ output and collision detection are implemented. In one embodiment two primitives are retained for the modeling of the robot body, namely rectangular parallelepipeds and cylinders. Part of robot modeling 118 typically includes using inverse kinematics, either analytically or numerically, to detect possible interferences between links of the robot. In other words, collision detection is carried out. For efficiency purposes, a dedicated interference detection method may include a hierarchical method based on direct collision tests between the different modeling primitives (cylinders and rectangular parallelepipeds), in addition to spheres. This method can be extended accordingly if the model is refined with more complex primitives. An analytic solution is used when there is the same number of degrees of freedom (dofs) and constraints, whereas a numerical solution is used when there are more dofs than constraints. In the latter case, artificial constraints are added to reflect the proximity between the arms, which would be of great significance when dealing with the problem of collision avoidance,
In another criteria, an admissible point may be eliminated if an angle 222 between a surface of the patient at the entry point and a line from the entry point to the target point 212 is too large, such that use of a tool through that entry point may cause damage to a nearby structure. Use of such an entry point in a heart operation, for example, may cause damage to a rib. Yet another criteria which might be used to eliminate an admissible point would be if the combination of the admissible entry point, surgical tool, attack direction and target point would result in the tool passing through an anatomic structure. For example, if the tool would pass through a lung on its way to the heart, that admissible point would be eliminated. Computer graphics hardware may be used to perform this test in a method similar to that described in “Real-time Collision Detection for Virtual Surgery,” by J. -C. Lombardo, M. P. Cani and F. Neyret, Computer Animation, Geneva, May 1999, the entire contents of which is hereby incorporated by reference.
Criteria such as those described above may be applied in various orders and by various means. In one embodiment, for example, identifying an advantageous triplet of entry port locations is accomplished in two basic steps: First an entry port for an endoscope is chosen based on various criteria, then admissible entry port locations for two (or another number) tools are ranked according to their combined quantitative grade and their position with respect to the endoscope. More precisely, the triplet (of endoscope port and two tool ports) is ranked to provide a desirable synametry between two robot arms and the endoscope, and to favor positions of the robot arms at maximum distances from the endoscope to provide the surgeon with a clear field of view.
Applying criteria in this way may involve several steps. For example, in one embodiment a first step involves eliminating admissible entry port location candidates that will not provide access to the target areas. In a next step, admissible sites for an endoscope are sorted to minimize the angle between the target normal and the line connecting the admissible point to the target point. This step gives precedence to entry ports for the endoscope that provide a direct view over the target areas and, therefore, ports which would create angles greater than a desired camera angle are eliminated. In applying such criteria, targets areas may be weighted according to their relative sizes. For robot arms, admissible entry points may be sorted in the same way as for the endoscope, but with the angle limitation relaxed. Admissible candidates that make too obtuse an angle between the tool and the skin may be eliminated. For example, an maximum angle of 600 may be chosen in a heart operation to avoid excessive stress on the ribs. Finally, a triplet combination of three entry ports (or however many entry ports are to by used) may be chosen to optimize the criteria discussed above while also maximizing the distances between the ports. This distance maximization criteria will prevent collision between robot arms and allow the surgeon to operate the robotic arms with a relatively wide range of movement.
Simulation 140 generally provides a surgeon or other operator of a robotic system an environment in which to practice a given operation to develop facility with the robot and to re-validate the selected, advantageous entry port locations, Generally, simulation 140 is carried out using robotic control mechanisms, a computer with a monitor, and computer software to enable the simulation. Thus, the validation 130 step just described and the simulation step 140 are typically carried out using computerized systems. Using simulation 140, a surgeon can essentially perform the operation as it would be performed on a live patient, as simulated on a 3-dimensional representation on a computer monitor, to practice use of the robotic system and to confirm that the selected combination of robot position and entry port locations is feasible.
As with the validation step 130, simulation 140 typically includes collision detection for possible collisions between the robot arms. Collisions are typically stratified as internal (between the manipulators) and external (with the anatomical entities), Internal collisions may be further divided into static and dynamic (continuous movement) collisions. In most embodiments, an algorithm is used to detect possible internal collisions, the algorithm detecting interferences between rectangular parallelepipeds and cylinders. FIG. 6, for example, shows a diagram of an internal collision detection logic which may be used. In that logic, testing for an intersection between two boxes is accomplished by looking for an overlap between one of the boxes and the sides of the other. The same can be done for two cylinders or between a box and a cylinder. For further details regarding this algorithm, see master's thesis of Louai Adhami, attached as Exhibit C to U.S. Provisional Patent Application No. 60/296808, which has previously been incorporated by reference. (Also see doctoral thesis of Louai Adhami, available from INRIA Sophia after Jul. 3, 2002.)
External collisions, on the other hand, are typically detected using graphics hardware and a method such as that suggested in “Real-time Collision Detection for Virtual Surgery,” by J. -C. Lombardo, M. P. Cani and F. Neyret, Computer Animation, Geneva, May 1999, previously incorporated herein by reference. Sufficient graphics hardware may include, for example, a personal computer work station with graphics capabilities, such as a PENTIUM III® or equivalent processor with a GEFORCE2® graphics card, and any suitable monitor.
Step 9: Defining admissible port set. This step involves defining a set of admissible ports for target and/or surgical procedure in relation to patient model, includes the entry point location and normal direction for each port. The choice of the admissible locations set stems from the characteristics of the intervention and/or anatomy of the patient, and is meant to cover all possible entry points from which optimal ports are to be chosen, Determining this set can either be done empirically or automatically using specialized segmentation algorithms.
Step 11: Determining optimized multiple-port combination. Step 11 may include applying an optimization algorithm to calculated optimization criteria for all feasible port combinations of the total arm number (e.g., all feasible 3-port combinations or triplets). Step 11 may also include adding more ports than arms for surgeon assistance (e.g. cardiac stabilizer), The total number of ports is often referred to as n-tuplet. Several ports may be chosen for the same arm (e.g. two different non-simultaneous positions of the endoscope). Alternatively, step 11 may include pre-selecting an endoscope port, and then optimizing other ports by considering all combinations of remaining feasible ports (in example below, remaining feasible port pairs), as in the following sub-steps:
3. optimizing tool(s) and/or endoscope(s) combinations. For 1 endoscope+2 tools, each combination is commonly referred to as a triplet. More generally, for 1 endoscope+n−1 tools, each combination is referred to as an n-tuplet. Note that the combination may include more than one endoscope, or an integrated multifunctional endoscope/tool. This step may include, for example, calculating the optimization criteria for each port combination; calculating cost function value; ranking the n-tuplet by cost function value; and selecting the n-tuplet which has the best cost function value,
Step 21; Transferring and registering planning results to patient body and surgical system. For both robotic and non-robotic surgical procedures, the results of planning are transferred to the patient, The model of the planned procedure may be registered to the patient's body in the operating room. Transfer and registration may include the marking of port locations, and reproducing the planned initial positions and alignment of the instruments and/or robotic arms. For example, the following sub-steps may be used:
53.A method for planning a surgical procedure, comprising:
receiving information of a target area in which a surgical procedure is to be performed within a patient;
receiving information for a plurality of robotic arms individually manipulatable in a plurality of degrees of freedom movement and individually adapted to hold a surgical device for performing the surgical procedure; and
determining whether any one of the plurality of robotic arms may collide with another one of the plurality of robotic arms during the surgical procedure using the received information of the target area and the received information of the plurality of robotic arms,
determining a plurality of entry points into the patient for inserting surgical devices held by the plurality of robotic arms so as avoid collisions between any two or more of the plurality of robotic arms during the surgical procedure.
determining set-up positions for the plurality of robotic arms so as avoid collisions between any two or more of the plurality of robotic arms during the surgical procedure.
56. The method according to claim 53, wherein the determination of whether any one of the plurality of robotic arms may collide with another one of the plurality of robotic arms during the surgical procedure, comprises:
simulating the surgical procedure by manipulating the plurality of robotic arms so as to perform the surgical procedure while viewing a virtual image of the target area generated from the received information of the target area.
57. The method according to claim 56, wherein the received information of the plurality of robotic arms includes information of a first set of entry points into the patient for inserting surgical devices held by the plurality of robotic arms, and further comprising:
re-simulating the surgical procedure for a second set of entry points having at least one entry point different than the first set of entry points, if a collision between any two or more of the plurality of robotic arms occurs during the simulation of surgical procedure using the first set of entry points.
58. The method according to claim 56, wherein the received information of the plurality of robotic arms includes information of a first set of set-tip positions for the plurality of robotic arms, and further comprising:
re-simulating the surgical procedure for a second set of set-up positions for the plurality of robotic arms having at least one set-up position different than the first set of set-up positions, if a collision between any two or more of the plurality of robotic arms occurs during the simulation of surgical procedure using the first set of set-up positions.
59. The method according to claim 58, wherein the first set of set-up positions is randomly drawn from corresponding articular spaces of the plurality of robotic arms.
60. The method according to claim 53, wherein the determination of whether any one of the plurality of robotic arms may collide with another one of the plurality of robotic arms during the surgical procedure, comprises:
validating the surgical procedure by executing a computer program including an interference detection algorithm that test sweeps a volume covered by the plurality of robotic arms to detect a possibility of collision between any two or more of the plurality of robotic arms during the surgical procedure.
61. A robotic surgical system comprising:
a processor configured to determine whether any two or more of the plurality of robotic arms may collide during the surgical procedure using information of a target area in which the surgical procedure is to be performed within a patient, and information mechanically characterizing the plurality of robotic arms.
62. The robotic surgical system according to claim 61, wherein the processor is further configured to determine a plurality of entry points into the patient for inserting surgical devices held by the plurality of robotic arms so as avoid collisions between any two or more of the plurality of robotic arms during the surgical procedure.
63. The robotic surgical system according to claim 61, wherein the processor is further configured to determine set-up positions for the plurality of robotic arms so as avoid collisions between any two or more of the plurality of robotic arms during the surgical procedure.
64. The robotic surgical system according to claim 61, wherein the processor is further configured to facilitate simulation of the surgical procedure by a surgeon in order to determine whether any two or more of the plurality of robotic arms may collide during the surgical procedure.
65. The robotic surgical system according to claim 64, wherein the received information of the plurality of robotic arms includes information of a first set of entry points into the patient for inserting surgical devices held by the plurality of robotic arms, and the processor is further configured to facilitate re-simulation of the surgical procedure for a second set of entry points having at least one entry point different than the first set of entry points.
66. The robotic surgical system according to claim 64, wherein the received information of the plurality of robotic arms includes information of a first set of set-up positions for the plurality of robotic arms, and the processor is further configured to facilitate re-simulation of the surgical procedure for a second set of set-up positions for the plurality of robotic arms having at least one set-up position different than the first set of set-up positions.
67. The robotic surgical system according to claim 66, wherein the processor is further configured to randomly draw the first set of set-up positions from corresponding articular spaces of the plurality of robotic arms.
68. The robotic surgical system according to claim 61, wherein the processor is configured to determine whether any one of the plurality of robotic arms may collide with another one of the plurality of robotic arms during the surgical procedure by executing a computer program including an interference detection algorithm that test sweeps a volume covered by the plurality of robotic arms to detect a possibility of collision between any two or more of the plurality of robotic arms during the surgical procedure.
69. A method for planning a surgical procedure, comprising:
determining whether or not one or more of the plurality of robotic arms is unable to be manipulated as required to perform the surgical procedure by simulating the surgical procedure using the received information of the target area and the received information of the plurality of robotic arms.
70. The method according to claim 69, wherein one of the plurality of robotic arms is determined to be unable to be manipulated as required to perform the surgical procedure if the robotic arm is commanded to exceed its mechanical range of motion while simulating the surgical procedure.
71. The method according to claim 69, wherein one of the plurality of robotic arms is determined to be unable to be manipulated as required to perform the surgical procedure if the robotic arm collides with another of the plurality of robotic arms while simulating the surgical procedure.
72. A robotic surgical system comprising:
a processor configured to facilitate simulating the surgical procedure using information of a target area in which the surgical procedure is to be performed within a patient, and information mechanically characterizing the plurality of robotic arms to determine whether or not one or more of the plurality of robotic arms is unable to be manipulated as required to perform the surgical procedure.
73. A robotic surgical system comprising:
a processor configured to validate the surgical procedure using information of a target area in which the surgical procedure is to be performed within a patient, and information mechanically characterizing the plurality of robotic arms to determine whether or not one or more of the plurality of robotic arms is unable to be manipulated as required to perform the surgical procedure.
US11677747 2001-06-07 2007-02-22 Methods and apparatus for surgical planning Active 2024-10-16 US8170716B2 (en)
US10165413 Division US7607440B2 (en) 2001-06-07 2002-06-06 Methods and apparatus for surgical planning
US13432305 Continuation US8571710B2 (en) 2001-06-07 2012-03-28 Methods and apparatus for surgical planning
US20070293734A1 true true US20070293734A1 (en) 2007-12-20
US8170716B2 US8170716B2 (en) 2012-05-01
WO2009104852A1 (en) * 2008-02-20 2009-08-27 Meerecompany Bed integrated surgical robot
US20120116416A1 (en) * 2010-11-08 2012-05-10 Kuka Laboratories Gmbh Medical Workstation
US20130085380A1 (en) * 2010-11-10 2013-04-04 Perfint Healthcare Private Limited Systems and methods for planning image guided interventional procedures
EP3288720A1 (en) * 2015-05-01 2018-03-07 Titan Medical Inc. Instrument collision detection and feedback
US20100286712A1 (en) * 2008-02-20 2010-11-11 Jong Seok Won Bed integrated surgical robot
US8828023B2 (en) * 2010-11-08 2014-09-09 Kuka Laboratories Gmbh Medical workstation
US7607440B2 (en) 2009-10-27 grant
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTUITIVE SURGICAL, INC.;REEL/FRAME:027945/0706