Source: http://www.google.com/patents/US20030109780?dq=6480844
Timestamp: 2017-04-25 10:08:10
Document Index: 223897814

Matched Legal Cases: ['art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300']

Patent US20030109780 - Methods and apparatus for surgical planning - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsMethods and apparatus for enhancing surgical planning provide enhanced planning of entry port placement and/or robot position for laparoscopic, robotic, and other minimally invasive surgery. Various embodiments may be used in robotic surgery systems to identify advantageous entry ports for multiple robotic...http://www.google.com/patents/US20030109780?utm_source=gb-gplus-sharePatent US20030109780 - Methods and apparatus for surgical planningAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS20030109780 A1Publication typeApplicationApplication numberUS 10/165,413Publication dateJun 12, 2003Filing dateJun 6, 2002Priority dateJun 7, 2001Also published asUS7607440, US8170716, US8571710, US9532838, US20070293734, US20120253515, US20140135985Publication number10165413, 165413, US 2003/0109780 A1, US 2003/109780 A1, US 20030109780 A1, US 20030109780A1, US 2003109780 A1, US 2003109780A1, US-A1-20030109780, US-A1-2003109780, US2003/0109780A1, US2003/109780A1, US20030109780 A1, US20030109780A1, US2003109780 A1, US2003109780A1InventorsEve Coste-Maniere, Louai Adhami, Jean-Daniel Boissonnat, Alain Carpentier, Gary GuthartOriginal AssigneeInria Roquencourt, Intuitive Surgical, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (11), Referenced by (135), Classifications (16), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetMethods and apparatus for surgical planning
DETAILED DESCRIPTION OF THE INVENTION [0035] The present invention generally provides methods and apparatus for enhancing planning of laparoscopic, robotic, and other minimally invasive surgery. More specifically, various embodiments provide methods and apparatus for planning advantageous locations for placement of two or more entry ports for accessing a defined volume, such as a patient, with surgical tools to perform a minimally invasive operation. In robotic surgery, robot position will typically also be planned. Additionally, many embodiments provide validation that selected entry port placements and/or robot positions will be feasible for a given operation. Many embodiments also provide simulation of a given operation, using selected entry port placements and/or robot positions, to allow a surgeon or other user to practice using the surgical system. [0036] Although the following description focuses on planning port placement and robot position in a robotic surgery context, specifically in a heart surgery context, many other applications are contemplated within the scope of the invention. As mentioned above, for example, various embodiments may be used in other surgical contexts, such as non-robotic laparoscopic/abdominal surgery, arthroscopy, retroperitoneoscopy, pelviscopy, nephroscopy, cystoscopy, cistemoscopy, sinoscopy, hysteroscopy, urethroscopy, and the like. Furthermore, non-surgical applications are contemplated, including but not limited to handling, disabling, maintaining and/or the like of munitions, hazardous materials, and/or other suitable materials. Within the surgical context, methods and apparatus of the present invention may be used with many different systems for conducting robotic or minimally invasive surgery. One example of a robotic surgical system which may incorporate methods and apparatus of the present invention is the DAVINCI™ system available from Intuitive Surgical, Inc. of Mountain View, Calif. Many other surgical systems and apparatus may be used, however. Therefore, the following description is provided for exemplary purposes only and should not limit the scope of the present invention as set forth in the appended claims. [0037] Referring now to FIG. 1, one example of a robotic surgical system 10, with which the methods and apparatus of the present invention may be used, includes a master control station 200 and a slave cart 300. Optionally, any of several other additional components may be included in the surgical system to enhance the capabilities of the robotic devices to perform complex surgical procedures. An operator O performs a minimally invasive surgical procedure at an internal surgical site within patient P using minimally invasive surgical instruments 100. Operator O works at master control station 200. Operator O views a display provided by the workstation and manipulates left and right input devices. The telesurgical system moves surgical instruments mounted on robotic arms of slave cart 300 in response to movement of the input devices. As described in co-pending U.S. patent application Ser. No. 09/436,527, filed on Dec. 14, 2001 (Attorney Docket No. 17516-002530), the full disclosure of which is incorporated herein by reference, a selectively designated “left” instrument is associated with the left input device in the left hand of operator O and a selectively designated “right” instrument is associated with the right input device in the right hand of the operator. [0038] As described in more detail in co-pending U.S. patent application Ser. No. 09/373,678 entitled “Camera Reference Control in a Minimally Invasive Surgical Apparatus,” filed Aug. 13, 1999 (the full disclosure of which is incorporated herein by reference) a processor of master controller 200 will preferably coordinate movement of the input devices with the movement of their associated instruments, so that the images of the surgical tools, as displayed to the operator O, appear substantially connected to the input devices in the hand of the operator. [0039] Optionally, an auxiliary cart A can support one or more additional surgical tools 100 for use during the procedure. One tool is shown here for the illustrative purposes only. A first assistant A1 is seated at an assistant control station 200A, the first assistant typically directing movement of one or more surgical instruments not actively being manipulated by operator O via master control station 200, such as a tissue stabilizer. A second assistant A2 may be disposed adjacent patient P to assist in swapping instruments 100 during the surgical procedure. Auxiliary cart A may also include one or more assistant input devices 12 (shown here as a simple joystick) to allow second assistant A2 to selectively manipulate one or more surgical instruments while viewing the internal surgical site via an assistant display 14. Preferably, the first assistant A1 seated at console 200A has the same image as the surgeon seated at console 200. [0040] Master control station 200, assistant controller 200A, cart 300, auxiliary cart 300A, and assistant display 14 (or subsets of these components) may allow complex surgeries to be performed by selectively handing off control of one or more robotic arms between operator O and one or more assistants. Alternatively, operator O may actively control two surgical tools while a third remains at a fixed position. For example, to stabilize and/or retract tissues, with the operator selectively operating the retracting or stabilizer only at designated times. In still further alternatives, a surgeon and an assistant can cooperate to conduct an operation without either passing control of instruments or being able to pass control of instruments with both instead manipulating his or her own instruments during the surgery. [0041] Although FIG. 1 depicts two surgeon consoles controlling the two cart structures, a preferred embodiment comprises only one console controlling four or more arms on two carts. The scope may optionally be mounted on the auxiliary cart, and three tissue manipulator arms may be mounted on the main cart. In some embodiments, one or more tools, particularly tissue stabilizers, may not be actively driven, instead being positioned by manually actuating a drive system of the tool and then locking the tool into position. [0042] Referring now to FIG. 2, master control station 200 includes a viewer 202 wherein an image of a surgical site is displayed in use. A support 204 is provided on which the operator, typically a surgeon, can rest his or her forearms while gripping two master controls, one in each hand. Master controls are positioned in a workspace 206 disposed inwardly beyond support 204. When using workstation 100, the surgeon typically sits in a chair in front of the workstation, positions his or her eyes in front of the viewer 202 and grips the master controls. [0043] [0043]FIG. 2 shows also the surgical manipulator slave or cart 300 of the telesurgical system. In use, cart 300 is positioned close to a patient for surgery, and the base of the cart is caused to remain stationary until the surgical procedure has been completed. Cart 300 here includes three robotic manipulator arm assemblies 302, each manipulator supporting an instrument 100. More specifically, one of the robotic arm assemblies supports an image capture device, such as an endoscope 306 (which is coupled to display 102 of the workstation). Each of the other two manipulator arms supports a tissue manipulation tool 308 having a surgical end effector for treating tissue. [0044] Finally, FIG. 2 shows a processor 400 coupled with master control station 200 and cart 300 and a tangible medium 410 embodying machine readable code, or software. The software typically includes instructions which enable various embodiments of the methods of the present invention. The tangible medium 410 may be coupled with the processor 400 for use. Generally, the software may be used with any suitable hardware, such as a personal computer work station with graphics capabilities, such as but not limited to a PENTIUM III® or equivalent processor with a GEFORCE2® graphics card. Other hardware which may be used with software of the present invention includes a display monitor, such as a 17″ monitor, a processor with 256 Mbytes of RAM and a 10 Gigabytes hard disk. Input devices will typically include a mouse and may also include a 3D mouse or a PHANTOM® arm. [0045] Although in some embodiments, as just described, hardware will include a stand-alone PC workstation or similar stand-along hardware, other embodiments will be integrated with an existing system. For example, hardware may be embedded in a dedicated apparatus such as a robotic surgical system. In one embodiment, hardware is embedded in a part of DAVINCI® robotic system (Intuitive Surgical, Inc., Sunnyvale, Calif.) such as the master control station 200. [0046] The robotic manipulator arms will move and articulate the surgical tools in response to motions of the input devices at the workstation, so that the surgeon can direct surgical procedures at internal surgical sites through minimally invasive surgical apertures. The workstation 200 is typically used within an operating room with the cart, but can be positioned remote from the cart, even miles away. An exemplary master control input device for manipulation by the surgeon is more fully described in co-pending U.S. patent application Ser. No. 09/398,507, entitled “Master Having Redundant Degrees of Freedom,” as filed on Sep. 17, 1999, the full disclosure of which is incorporated herein by reference. Exemplary manipulator arms are more fully described in co-pending U.S. patent application Ser. No. 09/368,309 as filed on Aug. 3, 1999, for a “Manipulator Positioning Linkage for Robotic Surgery,” (the full disclosure of which is also incorporated herein by reference), which also describes manually positionable linkages supporting the manipulators. It should be noted that a number of alternative robotic manipulator arms might be used, including those described in U.S. Pat. No. 5,855,583, the full disclosure of which is also incorporated herein by reference. [0047] Referring now to FIG. 3, a method for enhancing port placement 12 suitably includes four general steps or stages. In various alternative embodiments, certain steps may be combined, other steps may be added, and/or one or more steps may be eliminated, without significantly changing the overall result. That being said, four general stages used to plan entry port placement may include preliminary data processing 110, planning 120, validation 130 and simulation 140. [0048] Preliminary data processing 110 generally includes processing imaging data, such as radiological data from computed tomography (CT) and/or magnetic resonance imaging (MRI) scans. Such processing may include segmentation, 3D reconstruction, robot modeling and/or the like. Planning 120 generally includes choosing locations for two or more entry ports into a defined volumetric space, such as a patient, for allowing entry of surgical tools, robotic tools or arms, one or more endoscopes, retractors, and/or the like. Typically, planning 120 involves combining data in an optimization algorithm where mathematical criteria have been integrated. The criteria translate features such as collision avoidance between the manipulator arms and reachability of targeted organs. Validation 130 refers to a process of testing the feasibility of the operation by reproducing the expected movements of the surgeon and looking for collisions or other problems, such as an out of reach condition. Finally, simulation 140 allows a surgeon or other user to use the chosen entry ports and robot position to perform a practice operation. In many embodiments, if the surgeon judges the proposed ports and/or robot position less than optimal, the surgeon may reject the chosen locations and new ones may be chosen by the system. [0049] Each of the steps or processes described above may involve various components or steps in various embodiments. For a more detailed discussion of each step, see the master's thesis of Louai Adhami, attached as Exhibit C to U.S. Provisional Patent Application Ser. No. 60/296808, previously incorporated by reference. For example, with reference to FIG. 4, some embodiments include multiple stages or steps at the preliminary data acquisition 110 phase. In one embodiment, for example, steps include data acquisition 112, segmentation 114, reconstruction 116 and robot modeling 118. Again, in various embodiments these steps may be carried out in any suitable order and/or steps may be added, eliminated, and/or carried out simultaneously. [0050] Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement. [0051] Segmentation 114 generally first involves separating out different anatomical entities within the defined volume of the operation, such as various anatomical organs and tissues within a patient. For a TECAB procedure, for example, bones (such as ribs), heart, and left inferior mammary artery (LIMA) are typically segmented. Segmentation of bones from surrounding soft tissues is automatically performed, based on the significantly higher density of the bones, by the “extractcontour” computer software. (“Extractcontour” is software developed by INRIA Sophia Antipolis, and is in the public domain and available from INRIA.) Heart and LIMA segmentation are generally performed manually, such as by a radiologist or other suitable technician. The LIMA is approximated by a fixed-size circle on each CT slice, in an area specified manually. The heart is approximated by splines built around a set of points that are manually drawn. Typically, this process is invariant from one patient to another, meaning that it does not require adjustments by a radiologist or other radiology technician between patients. [0052] Another part of the segmentation step 114 is to define admissible points for entry into the defined volume, as well as admissible directions for entry. In other words a list is compiled of possible entry points and directions. Admissible points of entry are sites on a surface of the volume that allow the introduction of robot arms, an endoscope, and/or any other tools to be used for the operation. In a TECAB operation, for example, admissible points may include any points within the intercostal spaces (spaces between the ribs) of a patient. Points which would cause a tool to pass through bone, such as a rib, are typically eliminated as not being admissible. Admissible directions are directions generally pointing outward and perpendicular to the skin, which replicate directions of orientation that robotic arms, endoscopes and the like will have during the operation. [0053] Another component in preliminary data processing 110 is reconstruction 116. Reconstruction 116 generally refers to formation of acquired, segmented data into a 3-dimensional model of the defined volume which will be operated upon. Generally, such 3D models are constructed using computer software, such as the nuages software, described in Bernhard. Geiger, “Three Dimentional Modeling of Human Organs and its Application to Diagnosis and Surgical Planning,” Technical Report 2105, INRIA-Sophia, 1993, the entire contents of which is hereby incorporated by reference. A public version of nuages software is available at ftp://Hftp-sop.inria.fr/prisme/NUAGES/Nuages. Again, this software may be run on conventional, off-the-shelf hardware, such as a PENTIUM III® processor. The underlying algorithm used for reconstruction 116 via nuages software is based on projected Voronoï diagrams, where the input is a set of closed non-intersecting contours, and the output is a mesh of triangles representing the reconstructed surface in 3D. This algorithm has the advantages of outputting a relatively low, manageable number of triangles and of not being prone to distortive effects such as the staircase effect observed in marching cubes algorithms. [0054] Another aspect of preliminary data processing 110, in some embodiments, includes robot modeling 118. Generally, robot modeling 118 involves combining a geometric model of a robot with the acquired radiological data from the patient or other defined volume in an interactive interface. In the preliminary phase, for example, Denavit-Hartenberg (DH) models may be used, along with a generic C++ library, where OPENGL™ output and collision detection are implemented. In one embodiment two primitives are retained for the modeling of the robot body, namely rectangular parallelepipeds and cylinders. Part of robot modeling 118 typically includes using inverse kinematics, either analytically or numerically, to detect possible interferences between links of the robot. In other words, collision detection is carried out. For efficiency purposes, a dedicated interference detection method may include a hierarchical method based on direct collision tests between the different modeling primitives (cylinders and rectangular parallelepipeds), in addition to spheres. This method can be extended accordingly if the model is refined with more complex primitives.. An analytic solution is used when there is the same number of degrees of freedom (dofs) and constraints, whereas a numerical solution is used when there are more dofs than constraints. In the latter case, artificial constraints are added to reflect the proximity between the arms, which would be of great significance when dealing with the problem of collision avoidance. [0055] With reference again to FIG. 3, after preliminary data processing 110 planning 120 is performed. Planning 120 generally consists of identifying advantageous locations for two or more entry ports for accessing the defined volume to be operated upon. In many embodiments, planning 120 also includes planning one or more positions of a robot and/or its component parts for performing an operation, with the robot positioning being based on the advantageous locations of the two or more entry points. Typically, planning 120 is carried out to identify optimal or advantageous entry port locations for three tools, such as two robot arms and an endoscope. Multiple criteria are generally used to help identify such locations, and the locations are selected from among the admissible entry points described above. Thus, one embodiment involves choosing a “triplet” of three entry points that optimizes a set of predefined criteria. The criteria may be any suitable criteria, such as robot constraints, anatomical constraints, surgeon preferences, and/or the like. [0056] In one embodiment, for example, some criteria are derived from surgeon preferences. For example, a surgeon may specify target points within the patient or other defined volume on which the surgeon wants to operate, such as points on or in a heart in heart surgery. Target points may then be used to define a target area, within which the surgeon wishes to operate. The surgeon also typically defines one or more preferred “attack directions,” which are generally directions from which the surgeon prefers to access the target points. “Attack angles” may be derived from these attack directions. An attack angle is an angle between the attack direction at the target point on the one hand, and the straight line connecting the latter to an admissible point (on a surface of the patient) on the other. It reflects the ease with which the surgeon can operate on a given location with respect to the attack direction chosen by the surgeon. A “dexterity parameter” is another criteria which may be used. The dexterity parameter is proportional to the angle between the surface normal at the admissible point and a straight line connecting the latter to the target point. This measure of dexterity is typically interpreted in accordance with the robot capabilities. [0057] Other criteria which may be used in identifying advantageous locations for entry ports include both qualitative and quantitative criteria. Referring now to FIG. 5, qualitative criteria, for example, may relate to the reachability from an admissible point 210 to a target point 212 , with the admissible point being eliminated from consideration if a tool to be used in the operation is not long enough to reach across distance 214 to reach target point 212. [0058] In another criteria, an admissible point may be eliminated if an angle 222 between a surface of the patient at the entry point and a line from the entry point to the target point 212 is too large, such that use of a tool through that entry point may cause damage to a nearby structure. Use of such an entry point in a heart operation, for example, may cause damage to a rib. Yet another criteria which might be used to eliminate an admissible point would be if the combination of the admissible entry point, surgical tool, attack direction and target point would result in the tool passing through an anatomic structure. For example, if the tool would pass through a lung on its way to the heart, that admissible point would be eliminated. Computer graphics hardware may be used to perform this test in a method similar to that described in “Real-time Collision Detection for Virtual Surgery,” by J.-C. Lombardo, M. P. Cani and F. Neyret, Computer Animation, Geneva, May 1999, the entire contents of which is hereby incorporated by reference. [0059] Quantitative criteria generally relate to dexterity of the robot, where each admissible point is graded based on an angle 212 between the attack direction and the line relating target point 212 to admissible point 210. This measure translates the ease with which the surgeon will be able to operate on target areas from a given port in the case of a robotic tool, or the quality of viewing those areas via an endoscope. [0060] Criteria such as those described above may be applied in various orders and by various means. In one embodiment, for example, identifying an advantageous triplet of entry port locations is accomplished in two basic steps: First an entry port for an endoscope is chosen based on various criteria, then admissible entry port locations for two (or another number) tools are ranked according to their combined quantitative grade and their position with respect to the endoscope. More precisely, the triplet (of endoscope port and two tool ports) is ranked to provide a desirable symmetry between two robot arms and the endoscope, and to favor positions of the robot arms at maximum distances from the endoscope to provide the surgeon with a clear field of view. [0061] Applying criteria in this way may involve several steps. For example, in one embodiment a first step involves eliminating admissible entry port location candidates that will not provide access to the target areas. In a next step, admissible sites for an endoscope are sorted to minimize the angle between the target normal and the line connecting the admissible point to the target point. This step gives precedence to entry ports for the endoscope that provide a direct view over the target areas and, therefore, ports which would create angles greater than a desired camera angle are eliminated. In applying such criteria, targets areas may be weighted according to their relative sizes. For robot arms, admissible entry points may be sorted in the same way as for the endoscope, but with the angle limitation relaxed. Admissible candidates that make too obtuse an angle between the tool and the skin may be eliminated. For example, an maximum angle of 60° may be chosen in a heart operation to avoid excessive stress on the ribs. Finally, a triplet combination of three entry ports (or however many entry ports are to by used) may be chosen to optimize the criteria discussed above while also maximizing the distances between the ports. This distance maximization criteria will prevent collision between robot arms and allow the surgeon to operate the robotic arms with a relatively wide range of movement. [0062] Once a set of advantageous entry port locations has been selected, an advantageous position for placement of the robot to be used, in relation to the patient, is typically determined. Robot positioning will typically be based on the entry port placement and the robot configuration, such that the robot is positioned in a way that avoids collisions between robot arms and allows the arms to function in performing a given operation without violating any of a number of selected constraints. Constraints may include, for example, number of robot arms and number of degrees of freedom for each arm, potential collisions between the robot arms, potential collisions between an arm and the patient, other potential collisions (e.g. with anesthesia equipment or operating room table), and/or miscellaneous constraints (e.g. endoscope orientation for assistant surgeon). Certain constraints may be more subjective, such as surgeon preferences, operating room configurations and the like. [0063] To determine an advantageous robot position, one method uses a combined probabilistic and gradient descent approach, where configurations of the passive joints (including a translation of the base) are randomly drawn in robot articular space. To each configuration, a cost function is associated that depends on the constraints discussed above. A low cost function gives its corresponding robot configuration a high selection probability. This process is repeated until a configuration arrives at a cost function that is less than a given threshold. Once the cost function is low enough (i.e., the passive joints are close enough to the desired port), the active joints are moved over all the targets (using inverse kinematics), to verify that there are no collisions. [0064] Returning to FIG. 3, once a suitable triplet or other configuration of entry port locations has been identified, validation 130 is performed to verify that the identified locations are feasible for carrying out the given operation. In most embodiments, the robot is placed in the position which has been selected and movement of the robot as will be done during the operation is carried out to look for possible collisions between the robot arms. Generally, the trajectory between two target areas is a straight line, and this is the way a surgeon is expected to navigate. The possibility of collisions between discrete time steps is handled by an interference detection algorithm that tests sweeps the volume covered by the manipulator arms. In addition to interference detection, out of reach conditions and possible singularities are monitored and signaled. Finally, the endoscope is positioned relative to the tips of the tool arms at a predefined distance, in a way to guarantee a good visibility at all times. If no problem is detected during validation 130, then the triplet or other configuration is accepted and a surgeon may proceed with simulation 140. If there is a collision or other problem, the system may return to the planning stage 120 to select other entry port locations. [0065] Simulation 140 generally provides a surgeon or other operator of a robotic system an environment in which to practice a given operation to develop facility with the robot and to re-validate the selected, advantageous entry port locations. Generally, simulation 140 is carried out using robotic control mechanisms, a computer with a monitor, and computer software to enable the simulation. Thus, the validation 130 step just described and the simulation step 140 are typically carried out using computerized systems. Using simulation 140, a surgeon can essentially perform the operation as it would be performed on a live patient, as simulated on a 3-dimensional representation on a computer monitor, to practice use of the robotic system and to confirm that the selected combination of robot position and entry port locations is feasible. [0066] As with the validation step 130, simulation 140 typically includes collision detection for possible collisions between the robot arms. Collisions are typically stratified as internal (between the manipulators) and external (with the anatomical entities). Internal collisions may be further divided into static and dynamic (continuous movement) collisions. In most embodiments, an algorithm is used to detect possible internal collisions, the algorithm detecting interferences between rectangular parallelepipeds and cylinders. FIG. 6, for example, shows a diagram of an internal collision detection logic which may be used. In that logic, testing for an intersection between two boxes is accomplished by looking for an overlap between one of the boxes and the sides of the other. The same can be done for two cylinders or between a box and a cylinder. For further details regarding this algorithm, see master's thesis of Louai Adhami, attached as Exhibit C to U.S. Provisional Patent Application Ser. No. 60/296808, which has previously been incorporated by reference. (Also see doctoral thesis of Louai Adhami, available from INRIA Sophia after Jul. 3, 2002.) [0067] External collisions, on the other hand, are typically detected using graphics hardware and a method such as that suggested in “Real-time Collision Detection for Virtual Surgery,” by J.-C. Lombardo, M. P. Cani and F. Neyret, Computer Animation, Geneva, May 1999, previously incorporated herein by reference. Sufficient graphics hardware may include, for example, a personal computer work station with graphics capabilities, such as a PENTIUM III® or equivalent processor with a GEFORCE2® graphics card, and any suitable monitor. [0068] Referring now to FIGS. 7a-d, a system for planning, validating, and simulating port placement may first be calibrated, or experimentally adjusted, using a physical, structural modeling system. For example, FIGS. 7a and 7 b show an experimental validation of port placement and robot position for a TECAB procedure using a skeleton rib cage. FIGS. 7c and 7 d show a computerized representation of the same exerpeimental validation. Such physical validation on a skeleton, cadaver, or other structural model will not be necessary in typical use of the methods and apparatus of the present invention. Generally, validation 130 and simulation 140 steps will be carried out using the robotic system and a computerized system, and will not involve physical, structural models. If use of such models were found to be advantageous for general use of various embodiments, however, such models are within the scope of the present invention. [0069] [0069]FIGS. 8a and 8 b show screen shots representing an embodiment of an apparatus for simulation 140 of a surgical procedure. FIG. 8a is a screen shot side view of a computer interface for simulation of a TECAB procedure. The interface may allow a surgeon to manipulate the computerized view of the simulation, to view the simulation from other angles, for example. FIG. 8b is a similar screen shot perspective view of a computer interface for simulation of a TECAB procedure. [0070] Again referring to FIG. 3, once entry port placement and robot positioning have been validated 130 and simulated 140, they may be registered 150. Registration 150 generally refers to a process of transferring the entry port and robot placement from a simulator to an actual defined volume, such as a patient in an operating room (“OR”). In one embodiment, it is assumed that the robot base will be parallel to the OR table, and that the relative tilt between the skeleton used for simulation (or other simulation device) and the OR table is the same as the one in the CT scan. The translational pose is registered by identifying an particular point (such as the tip of the sternum) in the simulator using the endoscope, and reading the corresponding articular values. Then the robot base and its first translational joint (up/down) are moved so that the articular values read through an applications program interface (API) match the computed ones. Generally, an API is an interface including data from robot arm sensors, various other robot sensors, and the like, for registering robot positioning. [0071] Once the robot is registered to the simulation skeleton, positioning the ports may simply be achieved by moving the robot arms according to the precomputed articular values that correspond to having the remote center on the port. On the other hand, the results of the planning can also be expressed as a quantitative description of the positions of the port, for example endoscope arm at third intercostal space at the limit of the cartilage. This is a relatively accurate description since the ports are planned to be located in the intercostal spacing. When entry port locations are identified and the robot is positioned, a surgeon or other operator may begin the procedure. [0072] As described above, one embodiment of the present invention may be described as a method for surgical planning. The following is a more detailed description, set forth in a series of steps, of such an embodiment. Again, this description is provided for exemplary purposes only. In other embodiments, multiple steps may be added, eliminated, combined, performed in different orders, performed simultaneously, and/or the like, without departing from the scope of the invention. Therefore, the following description should not be interpreted to limit the scope of the invention as defined by the appended claims. Example Method [0073] Step 1: Robotic system modeling. Step 1 typically includes defining a model of the insertable surgical tool portion, including structure, range of motion (ROM) limits, and optionally tool-type specific properties. Step 1 also includes defining a model of the external portion or robotic tool and manipulator arm structure and ROM limits. Finally, step 1 includes defining a multiple-arm robotic system model. Optionally, the model may include adjacent OR equipment such as operating table and accessories. [0074] Step 2: Defining port feasibility criteria. Port feasibility may be defined, for example, relative to “reachability” criteria based on patient and tool models. Such criteria may include, for example: maximum acceptable port-to-target distance (i.e., tool shaft length, for endoscope, may include an objective lens offset); maximum acceptable entry angle, the angle between port-to-target path and the port direction—may depend on body wall properties, intercostal spacing, thickness, elasticity, etc., and may have different values in different body regions; determination that port-to-target path is clear of obstructions; and/or additional feasibility criteria (e.g., surgeon preference, procedure specific or tool-type specific criteria). [0075] Step 3: Defining port optimization criteria. Examples of criteria include: tool-to-target attack angle for each port; dexterity parameter; tool 1 to endoscope angle; tool 2 to endoscope angle; symmetry and alignment of Tool 1 and Tool 2 about Endoscope; port-to-port separation distance; and/or additional optimization criteria (e.g., surgeon preference, procedure specific or tool-type specific criteria). [0076] Step 4: Defining port optimization algorithm. For example, a cost function may be defined, wherein the function value is determined by weighted criteria values. [0077] Step 5: Defining collision/interference prediction algorithms. Step 5 involves, for example, defining collision and interference prediction/detection algorithms relative to robotic system and/or patient models. Examples of such algorithms may include: internal tool-tool and/or tool-organ collisions; external arm-arm, arm-equipment and/or arm-patient collisions. [0078] Step 6: Defining operative motion prediction algorithms. This step involves defining a predictive model of expected range of surgeon-commanded operational tool movements during surgical task (generic task, specific procedural and/or tool-types). [0079] Step 7: Modeling the patient. Various exemplary embodiments include the use of patient-specific data to characterize the body portion being treated. In some embodiments and surgical procedures, port optimization planning on a representative sample of patients will have sufficient generality to be useful as a generic port placement plan. Modeling a patient may involve several sub-steps, such as: [0080] 1. acquiring patient-specific data for at least a portion of patent's body via such modalities as CT, MRI, and or arterial angiograms; [0081] 2. segmenting acquired data to distinguish organ, bone, vessel and other tissue structures (may be automated, manual or a combination of these); [0082] 3. reconstructing segmented, acquired data to construct a 3D model for at least a portion of patent's body. Optionally, such a model may include additional overlaid patient data, a body cavity insufflation space model, and/or the like. [0083] Step 8: Defining target criteria. Step 8 includes defining one or more surgical target points (e.g., location in the body of a particular intended tissue manipulation) and target direction(s) in relation to patient model, (the direction(s) most convenient for performing the surgical task, e.g., normal to an organ surface or a preferred direction relative to an organ structure). Target directions may be different for endoscope and each tool. A process model of the interventional surgical procedure may be defined, specifying one or more relevant surgical targets. [0084] Step 9: Defining admissible port set. This step involves defining a set of admissible ports for target and/or surgical procedure in relation to patient model, includes the entry point location and normal direction for each port. The choice of the admissible locations set stems from the characteristics of the intervention and/or anatomy of the patient, and is meant to cover all possible entry points from which optimal ports are to be chosen. Determining this set can either be done empirically or automatically using specialized segmentation algorithms. [0085] Step 10: Determining feasible port set. This step includes calculating port feasibility criteria for each admissible port, testing port feasibility, and eliminating failed ports. [0086] Step 11: Determining optimized multiple-port combination. Step 11 may include applying an optimization algorithm to calculated optimization criteria for all feasible port combinations of the total arm number (e.g., all feasible 3-port combinations or triplets). Step 11 may also include adding more ports than arms for surgeon assistance (e.g. cardiac stabilizer). The total number of ports is often referred to as n-tuplet. Several ports may be chosen for the same arm (e.g. two different non-simultaneous positions of the endoscope). Alternatively, step 11 may include pre-selecting an endoscope port, and then optimizing other ports by considering all combinations of remaining feasible ports (in example below, remaining feasible port pairs), as in the following sub-steps: [0087] 1. optimizing endoscope port., e.g., selecting port for port-to-target path close to target direction; [0088] 2. optional port pair feasibility criteria, e.g., eliminating port pairs with less than a minimum port surface separation, to simplify optimization by avoiding highly probable internal and external collisions, and/or aberrant ports with regard to dexterity and/or visibility; and [0089] 3. optimizing tool(s) and/or endoscope(s) combinations. For 1 endoscope+2 tools, each combination is commonly referred to as a triplet. More generally, for 1 endoscope +n−1 tools, each combination is referred to as an n-tuplet. Note that the combination may include more than one endoscope, or an integrated multifunctional endoscope/tool. This step may include, for example., calculating the optimization criteria for each port combination; calculating cost function value; ranking the n-tuplet by cost function value; and selecting the n-tuplet which has the best cost function value. [0090] Step 12: Determining an advantageous robotic system pre-surgical set-up configuration. Although the robotic system pre-surgical set-up position(s) may be determined empirically, preferably optimization methods are employed according to the principals of the invention. This may include determining positions for a portion or all of the “passive” flexibility degrees of freedom (dofs) of the system (i.e., “passive” in the sense of fixed during surgical treatment manipulation), including the base support position(s), base support orientation(s), set-up joint position(s), and the like. Note that different robotic surgical systems vary considerably in the number of passive pre-surgical set-up dofs. An exemplary process (e.g., probabilistic and gradient descent) may include: [0091] 1. defining a set of constraints on the system based on port location and/or trajectory modeling the intervention; [0092] 2. defining a cost function based on a measure of goodness including, e.g., separation between the arms; separation from obstacles; maximizing dexterity and/or maneuverability at the end effector(s); [0093] 3. running probabilistic optimization to get a set of admissible (constraints realized) solutions (position/orientation of the base and/or values for set-up joints); and/or [0094] 4. running gradient descent optimization from the above initial solutions to optimize measure of goodness. [0095] Step 13: Performing validation. The validation step involves applying the predictive model of expected surgeon-commanded operational instrument movements for a surgical procedure during manipulations at a surgical target site within the body. (e.g., the range of motions of instrument end effector, wrist and shaft within the body cavity relative to the body model; the range of motions of robotic arms outside the body relative to system model and/or body model). Collision prediction algorithms are also applied to determine if collisions will occur. [0096] Step 14: Re-selecting ports based on validation. If port placements and/or robot positioning fail the validation step, the port/positioning combination is rejected and steps 11 through 13 are repeated to choose new port placement locations and/or robot positions. [0097] Step 15: Simulating surgical procedure. Step 15 involves performing interactive surgery rehearsal by the surgeon, including surgeon inputs for simulated robotic manipulations, applying collision prediction algorithms, and/or inputting surgeon subjective assessment of effectiveness. [0098] Step 16: Re-selecting ports based on simulation. If simulation is unsatisfactory, port placements and/or robot positioning may be rejected and steps 11 through 15 may be repeated to select and validate new placements and/or positions. Optionally, if simulation is unsatisfactory, the surgeon may fix one or more of current ports, and/or pre-select one or more ports, based on simulation/rehearsal experience (e.g., A desirable tool or endoscope port), and repeat steps 11-15 to re-optimize with reduced feasible port set. [0099] Step 17: Recording and analyzing simulation data. Recorded simulation history and computer data, including surgeon inputs, tool motions and robotic arm movements, may be used to refine models, optimization criteria, feasibility criteria and/or cost function terms. [0100] Step 18: Repeating steps 8-17 for additional targets. For complex or multi-site procedures, planning steps may be repeated for all necessary surgical targets. [0101] Step 19: Determining multi-target optimized robotic system base support position. [0102] Optionally, method steps may be employed to optimize the base position of a robotic system to permit advantageous access to all targets. Preferably, the robotic system base support(s) are pre-positioned so that the multi-target procedure may be performed with no re-positioning of base support(s). [0103] Step 20: Multi-target port optimization. Optionally, method steps may be used to re-optimize port triplets to use particular ports for more than one target (minimize total number of ports and reduce set up time when accessing multiple targets). [0104] Step 21: Transferring and registering planning results to patient body and surgical system. For both robotic and non-robotic surgical procedures, the results of planning are transferred to the patient. The model of the planned procedure may be registered to the patient's body in the operating room. Transfer and registration may include the marking of port locations, and reproducing the planned initial positions and alignment of the instruments and/or robotic arms. For example, the following sub-steps may be used: [0105] 1. selecting common reference point(s) and/or directional bearing for patient on operating table and for models (robotic and patient models); [0106] 2. superimposing alignment of models to patient coordinates; [0107] 3. aligning the robotic system to the reference point(s) and bearings; [0108] 4. determining actual port locations based on model relative to patient coordinates; and [0109] 5. making incisions at determined port locations for instrument insertion. [0110] Optionally, a robotic control system, joint position sensors and encoders may be employed to effect a transformation from patient reference coordinates to joint-space coordinates for the robotic system. In one embodiment, for example, a robotic arm may be positioned to touch one or more reference point(s) on the body surface. Robotic system coordinates may then be defined relative to the body reference point(s). Finally, joint position sensors may monitor the arm motions relative to the body reference point(s), to direct and/or confirm setup arm positioning according to the optimized procedure plan, and to direct and/or confirm instrument orientation and tip location to touch the body surface at a modeled port location and orientation. [0111] Step 22: Collision detection during surgical procedure. Optionally, the collision prediction/detection algorithms may be applied to real-time robotic arm and instrument positions and orientations to predict, warn of, and/or avoid collisions during the procedure. [0112] Step 23: Recording and analyzing operational data. Recorded procedure history and computer data, including surgeon inputs, tool motions and robotic arm movements, may be used to refine models, optimization criteria, feasibility criteria and/or cost function terms. The robotic surgical system may be provided with an Application Program Interface (API), or the equivalent, in communication with the robotic control system and/or endoscope imaging system, to permit recordation during the course of a surgical procedure (and/or real-time analysis) of sensor signals, encoder signals, motions, torques, power levels, rates, input commands, endoscope display images, and the like. [0113] While the above is a complete description of exemplary embodiments of the invention, various alternatives, modifications and equivalents may be used. For example, various steps or stages in any of the above methods may be combined. For example, in one embodiment the planning and validation steps may be combined. In other embodiments, steps may be added or eliminated. [0114] As described variously above, methods and apparatus of the present invention are not limited to robotic surgery, but may be applied to laparoscopic, minimally invasive, or other types of surgery. Furthermore, the present invention is not limited to any particular type or category of surgical procedure. Examples of surgical procedures (including veterinary surgical procedures) in which embodiments of the invention may be used include, but are not limited to robotic and non-robotic thoracic, abdominal, neurological, orthopedic, gynecological, urological surgical procedures, and/or the like. The surgical instruments and instrument combinations employed may include more than one endoscope, or may include an integrated multifunctional endoscope/tool. Likewise the instrument combinations may include interventional instruments used with or monitored by other modalities of medical imagery instead of, or in addition to, visual endoscopy, e.g., ultrasound, real-time MRI, CT, fluoroscopy, and the like [0115] When applied to robotic surgery, embodiments having aspects of the invention are not limited to any particular make or type of robotic surgical system. Thus, methods and apparatus according to the principles of the invention may include robotic systems having more or fewer than three robotic arms, surgical procedures employing two or more cooperative robotic systems, robotic surgical systems co-operated by two or more surgeons simultaneously, or the like. Embodiments having aspects of the invention may include surgical systems having passive center-of-motion robotic manipulators, computed center-of-motion robotic manipulators, and/or mechanically constrained remote center-of-motion robotic manipulators, and the like. Models of robotic systems employed in simulation and planning steps may include modeling of active manipulator links and joints (servo-operated and passively responding joints which move during tissue treatment operation). Robotic arm models may also include base support links and joints (set up or pre-positioning arms fixed during tissue treatment operation). Multiple-arm robotic systems employed in embodiments having aspects of the invention may include a plurality of robotic arms may have a single integrated support base (e.g., a multi-arm cart-type support base), or each arm may have an individual base (e.g., wherein each robotic arm is individually clamped to an operating table structure or rail), or combinations of these. [0116] Additionally, the present invention is not limited to surgical procedures on a human patient or animal patient, but may be employed in a variety of non-surgical or quasi-surgical procedures and operations. The principles of the invention are particularly suitable to operations usefully performed by remotely operated or robotic tools, where substantially similar modeling, planning and simulation methods are useful. Examples include operations on a defined target volume, such as deactivation of a suspected explosive device; remote inspection and operations within a container, vehicle, or the like; underwater operations; and rescue operations in a collapsed structures, mine and the like. In non-surgical and quasi-surgical target volumes, the modeling of the target volume may optionally be based, at least in part, on archival data, such as engineering data, architectural data, CAD file inputs, and the like, as well as a variety of different actively acquired data modalities. [0117] Therefore, the above description should not be taken as limiting the scope of the invention which is defined by the appended claims. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5205289 *Jun 8, 1990Apr 27, 1993Medical Instrumentation And Diagnostics CorporationThree-dimensional computer graphics simulation and computerized numerical optimization for dose delivery and treatment planningUS5279309 *Jul 27, 1992Jan 18, 1994International Business Machines CorporationSignaling device and method for monitoring positions in a surgical operationUS5445166 *Apr 6, 1994Aug 29, 1995International Business Machines CorporationSystem for advising a surgeonUS5572999 *Jan 26, 1995Nov 12, 1996International Business Machines CorporationRobotic system for positioning a surgical instrument relative to a patient's bodyUS5791908 *Jul 11, 1996Aug 11, 1998Gillio; Robert G.Apparatus and method for telesurgeryUS5871018 *Jun 6, 1997Feb 16, 1999Delp; Scott L.Computer-assisted surgical methodUS6244809 *Dec 30, 1997Jun 12, 2001Computer Motion, Inc.Method and apparatus for performing minimally invasive cardiac proceduresUS6390097 *Mar 24, 1998May 21, 2002International Business Machines CorporationMethod and system for providing artificial intelligence for planning and risk assessment of surgical pathsUS6505065 *Oct 27, 2000Jan 7, 2003Koninklijke Philips Electronics, N.V.Methods and apparatus for planning and executing minimally invasive procedures for in-vivo placement of objectsUS6529765 *May 24, 1999Mar 4, 2003Neutar L.L.C.Instrumented and actuated guidance fixture for sterotactic surgeryUS6786896 *Sep 18, 1998Sep 7, 2004Massachusetts Institute Of TechnologyRobotic apparatus* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS6757587 *Apr 4, 2003Jun 29, 2004Nokia CorporationMethod and apparatus for dynamically reprogramming remote autonomous agentsUS7035716 *Jan 29, 2002Apr 25, 2006The Acrobot Company LimitedActive-constraint robotsUS7713190Jun 14, 2007May 11, 2010Hansen Medical, Inc.Flexible instrumentUS7809176Aug 4, 2006Oct 5, 2010Siemens AktiengesellschaftDevice and method for automated planning of an access path for a percutaneous, minimally invasive interventionUS7813785 *Mar 11, 2004Oct 12, 2010General Electric CompanyCardiac imaging system and method for planning minimally invasive direct coronary artery bypass surgeryUS7860614 *Sep 11, 2006Dec 28, 2010The United States Of America As Represented By The Secretary Of The ArmyTrainer for robotic vehicleUS7904206 *Jan 24, 2005Mar 8, 2011Olympus CorporationMethod and apparatus for removably holding medical deviceUS7972298Mar 4, 2005Jul 5, 2011Hansen Medical, Inc.Robotic catheter systemUS7974681 *Jul 6, 2005Jul 5, 2011Hansen Medical, Inc.Robotic catheter systemUS7976539Jul 19, 2005Jul 12, 2011Hansen Medical, Inc.System and method for denaturing and fixing collagenous tissueUS7979157 *Jul 25, 2005Jul 12, 2011Mcmaster UniversityMulti-purpose robotic operating system and methodUS8004229May 19, 2005Aug 23, 2011Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8005537 *Aug 12, 2005Aug 23, 2011Hansen Medical, Inc.Robotically controlled intravascular tissue injection systemUS8005571Jul 3, 2006Aug 23, 2011Neuroarm Surgical Ltd.Microsurgical robot systemUS8041459Feb 6, 2008Oct 18, 2011Neuroarm Surgical Ltd.Methods relating to microsurgical robot systemUS8170717Feb 6, 2008May 1, 2012Neuroarm Surgical Ltd.Microsurgical robot systemUS8271135 *Oct 25, 2010Sep 18, 2012Olympus CorporationMethod and apparatus for removably holding medical deviceUS8333204Jun 22, 2009Dec 18, 2012Hansen Medical, Inc.Apparatus and methods for treating tissueUS8396598Nov 22, 2006Mar 12, 2013Neuroarm Surgical Ltd.Microsurgical robot systemUS8428781Nov 17, 2009Apr 23, 2013Energid Technologies, Inc.Systems and methods of coordination control for robot manipulationUS8437890Jul 15, 2010May 7, 2013Massachusetts Institute Of TechnologyIntegrated framework for vehicle operator assistance based on a trajectory prediction and threat assessmentUS8473031Dec 26, 2007Jun 25, 2013Intuitive Surgical Operations, Inc.Medical robotic system with functionality to determine and display a distance indicated by movement of a tool robotically manipulated by an operatorUS8523883Aug 18, 2011Sep 3, 2013Hansen Medical, Inc.Apparatus and methods for treating tissueUS8541970Jul 1, 2011Sep 24, 2013Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8543261Jul 15, 2010Sep 24, 2013Massachusetts Institute Of TechnologyMethods and apparati for predicting and quantifying threat being experienced by a modeled systemUS8560118Apr 16, 2008Oct 15, 2013Neuroarm Surgical Ltd.Methods, devices, and systems for non-mechanically restricting and/or programming movement of a tool of a manipulator along a single axisUS8620473May 14, 2010Dec 31, 2013Intuitive Surgical Operations, Inc.Medical robotic system with coupled control modesUS8624537Jul 1, 2011Jan 7, 2014Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8744648Apr 9, 2013Jun 3, 2014Massachusetts Institute Of TechnologyIntegrated framework for vehicle operator assistance based on a trajectory prediction and threat assessmentUS8749189Jul 1, 2011Jun 10, 2014Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8749190Jul 1, 2011Jun 10, 2014Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8761930 *Apr 30, 2013Jun 24, 2014Intuitive Surgical Operations, Inc.Tool grip calibration for robotic surgeryUS8786241Jul 1, 2011Jul 22, 2014Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8792963Sep 30, 2011Jul 29, 2014Intuitive Surgical Operations, Inc.Methods of determining tissue distances using both kinematic robotic tool position information and image-derived position informationUS8816628Jul 1, 2011Aug 26, 2014Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8823308Jul 1, 2011Sep 2, 2014Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS8843234 *Apr 13, 2007Sep 23, 2014Fanuc America CorporationDynamic space check for multi-arm system moving on a railUS8864652Dec 17, 2008Oct 21, 2014Intuitive Surgical Operations, Inc.Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the positioning and orienting of its tipUS8903546Aug 15, 2009Dec 2, 2014Intuitive Surgical Operations, Inc.Smooth control of an articulated instrument across areas with different work space conditionsUS8918211Feb 12, 2010Dec 23, 2014Intuitive Surgical Operations, Inc.Medical robotic system providing sensory feedback indicating a difference between a commanded state and a preferred pose of an articulated instrumentUS8926603Mar 9, 2011Jan 6, 2015Hansen Medical, Inc.System and method for denaturing and fixing collagenous tissueUS8944070Mar 30, 2005Feb 3, 2015Intuitive Surgical Operations, Inc.Non-force reflecting method for providing tool force information to a user of a telesurgical systemUS9084623Nov 5, 2009Jul 21, 2015Intuitive Surgical Operations, Inc.Controller assisted reconfiguration of an articulated instrument during movement into and out of an entry guideUS9089256Jun 23, 2009Jul 28, 2015Intuitive Surgical Operations, Inc.Medical robotic system providing an auxiliary view including range of motion limitations for articulatable instruments extending out of a distal end of an entry guideUS9101397Aug 21, 2013Aug 11, 2015Intuitive Surgical Operations, Inc.Real-time generation of three-dimensional ultrasound image using a two-dimensional ultrasound transducer in a robotic systemUS9131986Oct 15, 2013Sep 15, 2015Neuroarm Surgical Ltd.Methods, devices, and systems for non-mechanically restricting and/or programming movement of a tool of a manipulator along a single axisUS9138129Nov 11, 2011Sep 22, 2015Intuitive Surgical Operations, Inc.Method and system for moving a plurality of articulated instruments in tandem back towards an entry guideUS9192788 *Jan 12, 2012Nov 24, 2015Koninklijke Philips N.V.Therapeutic apparatus, computer program product, and method for determining an achievable target region for high intensity focused ultrasoundUS9220567Apr 16, 2007Dec 29, 2015Neuroarm Surgical Ltd.Microsurgical robot systemUS9232981 *Jul 18, 2014Jan 12, 2016National Taiwan University Of Science And TechnologySurgical holderUS9232984Nov 10, 2010Jan 12, 2016Intuitive Surgical Operations, Inc.Real-time generation of three-dimensional ultrasound image using a two-dimensional ultrasound transducer in a robotic systemUS9259276Aug 22, 2011Feb 16, 2016Intuitive Surgical Operations, Inc.Indicator for tool state and communication in multiarm telesurgeryUS9320421Jun 16, 2011Apr 26, 2016Smith & Nephew, Inc.Method of determination of access areas from 3D patient imagesUS9333042Dec 3, 2013May 10, 2016Intuitive Surgical Operations, Inc.Medical robotic system with coupled control modesUS9345387Jun 13, 2007May 24, 2016Intuitive Surgical Operations, Inc.Preventing instrument/tissue collisionsUS9345544May 31, 2013May 24, 2016Intuitive Surgical Operations, Inc.Systems and methods for avoiding collisions between manipulator arms using a null-spaceUS9358076Apr 13, 2015Jun 7, 2016Hansen Medical, Inc.System and method for endoluminal and translumenal therapyUS9364171 *Dec 22, 2011Jun 14, 2016Veebot Systems, Inc.Systems and methods for autonomous intravenous needle insertionUS9408669Mar 15, 2013Aug 9, 2016Hansen Medical, Inc.Active drive mechanism with finite range of motionUS9457168Jun 19, 2014Oct 4, 2016Hansen Medical, Inc.Robotic catheter system and methodsUS9469034Mar 7, 2013Oct 18, 2016Intuitive Surgical Operations, Inc.Method and system for switching modes of a robotic systemUS9492235May 31, 2013Nov 15, 2016Intuitive Surgical Operations, Inc.Manipulator arm-to-patient collision avoidance using a null-spaceUS9492927Nov 9, 2011Nov 15, 2016Intuitive Surgical Operations, Inc.Application of force feedback on an input device to urge its operator to command an articulated instrument to a preferred poseUS9516996Sep 17, 2014Dec 13, 2016Intuitive Surgical Operations, Inc.Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the position and orienting of its tipUS9517106May 31, 2013Dec 13, 2016Intuitive Surgical Operations, Inc.Systems and methods for commanded reconfiguration of a surgical manipulator using the null-spaceUS9554859Aug 18, 2014Jan 31, 2017Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesUS20030090490 *Nov 8, 2002May 15, 2003Fanuc Ltd.Simulation deviceUS20040128026 *Jan 29, 2002Jul 1, 2004Harris Simon JamesActive-constraint robotsUS20050004443 *Mar 11, 2004Jan 6, 2005General Electric CompnayCardiac imaging system and method for planning minimally invasive direct coronary artery bypass surgeryUS20050137661 *Dec 17, 2004Jun 23, 2005Sra Jasbir S.Method and system of treatment of cardiac arrhythmias using 4D imagingUS20050143777 *Dec 17, 2004Jun 30, 2005Sra Jasbir S.Method and system of treatment of heart failure using 4D imagingUS20050165271 *Jan 24, 2005Jul 28, 2005Keiji ShiodaMethod and apparatus for removably holding medical deviceUS20050200324 *Mar 30, 2005Sep 15, 2005Intuitive Surgical Inc.Non-force reflecting method for providing tool force information to a user of a telesurgical systemUS20050222554 *Mar 4, 2005Oct 6, 2005Wallace Daniel TRobotic catheter systemUS20060111692 *Aug 12, 2005May 25, 2006Hlavka Edwin JRobotically controlled intravascular tissue injection systemUS20060149418 *Jul 25, 2005Jul 6, 2006Mehran AnvariMulti-purpose robotic operating system and methodUS20060293643 *Jul 6, 2005Dec 28, 2006Wallace Daniel TRobotic catheter systemUS20070013336 *May 19, 2005Jan 18, 2007Intuitive Surgical Inc.Software center and highly configurable robotic systems for surgery and other usesUS20070049861 *Aug 4, 2006Mar 1, 2007Lutz GundelDevice and method for automated planning of an access path for a percutaneous, minimally invasive interventionUS20070118101 *Nov 23, 2005May 24, 2007Prakash MaheshMethods and systems for facilitating planning of surgical proceduresUS20080004603 *Jun 29, 2006Jan 3, 2008Intuitive Surgical Inc.Tool position and identification indicator displayed in a boundary area of a computer display screenUS20080065109 *Jun 13, 2007Mar 13, 2008Intuitive Surgical, Inc.Preventing instrument/tissue collisionsUS20090171371 *Dec 26, 2007Jul 2, 2009Intuitive Surgical, Inc.Medical robotic system with functionality to determine and display a distance indicated by movement of a tool robotically manipulated by an operatorUS20090192523 *Mar 31, 2009Jul 30, 2009Intuitive Surgical, Inc.Synthetic representation of a surgical instrumentUS20090192524 *Mar 31, 2009Jul 30, 2009Intuitive Surgical, Inc.Synthetic representation of a surgical robotUS20090204258 *Apr 13, 2007Aug 13, 2009Jianming TaoDynamic space check for multi-arm system moving on a railUS20090326318 *Jun 23, 2009Dec 31, 2009Intuitive Surgical, Inc.Medical robotic system providing an auxilary view including range of motion limitations for articulatable instruments extending out of a distal end of an entry guideUS20090326324 *Jul 18, 2007Dec 31, 2009Universidad De MalagaRobotic system for assisting in minimally-invasive surgery, which can position a surgical instrument in response to orders from a surgeon, is not attached to the operating table and does not require pre-calibration of the insertion pointUS20090326553 *Jun 27, 2008Dec 31, 2009Intuitive Surgical, Inc.Medical robotic system providing an auxiliary view of articulatable instruments extending out of a distal end of an entry guideUS20090326556 *Dec 17, 2008Dec 31, 2009Intuitive Surgical, Inc.Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the positioning and orienting of its tipUS20100125284 *Jul 22, 2009May 20, 2010Hansen Medical, Inc.Registered instrument movement integrationUS20100152899 *Nov 17, 2009Jun 17, 2010Energid Technologies, Inc.Systems and methods of coordination control for robot manipulationUS20100198402 *Apr 16, 2008Aug 5, 2010Alexander GreerMethods, devices, and systems for non-mechanically restricting and/or programming movement of a tool of a manipulator along a single axisUS20100228427 *Feb 24, 2010Sep 9, 2010Massachusetts Institute Of TechnologyPredictive semi-autonomous vehicle navigation systemUS20100274087 *May 14, 2010Oct 28, 2010Intuitive Surgical Operations, Inc.Medical robotic system with coupled control modesUS20110040406 *Oct 25, 2010Feb 17, 2011Olympus CorporationMethod and apparatus for removably holding medical deviceUS20110105898 *Nov 10, 2010May 5, 2011Intuitive Surgical Operations, Inc.Real-Time Generation of Three-Dimensional Ultrasound image using a Two-Dimensional Ultrasound Transducer in a Robotic SystemUS20110144658 *Feb 24, 2011Jun 16, 2011Corindus Inc.Catheter simulation and assistance systemUS20110190937 *Aug 19, 2009Aug 4, 2011Kuka Laboratories GmbhMedical Work Station And Operating Device For Manually Moving A Robot Arm Of A Medical Work StationUS20110202068 *Feb 12, 2010Aug 18, 2011Intuitive Surgical Operations, Inc.Medical robotic system providing sensory feedback indicating a difference between a commanded state and a preferred pose of an articulated instrumentUS20110238080 *Mar 22, 2011Sep 29, 2011Date RanjitRobotic Surgical Instrument SystemUS20120190981 *Dec 22, 2011Jul 26, 2012Veebot, LlcSystems and methods for autonomous intravenous needle insertionUS20130296742 *Jan 12, 2012Nov 7, 2013Koninklijke Philips Electronics N.V.Therapeutic apparatus, computer program product, and method for determining an achievable target region for high intensity focused ultrasoundUS20140212025 *Sep 5, 2012Jul 31, 2014Koninklijke Philips Electronics N.V.Automatic online registration between a robot and imagesUS20150136926 *Jul 18, 2014May 21, 2015National Taiwan University Of Science And TechnologySurgical holderUS20160015471 *Mar 14, 2015Jan 21, 2016Synaptive Medical (Barbados) Inc.Context aware surgical systemsCN102448680A *Mar 26, 2010May 9, 2012直观外科手术操作公司Synthetic representation of a surgical robotDE102005037000B4 *Aug 5, 2005Jun 1, 2011Siemens AgVorrichtung zur automatisierten Planung eines Zugangspfades für einen perkutanen, minimalinvasiven EingriffEP1814050A2 *Nov 23, 2006Aug 1, 2007General Electric CompanyMethods and systems for facilitating planning of surgical proceduresEP1814050A3 *Nov 23, 2006Aug 8, 2007General Electric CompanyMethods and systems for facilitating planning of surgical proceduresEP2142132A2 *Apr 16, 2008Jan 13, 2010Neuroarm Surgical Ltd.Methods, devices, and systems for non-mechanically restricting and/or programming movement of a tool of a manipulator along a single axisEP2142132A4 *Apr 16, 2008Mar 2, 2011Neuroarm Surgical LtdMethods, devices, and systems for non-mechanically restricting and/or programming movement of a tool of a manipulator along a single axisEP2332477A3 *May 8, 2006Aug 22, 2012Intuitive Surgical Operations, Inc.Software Center and Highly Configurable Robotic Systems for Surgery and other usesEP2332478A3 *May 8, 2006Nov 14, 2012Intuitive Surgical Operations, Inc.Software Center and Highly Configurable Robotic Systems for Surgery and other usesEP2332479A3 *May 8, 2006Aug 22, 2012Intuitive Surgical Operations, Inc.Software Center and Highly Configurable Robotic Systems for Surgery and other usesEP2332480A3 *May 8, 2006Oct 10, 2012Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesEP2332481A3 *May 8, 2006Aug 29, 2012Intuitive Surgical Operations, Inc.Software center and highly configurable robotic systems for surgery and other usesEP2332482A3 *May 8, 2006Nov 28, 2012Intuitive Surgical Operations, Inc.Software Center and Highly Configurable Robotic Systems for Surgery and other usesEP2332483A3 *May 8, 2006Nov 14, 2012Intuitive Surgical Operations, Inc.Software Center and Highly Configurable Robotic Systems for Surgery and other usesEP2332484A3 *May 8, 2006Sep 26, 2012Intuitive Surgical Operations, Inc.Software Center and Highly Configurable Robotic Systems for Surgery and other usesEP2384716A3 *Dec 10, 2008Jul 6, 2016Intuitive Surgical Operations, Inc.Medical robotic system with functionality to determine and display a distance indicated by movement of a tool robotically manipulated by an operatorEP2465461A3 *Dec 10, 2008Jul 20, 2016Intuitive Surgical Operations, Inc.Medical robotic system with functionality to determine and display a distance indicated by movement of a tool robotically manipulated by an operatorWO2006124390A3 *May 8, 2006Mar 1, 2007Intuitive Surgical IncSoftware center and highly configurable robotic systems for surgery and other usesWO2009085616A1 *Dec 10, 2008Jul 9, 2009Intuitive Surgical, Inc.Medical robotic system with functionality to determine and display a distance indicated by movement of a tool robotically manipulated by an operatorWO2009114613A3 *Mar 11, 2009Dec 10, 2009Health Research Inc.System and method for robotic surgery simulationWO2010022884A1 *Aug 19, 2009Mar 4, 2010Kuka Roboter GmbhMedical work station and operating device for manually moving a robot arm of a medical work stationWO2010117685A2 *Mar 26, 2010Oct 14, 2010Intuitive Surgical Operations, Inc.Synthetic representation of a surgical robotWO2010117685A3 *Mar 26, 2010Feb 24, 2011Intuitive Surgical Operations, Inc.Synthetic representation of a surgical robotWO2011009011A1 *Jul 15, 2010Jan 20, 2011Massachusetts Institute Of TechnologyAn integrated framework for vehicle operator assistance based on a trajectory prediction and threat assessmentWO2011150254A2 *May 26, 2011Dec 1, 2011Health Research Inc.Method and system for automatic tool position determination for minimally-invasive surgery trainingWO2011150254A3 *May 26, 2011Mar 8, 2012Health Research Inc.Method and system for automatic tool position determination for minimally-invasive surgery trainingWO2011150257A3 *May 26, 2011Mar 8, 2012Health Research Inc.Method and system for minimally-invasive surgery training using tracking dataWO2011158115A3 *Jun 16, 2011Nov 22, 2012A2 SurgicalMethod of determination of access areas from 3d patient imagesWO2016176755A1 *Dec 15, 2015Nov 10, 2016Titan Medical Inc.Instrument collision detection and feedbackWO2017030848A1 *Aug 10, 2016Feb 23, 2017Ethicon Endo-Surgery, LlcGathering and analyzing data for robotic surgical systems* Cited by examinerClassifications U.S. Classification600/407International ClassificationA61B19/00Cooperative ClassificationA61B34/37, G05B2219/40418, B25J9/1671, A61B2034/107, A61B90/361, A61B34/70, A61B34/35, A61B34/30, A61B34/25, A61B90/37, A61B34/10European ClassificationB25J9/16P5, A61B19/22B, A61B19/22Legal EventsDateCodeEventDescriptionSep 25, 2002ASAssignmentOwner name: INRIA ROQUENCOURT, FRANCEFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:COSTE-MANIERE, EVE C.;ADHAMI, LOUAI;BOISSONNAT, JEAN-DANIEL;AND OTHERS;REEL/FRAME:013319/0227;SIGNING DATES FROM 20020828 TO 20020902Owner name: INTUITIVE SURGICAL, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:COSTE-MANIERE, EVE C.;ADHAMI, LOUAI;BOISSONNAT, JEAN-DANIEL;AND OTHERS;REEL/FRAME:013319/0227;SIGNING DATES FROM 20020828 TO 20020902Mar 14, 2013FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services