Source: https://patents.justia.com/patent/20110137322
Timestamp: 2019-10-19 15:43:40
Document Index: 485869844

Matched Legal Cases: ['Application No. 60', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300', 'art 300']

US Patent Application for Cooperative Minimally Invasive Telesurgical System Patent Application (Application #20110137322 issued June 9, 2011) - Justia Patents Search
Justia Patents Stereotaxic DeviceUS Patent Application for Cooperative Minimally Invasive Telesurgical System Patent Application (Application #20110137322)
Dec 3, 2010 - Intuitive Surgical Operations
Latest Intuitive Surgical Operations Patents:
The present application is a continuation of application Ser. No. 09/433,120 (Attorney Docket No. 017516-004720), filed Nov. 3, 1999, for a “Cooperative Minimally Invasive Telesurgical System”, which was a continuation-in-part of and claims the benefit of priority from application Ser. No. 09/399,457 (Attorney Docket No. 017516-004710), filed Sep. 17, 1999 for a “Cooperative Minimally Invasive Telesurgical System”; application Ser. No. 09/374,643 (Attorney Docket No. 017516-005900) (Abandoned), filed Aug. 16, 1999 for a “Cooperative Minimally Invasive Telesurgical System”; and also claims the benefit of priority from Provisional Application Ser. No. 60/116,891 (Attorney Docket No. 017516-004700), filed Jan. 22, 1999, for “Dynamic Association of Master and Slave in a Minimally Invasive Telesurgical System”; Provisional Application Ser. No. 60/116,842 (Attorney Docket No. 017516-001300), filed Jan. 22, 1999, for “Repositioning and Reorientation of Master/Slave Relationship in Minimally Invasive Telesurgery”; and Provisional Application Ser. No. 60/109,359 (Attorney Docket No. 017516-002500), filed Nov. 20, 1998, for “Apparatus and Method for Tracking and Controlling Cardiac Motion During Cardiac Surgery Without Cardioplegia”; the full disclosures of which are incorporated herein by reference.
The most common form of minimally invasive surgery may be endoscopy. Probably the most common form of endoscopy is laparoscopy, which is minimally invasive inspection and surgery inside the abdominal cavity. In standard laparoscopic surgery, a patient's abdomen is insufflated with gas, and cannula sleeves are passed through small (approximately ½ inch or less) incisions to provide entry ports for laparoscopic surgical instruments. The laparoscopic surgical instruments generally include a laparoscope (for viewing the surgical field) and working tools. The working tools are similar to those used in conventional (open) surgery, except that the working end or end effector of each tool is separated from its handle by an extension tube. As used herein, the term “end effector” means the actual working part of the surgical instrument and can include clamps, graspers, scissors, staplers, image capture lenses, and needle holders, for example. To perform surgical procedures, the surgeon passes these working tools or instruments through the cannula sleeves to an internal surgical site and manipulates them from outside the abdomen. The surgeon monitors the procedure by means of a monitor that displays an image of the surgical site taken from the laparoscope. Similar endoscopic techniques are employed in, e.g., arthroscopy, retroperitoneoscopy, pelviscopy, nephroscopy, cystoscopy, cisternoscopy, sinoscopy, hysteroscopy, urethroscopy, and the like.
The present invention provides improved robotic surgical systems, devices, and methods. The robotic surgical systems of the present invention will often include a plurality of input devices and/or a plurality of robotic manipulator arms for moving surgical instruments. A processor will often selectably couple a selected input device to a selected manipulator arm, and allows modification of the operative association so that that same input device can be coupled to a different manipulator arm, and/or so that that same manipulator arm can be controlled by a different input device. This selective coupling, for example, allows the controller to properly assign left and right surgical end effectors to left and right input devices for use by an operator viewing the procedure via an image capture device. When the image capture devices moves, the operative associations can be revised. In some embodiments, the image capture device may be removed from one of the manipulator arms and instead mounted to another of the manipulator arms, with the left and right input devices reassigned so as to avoid an awkward surgical environment for the system operator sitting at a master control station.
The systems of the present invention will often include more manipulator arms than will be moved simultaneously by a single surgeon. In addition to an imaging arm (movably supporting an image capture device) and two manipulator arms (holding selectably designated “left” and “right” surgical tools for manipulation by left and right hands of the system operator, e.g.), one or more additional manipulator arms will often be provided to position associated surgical instrument(s). At least one of the additional manipulator arms may be maintained in a stationary configuration to stabilize or retract tissue while the operator moves left and right input devices with his or her left and right hands to manipulate tissues with the associated surgical tools. The one or more additional arms may also be used to support another image capture device, often a second endoscope to view the internal surgical site from an alternative vantage point. Additional arms may optionally provide one or more additional surgical tools for manipulating tissue or aiding in the performance of a procedure at a surgical site. To manipulate this additional surgical instrument, an assistant input device may optionally be provided so that an assistant (such as an assisting surgeon at another workstation, a surgical nurse at the patient's side, or the like) can position the additional robotic arm(s). Regardless of the presence or absence of such an assistant input device, the robotic surgical systems of the present invention will preferably allow the system operator to selectively associate the right and/or left input devices of the surgical workstation with any of a plurality of surgical instruments.
In a first aspect, the invention provides a robotic surgical system comprising a first input device manipulatable by a hand of an operator. A first robotic arm assembly includes a first manipulator arm for moving a first surgical instrument. A second robotic arm assembly includes a second manipulator arm for moving a second surgical instrument. A control system couples the first input device to the first and second robotic arm assemblies. The control system permits selective operative association of the first input device with the first robotic arm assembly, and also permits selective operative association of the first input device with the second robotic arm assembly.
Typically, the control system will have a plurality of selectable modes, with manipulation of the first input device effecting corresponding movement of the first surgical instrument in one mode, and with the same manipulation of the first input device effecting corresponding movement of the second surgical instrument in a second mode. In some embodiments, a second input device may be used to effect corresponding movement of the second surgical instrument when the control system is in the first mode, and effect corresponding movement of the first surgical instrument when the control system is in the second mode, allowing swapping control of the surgical instruments between the input devices. This is particularly useful when the system operator is controlling the first and second input devices using left and right hands with reference to an image of an internal surgical site, as it allows the system operator to switch tools when the image capture device providing the image moves to what would otherwise be an awkward position.
In another aspect, the invention provides a robotic surgical system comprising a plurality of input devices and a plurality of manipulator arms, each manipulator arm having an instrument holder. A plurality of surgical instruments are mountable to the instrument holders, the surgical instruments including an image capture device and a tool having a surgical end effector for treating tissue. A control system couples the input devices with the manipulator arms. The control system selectably associates each input device with a manipulator arm.
In some embodiments, the image capture device may be removed from one manipulator arm and mounted to an alternative manipulator arm, often with the control system being reconfigurable so that the input devices are operatively associatable with manipulator arms holding tools for treating tissue. A particularly advantageous control system is provided which allows this flexible pairing of input devices and manipulator arms.
In a specific aspect, the invention provides a minimally invasive robotic surgical system comprising two input devices and at least two medical instrument robotic arm assemblies. One of the input devices is operatively associated with one of the robotic arm assemblies to cause movement of the robotic arm assembly in response to inputs on the input device. The other input device is operatively associated with another of the robotic arm assemblies to cause movement of that other robotic arm assembly in response to inputs on that other input device. A control system couples the input devices with the robotic arm assemblies. The control system enables selective swapping so as to cause the input device to be operatively associated with the robotic arm assembly which was operatively associated with the other input device, and to cause the other input device to be operatively associated with the robotic arm assembly which was operatively associated with the input device. Related systems allow selective operative association between at least one of the input devices and an image capture robotic arm assembly to permit the position of an image capture device to be changed using the at least one input device.
In a method aspect, the invention provides a robotic surgical method comprising robotically moving a first surgical instrument using a first manipulator arm by manipulating a first input device with a hand. A control system is reconfigured by entering a command. The control system couples the first input device with the first manipulator arm, and also with a second manipulator arm. A second surgical instrument is moved robotically using the second manipulator arm by manipulating the input device with the hand after the reconfiguring step.
In a related method, a robotic surgical method comprises robotically moving a surgical instrument using a manipulator arm by manipulating a first input device with a first hand. A control system is reconfigured by entering a command. The control system couples the first input device, and also a second input device, with the manipulator arm. The surgical instrument is robotically moved using the manipulator arm by manipulating the second input device with a second hand after the reconfiguring step.
In another aspect, the invention provides a robotic surgical system comprising an imaging system transmitting an image from an image capture device to a display. First, second, and third manipulator arms each support an associated surgical instrument. A master controller is disposed adjacent the display. The master has a first input device manipulatable by a first hand of an operator, and a second input device manipulatable by a second hand of the operator. A processor operatively couples each input device of the master to an associated manipulator arm so that movement of the input device effects movement of the associated surgical instrument.
In some embodiments, the surgical instrument associated with the third arm will comprise another image capture device. Where two image capture devices are included in the system, the processor will preferably transmit arm movement command signals to the arms according to different coordinate system transformations depending on which image capture device is providing the image currently shown on the display. This allows the processor to correlate between a direction of movement of the input device and the movement of the surgical instrument when switching between two different endoscopes having different fields of view.
Preferably, at least the third robotic instrument arm should be configurable to maintain a stationary configuration under some circumstances, with the arm in the stationary configuration inhibiting movement of the associated surgical instrument despite movement of the input devices. Such a stationary configuration is particularly useful when the surgical instrument mounted on the third arm comprises a stabilizer (such as a coronary tissue stabilizer used for beating heart surgery) or a retractor (for example, to retract tissue to expose a desired area of the cystic duct to the surgeon during cholecystectomy). The third arm will often comprise a linkage having a series of joints, and a brake system coupled to the joints to releasably inhibit articulation of the linkage. The third arm linkage will preferably also have a repositionable configuration allowing manual articulation of the arm, and at least some of the arms will often remain stationary and/or be repositionable in response to a signal. In addition to comprising a robotic manipulator arm having a driven configurations, the third arm may alternatively comprise a simple passive linkage with a brake system but without actuators.
Preferably, the surgical system will include four or more robotic manipulator arms. One of the additional arms may support an image capture device of the imaging system. To allow the operator to selectively manipulate all of these surgical instruments, including the image capture device, the processor will have an operation mode in which the first arm moves its associated surgical instrument at the surgical site in response to movement to the first input device, while the second arm moves its associated surgical instrument in response to movement of the second input device. In response to an arm selection signal from at least one arm selector input coupled to the processor, the processor can selectively change operating modes by decoupling the first arm from the first input device, and instead operatively couple the first input device with the second arm, the third arm, or the fourth arm. The processor will maintain some (or ideally all) of the decoupled arms in the stationary configuration under some circumstances, although a decoupled arm could be controlled to move in a repetitive or automated manner until re-coupled to the surgeon's input device. An example of such automated motion of a decoupled robotic arm includes motion tracking of a beating heart.
The robotic surgical system will often include an assistant input device. The processor can selectively associate one or more of the arms with the assistant input device, or with an input device of the surgeon, so that the one arm moves in response to movement of the selected input device. Hence, the processor can “hand-off” control of at least one arm (and its associated surgical instrument) between surgeon and assistant input stations. This may be useful when the assistant is removing and replacing the surgical instrument from the arm or when the assistant at a second console is to perform a part of the surgical procedure such as “closing” a portion of the surgical site. This also allows the surgeon to selectively assign direct control over an instrument based on the skill required to use the instrument for a given task, thereby enhancing robotic team effectiveness. The assistant may optionally be working at an assistant control station that can correlate direction of movement of the assistant input device with an image of the end effector shown in an assistant display. The “assistant” image might be either a different image from a second endoscope or a shared image from the primary console's endoscope, thereby enabling the surgeon and assistant to view the same image of the surgical site and manipulate each of their assigned and coupled instruments to cooperate in performing a surgical procedure. Alternatively, a simple video monitor and assistant input device (such as a handle within the sterile field) may be provided for the assistant, particularly for a patient-side assistant performing tool swaps, intermittent irrigation, or the like. In other embodiments, the surgeon's ability to select from among three or more surgical instruments, and to selectively associate each instrument with each of the input devices, will reduce and/or eliminate the need for surgical assistants. Decreasing the use of surgical assistants (and the time to continually direct and oversee the assistant's movements) can significantly decrease the time and expense of a surgical procedure.
Typically, for a system having four arms, for example, three surgical instruments and the image capture device will be supported by four manipulators each comprising an endoscopic instrument having an elongate shaft with a proximal end adjacent the manipulator and a distal end insertable into an internal surgical site within a patient body through a minimally invasive surgical aperture. Preferably, the manipulators will support the surgical instrument so that the shafts extend radially outwardly from a pattern of apertures (typically incisions) in a “spoked wheel”-type arrangement. In an exemplary arrangement for performing cardiac surgery, the four shafts will be sufficiently long to enter apertures on the right side of the patient body, and to extend toward the left side of the patient body for treating the heart. During at least a portion of the procedure, top and bottom apertures of the pattern may accommodate first and second endoscopes to provide flexibility in the field of view, particularly for the separate steps of harvesting a suitable supply artery and forming of the anastomosis during coronary artery bypass grafting (CABG). Additional ports can support additional surgical tools. When performing a CABG procedure without cardioplegia on a beating heart, at least one of the apertures of the pattern will often accommodate a tissue stabilizer during at least a portion of the procedure.
In another aspect, the invention provides a robotic surgical system comprising a plurality of manipulator arms and a plurality of surgical instruments, each instrument mounted to an associated arm. A master controller station has a master display for viewing by an operator, a first input device for manipulation by a first hand of the operator, and a second input device for manipulation by a second hand of the operator. An assistant input device for manipulation by a hand of an assistant is also provided. Preferably, a processor selectively operatively couples the controllers to the arms to effect movement to the surgical instruments in response to movement of the input devices.
In a method aspect, the invention provides a robotic surgical method comprising robotically moving first and second surgical instruments at a surgical site with first and second robotic manipulator arms by manipulating first and second input devices with first and second hands of the operator, respectively. The first input device is selectively associatable with a third manipulator arm, so that a third surgical instrument can be robotically moved at the surgical site with the third manipulator arm by manipulating the first input device with the first hand of the operator.
In another method aspect, the invention provides a robotic surgical method comprising robotically moving first and second surgical instruments at a surgical site with first and second robotic manipulator arms by manipulating first and second input devices with first and second hands of the operator, respectively. A third surgical instrument is positioned at the surgical site by articulating a linkage of a third manipulator. Movement of the positioned third surgical instrument is impeded at the surgical site by inhibiting movement of the third manipulator.
In yet another method aspect, the invention provides a robotic surgical method comprising robotically positioning a surgical instrument at a surgical site with a manipulator arm by manipulating a first input device with a hand of a first operator. The manipulator arm is selectively associated with a second input device, and the surgical instrument is robotically moved at the surgical site with the manipulator arm by manipulating the second input device with a hand of a second operator.
In a still further method aspect, the invention provides a robotic surgical method comprising showing, on a display, a first view of a surgical site from a first image capture device. A surgical instrument is robotically removed at the surgical site with a manipulator arm by manipulating an input device with a hand of an operator while the operator views the first view of the surgical site on the display. A second image capture device is selectively associated with the display, and the surgical instrument is robotically manipulated at the surgical site with the arm by manipulating the input device while the operator views a second view of the surgical site from the second image capture device on the display.
In yet another method aspect, the invention provides a robotic coronary artery bypass grafting method comprising introducing an image capture device into a chest cavity of a patient through an aperture disposed along a right side of the patient. An image of a surgical site adjacent the heart is displayed from the image capture device to an operator. A surgical procedure is performed on the heart by moving a surgical instrument at the surgical site with at least one robotic manipulator arm while the surgical instrument extends through another aperture disposed along the right side of the patient. Preferably, tissue manipulation during the surgical procedure will primarily be performed by surgical tools extending through a pattern of apertures along the right side of the patient.
In another aspect, first and second robotic manipulators are controlled by a first operator with first and second controllers, and third and fourth manipulators are controlled by a second operator with third and fourth controllers. Both operators view an image of the operating site captured by a single image capture device, and both cooperate to perform a surgical procedure. Each operator may have a separate dedicated viewing station for his use. Such cooperation may include helping each other perform the procedure, passing objects back and forth between manipulators during the procedure, and passing control of various of the manipulator arms. Both sets of manipulators may share the same reference point so that control may be transferred without loss of context.
This application is related to the following patents and patent applications, the full disclosures of which are incorporated herein by reference: PCT International Application No. PCT/US98/19508, entitled “Robotic Apparatus”, filed on Sep. 18, 1998 (Attorney Docket No. 17516-005510PC), U.S. Patent Application Ser. No. 60/111,713, entitled “Surgical Robotic Tools, Data Architecture, and Use” (Attorney Docket No. 17516-003200), filed on Dec. 8, 1998; U.S. Patent Application Ser. No. 60/111,711, entitled “Image Shifting for a Telerobotic System” (Attorney Docket No. 17516-002700), filed on Dec. 8, 1998; U.S. Patent Application Ser. No. 60/111,714, entitled “Stereo Viewer System for Use in Telerobotic System” (Attorney Docket No. 17516-001500), filed on Dec. 8, 1998; U.S. Patent Application Ser. No. 60/111,710, entitled “Master Having Redundant Degrees of Freedom” (Attorney Docket No. 17516-001400), filed on Dec. 8, 1998, U.S. Patent Application No. 60/116,891, entitled “Dynamic Association of Master and Slave in a Minimally Invasive Telesurgery System” (Attorney Docket No. 17516-004700), filed on Jan. 22, 1999; and U.S. Pat. No. 5,808,665, entitled “Endoscopic Surgical Instrument and Method for Use,” issued on Sep. 15, 1998.
As used herein, first and second objects (and/or their images) appear “substantially connected” if a direction of an incremental positional movement of the first object matches the direction of an incremental positional movement of the second object (often as seen in an image), regardless of scaling between the movements. Matching directions need not be exactly equal, as the objects (or the object and the image) may be perceived as being connected if the angular deviation between the movements remains less than about ten degrees, preferably being less than about five degrees. Similarly, objects and/or images may be perceived as being “substantially and orientationally connected” if they are substantially connected and if the direction of an incremental orientational movement of the first object is matched by the direction of an incremental orientational movement of the second object (often as seen in an image displayed near the first object), regardless of scaling between the movements.
Referring now to FIG. 1, a robotic surgical network 10 includes a master control station 200 and a slave cart 300, along with any of several other additional components to enhance the capabilities of the robotic devices to perform complex surgical procedures. An operator O performs a minimally invasive surgical procedure at an internal surgical site within patient P using minimally invasive surgical instruments 100. Operator O works at master control station 200. Operator O views a display provided by the workstation and manipulates left and right input devices. The telesurgical system moves surgical instruments mounted on robotic arms of slave cart 300 in response to movement of the input devices. As will be described in detail below, a selectably designated “left” instrument is associated with the left input device in the left hand of operator O, and a selectably designated “right” instrument is associated with the right input device in the right hand of the operator.
As described in more detail in co-pending U.S. patent application Ser. No. 09/373,678 entitled “Camera Referenced Control In A Minimally Invasive Surgical Apparatus” and filed Aug. 13, 1999, (Attorney Docket No. 17516-002110), the full disclosure of which incorporated herein by reference, a processor of master controller 200 will preferably coordinate movement of the input devices with the movement of their associated instruments so that the images of the surgical tools 100, as displayed to the operator, appear at least substantially connected to the input devices in the hands of the operator. Further levels of connection will also often be provided to enhance the operator's dexterity and ease of use of surgical instruments 100.
Introducing some of the other components of network 10, an auxiliary cart 300A can support one or more additional surgical tools 100 for use during the procedure. One tool is shown here for illustrative purposes only. A first assistant A1 is seated at an assistant control station 200A, the first assistant typically directing movements of one or more surgical instruments not actively being manipulated by operator O via master control station 200. A second assistant A2 may be disposed adjacent patient P to assist in swapping instruments 100 during the surgical procedure. Auxiliary cart 300A may also include one or more assistant input devices 12 (shown here as a simple joy stick) to allow second assistant A2 to selectively manipulate the one or more surgical instruments while viewing the internal surgical site via an assistant display 14. Preferably, the first assistant A1 seated at console 200A views the same image as surgeon seated at console 200. Further preferably, both the instruments of cart 300 and the “assistant” instruments of cart 300A are controlled according to the same camera reference point, such that both surgeon and assistant are able to be “immersed” into the image of the surgical field when manipulating any of the tools.
Component Descriptions. Referring to FIG. 2 of the drawings, the control station of a minimally invasive telesurgical system in accordance with the invention is generally indicated by reference numeral 200. The control station 200 includes a viewer or display 202 where an image of a surgical site is displayed in use. A support 204 is provided on which an operator, typically a surgeon, can rest his or her forearms while gripping two master controls (FIGS. 3A and 3B), one in each hand. The master controls are positioned in a space 206 inwardly beyond the support 204. When using control station 200, the surgeon typically sits in a chair in front of the control station 200, positions her eyes in front of the viewer 202, and grips the master controls, one in each hand, while resting her forearms on the support 204.
The articulated arm 210A includes a plurality of links 220 connected together at joints 222. Articulated arm 210A has appropriately positioned electric motors to provide for feedback as described in greater detail below. Furthermore, appropriately positioned positional sensors, e.g., encoders, or potentiometers, or the like, are positioned on the joints 222 so as to enable joint positions of the master control to be determined as further described herein below. Axes A, B, and C indicate the positional degrees of freedom of articulated arm 210A. In general, movement about joints of the master control 210B primarily accommodates and senses orientational movement of the end effector, and movement about the joints of arm 210A primarily accommodates and senses translational movement of the end effector. The master control 210 is described in greater detail in U.S. Provisional Patent Application Ser. No. 60/111,710, and in U.S. patent application Ser. No. 09/398,507, filed concurrently herewith (Attorney Docket No. 17516-001410), the full disclosures of which are incorporated herein by reference.
The positioning linkages or “set-up joints” are described in Provisional Application Ser. No. 60/095,303, the full disclosure of which is incorporated herein by reference. Preferably, the set-up joints include joint sensors which transmit signals to the processor indicating the position of the remote center of rotation. It should be noted that the manipulator arm assemblies need not be supported by a single cart. Some or all of the manipulators may be mounted to a wall or ceiling of an operating room, separate carts, or the like. Regardless of the specific manipulator structures or their mounting arrangement, it is generally preferable to provide information to the processor regarding the location of insertion/pivot points of the surgical instruments into the patient body. The set-up joint linkages need not have joint drive systems but will often include a joint brake system, as they will often hold the manipulators in a fixed position during some or all of a surgical procedure.
While this “remote” center of motion-type arrangement for robotic manipulation is described in connection with the preferred embodiments of this invention, the scope of the inventions disclosed herein is not so limited, encompassing other types of arrangements such as manipulator arms having passive or natural centers of motion at the point of insertion into a patient body.
Tissue stabilizer end effectors 120a, b, and c, referred to generally as tissue stabilizers 120, are illustrated in FIGS. 8A-C. Tissue stabilizers 120 may have one or two end effector elements 122 that preferably are pivotally attached to the distal end of the shaft or wrist of a surgical instrument and are moveable with respect to one another, and that preferably comprise tissue-engaging surfaces 124. The tissue-engaging surfaces optionally include protrusions, ridges, vacuum ports, or other surfaces adapted so as to inhibit movement between the engaged tissue and the stabilizer, either through pressure applied to the engaged tissue or vacuum applied to draw the tissue into an at least partially stabilized position, or a combination of both pressure and vacuum. The ideal tissue engaging surface will constrain and/or reduce motion of the engaged tissue in the two lateral (sometimes referred to as the X and Y) axes, along the tissue-engaging surface, and the stabilizer configuration and engagement with the tissue will at least partially decrease motion normal to the surface. Other configurations for traditional stabilizers are known to those of skill in the art, such as the Octopus II of Medtronic, Inc. and various HeartPort, Inc. and CardioThoracic Systems stabilizers having multipronged and doughnut configurations. These manners of contacting tissue allow stabilizers 120 to firmly engage a moving tissue such as a beating heart of a patient and reduce movement of the tissue adjacent the stabilizer.
As seen most clearly in FIG. 9E, the rotational position of discs 374 can be changed by manually rotating adjustment knobs 376, which are rotationally coupled to the discs. Once the instrument 100 is in the desired configuration, lock nuts 378 may be tightened against washers 379 to rotationally affix knobs 376 and discs 374. In the exemplary embodiment, bracket 372 comprises a polymer, while knobs 376 and nuts 378 may be polymeric and/or metallic. Washer 379 may comprise a low friction polymer, ideally comprising a PTFE such as Teflon™, or the like. While the disclosure herein shows a preferred embodiment for manual manipulation of a stabilizer by a surgical assistant, it should be apparent that the stabilizer might just as easily be controlled from a remote robotic control console, from which the operator would manipulate the stabilizer and any associated wrist in the same way as other instruments are controlled, as herein described.
Telesurgical Methods and Component Interactions. In use, the surgeon views the surgical site through the viewer 202. The end effector 102 carried on each arm 312, 302, 302A is caused to perform movements and actions in response to movement and action inputs of its associated master control. It will be appreciated that during a surgical procedure images of the end effectors are captured by the endoscope together with the surgical site and are displayed on the viewer so that the surgeon sees the movements and actions of the end effectors as he or she controls such movements and actions by means of the master control devices. The relationship between the end effectors at the surgical site relative to the endoscope tip as viewed through the viewer and the position of the master controls in the hands of the surgeon relative to the surgeon's eyes at the viewer provides an appearance of at least a substantial connection between the master controls and the surgical instrument for the surgeon.
Master-Slave Controller. In FIG. 10, the Cartesian space coordinate system is indicated generally by reference numeral 902. The origin of the system is indicated at 904. The system 902 is shown at a position removed from the endoscope 304. In the minimally invasive telesurgical system of the invention, and for purposes of identifying positions in Cartesian space, the origin 904 is conveniently positioned at the viewing end 306. One of the axes, in this case the Z-Z axis, is coincident with the viewing axis 307 of the endoscope. Accordingly, the X-X and Y-Y axes extend outwardly in directions perpendicular to the viewing axis 307.
In this specification, for the sake of clarity, positions sensed by the encoders on the master which relate to joint positions are referred to as “joint space” positions. Similarly, for the sensors on the joints of the robotic arm and the wrist mechanism, positions determined by these sensors are also referred to as “joint space” positions. The robotic arm and wrist mechanism will be referred to as the slave in the description which follows. Furthermore, references to positions and positioned signals may include orientation, location, and/or their associated signals. Similarly, forces and force signals may generally include both force and torque in their associated signals.
As the master is moved, signals em from the encoders on the master is input to a master input controller at 406 as indicated by arrow AB2. At the master input controller 406, the signals em are converted to a joint space position θm corresponding to the new position of the master. The joint space position θm is then input to a master kinematics converter 408 as indicated by arrow AB3. At 408 the joint position θm is transformed into an equivalent Cartesian space position xm. This is optionally performed by a kinematic algorithm including a Jacobian transformation matrix, inverse Jacobian (J−1), or the like. The equivalent Cartesian space position xm is then input to a bilateral controller at 410 as indicated by arrow AB4.
An exemplary controller block diagram and data flow to flexibly couple pairs of master controllers with manipulator arms are shown in FIGS. 11A-11D. As described above, the operator 402 manipulates manipulators 404, here inputting actuation forces against both the left and right master manipulators fh(L, R). Similarly, both left and right positions of the master input devices will also be accommodated by the control system, as will forces and positions of four or more slave manipulator arms fe(1, 2, 3, and 4), xe(1, 2, 3, and 4). Similar left, right, and slave notations apply throughout FIGS. 11A-11D.
During pair re-assignment, appropriate data sets and/or transformations reflecting the kinematics of the master/slave pairs, the relationship of the image capture device with the end effectors, and the like, may be transmitted to the controller. To facilitate swapping the image capture device from one manipulator to another, it may be beneficial to maintain a common manipulator structure throughout the system, so that each manipulator includes drive motors for articulating tools, endoscope image transfer connectors, and the like. Ideally, mounting of a particular tool on a manipulator will automatically transmit signals identifying the tool to the control system, as described in co-pending U.S. Patent Application Ser. No. 60/111,719, filed on Dec. 8, 1998, (Attorney Docket No. 17516-003210) entitled “Surgical Robotic Tools, Data Architecture, and Use.” This facilitates changing of tools during a surgical procedure.
The master/slave interaction between master control station 200 and cart 300 is generally maintained while the operator O is actively manipulating tissues with surgical instruments associated with his or her left and right hands. During the course of a surgical procedure, this master/slave interaction will be interrupted and/or modified for a variety of reasons. The following sections describes selected interruptions of the master/slave control interaction, and are useful for understanding how similar interruptions and reconfigurations of the telesurgical robotic network may be provided to enhance the capabilities of the overall robotic system. The exemplary interruptions include “clutching” (repositioning of a master control relative to a slave), repositioning of an endoscope, and a left-right tool swap (in which a tool previously associated with a master control input device in a right hand of a surgeon is instead associated with an input device in a left hand of the surgeon, and vice versa.) It should be understood that a variety of additional interruptions may occur, including during removal and replacement of a tool, during manual repositioning of a tool, and the like.
Clutching. In the course of performing a surgical procedure, the surgeon may wish to translationally reposition one or both of the master controls relative to the position or positions of a corresponding end effector or effectors as displayed in the image. The surgeon's dexterity is generally enhanced by maintaining an ergonomic orientational alignment between the input device and the image of the end effector. The surgeon may reposition the master relative to the end effector by simply interrupting the control loop and re-establishing the control loop in the desired position, but this can leave the end effector in an awkward orientation, so that the surgeon repeatedly opens the control loop to reorient the end effectors for each translational repositioning. Advantageously, the ergonomic rotational alignment between input devices and the images of the end effectors can be preserved after the master control or controls have been repositioned by a modified clutching procedure, which will now be described with reference to FIGS. 12 and 13.
Referring to FIG. 12, a block diagram indicating the repositioning of one of the master controls is indicated generally by reference numeral 450 and will now be described. It will be appreciated that both master controls can be re-positioned simultaneously. However, for ease of description, the repositioning of a single master control will be described. To reposition the master control relative to its associated slave, the surgeon causes the control loop 400 linking master control movement with corresponding slave movement to be interrupted. This is accomplished by activation by the surgeon of a suitable input device, labeled “Depress Master Clutch Button” at 452 in FIG. 12. It has been found that such a suitable input device can advantageously be in the form of a foot pedal as indicated at 208 in FIG. 2. It will be appreciated that any suitable input can be provided such as voice control input, a finger button, or the like. It is advantageous to provide an input device which does not require the surgeon to remove his or her hands from the master controls so as to preserve continuity of master control operation. Thus, the input device can be incorporated on the master control device itself instead of having a foot pedal.
Where substantially instantaneous changes in perimetric values and/or configuration are imposed, it is possible that a sudden change in motor currents may result, causing the system to jerk. Such inadvertent instantaneous movements of the system may be transmitted to the surgeon or other system operator, and can be disconcerting and/or reduce the overall feel of control the operator has over the system. Additionally, unexpected rapid movements of a surgical instrument at a surgical site are preferably minimized and/or avoided. Hence, rather than effecting these changes in perimetric values and/or configuration instantaneously, the changes will preferably be timed and executed in a manner so as to avoid significant instantaneous changes in the computed motor currents applied before, during, and after the change in configuration. This smooth change of perimetric values and/or controller configurations may be provided by a “no-jerk” algorithm which will be described with reference to FIG. 19A.
In FIG. 14 a block diagram indicating steps involved in repositioning a slave relative to its associated master is generally indicated by reference numeral 500. When it is desired to move the end effector of a slave to a new position, a suitable input is activated to interrupt the control loop 400 between the master and the slave. Such a suitable input can be in the form of a button on the robotic arm as indicated at 480 in FIG. 5A. Depressing such a button to interrupt the control loop 400 is indicated by the term “Depress Slave Clutch Button” at 504 in FIG. 14. Once the button is depressed, the control between master and slave is interrupted to cause the translational movements of the slave to float while the orientation of the end effector is locked as indicated at 502 in FIG. 14.
Endoscope Movement. Referring now to FIGS. 11, 16, and 17, repositioning of the endoscope to capture a different view of the surgical site will now be described. As the surgeon may wish to view the surgical site from another position, endoscope arm 302 can selectively be caused to vary its position so as to enable the surgical site to be viewed from different positions and angular orientations. The arm 302 includes appropriately positioned electrical motors controllable from the control station 200. The endoscope arm can thus be regarded as a slave and is typically controllable in a control loop similar to that shown in FIG. 11. Regarding the endoscope as another slave, cart 300 has three slaves, the robotic arm assemblies 310 and 304, and two masters 210.
An exemplary method and system for robotic movement of the endoscope using both of the master controllers is described in more detail in Application Ser. No. 60/111,711, filed on Dec. 8, 1998, and entitled “Image Shifting for a Telerobotic System,” the full disclosure of which is incorporated herein by reference.
The suitable input device is typically in the form of a depressible button on the endoscope arm 302. However other methods such as voice control or the like can be used instead. The button is similar to the button on the arm 312 as described above. The depressing of such a button is indicated at 602 in FIG. 16 and is labeled “Depress camera slave clutch button”. Upon activation of the input button the tool slaves and masters are servo locked at the positions they were at immediately before activation of the input button.
Left-Right Tool Swap. Referring now to FIG. 20 of the drawings, in which like reference numerals are used to designate similar parts unless otherwise stated, an image as viewed by the surgeon, and as captured by the endoscope, is generally indicated by reference numeral 800.
Robotic Network. Referring now to FIGS. 1, 23A and 23B, many of the above steps may be used to selectively associate any of a plurality of tools with any of a plurality of input devices. Operator O may initiate a tool selection subroutine 910 by actuating a tool selector input, such as by depressing foot activated button 208a of workstation 200 (illustrated in FIG. 2). Assuming operator O is initially manipulating tools A and B with input devices 210L and 210R using his or her left and right hands LH and RH, respectively, tool selector procedure 910 will be described with reference to a change of association so that input device 210L is instead associated with a tool C, here comprising a tissue stabilizer 120.
Once the tool selector subroutine is activated, the operator will generally select the desired tools to be actively driven by the robotic system. The surgeon here intends to maintain control over Tool B, but wishes to reposition stabilizer 120. Optionally, operator O will select between the left and right input devices for association with the newly selected tool. Alternatively, the processor may determine the appropriate left/right association based on factors more fully described in co-pending U.S. Patent Application Ser. No. 60/116,891, filed on Jan. 22, 1999, and entitled “Dynamic Association Of Master And Slave In A Minimally Invasive Telesurgical System,” (Attorney Docket No. 17516-004700) the full disclosure of which is incorporated herein by reference.
Optionally, operator O may select the desired tools for use by sequentially depressing selector input 208a, with the processor sequentially indicating selection of, for example, Tools A and B, then B and C, then A and C, and the like. Controller station 200 may indicate which tools are selected on display 800, audibly, or the like. For example, the image of the selected tools viewable by the surgeon may be colored green to designate active manipulation status, and/or the deselected tools may be colored red. Preferably, any deselected tools (for example, Tool A) will be maintained in a fixed position per step 914. The tools may be held in position using a brake system and/or by providing appropriate signals to the drive motors of the tool and arm actuation system to inhibit movement of the tool. The tool fixation step 914 will preferably be initiated before a master input device is decoupled from the tool, so that no tool moves absent an instruction from an associated master. Tool fixation may occur simultaneously with tool selection. The selected master may be allowed to float, step 916, during and/or after tool fixation and tool selection.
A number of alternative specific procedures may be used to implement the method outlined in FIG. 23B. Optionally, the interface may allow the operator to manually move the input devices into apparent alignment with the desired tools while the tool selector button is depressed. In FIG. 23A, the surgeon might manually move master 210L from alignment with tool B into approximate alignment with tool C. The processor could then determine the tools to be driven based on the position of the input devices when the button is released, thereby allowing the operator to “grab” the tools of interest. Some or all of the tools (Tools A, B, and C) may optionally be maintained in a fixed configuration when the operator is moving the master controllers to grab the tools.
Advantageously, providing a “redundant” manipulator may reduce the need for a laparoscopic surgical assistant who might otherwise be called on to perform intermittent functions by manually manipulating a tool handle extending from an aperture adjacent the manipulator arms. This can help avoid interference between manual tools, personnel, and the moving manipulator arms, and may have economic advantages by limiting the number of highly skilled personnel involved in a robotic surgical procedure. The procedure time may also be decreased by avoiding the time generally taken for a lead surgeon to verbally direct an assistant.
Tool Hand-Off. Many of the steps described above will also be used when “handing-off” control of a tool between two masters in a tool hand-off subroutine 920, as illustrated in FIG. 24. Tool hand-off is again initiated by actuating an appropriate input device, such as by depressing foot pedal 208b shown in FIG. 2.
Camera Switch and Right Access Robotic CABG. The following pertains to an exemplary robotic surgery procedure that may be performed with the foregoing apparatuses and methods. Referring now to FIGS. 1, 25A, and 25B, a single complex minimally invasive surgery will often involve interactions with tissues that are best viewed and directed from different viewing angles. For example, in performing a Coronary Artery Bypass Grafting (CABG) procedure on patient P, a portion of the internal mammary artery IMA will be harvested from along the internal surface of the abdominal wall. The internal mammary artery IMA can be used to supply blood to coronary artery CA downstream of an occlusion, often using an end-to-side anastomosis coupling the harvested end of the IMA to an incision in the side of the occluded coronary artery. To provide appropriate images to Operator O at master control station 200, the operator may sequentially select images provided by either a first scope 306a or a second scope 306b for showing on display 800 of the workstation. The camera switch procedure can be understood through a description of an exemplary CABG procedure in which different camera views may be used. Two scopes are shown in FIG. 25A for illustrative purposes only. If only one image is desired, however, the procedure need not employ two endoscopes but instead need only use one together with various instruments for actually performing the procedure.
By inserting the elongate shafts of instruments 100 through the right side of the patient, the apertures will be further away from the target anatomy, including the left internal mammary artery (LIMA) and heart H. This approach can allow the camera to be separated from the target tissues by a greater distance, such as when a panorama or “big-picture” view is desired, while the resolution of robotic movement maintains the surgeon's dexterity when the scope and tools extend across the chest to the heart tissues for close-up views and work. The right-side approach may also increase the speed with which tools can be changed, as the additional separation between the aperture and the heart helps to ensure that the heart is not in the way when delivering tools to harvest the IMA. When performing multi-vessel cases with the right-side approach, the heart can also be repeatedly retracted and repositioned so as to sequentially expose target regions of the heart to the significant working volume available. Hence, different coronary vessels may selectively be present to operator O for bypassing.
As can be seen most clearly in FIGS. 1 and 25B, cart 300 supports first and second tools 100a, 100b for manipulating tissues (more may be used but are not shown) and first scope 306a, while auxiliary cart 300A supports second scope 306b (and/or other manipulator tools, not shown). The arms of cart 300 preferably extend over the patient from the patient's left side, and the instruments extend through aperture pattern 930. The instrument shafts are generally angled to extend radially outwardly from aperture pattern 930 in a “spoked wheel” arrangement to minimize interference between the manipulators. The exemplary arrangement has scopes 306a, 306b extending through apertures defining the top and bottom (anterior and posterior relative to the patient) positions of aperture pattern 930, while the manipulation tool shafts define left and right (inferior and superior relative to the patient) positions. Second scope 306b may be positioned through a lower, more dorsal aperture than shown, with the patient optionally being supported on a table having an edge RE which is recessed adjacent aperture pattern 930 to avoid interference between the auxiliary cart manipulator and the table.
4. Camera aperture for second scope 306b is cut in an appropriate innerspace (usually the 5th intercostal space) on approximately the anterior axillary line. The first camera port may be positioned more medially for directing anastomosis, and the like. If provided or desired, the camera aperture for the first scope 306a is cut in an appropriate innerspace (usually the 4th or 5th intercostal space) slightly posterior to the midclavicular line. Obviously, port placement for the endoscopic tools as well as other portions of this procedure may vary depending upon the anatomy of the particular patient in question.
6. Manipulation tool apertures are cut as appropriate (usually in the 3rd and 6th or 7th intercostal spaces for manipulation instruments 100a, 100b) a few centimeters medial to the anterior auxiliary line. Additional tool ports are placed as desired.
7. Robotic instruments 100a, 100b, 306a, 306b introduced through apertures and robotic telesurgical control system is initiated.
8. LIMA harvesting is initiated by locating midline to establish the beginning of dissection. Harvesting may be viewed and directed using second scope 306b.
9. LIMA is located by moving laterally using blunt dissection and cautery as desired.
16. Anastomosis may be performed using needle-grasping tools 100a, 100b while viewing display 800, as illustrated in FIG. 23A.
Suturing and exposure of the aorta and coronary artery or arteries may at least in part be performed while viewing the more anterior-to-posterior field of view provided from first scope 306a, as may portions of all other steps throughout the CABG procedure. When the surgeon desires to change views between first and second image capture devices, the surgeon may initiate the view change procedure by activating a view change input device, possibly in the form of yet another foot switch. The tissue manipulation tools will be briefly fixed in position, and the display will shift between the image capture devices—for example, from the image provided from first scope 306a, to the image provided from second scope 306b.
Optionally, the processor can reconfigure the coordinate transformations between the masters and the end effectors when changing between two different image capture devices to re-establish an at least substantially connected relationship. This transformation modification is similar to the process described above for a change in scope position, but will generally also accommodate the differences in support structure of the image capture devices. In other words, for example, the master and/or slave kinematics 408, 412 (see FIG. 11) may be redefined to maintain a correlation between a direction of movement of the input device 210 and a direction of movement of an image of the end effector 102 as shown in display 202 when viewing the end effector from a different scope. Similarly, when moving second scope 306b (supported by auxiliary cart 300A) as a slave after a scope change from scope 306a (which is supported by cart 300), the slave kinematics 412, slave input/output 414, and slave manipulator geometry 416 may all be different, so that the control logic between the master and slave may be revised as appropriate.
More easily implemented approaches might allow the operator O to switch views between scopes 306a and 306b without major software revisions. Using software developed to perform telesurgery with a single master control station 200 coupled to a single three arm cart 300 (see FIG. 1), switching the view to scope 306b from scope 306a might be accomplished while maintaining the substantially connected relationship by “fooling” the processor of the master control station into believing that it is still viewing the surgery through scope 306a. More accurately, the processor may be fed signals which indicate that the middle set-up joint 395 and/or manipulator arm 302 of cart 300 are supporting scope 306a at the actual orientation of scope 306b. This may be accomplished by decoupling the position sensing circuitry of the middle set-up joint and/or manipulator of cart 300 from the processor, and instead coupling an alternative circuit that transmits the desired signals. The alternative “fooling” circuit may optionally be in the form of a sensor system of an alternative set-up joint and/or manipulator 302, which might be manually configured to hold a scope at the orientation of scope 306b relative to cart 300, but which need not actually support anything. The image may then be taken from scope 306b supported by auxiliary cart 300A, while the slave position signals xs (See FIG. 11) are taken from the alternative set-up joint. As described above, so long as the orientation of the end effectors relative to the scope are accurately known, the system can easily accommodate positional corrections (such as by the translational clutching procedure described above).
A slightly more complicated arrangement of surgical manipulators on two systems within the scope of the present invention, occurs when operators are provided with the ability to “swap” control of manipulator arms. For example, the first operator is able to procure control over a manipulator arm that is directly connected to the second operator's system. Such an arrangement is depicted in FIG. 26.
In addition to enabling cooperative surgery between two or more surgeons, operatively hooking two or more operator control stations together in a telesurgical networking system also may be useful for surgical training. A first useful feature for training students or surgeons how to perform surgical procedures would take advantage of a “playback” system for the student to learn from a previous operation. For example, while performing a surgical procedure of interest, a surgeon would record all of the video information and all of the data concerning manipulation of the master controls on a tangible machine readable media. Appropriate recording media are known in the art, and include videocassette or Digital Video Disk (DVD) for the video images and/or control data, and Compact Disk (CD), e.g., for the servo data representing the various movements of the master controls.
During playback of the operation, a student could place his hands on the master controls and “experience” the surgery, without actually performing any surgical manipulations, by having his hands guided by the master controls through the motions of the slave manipulators shown on the video display. Such playback might be useful, for example, in teaching a student repetitive motions, such as during suturing. In such a situation, the student would experience over and over how the masters might be moved to move the slaves in such a way as to tie sutures, and thus hopefully would learn how better to drive the telesurgical system before having to perform an operation.
The principles behind this playback feature can be built upon by using a live hand of a second operator instead of simple data playback. For example, two master control consoles may be connected together in such a way that both masters are assigned to a single set of surgical instruments. The master controls at the subordinate console would follow or map the movements of the masters at the primary console, but would preferably have no ability to control any of the instruments or to influence the masters at the primary console. Thus, the student seated at the subordinate console again could “experience” a live surgery by viewing the same image as the surgeon and experiencing how the master controls are moved to achieve desired manipulation of the slaves.
An alternative to this “on-off” clutching—whereby the instructor surgeon is either subordinate to the student or in command—would be a variable clutch arrangement. For example, again the instructor is subordinate to the student's performance of a procedure, and has his masters follow the movement of the student's master controls. When the instructor desires to participate in the procedure, but does not desire to wrest all control from the student, the instructor could begin to exert some control over the procedure by partially clutching and guiding the student through a certain step. If the partial control was insufficient to achieve the instructor's desired result, the instructor could then completely clutch in and demonstrate the desired move, as above. Variable clutching could be achieved by adjusting an input device, such as a dial or a foot pedal having a number of discrete settings corresponding to the percentage of control desired by the instructor. When the instructor desires some control, he or she could operate the input device to achieve a setting of, for example, 50 percent control, in order to begin to guide the student's movements. Software could be used to calculate the movements of the end effectors based on the desired proportionate influence of the instructor's movements over the student's. In the case of 50% control, for example, the software would average the movements of the two sets of master controls and then move the end effectors accordingly, producing resistance to the student's desired movement, thereby causing the student to realize his error. As the surgeon desires more control, he or she could ratchet the input device to a higher percentage of control, finally taking complete control as desired.
44. A medical robotic system comprising:
a plurality of robotic arms;
a first control station having first and second input devices;
a second control station having a third input device; and
a controller configured to switch associations between the plurality of robotic arms and the input devices of the first and second control stations in response to selection inputs received from an operator of the first control station, during the performance of a medical procedure on a patient, to operatively couple the operator selected manipulator arms to the operator selected input devices.
45. The medical robotic system of claim 44, further comprising an arm selector coupled to the controller for selecting between a plurality of modes, wherein a first arm is operatively associated with the second control station when the controller is in a first mode, and wherein the first arm is operatively associated with the first control station when the controller is in a second mode, the operatively associated arm moving an associated tool in response to movement of the operatively associated input device.
46. The medical robotic system of claim 44, wherein a second arm is operatively associated with the first control station when the controller is in the first mode, and wherein the second arm is operatively associated with the second control station when the controller is in the second mode.
47. The medical robotic system of claim 44, further comprising:
an association interrupting input device;
wherein the controller is configured to interrupt control of one of the plurality of robotic arms by a currently associated one of the input devices of the first and second control stations upon operator activation of the association interrupting input device, accept an operator selection of another one of the plurality robotic arms to be associated with the currently associated one of the input devices of the first and second control stations while disassociating the another one of the plurality of robotic arms from a previously associated one of the input devices of the first and second control stations, and associate the currently associated one of the input devices of the first and second control stations to the plurality of robotic arms so that a tool associated with the another one of the plurality of robotic arms is manipulatable according to manipulation of the previously associated one of the input devices of the first and second control stations following user deactivation of the association interrupting input device.
48. The medical robotic system according to claim 47, wherein the association interrupting device is a button manually activated by the operator.
49. The medical robotic system according to claim 47, wherein the association interrupting device is a foot pedal manually activated by the operator.
50. The medical robotic system according to claim 47, wherein the association interrupting device is voice controlled.
51. The medical robotic system according to claim 47, wherein the association interrupting device is a computer mouse.
52. The medical robotic system according to claim 47, wherein the tool is a surgical instrument.
53. The medical robotic system according to claim 47, wherein the tool is an endoscope.
Publication number: 20110137322
Patent Grant number: 8489235
Applicant: Intuitive Surgical Operations (Sunnyvale, CA)
Inventors: Frederic H. Moll (Woodside, CA), David J. Rosa (San Jose, CA), Andris D. Ramans (Mountain View, CA), Steven J. Blumenkranz (Redwood City, CA), Gary S. Guthart (Foster City, CA), Gunter D. Niemeyer (Mountain View, CA), William C. Nowlin (Los Altos, CA), J. Kenneth Salisbury, JR. (Los Altos, CA), Michael J. Tierney (Pleasanton, CA)
Application Number: 12/959,704