Patent Description:
Robotic surgical systems can have a number of robotic arms that move attached instruments or tools, such as an image capturing device, a stapler, an electrosurgical instrument, etc., in response to movement of input devices by a surgeon viewing images captured by the image capturing device of a surgical site. During a robotic surgical procedure, each of the tools is inserted through an opening, either natural or an incision, into the patient and positioned to manipulate tissue at a surgical site. The openings are placed about the patient's body so that the surgical instruments may be used to cooperatively perform a robotic surgical procedure and the image capturing device may view the surgical site.

During a robotic surgical procedure it is important to accurately know and control the position of the tools within the surgical site. Accordingly, there is a continuing need for systems and methods for detecting and controlling the position of tools within a surgical site during robotic surgical procedures. Related prior art can be found in <CIT> and <CIT>.

The methods mentioned below do not form part of the claimed subject-matter. This disclosure relates controlling a surgical robot from visually capturing a tool pose within a surgical site. In disclosed methods, an imaging device captures a tool pose of a tool within a surgical site. The surgical robot then determines an arm pose of a linkage supporting the tool. The surgical robot then manipulates the arm to move the tool to a desired tool pose in response to input from a clinician. Visually determining the tool pose provides improved accuracy and resolution to the position of the tool within the surgical site. The improved accuracy and resolution can be used to complete precision movements and may assist in completing automated actions. Visually determining the tool pose also eliminates discrepancies in kinematic models that are induced by loads and dynamic performance of joints.

In some methods disclosed herein functions of tools are enabled and disabled based on a visually captured tool pose. The function of the tool can be enabled as the end effector of the tool is within an enabled zone of the surgical site. The enabled zone can be determined based on the location of targeted tissue, the size of targeted tissue, the proximity of nontargeted tissue relative to targeted tissue, or the type of function of the tool. The surgical robot can be controlled by a user interface which includes a display. The display shows a graphical representation of the surgical site and can provide visual indicia of the enabled zone and the enabled/disabled status of the function of the tool. For example, the display can have a border that shows one color when the function of the tool is enabled and another color when the function of the tool is disabled. Additionally or alternatively, the display can show the tool in one color when the function of the tool is enabled and show the tool in another color when the function of the tool is disabled. It is contemplated that when the tool is in the enabled zone, the surgical robot can complete automated tasks within the surgical site with the tool. In particular methods, the user interface can track the gaze of the clinician to verify that the clinician's gaze is focused or directed to a representation of the enabled zone on the display before enabling the function of the tool. It is contemplated that by requiring a tool to be within the enabled zone and/or that the clinician's gaze is directed to the enabled zone before activating a function of the tool can increase safety during a robotic surgical procedure by reducing inadvertent or unintended activations of tool functions when the tool is outside of an enabled zone.

In certain methods, a center of view of an imaging device can automatically track a centroid during a surgical procedure. The tracked centroid can be a centroid of a tool, a point between a centroid of a tool and targeted tissue, or a point between centroids of multiple tools. The tracked centroid can be automatically assigned to active tools within the surgical site or can be selectively assigned by the clinician. By automatically tracking a centroid during a surgical procedure, a clinician can concentrate on the procedure without having to focus on moving the imaging device. The method may include receiving the control signal from a user interface of a robotic surgical system.

In an aspect of the present disclosure, a method of controlling a surgical robot includes visually capturing a first tool pose of a first tool within a surgical site in a fixed frame of reference with an imagining device, determining a first arm pose of a first linkage supporting the first tool from known geometries of the first linkage in the fixed frame of reference, and manipulating the first linkage to move the first tool to a desired first tool pose in the fixed frame of reference in response to a first control signal.

In aspects, visually capturing the first tool pose of the first tool in the fixed frame of reference includes defining the fixed frame of reference in a frame defined by the imaging device. Visually capturing the first tool pose of the first tool in the fixed frame of reference may include capturing the first tool pose with both a first lens and a second lens of the imaging device.

In some aspects, visually capturing the first tool pose includes identifying the position of one or more markers on the first tool. Visually capturing the first tool pose may include capturing the position of the one or more markers within an infrared spectrum of light.

In certain aspects, the method includes visually capturing a second tool pose of a second tool within the surgical site in the fixed frame of reference with the imaging device, determining a second arm pose of a second linkage supporting the second tool from known geometries of the second linkage in the fixed frame of reference, and manipulating the second linkage to move the second tool to a desired second tool pose in the fixed frame of reference in response to a second control signal. Determining the first arm pose and determining the second arm pose may occur entirely within the fixed frame of reference.

In another aspect of the present disclosure, a method of controlling a function of a tool of a surgical system including capturing images of a surgical site with an imaging device, determining a distance of the tool within the surgical site relative to targeted tissue, enabling activation of a function of the tool when the tool is within a predetermined distance from the targeted tissue, and activating the function of the tool to manipulate the tool in response to a control signal.

In aspects, the enabling activation of the function of the tool includes providing visual indicia to a clinic engaged with the surgical system that the function is enabled. Providing visual indicia may include changing a color of a border of a display of the surgical system.

In some aspects, the method may include disabling activation of the function of the tool when the tool is beyond the predetermined distance from the targeted tissue. Disabling activation of the function may include providing visual indicia to a clinician engaged with the surgical system that the function is disabled. Providing visual indicia may include changing a color of a border of a display of the surgical system.

In certain aspects, the method may include the surgical system completing an automated task within the surgical site with the tool when the tool is within the predetermined distance from the target tissue. Completing the automated task may include suturing the targeted tissue when the tool is within the predetermined distance from the targeted tissue.

In particular aspects, the method may include verifying that a gaze of a clinician interfacing with the surgical system is directed to an enabled zone on a display of the surgical system before enabling activation of the function of the tool. Activating the function of the tool to manipulate tissue with the tool may include at least one of clamping tissue with the tool, delivering electrosurgical energy to tissue with the tool, stapling tissue with the tool, suturing tissue with the tool, or advancing a cutting edge or knife of the tool through tissue.

In another aspect of the present disclosure, a surgical system includes an imaging device, a tool, and a processing unit. The imaging device is configured to capture images of a surgical site. The tool has a function that is configured to manipulate tissue in response to a control signal. The processing unit is in communication with the imaging device and the tool and is configured to determine a distance of the tool relative to targeted tissue from the captured images and enable activation of the function of the tool when the tool is positioned within a predetermined distance of the targeted tissue.

In aspects, the surgical system includes a display that is configured to provide a representation of the surgical site. The processing unit may be configured to provide a representation of an enablement zone defined by the predetermined distance within the representation of the surgical site. The processing unit may be configured to provide visual indicia on the display when the function of the tool is enabled. The display may be configured to change a color of a border of the display when the function of the tool is enabled.

In some aspects, the surgical system includes a display that is configured to provide a representation of the surgical site. The processing unit may be configured to verify that a gaze of a clinician is directed to the display before enabling activation of the function of the tool. The processing unit may be configured to complete an automated task when the tool is within the predetermined distance of the targeted tissue. The processing unit may be configured to prevent activation of the function of the tool when the tool is positioned beyond the predetermined distance from the targeted tissue.

In another aspect of the present disclosure, a method of manipulating an imaging device includes identifying a tracked centroid within a field of view of the imaging device, manipulating a pose of the imaging device to posing the tracked centroid at a center of the field of view of the imaging device, moving a first tool within the field of view such that the tracked centroid is moved within the field of view of the imaging device, and tracking the tracked centroid as the first tool is moved within the field of view and maintaining the tracked centroid at the center of the field of view of the imaging device.

In aspects, identifying the tracked centroid includes defining the tracked centroid as a first tool centroid of the first tool. Alternatively, identifying the tracked centroid may include defining the tracked centroid as a point between a first tool centroid of the first tool and targeted tissue. The tracked centroid may be a midpoint of a line between the first tool centroid and a centroid of the targeted tissue.

In some aspects, the method includes moving a second tool within the field of view such that the tracked centroid is moved within the field of view of the imaging device. Identifying the tracked centroid may include defining the tracked centroid as appoint between a first tool centroid of the first tool and a second tool centroid of the second tool. The tracked centroid may be a midpoint of a line between the first tool centroid and the second tool centroid. Alternatively, identifying the tracked centroid may include defining the tracked centroid as a point triangulated between a first tool centroid of the first tool, a second tool centroid of the second tool, and targeted tissue.

In certain aspects, manipulating the pose of the imaging device includes moving an arm of a surgical robot supporting the imaging device. Tracking the tracked centroid may include delaying reentering of the field of view of the imaging device until the tracked centroid is misaligned a predetermined distance from the center of the field of view. Tracking the tracked centroid may include limiting a velocity of movement of the field of view of the imaging device.

In another aspect of the present disclosure, a surgical system includes a first tool, an arm, and an imaging device. The first tool is at least partially defines a tracked centroid. The arm is movable within a surgical site. The imaging device is supported on the arm and has a field of view. The imaging device is configured to be manipulated to maintain the tracked centroid at a center of the field of view.

In aspects, the tracked centroid is defined at a first tool centroid of the first tool. Alternatively, the tracked centroid may be defined at a point between a first tool centroid of the first tool and a target within the surgical site.

In some aspects, the surgical system includes a second tool. The tracked centroid may be defined at a point between a first tool centroid of the first tool and a second tool centroid of the second tool. The tracked centroid may be triangulated at a point between a first tool centroid of the first tool, a second tool centroid of the second tool, and a target within the surgical site.

It is envisioned that the methods herein can be implemented in the software of existing robotic surgical systems to improve the efficacy of the existing system. In addition, some of the methods detailed herein can be enhanced with specialized equipment.

Further, to the extent consistent, any of the aspects described herein may be used in conjunction with any or all of the other aspects described herein.

Various aspects of the present disclosure are described hereinbelow with reference to the drawings, which are incorporated in and constitute a part of this specification, wherein:.

Embodiments of the present disclosure are now described in detail with reference to the drawings in which like reference numerals designate identical or corresponding elements in each of the several views. As used herein, the term "clinician" refers to a doctor, a nurse, or any other care provider and may include support personnel. Throughout this description, the term "proximal" refers to the portion of the device or component thereof that is closest to the clinician or surgical robot arm and the term "distal" refers to the portion of the device or component thereof that is farthest from the clinician or surgical robot arm.

Referring to <FIG>, a robotic surgical system <NUM> in accordance with the present disclosure is shown generally as a robotic system <NUM>, a processing unit <NUM>, and a user interface <NUM>. The robotic system <NUM> generally includes linkages or arms <NUM> and a robot base <NUM>. The arms <NUM> moveably support a tool <NUM> having an end effector <NUM> which is configured to act on tissue. The arms <NUM> each have an end <NUM> that supports tool <NUM>. In addition, the ends <NUM> of the arms <NUM> may include an imaging device <NUM> for imaging a surgical site "S". The user interface <NUM> is in communication with robot base <NUM> through the processing unit <NUM>.

The user interface <NUM> includes a display device <NUM> which is configured to display three-dimensional images. The display device <NUM> displays three-dimensional images of the surgical site "S" which may include data captured by imaging devices <NUM> positioned on the ends <NUM> of the arms <NUM> and/or include data captured by imaging devices that are positioned about the surgical theater (e.g., an imaging device positioned within the surgical site "S", an imaging device positioned adjacent the patient, imaging device <NUM> positioned at a distal end of an imaging linkage or arm <NUM>). The imaging devices (e.g., imaging devices <NUM>, <NUM>) may capture visual images, infra-red images, ultrasound images, X-ray images, thermal images, and/or any other known real-time images of the surgical site "S". The imaging devices transmit captured imaging data to the processing unit <NUM> which creates three-dimensional images of the surgical site "S" in real-time from the imaging data and transmits the three-dimensional images to the display device <NUM> for display.

The user interface <NUM> also includes input handles <NUM> which are supported on control arms <NUM> which allow a clinician to manipulate the robotic system <NUM> (e.g., move the arms <NUM>, the ends <NUM> of the arms <NUM>, and/or the tools <NUM>). Each of the input handles <NUM> is in communication with the processing unit <NUM> to transmit control signals thereto and to receive feedback signals therefrom. Additionally or alternatively, each of the input handles <NUM> may include input devices (not shown) which allow the surgeon to manipulate (e.g., clamp, grasp, fire, open, close, rotate, thrust, slice, etc.) the end effectors <NUM> of the tools <NUM> supported at the ends <NUM> of the arms <NUM>.

Each of the input handles <NUM> is movable through a predefined workspace to move the ends <NUM> of the arms <NUM> within a surgical site "S". The three-dimensional images on the display device <NUM> are orientated such that movement of the input handle <NUM> moves the ends <NUM> of the arms <NUM> as viewed on the display device <NUM>. It will be appreciated that the orientation of the three-dimensional images on the display device may be mirrored or rotated relative to view from above the patient. In addition, it will be appreciated that the size of the three-dimensional images on the display device <NUM> may be scaled to be larger or smaller than the actual structures of the surgical site "S" permitting a clinician to have a better view of structures within the surgical site "S". As the input handles <NUM> are moved, the tools <NUM>, and thus the end effectors <NUM>, are moved within the surgical site "S" as detailed below. As detailed herein, movement of the tools <NUM> may also include movement of the ends <NUM> of the arms <NUM> which support the tools <NUM>.

For a detailed discussion of the construction and operation of a robotic surgical system <NUM>, reference may be made to <CIT>.

With reference to <FIG>, the robotic system <NUM> is configured to support the tool <NUM> (<FIG>) thereon and to selectively move the tool <NUM> in a plurality of orientations relative to a small incision in a patient "P" (<FIG>) while maintaining the tool <NUM> within the small incision. The arm <NUM> includes a plurality of elongate members or links <NUM>, <NUM>, <NUM>, <NUM> pivotably connected to one another to provide varying degrees of freedom to the arm <NUM>. In particular, the arm <NUM> includes a first link <NUM>, a second link <NUM>, a third link <NUM>, and a fourth link <NUM>.

The first link <NUM> has a first end 110a and a second end 110b. The first end 110a is rotatably coupled to a fixed structure. The fixed structure can be a movable cart <NUM> locked in position, a surgical table, a stanchion, an operating room wall, or other structure present in the operating room. A first motor "M1" is operably coupled to first end 110a to rotate the first link <NUM> about a first axis of rotation A<NUM> that passes through the first end 110a transverse to a longitudinal axis of the first link <NUM>. The second end 110b of first link <NUM> has a second motor "M2" operably coupled to a first end of 120a of the second link <NUM> such that actuation of motor "M2" effects a rotation of the second link <NUM> relative to first link <NUM> about a second axis of rotation A<NUM> defined through the second end 110b of first link <NUM> and a first end 120a of second link <NUM>. It is envisioned the second axis of rotation A<NUM> can be transverse to the longitudinal axis of the first link <NUM> and a longitudinal axis of the second link <NUM>.

A second end 120b of the second link <NUM> is operably coupled to the first end 130a of the third link <NUM> such that the third link <NUM> rotates relative to the second link <NUM> about a third axis of rotation As that passes through the second end 120b of the second link and the first end 130a of the third link <NUM>. The third axis of rotation As is parallel to the second axis of rotation A<NUM>. Rotation of the second link <NUM> about the second axis of rotation A<NUM> affects rotation of the third link <NUM> about the third axis of rotation As such that the first and third links <NUM>, <NUM> maintain a substantially parallel relationship with one another. For a detailed description of exemplary mechanisms to maintain the substantially parallel relationship between the first and third links, reference may be made to <CIT>, and entitled "SURGICAL ROBOTIC ARMS AND PULLEY ASSEMBLIES THEREOF" and PCT Patent Application No. <CIT>, and entitled "SURGICAL ROBOTIC ARMS AND PULLEY ASSEMBLIES THEREOF,".

A second end 130b of the third link <NUM> is operably coupled to a first end 140a of the fourth link <NUM>. The fourth link <NUM> is rotatable relative to the third link <NUM> about a fourth axis of rotation A<NUM> that passes through the second end 130b of the third link <NUM> and the first end 140a of the fourth link <NUM>.

With additional reference to <FIG>, the fourth link <NUM> can be in the form of a rail that supports a slider <NUM>. The slider <NUM> is slidable along an axis parallel to the longitudinal axis of the fourth link <NUM> and supports the tool <NUM>.

During a surgical procedure, the robotic system <NUM> receives input commands from the user interface <NUM> to move the tool <NUM> such that the end effector <NUM> is moved to manipulate and/or act on tissue within the surgical site "S". Specifically, the links <NUM>, <NUM>, <NUM>, <NUM> of the robot arm <NUM> are rotated relative to one another and the slider <NUM> is translated to position and orientate the tool <NUM> within the surgical site "S" in response to the input commands. To control the robot arm <NUM>, the robotic system <NUM> calculates a desired tool pose of the tool <NUM> from the input commands, captures a tool pose of the tool <NUM>, and manipulates the robot arm <NUM> to move the tool <NUM> to the desired tool pose. From the desired tool pose, the robotic system <NUM> calculates a required arm pose of the robot arm <NUM> to achieve the desired tool pose. The robot arm <NUM> then determines which links <NUM>, <NUM>, <NUM>, <NUM> to manipulate to reach the required arm pose and thus, the desired tool pose of the tool <NUM> within the surgical site "S" in response to input captured by the user interface <NUM> (<FIG>).

To determine the arm pose of the robot arm <NUM>, the robot system <NUM> uses an imaging device or endoscope <NUM> positioned within the surgical site "S" to capture the position and orientation or tool pose of the tool <NUM> within the surgical site "S". As detailed herein below, the endoscope <NUM> is described as capturing the tool pose within the surgical site; however, it is contemplated that imaging devices can be used and that each one of the imaging devices can include a single or multiple lenses to capture two or three dimensional images.

The endoscope <NUM> can be stationary within the surgical site "S", can be manipulated by a clinician within the surgical theater, or can be attached to another robot arm <NUM> such that the position and orientation of the endoscope <NUM> can be manipulated during a surgical procedure. The robotic system <NUM> uses the endoscope <NUM> to visually capture the tool pose of the tool <NUM> within the surgical site "S" using known techniques. The tool <NUM> may include indicia to aid in capturing the tool pose, which may include, but are not limited to, using distinct colors, distinct markings, distinct shapes, or combinations thereof. The tool pose of the tool <NUM> is captured in a camera frame relative to the endoscope <NUM> and can be translated to a frame of the surgical site "S", a frame of the tool <NUM>, a frame of the robot arm <NUM>, or any other desired frame of reference. It is envisioned that it may be beneficial to translate the tool pose of the tool <NUM> to a fixed frame.

From the tool pose of the tool <NUM>, the robotic system <NUM> can use known kinematics of the robot arm <NUM> to calculate an arm pose of the robot arm <NUM> starting from the tool pose of the tool <NUM> and working towards the first link <NUM>. By calculating the arm pose of the robot arm <NUM> from tool pose of the tool <NUM>, a solution to move the tool <NUM> to a desired tool pose within the surgical site "S" accounts for any deformations of the robot arm <NUM> or the tool <NUM> when under load. In addition, by calculating the arm pose from the tool pose, it is unnecessary to know the position of the fixed structure (e.g., movable cart <NUM>), to which the first link <NUM> (<FIG>) of the arm <NUM> is coupled, to determine a solution to move the tool <NUM> to the desired tool pose. In calculating the solution, the robotic system <NUM> accounts for any possible collisions of the arm <NUM> with other arms <NUM>, clinicians within the surgical theater, the patient, or other structures within the surgical theater. Further, by calculating the tool pose and/or the arm pose in a common frame, e.g., the camera frame of a single endoscope, the poses of the tools and/or arms can be computed at the same time by using the kinematics of each of the arms, e.g., arm <NUM>, to calculate the locations of the links, e.g., link <NUM>, to estimate possible collisions of the arm <NUM>.

It is contemplated that the robot system <NUM> can be used to simultaneously capture the tool pose of multiple tools <NUM> with the endoscope <NUM>. By capturing the tool pose of multiple tools <NUM>, the interaction of the tools <NUM> and the end effectors <NUM> of the tools <NUM> can be controlled with high precision. This high precision control can be used to complete automated tasks; for example, suturing tissue. It is envisioned that by using a single endoscope <NUM> to capture the tool poses of multiple tools <NUM>, the speed and accuracy of automated tasks can be increased by reducing the need to translate the high precision tool poses from a camera frame to another frame for the duration of the automated task.

It is contemplated that more than one camera and/or endoscope <NUM> can be used to simultaneously capture the tool pose of the tool <NUM> within the surgical site "S". It will be appreciated that when multiple cameras are used that it may be beneficial to translate the position and orientation of the tool <NUM> to a frame other than a frame defined by one of the cameras.

It is contemplated that determining the arm pose from the captured tool pose allows for determining the position of movable carts <NUM> supporting each of the arms <NUM> from the captured tool pose and the kinematics of the arms <NUM>. After the surgical procedure is completed, the efficiency of the surgical procedure can be determined and the position of the movable carts <NUM> recorded. By comparing the position of movable carts <NUM> during surgical procedures with high efficiency ratings, a guide or recommended locations of the movable carts <NUM> for a given procedure can be provided to increase the efficiency of future surgical procedures. Increased efficiency of surgical procedures can reduce cost, surgical time, and recovery time while improving surgical outcome.

Continuing to refer to <FIG> and additionally to <FIG>, a method of enabling and disabling a function of a tool <NUM> in response to a captured pose is disclosed in accordance with the present disclosure. During a surgical procedure, the endoscope <NUM> is used to determine the position of anatomical structures within a surgical site "S". The positions of the anatomical structures can be registered to presurgical scans such that targeted tissue "T" can be identified within the surgical site "S". The targeted tissue "T" can be identified prior to and/or during the surgical procedure and can be identified by the clinician performing the surgery or a clinician remote to the surgical procedure.

With the targeted tissue "T" identified, an enabled zone "EZ" is created about the targeted tissue "T" such that activation of a function of the tool <NUM> is limited to when the end effector <NUM> of the tool <NUM> is within the enabled zone "EZ". By limiting activation of the function of the tool <NUM> to when the end effector <NUM> is within the enabled zone "EZ" can prevent inadvertent or unintentional activation of the tool <NUM>.

The enabled zone "EZ" is based on geometric locales within the surgical site "S" adjacent the targeted tissue "T". The size of the enabled zone "EZ" can be based on the function or functions (e.g., clamping, delivery of electrosurgical energy, stapling, suturing, advancement of a knife, etc.) of the tool <NUM>. The enabled zone "EZ" can also be based on the proximity of other anatomical structures to the targeted tissue "T". For example, when other anatomical structures are spaced apart from the targeted tissue "T" the enabled zone "EZ" may be larger than when other anatomical structures are close to or in contact with the targeted tissue "T". The enabled zone "EZ" can be set manually before or during the surgical procedure or can be set automatically based on the function of the tool <NUM>. It is contemplated that for a given surgical procedure, targeted tissue "T" can be in more than one location with the surgical site "S". During such a surgical procedure, the surgical site "S" can include an enabled zone "EZ" about each targeted tissue "T".

With additional reference to <FIG>, a graphical representation of the enabled zone "EZ" can be shown on the display <NUM>. The enabled zone "EZ" can be shown as a cloud about the targeted tissue "T" or can be shown clear with the area outside the enabled zone "EZ" appearing as clouded. Additionally or alternatively, the enabled zone "EZ" can be represented by another form of visual delineation on the display <NUM>.

During the surgical procedure, the tool pose of the tool <NUM>, is captured by the camera as detailed above. When the robotic system <NUM> determines, from the tool pose, that the end effector <NUM> is outside of the enabled zone "EZ", the robotic system <NUM> prevents a clinician from activating a function of the tool <NUM>. It is contemplated that the display <NUM> may also provide a visual indication that the function of the tool <NUM> is disabled. For example, a border <NUM> on the display <NUM> may be red in color, the tool <NUM> or a portion of the tool <NUM> (e.g., the end effector <NUM>) may be red in color. Additionally or alternatively, an activation button (not shown) on the input handle <NUM> (<FIG>) of the user interface <NUM> may provide a visual indication (e.g., be backlit in the color red) that the function of the tool <NUM> is disabled.

As the robotic system <NUM> determines, from the tool pose, that the end effector <NUM> enters the enabled zone "EZ", the robotic system <NUM> enables the function of the tool <NUM>. As the tool <NUM> enters the enabled zone "EZ", the display <NUM> may provide a visual indication that the function of the tool <NUM> is enabled. For example, the border <NUM> on the display <NUM> may be green, the tool <NUM> or a portion of the tool <NUM> (e.g., the end effector <NUM>) may be green. Additionally or alternatively, an activation button (not shown) on the input handle <NUM> (<FIG>) of the user interface <NUM> may provide a visual indication (e.g., be backlit in green) that the function of the tool <NUM> is enabled.

It is envisioned that during a surgical procedure where multiple tools <NUM> are within the surgical site "S". The tools <NUM> may independently enable a function of the tool <NUM> based on the position of the end effector <NUM> of the respective tool relative to the enabled zone "EZ" of the targeted tissue "T". Alternatively, it is contemplated that where multiple tools are within the surgical site "S", that functions of the tools <NUM> may only be enabled when both end effectors <NUM> are positioned within the enabled zone "EZ". Limiting enablement of the functions in such a manner may be preferred when the tools <NUM> cooperate together to act on the targeted tissue "T".

The method may include verifying a gaze of a clinician engaged with the user interface <NUM> is directed to the enabled zone "EZ" on the display <NUM> before enabling a function of the tool <NUM>. Specifically, during the surgical procedure, the user interface <NUM> tracks the gaze of the clinician engaged therewith. As the endoscope <NUM> determines that the end effector <NUM> of one of the tools <NUM> enters the enabled zone "EZ", the user interface <NUM> verifies that the gaze of the clinician engaged with the user interface <NUM> is directed to a portion of the display <NUM> including a representation of the targeted tissue "T" and/or the enabled zone "EZ". Requiring the clinician's gaze to be directed to the target tissue "T" or the enabled zone "EZ" before enabling the function of the tool <NUM> provides an additional level of safety to the surgical procedure.

As detailed above, the endoscope <NUM> can be movable about the surgical site "S". It is contemplated that as the endoscope <NUM> is moved about the surgical site "S" the function of the tool <NUM> would be disabled until the position of the endoscope <NUM> is stationary. By disabling the function of the tool <NUM> as the endoscope <NUM> is moved about the surgical site "S" provides an additional level of safety to the surgical procedure.

Referring to <FIG>, a method for changing a center of view and/or field of view of the imaging device or endoscope <NUM> during a surgical procedure is described utilizing the robotic surgical system <NUM> detailed above. During a surgical procedure the view of the surgical site "S" can track the end effector <NUM> of a tool <NUM> within the surgical site "S". By tracking the end effector <NUM> of the tool <NUM>, the attention or focus of a clinician engaged with the user interface <NUM> can be directed to the surgical procedure and not be distracted or consumed by directing the center of view of the endoscope <NUM>.

The endoscope <NUM> is disposed on an arm <NUM> of the robotic system <NUM> (<FIG>) such that the robotic system <NUM> can manipulate the endoscope <NUM> during a surgical procedure. Initially the endoscope <NUM> is introduced into the surgical site "S" with a center of view of the endoscope directed towards an area of interest. The area of interest can be an entry point of the end effector <NUM> of the tool <NUM> into the surgical site "S" or can be directed to targeted tissue "T" within the surgical site "S". In addition, a field of view of the endoscope <NUM> can be set to encompass a large area of the surgical site "S".

With the field and center of view "CV" of the endoscope <NUM> set, the end effector <NUM> of the tool <NUM> is brought within a field of view "FV" the endoscope <NUM>. The endoscope <NUM> is then used to determine a tool pose of the tool <NUM> as detailed above. From the tool pose, a centroid "Ci" of the end effector <NUM> can be determined. The center of view of the endoscope <NUM> is re-centered to be directed to the centroid "Ci" of the end effector <NUM> such that the centroid "Ci" is tracked by the endoscope <NUM>. As used herein, it is understood that centroid may include features or the like, or locations that are mathematically computable.

During the surgical procedure it will be appreciated that the centroid "Ci" of the end effector <NUM> is moved about the surgical site "S". As the centroid "Ci" is moved, the center of view "CV" of the endoscope <NUM> re-centers to track the centroid "Ci" of the end effector <NUM>. By re-centering the center of view "CV" of the endoscope <NUM> during the surgical procedure, the attention and focus of the clinician can be directed to the surgical procedure.

The re-centering can be done in a manner such that the center of view "CV" of the moves in a manner that does not distract a clinician viewing the display <NUM> (<FIG>). The re-centering is done in a matter such that the velocity of the re-centering can be controlled to ensure a perceptually appropriate experience during re-centering of the center of view "CV". The re-centering can implement a dwell time of the centroid "Ci" such that the re-centering occurs in a smooth manner. The re-centering can also have a maximum velocity of the center of view "CV" of the endoscope <NUM>. In addition, the re-centering can also shape an acceleration/deceleration of the center of view "CV" of the endoscope <NUM> at the start and end of movement to keep the re-centering of the center of view "CV" to be comfortable for the clinician engaging the user interface <NUM>. Further, the re-centering may incorporate a form of hysteresis to prevent continuous chasing of the center of view "CV" to the end effector <NUM>. For example, the re-centering may not occur until the centroid of the end effector <NUM>, e.g., centroid "Ci", is offset from the center of view "CV" of the endoscope by a predefined distance, e.g., about <NUM>. It is envisioned that the re-centering can be fully automated or be selectively activated at the discretion of a clinician.

It is contemplated that, the center of view of the endoscope <NUM> can track a centroid "C<NUM>" which is centered between the centroid "Ci" of the end effector <NUM> and the targeted tissue "T". In addition, the endoscope <NUM> can adjust its field of view "FV" based on the distance between the centroid "Ci" of the end effector <NUM> and the targeted tissue "T" such that as the end effector <NUM> approaches the targeted tissue "T", the endoscope <NUM> zooms in or reduces the size of the field of view "FV". In addition, as the end effector <NUM> moves away from the targeted tissue "T", the endoscope <NUM> zooms out or increased the size of the field of view "FV".

During surgical procedures with two tools <NUM> within the surgical site "S", the center of view "CV" of the endoscope <NUM> can track a centroid "C<NUM>" that is a centered between the centroid "Ci" of a first end effector <NUM> and a centroid "C<NUM>" of a second end effector <NUM>. By tracking the centroid "C<NUM>" with the center of view "CV" of the endoscope <NUM>, interactions of the first and second end effectors <NUM> can be viewed by the clinician. In addition, the endoscope <NUM> can change its field of view "FV" to zoom in and out based on the distance between the centroids "C<NUM>" and "C<NUM>". Alternatively, the center of view "CV" of the endoscope <NUM> can track the centroid "C<NUM>" centered between the centroid "Ci" of the first end effector <NUM> and the targeted tissue "T" and disregard the centroid "C<NUM>" of the second end effector <NUM>. It will be appreciated that a form of hysteresis may also be introduced to the tracking of the centroid "C<NUM>" with the center of view "CV" of the endoscope <NUM>.

During surgical procedures with more than two tools <NUM> within the surgical site "S", the center of view "CV" of the endoscope <NUM> can track a centroid (e.g., centroid "C<NUM>") that is centered between centroids (e.g., centroids "C<NUM>" and "C<NUM>") of active end effectors <NUM> in a manner similar to that detailed above with respect to two tools <NUM>. As the active end effectors <NUM> change within the surgical site "S", the center of view "CV" of the endoscope <NUM> re-centers on a centroid between the active end effectors <NUM>.

Claim 1:
A surgical system comprising:
an imaging device configured to capture images of a surgical site the imaging device movable within the surgical site;
a tool having a function, the function configured to manipulate tissue in response to a control signal; and
a processing unit in communication with the imaging device and the tool, the processing unit configured to:
determine a distance of the tool relative to targeted tissue from the captured images; and
characterized in that, the processing unit is further configured to enable activation of the function of the tool when the tool is positioned within a predetermined distance of the targeted tissue and it is determined that the imaging device is stationary within the surgical site.