Patent Description:
Robotic surgical systems typically used a scaling factor to scale down the motions of the surgeons hands to determine the desired position of the end effector within the patient so that the surgeon could more precisely move the end effector inside the patient. However, the larger the scaling factor, the farther the surgeon had to move the input device handle to move the end effector the same distance. Since the input device handle has a fixed range of motion, this meant that for larger scaling factors the surgeon may have reached an end of the range of motion of an input handle more often.

In addition, during a medical procedure a surgeon needs to rotate the end effector about a roll axis, a pitch axis, and a yaw axis to properly position the end effector to act on tissue. Further, during a medical procedure, clutching of movement of the input handle relative to the input handle may cause the input handle to become misaligned with the end effector about one or more of the roll axis, the pitch axis, and/or the yaw axis.

There is a continuing need for a robotic surgical system that realigns the input handle with the end effector during a medical procedure.

<CIT> discloses a robotic surgical system which includes a linkage, an input device, and a processing unit. The linkage moveably supports a surgical tool relative to a base. The input device is rotatable about a first axis of rotation and a second axis of rotation. The processing unit is in communication with the input device and is operatively associated with the linkage to rotate the surgical tool about a first axis of movement based on a scaled rotation of the input device about the first axis of rotation by a first scaling factor and to rotate the surgical tool about a second axis of movement based on a scaled rotation of the input device about the second axis of rotation by a second scaling factor that is different from the first scaling factor.

<CIT> discloses a minimally-invasive surgical system which includes a slave surgical instrument having a slave surgical instrument tip and a master grip. The slave surgical instrument tip has an alignment in a common frame of reference and the master grip, which is coupled to the slave surgical instrument, has an alignment in the common frame of reference. An alignment error, in the common frame of reference, is a difference in alignment between the alignment of the slave surgical instrument tip and the alignment of the master grip. A ratcheting system (i) coupled to the master grip to receive the alignment of the master grip and (ii) coupled to the slave surgical instrument, to control motion of the slave by continuously reducing the alignment error, as the master grip moves, without autonomous motion of the slave surgical instrument tip and without autonomous motion of the master grip.

<CIT> discloses a master interface for a surgical robot, mounted on a master robot for controlling a slave robot, may include two or more robot arms each having a mounted surgical instrument. The master interface may include: a screen display unit configured to display an on-screen image corresponding to a picture signal inputted from a surgical endoscope; two or more arm manipulation units equipped for controlling the two or more robot arms, respectively; and a control unit configured to provide control such that the on-screen image is rotated or mirrored in a pre-designated direction according to a user manipulation, and configured to provide control such that control conditions for the robot arm are renewed to match the rotated or mirrored on-screen image. Thus, the display screen on a surgical monitor can be suitably controlled according to the intent of the surgeon, to remove the non-intuitiveness of a surgical procedure.

This disclosure generally relates to the scaling of movement of an input device of a user interface to movement of a tool of a robotic system during a surgical procedure about one or more of roll, pitch, and yaw axis including a "trim" and/or a "flip" algorithm.

According to the invention, a robotic surgical system or simulator includes an input device, a display device, and a processing unit. The input device is rotatable about a first axis of rotation. The display device includes a representation of a surgical tool that is operably associated with the input device. The processing unit is in communication with the input device and is operatively associated with the representation of a surgical tool to rotate the representation of a surgical tool about a first axis of movement based on a scaled rotation of the input device about the first axis of rotation. The input device has an aligned configuration in which the input device is aligned with the representation of a surgical tool about the first axis of rotation. When the input device is misaligned with the representation of a surgical tool about the first axis of rotation, the processing unit varies the scaled rotation of the input device about the first axis of rotation to return the input device towards the first aligned configuration, by increasing the scale factor when the input device is moved away from the aligned location and/or decreasing the scaling factor when the input device is moved towards the aligned configuration, until the input device is misaligned about the first axis of rotation by a first predetermined offset from the aligned configuration. The first predetermined offset may be in a range of about <NUM>° to about <NUM>°.

In aspects, the input device is rotatable about a second axis of rotation that is perpendicular to the first axis of rotation. The processing unit may be in communication with the input device and may be operably associated with the representation of a surgical tool to rotate the representation of a surgical tool about a second axis of movement based on a scaled rotation of the input device about the second axis of rotation. In the aligned configuration, the input device is aligned with the representation of a surgical tool about the second axis of rotation. When the input device is misaligned with the representation of a surgical tool about the second axis of rotation, the processing unit varies the scaled rotation of the input device about the second axis of rotation to return the input device towards the aligned configuration, by increasing the scale factor when the input device is moved away from the aligned location and/or decreasing the scaling factor when the input device is moved towards the aligned configuration, until the input device is misaligned about the second axis of rotation by a second predetermined offset from the aligned configuration. The first predetermined offset may be equal to the second predetermined offset. Alternatively, the first predetermined offset may be greater or less than the second predetermined offset. For example, the first predetermined offset may be in a range of about16° to about <NUM>° and the second predetermined offset may be in a range of about <NUM>° to about <NUM>°.

In some aspects, the input device is rotatable about a third axis of rotation that is perpendicular to the first and second axes of rotation. The processing unit may be in communication with the input device and may be operably associated with the representation of a surgical tool to rotate the representation of a surgical tool about a third axis of movement based on a scaled rotation of the input device about the third axis of rotation. In the aligned configuration the input device is aligned with the representation of a surgical tool about the third axis of rotation. When the input device is misaligned with the representation of a surgical tool about the third axis of rotation, the processing unit varies the scaled rotation of the input device about the third axis of rotation to return the input device towards the aligned configuration, by increasing the scale factor when the input device is moved away from the aligned location and/or decreasing the scaling factor when the input device is moved towards the aligned configuration, until the input device is misaligned about the third axis of rotation by a third predetermined offset from the aligned configuration.

In particular aspects, the first predetermined offset is equal to each of the second and third predetermined offsets. Alternatively, the first predetermined offset may be greater or less than the second or third predetermined offsets. Additionally, the second predetermined offset may be greater or less than the second predetermined offset. The first, second, and/or third predetermined offset may be selectable by a user. Additionally or alternatively, the first, second, and/or third predetermined offsets may be at least partially determined based on the representation of a tool. When the input device is misaligned with the representation of a surgical tool about the first, second, and/or third axis of rotation by an amount greater than a predetermined misalignment, the aligned configuration about a respective one of the first, second, and/or third axis of rotation may be flipped <NUM>° about the respective axis of rotation.

In another aspect of the present disclosure, a method of operating a surgical robot or surgical simulator includes a processing unit receiving a rotation of an input device of a robotic surgical system about a first axis of rotation and scaling the rotation of the input device to a rotation of a representation of a tool on a display device about a first axis of movement. The processing unit scales down rotation of the input device when the input device is moved away from an aligned configuration and/or up scaling rotation of the input device when the input device is moved towards the aligned configuration to realign the input device with the representation of a tool until the input device is within a predetermined offset with the representation of a tool about the first axis of rotation.

In aspects, the method may include selecting the predetermined offset. Additionally or alternatively, selecting the predetermined offset may include the processing unit determining the predetermined offset based on the representation of a tool.

In some aspects, the method includes flipping the aligned configuration of the input device <NUM>° about the first axis of rotation when the input device is misaligned with the representation of the tool greater than a predetermined misalignment.

Various aspects of the present disclosure are described herein below with reference to the drawings, which are incorporated in and constitute a part of this specification, wherein:.

Embodiments of the present disclosure are now described in detail with reference to the drawings in which like reference numerals designate identical or corresponding elements in each of the several views. As used herein, the term "clinician" refers to a doctor, a nurse, or any other care provider and may include support personnel. Throughout this description, the term "proximal" refers to the portion of the device or component thereof that is closest to the clinician and the term "distal" refers to the portion of the device or component thereof that is farthest from the clinician. In addition, as used herein the term "neutral" is understood to mean non-scaled.

This disclosure generally relates to the scaling of movement of an input device of a user interface for movement of a tool of a robotic system during a surgical procedure. In particular, this disclosure relates to the scaling of movement about a roll axis, a pitch axis, and a yaw axis of the tool. The robotic system includes an alignment algorithm configured to realign the input device of a user interface with the position of the tool about one or more of the roll, pitch, and yaw axes. In addition, the robotic system includes a "trim" algorithm that allows the input device to remain offset at a predetermined offset about one or more of the roll, pitch, and yaw axes instead of fully aligning the input device about each of the roll, pitch, and yaw axes. Additionally the robotic system may include a "flip" algorithm that rotates an aligned configuration of the input device relative to the tool <NUM>° about a respective one of the roll, pitch, and yaw axes when the input device is misaligned greater than a predetermined misalignment about a respective one of the roll, pitch, and yaw axes.

Referring to <FIG>, a robotic surgical system <NUM> in accordance with the present disclosure is shown generally as a robotic system <NUM>, a processing unit <NUM>, and a user interface <NUM>. The robotic system <NUM> generally includes linkages <NUM> and a robot base <NUM>. The linkages <NUM> moveably support an end effector or tool <NUM> which is configured to act on tissue. The linkages <NUM> may be in the form of arms each having an end <NUM> that supports an end effector or tool <NUM> which is configured to act on tissue. In addition, the ends <NUM> of the arms <NUM> may include an imaging device <NUM> for imaging a surgical site "S". The user interface <NUM> is in communication with robot base <NUM> through the processing unit <NUM>.

The user interface <NUM> includes a display device <NUM> which is configured to display three-dimensional images. The display device <NUM> displays three-dimensional images of the surgical site "S" which may include data captured by imaging devices <NUM> positioned on the ends <NUM> of the arms <NUM> and/or include data captured by imaging devices that are positioned about the surgical theater (e.g., an imaging device positioned within the surgical site "S", an imaging device positioned adjacent the patient "P", imaging device <NUM> positioned at a distal end of an imaging arm <NUM>). The imaging devices (e.g., imaging devices <NUM>, <NUM>) may capture visual images, infra-red images, ultrasound images, X-ray images, thermal images, and/or any other known real-time images of the surgical site "S". The imaging devices transmit captured imaging data to the processing unit <NUM> which creates three-dimensional images of the surgical site "S" in real-time from the imaging data and transmits the three-dimensional images to the display device <NUM> for display.

The user interface <NUM> also includes input handles <NUM> which are supported on control arms <NUM> which allow a clinician to manipulate the robotic system <NUM> (e.g., move the arms <NUM>, the ends <NUM> of the arms <NUM>, and/or the tools <NUM>). Each of the input handles <NUM> is in communication with the processing unit <NUM> to transmit control signals thereto and to receive feedback signals therefrom. Additionally or alternatively, each of the input handles <NUM> may include input devices <NUM> (<FIG>) which allow the surgeon to manipulate (e.g., clamp, grasp, fire, open, close, rotate, thrust, slice, etc.) the tools <NUM> supported at the ends <NUM> of the arms <NUM>.

With additional reference to <FIG>, each of the input handles <NUM> is moveable through a predefined workspace to move the ends <NUM> of the arms <NUM>, e.g., tools <NUM>, within a surgical site "S". The three-dimensional images on the display device <NUM> are orientated such that the movement of the input handles <NUM> move the ends <NUM> of the arms <NUM> as viewed on the display device <NUM>. The three-dimensional images remain stationary while movement of the input handles <NUM> is scaled to movement of the ends <NUM> of the arms <NUM> within the three-dimensional images. To maintain an orientation of the three-dimensional images, kinematic mapping of the input handles <NUM> is based on a camera orientation relative to an orientation of the ends <NUM> of the arms <NUM>. The orientation of the three-dimensional images on the display device <NUM> may be mirrored or rotated relative to view from above the patient "P". In addition, the size of the three-dimensional images on the display device <NUM> may be scaled to be larger or smaller than the actual structures of the surgical site permitting a clinician to have a better view of structures within the surgical site "S". As the input handles <NUM> are moved, the tools <NUM> are moved within the surgical site "S" as detailed below. Movement of the tools <NUM> may also include movement of the ends <NUM> of the arms <NUM> which support the tools <NUM>.

For a detailed discussion of the construction and operation of a robotic surgical system <NUM>, reference may be made to <CIT>.

As detailed above, the user interface <NUM> is in operable communication with the robotic system <NUM> to perform a surgical procedure on a patient; however, it is envisioned that the user interface <NUM> may be in operable communication with a surgical simulator (not shown) to virtually actuate a robotic system and/or tool in a simulated environment. For example, the robotic surgical system <NUM> may have a first mode in which the user interface <NUM> is coupled to actuate the robotic system <NUM> and a second mode in which the user interface <NUM> is coupled to the surgical simulator to virtually actuate a robotic system. The surgical simulator may be a standalone unit or be integrated into the processing unit <NUM>. The surgical simulator virtually responds to a clinician interfacing with the user interface <NUM> by providing visual, audible, force, and/or haptic feedback to a clinician through the user interface <NUM>. For example, as a clinician interfaces with the input handles <NUM>, the surgical simulator moves representative tools that are virtually acting on tissue. It is envisioned that the surgical simulator may allow a clinician to practice a surgical procedure before performing the surgical procedure on a patient. In addition, the surgical simulator may be used to train a clinician on a surgical procedure. Further, the surgical simulator may simulate "complications" during a proposed surgical procedure to permit a clinician to plan a surgical procedure.

The movement of the tools <NUM> is scaled relative to the movement of the input handles <NUM>. When the input handles <NUM> are moved within a predefined workspace, the input handles <NUM> send control signals to the processing unit <NUM>. The processing unit <NUM> analyzes the control signals to move the tools <NUM> in response to the control signals. The processing unit <NUM> transmits scaled control signals to the robot base <NUM> to move the tools <NUM> in response to the movement of the input handles <NUM>. The processing unit <NUM> scales the control signals by dividing an Inputdistance (e.g., the distance moved by one of the input handles <NUM>) by a scaling factor SF to arrive at a scaled Outputdistance (e.g., the distance that one of the ends <NUM> is moved). The scaling factor SF is in a range between about <NUM> and about <NUM> (e.g., <NUM>). This scaling is represented by the following equation: <MAT> It will be appreciated that the larger the scaling factor SF the smaller the movement of the tools <NUM> relative to the movement of the input handles <NUM>.

For a detailed description of scaling movement of the input handle <NUM> along the X, Y, and Z coordinate axes to movement of the tool <NUM>, reference may be made to commonly owned International Patent Application Serial No. <CIT> (<CIT>, and entitled "Dynamic Input Scaling for Controls of Robotic Surgical System," and International Patent Application No. <CIT> (<CIT>.

Referring also to <FIG> and <FIG>, the rotation of the input device <NUM> relative to each of the X, Y, and Z coordinate axes may be scaled to rotation of the tool <NUM> about a roll axis "R", a pitch axis "P", and a yaw axis "Y" (RPY). The roll axis "R" is aligned with an end effector of a tool, e.g., tool <NUM>, as displayed on the display device <NUM> while pitch and yaw axes "P", "Y" are orientated to the camera frame as displayed on the display device <NUM> such that motions of the handles <NUM> and/or input device <NUM> are relative to a clinician's view of the display device <NUM>. Specifically, the pitch axis "P" is about the X coordinate axis and the yaw axis "Y" is about the Y coordinate axis of a neutral frame of the handle. The scaling of rotation of the input device <NUM> about each of the RPY axes may be scaled up, down, or neutral manner. By scaling rotation up, a clinician is able to reduce rotation of the input device <NUM> about a particular one of the RPY axes to achieve a desired rotation of the tool <NUM> about the respective RPY axis. This up scaling may allow a clinician to have dexterity beyond a natural movement of the human body. For example, a clinician may roll a tool <NUM> beyond what is possible with the movement of the clinician's wrist without releasing the input device <NUM>. In contrast, by scaling rotation down, a clinician is able to more precisely control rotation of the tool <NUM> about a particular one of the RPY axes of the tool <NUM> in response to rotation of the input device <NUM>.

Rotation of the input device <NUM> about each of the RPY axes may be scaled in a different manner to rotation of the tool <NUM>. For example, rotation of the input device <NUM> about the control shaft <NUM>, i.e., rotation about the roll axis "R", may be scaled up, rotation of the input device <NUM> about the pitch axis "P" may be scaled in a neutral manner, and rotation of the input device <NUM> about the yaw axis "Y" may be scaled down. Any other combinations of scaling are contemplated herein and form a part of the present disclosure.

During a surgical procedure, rotation of the input device <NUM> may be "clutched" relative to rotation of the tool <NUM> about the RPY axes. The "clutching" of the input device <NUM> relative to the tool <NUM> may be manually selected by a clinician or may be automatically selected by the robotic surgical system <NUM>. When input device <NUM> is reassociated or "declutched" with rotation of the tool <NUM> about the RPY axes, the input device <NUM> may be misaligned with the orientation of the tool <NUM> about one or more of the RPY axes.

The robotic surgical system <NUM> may vary the RPY scaling factors to realign the input device <NUM> with the tool <NUM> in a manner which is imperceptible to a clinician engaged with the input device <NUM>. To realign the input device <NUM> with the tool <NUM> the RPY scaling factors in a direction away from an aligned or centered position may be scaled down more than when the clinician moves the input handle <NUM> towards the aligned configuration until the tool is aligned with the input device <NUM>. By scaling down movement of the input handle <NUM> as the input handle <NUM> is moved towards the aligned configuration, the robotic surgical system <NUM> allows the input device <NUM> to "catch up" to the position of the tool <NUM> in a manner which is indiscernible to a clinician interfacing with the input device <NUM>. For example, one or more of the RPY scaling factors may be increased in a range of about <NUM>% to about <NUM>% (e.g., <NUM>% or <NUM>%) when the input device <NUM> is moved away from the aligned configuration and the RPY scaling factor may remain unchanged when the input device <NUM> is moved towards the aligned configuration. Additionally or alternatively, one or more of the RPY scaling factors may be reduced in a range of about <NUM>% to about <NUM>% (e.g., <NUM>% or <NUM>%) when the input device <NUM> is moved towards the aligned configuration which causes the tool <NUM> to catch up with the position of the input device <NUM>. When the tool <NUM> is aligned with the input device <NUM>, the RPY scaling factors return to operating in a symmetrical manner which may be up, down, or neutral, e.g., have the same scaling factor. It will be appreciated that by scaling movement of the input device <NUM> relative to the tool <NUM> in this manner, the tool <NUM> remains stationary when the input device <NUM> is stationary and tool <NUM> only moves in response to a clinician moving the input device <NUM>.

In an embodiment, the RPY scaling factors may vary based on the amount of misalignment of the respective RPY axis. For example, when the roll axis "R" is misaligned about <NUM> degrees, the scaling factor of the roll axis "R" may be scaled down about <NUM>% for movement of the input device <NUM> towards the aligned configuration, and when the roll axis "R" is misaligned about <NUM> degrees, the roll axis "R" may be scaled down about <NUM>% for movement of the input device <NUM> towards the aligned configuration. It is contemplated that the varying of the scaling factors may be a linear, exponential, polynomial, linear step, or other mathematical relationship. For a detailed description of varying scaling factors based on a distance or amount of misalignment, reference can be made to International Patent Application No. <CIT> (<CIT>.

The robotic surgical system <NUM> varies the RPY scaling factors to realign the input device <NUM> with the tool <NUM> until the misalignment of the input device <NUM> with the tool <NUM> is at or within a predetermined offset. For example, the input device <NUM> may be misaligned with the tool <NUM> about a roll axis by <NUM>°. The robotic surgical system <NUM> varies a first axis of rotation scaling factors, e.g. the roll scaling factors (according to the invention, to increase the roll scaling factor when the input device <NUM> is moved away from an aligned configuration and/or decrease the roll scaling factor when the input device <NUM> is moved towards the aligned configuration) until the input device <NUM> is at a predetermined offset or "trim" e.g. in a range of about <NUM>° to about <NUM>° (e.g., about <NUM>°) relative to the input device <NUM>. Once the input device <NUM> is at the predetermined offset relative to the input device <NUM>, the robotic surgical system <NUM> ceases to vary the roll scaling factors to realign the input device <NUM> with the tool <NUM>.

By allowing the input device <NUM> to remain offset from the tool <NUM> about one or more of the RPY axes, the position may allow the clinician to maintain a more comfortable hand and/or arm position during the surgical procedure. The predetermined offset may be the same or different about each of the RPY axes. For example, the predetermined offset may be in a range of about <NUM>° to about <NUM>° (e.g., about <NUM>°) about each of the RPY axes. Alternatively, the predetermined offset about the roll axis may be about <NUM>° and the predetermined offset about the pitch and yaw axes may be about <NUM>°.

The predetermined offsets may be set by the robotic surgical system <NUM> or may be user selected. The predetermined offsets may be set by the robotic surgical system <NUM> based on the type of tool <NUM> connected to an arm <NUM> associated with the input device <NUM> and/or may be set by the robotic surgical system <NUM> based on the type of surgical procedure. It will be appreciated that as the predetermined offset increases, the control of the tool <NUM> may become more difficult. Accordingly, there may be limits to the maximum limits set for the user selected predetermined offsets.

Referring to <FIG>, it may be beneficial to "flip" one or more of the RPY axes if the misalignment is greater than a predetermined misalignment. One or more of the RPY axes is "flipped" when the aligned configuration about a respective axis is rotated <NUM>° about the respective axis. A "flip" about a respective axis may be beneficial for control of a tool <NUM> when the tool <NUM> is asymmetrical or directional (e.g., a hook, curved scissors, curved dissector, etc.). For example, when a tool <NUM> is a hook with an opening <NUM> facing right as shown on the display device <NUM> in <FIG>, the tool <NUM> can be "flipped" about the roll axis "R" such that the opening <NUM> faces left as shown on the display device <NUM> as shown in <FIG>. The predetermined misalignment may be in a range of about <NUM>° to about <NUM>° (e.g., about <NUM>°). Additionally or alternatively, a clinician may "flip" one or more of the RPY axes using a button, voice command, or GUI input. It is contemplated the robotic surgical system <NUM> may incorporate both a "trim" and a "flip" algorithm.

Additionally or alternatively, the robotic surgical system <NUM> may include a "snap" algorithm which is similar to the "flip" algorithm. A "snap" would occur when one or more of the RPY axes is misaligned greater than a predetermined misalignment. For example, when the roll axis "R" is misaligned about <NUM>°, the trim of the roll axis "R" may snap to an offset of <NUM>°. The snap algorithm may include a plurality of sequential offsets such that each time the particular RPY axes exceeds a predetermined misalignment, the respective RPY axis "snaps" to an offset associated with the predetermined misalignment. For example, the roll axis "R" may have predetermined snap points spaced <NUM>, <NUM>, or <NUM> degrees about the roll axis "R".

Referring back to <FIG>, the input device <NUM> includes a button <NUM> to alter the scaling of one or more of the RPY scaling factors. For example, when the button <NUM> is depressed, the scaling factor about the roll axis "R" can be scaled up or scaled down to a predetermined value. Alternatively, when the button <NUM> is depressed, the input device <NUM> can be clutched out about the roll axis "R" while the other axes remain related to movement of the input device <NUM>.

While several embodiments of the disclosure have been shown in the drawings, it is not intended that the disclosure be limited thereto, as it is intended that the disclosure be as broad in scope as the art will allow and that the specification be read likewise.

Claim 1:
A robotic surgical system (<NUM>) or simulator comprising:
an input device (<NUM>) rotatable about a first axis of rotation;
a display device (<NUM>) including a representation of a surgical tool (<NUM>) operably associated with the input device; and
a processing unit (<NUM>) in communication with the input device and operatively associated with the representation of a surgical tool to rotate the representation of a surgical tool about a first axis of movement based on a scaled rotation of the input device about the first axis of rotation, the input device having an aligned configuration in which the input device is aligned with the representation of a surgical tool about the first axis of rotation,
wherein when the input device is misaligned with the representation of a surgical tool about the first axis of rotation, the processing unit varies the scaled rotation of the input device about the first axis of rotation to return the input device towards the first aligned configuration, by increasing the scaling
factor when the input device is moved away from the aligned location and/or decreasing the scaling factor when the input device is moved towards the aligned configuration, until the input device is misaligned about the first axis of rotation by a first predetermined offset from the aligned configuration.