Patent Description:
Another type of a steerable device used in surgery is a surgical image acquisition device, such as an endoscope, with a deflecting tip, or a robotic endoscope. Such an endoscope is a thin, elongated camera assembly that allows a clinician to view the internal anatomy of a patient without the need to surgically expose the anatomy for a direct view. Endoscopes can fit through narrow natural orifices or small incisions in the skin, resulting in reduced trauma to the patient as compared to visualization and intervention without the aid of an endoscope.

Control of known dexterous devices is challenging. The clinician (e.g., surgeon or other user) has to combine the motion of non-dexterous proximal end, which is usually around a fulcrum point (e.g., entry point to the body), and complex dexterous motion inside the body. An approach to this problem is robotic positioning of the dexterous device which increases footprint in the operating room and increases cost and duration of surgery. This problem is amplified if the proximal end is not within field-of-view of imaging devices (e.g. an endoscope takes images only on the inside of the patient, and the field of view in a portable imaging system, such as a C-arm, is too small to image the entire device and can cause radiation exposure to the operator). In addition, once the position is achieved with the dexterous device, hand tremors and involuntary motion of the hand can cause misalignment.

For example, aortic valve replacement is a procedure in which a patient's heart valve is replaced with a prosthetic (artificial) valve. Minimally invasive surgery for valve replacement includes deployment of the artificial valve in the beating heart of the patient through a small incision in the chest. A conventional procedure for aortic valve replacement includes, in part, creating a puncture from outside of the heart through a left ventricle wall of the patient, inserting an introducer sheath through the puncture, passing a balloon catheter through the introducer sheath into the left ventricle, and using a pusher and sleeve to advance the balloon catheter and prosthetic valve mounted thereon so that the prosthetic valve is positioned within the aortic annulus. However, the conventional procedure assumes a manual, straight line deployment of the valve from the surface of the patient (at the small incision), between the patient's ribs, through the heart and to the aortic valve (in a beating heart). This is very challenging for a number of reasons. For example, a straight line is a poor approximation of the anatomical environment. That is, there are three primary locations of interest for an aortic valve replacement: a patient entry location, a heart entry location, and the location of the valve itself. These three primary locations are not co-linear. Also, deployment along a straight line may be constrained by other anatomical features, such as ribs, heart muscle, trabeculations inside the heart, and the like. In addition, it is difficult for the surgeon to map all anatomical landmarks in order to plan the straight line path, especially under poor visual feedback and where the anatomy is constantly moving (e.g., heart beating and valve flapping). Further, a field of view of X-ray and ultrasound is typically limited to image the heart only, so there is no image guidance outside the heart.

Accordingly, it maybe desirable to provide an apparatus, systems, methods, and computer-readable storage medium for control of a surgical robot having a rigid proximal portion and a flexible distal portion using a combination of medical imagery and tracking information.

<CIT> describes a planning system for minimally invasive therapy. A system is provided that plans a pathway to a target location in a tissue within a patient's body. The system consists of a storage medium to store pre-operative imaging volumes, a surgical outcome criteria associated with anatomical portions of the body, and a processor in communication with the storage medium and outcome criteria to identify, score and save one or more surgical trajectory paths of a surgical tool.

According to a representative embodiment, a control unit is provided according to claim <NUM>.

According to another representative embodiment, a robot system includes a robot and a control unit according to claim <NUM>. The robot comprises a rigid proximal portion having a remote center of motion (RCM), a flexible distal portion, and at least one image acquisition device.

According to an example arrangement, a non-transitory computer-readable storage medium is provided, having stored therein machine readable instructions configured to be executed by a processor to control a robot system. The robot system has a control unit and a robot with a rigid proximal portion having a RCM, a flexible distal portion and at least one image acquisition device. The machine readable instructions are configured to perform a method of controlling the rigid proximal portion and the flexible distal portion of the robot to access a target, the method including generating a first deployment path for the rigid proximal portion to a region of interest within a surgical site of a patient; moving the rigid proximal portion on first deployment path to the region of interest; deploying the flexible distal portion through the rigid proximal portion; tracking a position of the flexible distal portion within the region of interest; generating a second deployment path for flexible distal portion to the target; and moving the flexible distal portion on second deployment path to the target.

The representative embodiments are best understood from the following detailed description when read with the accompanying drawing figures. It is emphasized that the various features are not necessarily drawn to scale. In fact, the dimensions may be arbitrarily increased or decreased for clarity of discussion. Wherever applicable and practical, like reference numerals refer to like elements.

In the following detailed description, for purposes of explanation and not limitation, representative embodiments disclosing specific details are set forth in order to provide a thorough understanding of the present teachings. However, it will be apparent to one having ordinary skill in the art having had the benefit of the present disclosure that other embodiments according to the present teachings that depart from the specific details disclosed herein remain within the scope of the appended claims. Moreover, descriptions of well-known apparatuses and methods may be omitted so as to not obscure the description of the representative embodiments. Such methods and apparatuses are clearly within the scope of the present teachings.

It is to be understood that the terminology used herein is for purposes of describing particular embodiments only, and is not intended to be limiting. Any defined terms are in addition to the technical and scientific meanings of the defined terms as commonly understood and accepted in the technical field of the present teachings.

As used in the specification and appended claims, the terms "a", "an" and "the" comprises both singular and plural referents, unless the context clearly dictates otherwise. Thus, for example, "a device" includes one device and plural devices.

As used herein, the statement that two or more parts or components are "coupled" shall mean that the parts are joined or operate together either directly or indirectly, i.e., through one or more intermediate parts or components, so long as a link occurs.

Directional terms/phrases and relative terms/phrases may be used to describe the various elements" relationships to one another, as illustrated in the accompanying drawings. These terms/phrases are intended to encompass different orientations of the device and/or elements in addition to the orientation depicted in the drawings.

Relative terms, such as "above," "below," "top," "bottom," "upper" and "lower" may be used to describe the various elements" relationships to one another, as illustrated in the accompanying drawings. These relative terms are intended to encompass different orientations of the device and/or elements in addition to the orientation depicted in the drawings. For example, if the device were inverted with respect to the view in the drawings, an element described as "above" another element, for example, would now be "below" that element. Similarly, if the device were rotated by <NUM>° with respect to the view in the drawings, an element described "above" or "below" another element would now be "adjacent" to the other element; where "adjacent" means either abutting the other element, or having one or more layers, materials, structures, etc., between the elements.

As used in the specification and appended claims, and in addition to their ordinary meanings, the terms 'substantial" or "substantially" mean to with acceptable limits or degree. For example, 'substantially cancelled" means that one skilled in the art would consider the cancellation to be acceptable.

Referring initially to <FIG>, a surgical robot system <NUM> in accordance with features of the present teachings will be described. In particular, the surgical robot system <NUM> may be utilized for medical procedures including, but are not limited to, minimally invasive cardiac surgery, such as coronary artery bypass grafting and mitral valve replacement; minimally invasive abdominal surgery, such as laparoscopy to perform prostatectomy or cholecystectomy; and natural orifice translumenal endoscopic surgery, for example.

<FIG> schematically illustrates the surgical robot system <NUM> comprising a hybrid robot <NUM> having a rigid proximal portion <NUM> and a flexible distal portion <NUM>. Generally, the rigid proximal portion <NUM> is advanced into a body cavity of the patient P through a first entry location E1 at the surface of the patient's body. The rigid proximal portion <NUM> is guided using imagery to a region of interest ROI (e.g., an internal organ, such as the patient's heart) at a surgical site S, and the flexible distal portion <NUM> is deployed from the rigid proximal portion <NUM> into the region of interest ROI through a second entry location E2 (also referred to as a first target position). The flexible distal portion <NUM> is then guided using imagery to a target T (e.g., a feature or object present in the internal organ, such as an aortic valve of the patient's heart) (also referred to as a second target position) in the region of interest ROI. Once at the target T, the flexible distal portion <NUM> is able to perform a variety of diagnostic, surgical or treatment procedures (e.g., such as aortic valve replacement). The rigid proximal portion <NUM> and the flexible distal portion <NUM> of the robot <NUM> may be operated under control of a control unit <NUM>, as discussed below, or in an embodiment, the rigid proximal portion <NUM> may be a handheld introducer manually positioned at the first entry location E1 to the body of the patient P and manipulated by the clinician (user).

The rigid proximal portion <NUM> of the robot <NUM> may have a remote center of motion (RCM) at the first entry location E1, enabling the rigid proximal portion <NUM> to pivot around the first entry location E1 ("pivot point"). In other words, the RCM located at the surface of the body of the patient P, and is configured for movement along a number of degrees of freedom. In an alternative embodiment, the rigid proximal portion <NUM> of the robot <NUM> may have a RCM at the second entry location E2, such that the RCM is located at the surface of the RIO, enabling movement along a number of degrees of freedom. In the embodiment where the RCM is locate at the surface of the body P, the opening at the first entry location E1 may be smaller than in the embodiment where the RCM is located at the surface of the RIO. This is because, when the RCM is located at the surface of the RIO, the rigid proximal portion <NUM> pivots inside the patient P, and thus there must be room at the first entry location E1 for the rigid proximal portion <NUM> to move distances at the surface of the body of the patient P corresponding to the angle of the movement and the distance between the first entry location E1 and the second entry location E2. In the various embodiments, the rigid proximal portion <NUM> is manipulated by instructions from the control unit <NUM>, received via an input/output (I/O) circuit <NUM>, or manipulated manually by the clinician, to guide the rigid proximal portion <NUM> to the desired region of interest ROI in the surgical site S.

In accordance with the present teachings, movement of the rigid proximal portion <NUM> is tracked, and based on this movement, the location of the distal portion <NUM> can be adjusted to ensure accurate location of the distal portion <NUM> relative to the target T. The proximal portion <NUM> illustratively comprises a tubular portion through which the flexible distal portion <NUM>, as well as components useful in effecting the particular surgical or therapeutic procedure, can be guided. By way of example, such components may include various end-effectors, imaging devices, and other components or devices (e.g., prosthetic heart valves or stents) that can be fed through the proximal portion <NUM>.

The surgical robot system <NUM> comprises a display <NUM>, which provides real-time images of the location of at least a portion of the rigid proximal portion <NUM> and the flexible distal portion <NUM> of the robot <NUM> within the patient P. The display <NUM> may receive the real-time images from the image acquisition device <NUM>, via the I/O circuitry <NUM> and/or a processor <NUM>, as described more fully below. The image acquisition device <NUM> may be configured to acquire a live image or live images of the proximal portion <NUM> and the flexible distal portion <NUM>, the flexible distal portion <NUM>, the region of interest ROI (e.g., an anatomical object, such as the heart or another organ) and/or the target T (e.g., a location within or a part of the anatomical object) at the surgical site S.

The image acquisition device <NUM> may comprise a C-arm, for example, which is an imaging scanner intensifier, so named because of its C configuration. A C-arm has radiographic capabilities, and may be used for fluoroscopic imaging during surgical procedures, as is would be apparent to those skilled in the art. More generally, the image acquisition device <NUM> may comprise one of a variety of inter-operative imaging devices within the purview of one of ordinary skill in the art to provide real-time imaging. These real-time (i.e., inter-operative) images may be used in connection with the pre-operative images to effect registration as described below. Contemplated imaging devices for the second image acquisition device <NUM> include, but are not limited to computed tomography (CT) devices, ultrasound imaging devices, magnetic resonance imaging (MRI) devices, positron emission tomography (PET) devices, single-photon emission computed tomography (SPECT) imaging devices. These images may be acquired real-time.

Generally, the flexible distal portion <NUM> comprises at least two links, and at least one joint therebetween. As described more fully below in connection with representative embodiments, the robot <NUM> is structurally configured to control one or more joints of the flexible distal portion <NUM> for maneuvering in one or more degrees of motion, e.g., within the region of interest ROI. Illustratively, the flexible distal portion <NUM> may be one of a number of devices, including but not limited to a two-linkage, one joint device, a snake-like device, or a steerable catheter. In practice, as would be appreciated by those skilled in the art, the flexible distal portion <NUM> is configured to move in one or more degrees of freedom. Generally, the flexible distal portion <NUM> linearly advances (i.e., one degree-of-freedom), move in a plane (i.e., two degrees-of-freedom), and rotate. More generally, the flexible distal portion <NUM> may have a plurality of degrees-of-freedom, e.g., six (<NUM>) or more in the case of snake like robots.

By way of example, the flexible distal portion <NUM> comprises a plurality of links and joints, which are controlled to properly locate a distal end <NUM>' of the flexible distal portion <NUM> in a desired location (e.g., target T). An example of the flexible distal potion <NUM> as a multiple link, multiple joint device is depicted in <FIG>.

Referring to <FIG>, the flexible distal portion <NUM> is a snake-like robot portion of the robot <NUM> according to a representative embodiment comprises a plurality of links <NUM>, each connected by a respective joint <NUM>. Each of the plurality of links <NUM> comprises a rigid segment, and each of the joints <NUM> may comprise a geared assembly. Illustratively, each joint <NUM> can be implemented between one and three degrees of freedom (roll, pitch, and yaw). As described more fully below, the control unit <NUM> is configured to perform motor control and collect position and orientation data of the flexible distal portion <NUM>. Alternatively, the flexible distal portion <NUM> may be a catheter robot, such as described, for example, by <CIT>).

As will be appreciated, the flexible distal portion <NUM> may comprise an end-effector (not shown) as desired for the particular robotic procedure. By way of example, the end-effector connected to the flexible distal portion <NUM> may comprise a gripper or a tool holder. Similarly, the flexible distal portion <NUM> may comprise a tool such as a laparoscopic instrument, laparoscope, a tool for screw placement, a forward-looking camera, or a needle for biopsy or therapy. Other surgical devices and tools within the purview of one of ordinary skill in the art are also contemplated to be used with the flexible distal portion <NUM>.

The display <NUM> comprises an output device and/or a user interface adapted for displaying images and data, as described more fully herein. The display <NUM> may include one or more displays that may be co-located near the clinician positioned adjacent to various elements of the surgical robot system <NUM>. The display <NUM> is configured to display live and/or preoperative images of the surgical site S provided, for example, by the image acquisition device <NUM>. The display <NUM> may output visual, audio, and/or tactile data. Examples of the display <NUM> include, but are not limited to, a computer monitor, a television screen, a touch screen, tactile electronic display, Braille screen, cathode ray tube (CRT), storage tube, bistable display, electronic paper, vector display, flat panel display, vacuum fluorescent display (VF), light-emitting diode (LED) displays, electroluminescent display (ELD), plasma display panels (PDP), liquid crystal display (LCD), organic light-emitting diode displays (OLED), a projector, and a head-mounted display (HMD).

The control unit <NUM> is configured to receive input from various components of the surgical robot system <NUM>, and to provide outputs (e.g., instructions or commands) thereto. The control unit <NUM> comprises the I/O circuitry <NUM>, which receives inputs from various components of the surgical robot system <NUM>, and provides outputs to and receives inputs from the processor <NUM>, as is described more fully below. The processor <NUM> also comprises a memory <NUM> for enabling processing and a computer readable medium (CRM) <NUM> (which may be the same or separate memories). The processor <NUM> is generally configured to receive images, e.g., from the image acquisition device <NUM>, via the I/O circuitry <NUM>, and to process and store the acquired images in the memory <NUM> and/or the CRM <NUM>. The processor <NUM> is therefore able to build a database essentially visually mapping interior portions of the patient P captured by the image acquisition device <NUM>. In alternative embodiments, an additional image acquisition device may be included to provide internal images. Such additional image acquisition devices may include, for example, a transesophageal echocardiography (TEE) probe or an endoscope, and the internal images may include interior portions of the patient P traversed by the additional image acquisition device. The database built by the processor <NUM> may be used to determine paths from the first entry location E1 to the second entry location E2 (e.g., a first deployment path DP1 to a first target position), and from the second entry location E2 to the target T within the region of interest ROI (e.g., a second deployment path DP2 to a second target position). The processor <NUM> transmits the images to the display <NUM> via the I/O circuitry <NUM> for display.

When an additional image acquisition device comprises an endoscope (not shown), it provides endoscopic images. The endoscope may be controlled by an endoscopic controller, operated independently or under control of the control unit <NUM>. In certain representative embodiments, the endoscope may include a tube, and a light delivery system to illuminate the organ or object under inspection, for example, the light source is normally outside the body and the light is typically directed via an optical fiber system. Also included may be a lens system transmitting the image from the objective lens to the viewer, typically a relay lens system in the case of rigid endoscopes or a bundle of fiberoptics in the case of a fiberscope. Also contemplated are videoscopes, with no eyepiece, in which a camera transmits images to a screen (e.g., display <NUM>) for image capture. An additional channel of the endoscope may allow entry of medical instruments or manipulators.

Examples of an endoscope for purposes of the present invention include, but are not limited to, any type of scope, flexible or rigid (e.g., endoscope, arthroscope, bronchoscope, choledochoscope, colonoscope, cystoscope, duodenoscope, gastroscope, hysteroscope, laparoscope, laryngoscope, neuroscope, otoscope, push enteroscope, rhinolaryngoscope, sigmoidoscope, sinuscope, thorascope, etc.) and any device similar to a scope that is equipped with an image system. The imaging is local, and surface images may be obtained optically with fiber optics, lenses, or miniaturized (e.g. CCD based) imaging systems.

The additional image acquisition device may be connected to, and may be a component of the control unit <NUM>. The additional image acquisition device provides images ultimately provided to the display <NUM>, and may include any type of camera having a forward optical view or an oblique optical view, and may be capable of acquiring a sequence of two-dimensional digital video frames at a predefined frame rate (e.g., <NUM> frames per second) and capable of providing each digital video frame to the control unit <NUM> via the I/O circuitry <NUM>. In particular, the additional image acquisition device may be positioned and oriented such that within its field of view it can capture images of the flexible distal portion <NUM>. In some embodiments, additional image acquisition device includes a camera which is actuated by a motor and can be positioned along a planned instrument path for the robot <NUM>.

The processor <NUM> may comprise one or more microprocessors that may be programmed using software (e.g., microcode) to perform various functions discussed herein. Notably, the processor <NUM> may comprise more than one processor or processing core. The processor <NUM> may for instance be a multi-core processor. The control unit <NUM> may also comprise a collection of processors within a single computer system (not shown) or distributed among multiple computer systems (not shown) associated with the surgical robot system <NUM>. As will be appreciated, many programs have their instructions performed by the processor <NUM> that may be within the same computing device or which may even be distributed across multiple computing devices. Examples of components that may be employed as the processor <NUM> in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, microcontrol units, application specific integrated circuits (ASICs), and/or field-programmable gate arrays (FPGAs).

The memory <NUM> and/or the CRM <NUM> may be configured to store various types of data gathered during the course of the function of the various components of the surgical robot system <NUM>. These data include image data and tracking data gathered as described more fully below. The memory <NUM> and/or the CRM <NUM> may also store pre-operative data, such as pre-operative image data. As described more fully below, these data can be used to track the locations of the rigid proximal portion <NUM> and the flexible distal portion <NUM> during operation of the robot <NUM>. Furthermore, each of the memory <NUM> and the CRM <NUM> comprises a non-transitory computer readable medium, which stores machine readable instructions configured to be executed by the processor <NUM> to control the surgical robot system <NUM>, and to store various data, including image data and tracking data. By way of example, these instructions (programs) are encoded in the memory <NUM>, and when executed on the processor <NUM>, perform at least some of the functions discussed herein. Notably, the terms "program" or "computer program" are used herein in a generic sense to refer to various types of computer code (e.g., software or microcode) that can be employed to program the control unit <NUM>.

The memory <NUM> and the CRM <NUM> may comprise non-volatile computer memory, or volatile computer memory, or both, including, but not limited to: such as random-access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), electrically programmable read-only memory (EPROM), electrically erasable and programmable read only memory (EEPROM), universal serial bus (USB) drive, floppy disks, compact disks (CDs), optical disks, magnetic tape, etc.), a smart card, a digital video disc (DVD), a CD-ROM, and a solid state hard drive. Various storage media, such as the memory <NUM>, for example, may be fixed within the processor <NUM> or may be transportable, such that the one or more programs stored thereon can be loaded into the processor <NUM> so as to implement various aspects of the present teachings discussed herein.

The robot <NUM> introduces an additional coordinate system, and therefore alignment (position and orientation) of the robot <NUM> with some desired frame of reference is difficult to guarantee and maintain because it is positioned with workspace and motion constraints. As described more fully below, misalignment of disparate coordinate systems so that the same alignment need not be performed mentally by clinicians is effected using known registration methods and apparatuses. To this end, a variety of current methods and apparatuses exist to register the robot <NUM> and the particular components thereof to the imaging system. By way of example, registration can be performed by matching features of the flexible distal portion <NUM> visible in the images with corresponding features gathered preoperatively. The target T may be identified by the clinician or surgeon by marking the location of the target T in the images. In another embodiment, the target T can be automatically detected by means of feature matching and object recognition known in art. The target T may then be computed from the image to the robot coordinate system using registration. Examples of registration are described in commonly owned <CIT>); and <CIT>), <CIT>), <CIT>, <CIT>), <CIT>), and<CIT>).

A tracking system <NUM> is configured to generate tracking information with respect to the rigid proximal portion <NUM> and the flexible distal portion <NUM> of the robot <NUM>. The tracking system <NUM> may be one or more of an optical tracking system, mechanical tracking system, an electromagnetic tracking system, and a shape sensing tracking system, as would be appreciated by those skilled in the art. A sensor or tag, such as a radio frequency (RF) sensor, LED sensor, passive markers, reflective markers, may be included at the proximal portion <NUM> of the robot <NUM>, or proximal to the end <NUM>' of the flexible distal portion <NUM>, or both to cooperate with the tracking system <NUM>. Shape sensing tracking systems are described, for example, by <CIT>.

The tracking system <NUM> provides information to the control unit <NUM> to provide feedback of the current position of the rigid proximal portion <NUM> and the flexible distal portion <NUM>. This allows adjustment of the position of the rigid proximal portion <NUM> relative to the region of interest ROI, and adjustment of the position of the flexible distal portion <NUM> relative to the target T. Through tracking of the rigid proximal portion <NUM> and the flexible distal portion <NUM>, and data from the registration realized by the image acquisition device <NUM>, the processor <NUM> is configured to ultimately determine the location of the flexible distal portion <NUM> relative to the target T. Notably, software in the memory <NUM>, for example, enables the calculation by the processor <NUM> of the current location of the flexible distal portion <NUM> relative to the target T. Based on these calculations, the processor <NUM> provides instructions/commands to the flexible distal portion <NUM> to move as needed to be in better position relative to the target T. In one embodiment, these commands function to compensate for tremor-induced motion in the rigid proximal portion <NUM> or from the clinician's hands to compensate (and substantially nullify) any undesired movement by the flexible distal portion <NUM>.

The surgical robot system <NUM> comprises a user interface <NUM>. The user interface <NUM>, like the display <NUM>, is illustratively coupled to the control unit <NUM> via a hardware interface (not shown) and the I/O circuitry <NUM>. The hardware interface enables the processor <NUM> to interact with various components of the surgical system, as well as control an external computing device (not shown) and/or apparatus. The hardware interface may allow a processor to send control signals or instructions to various components of the surgical robot system <NUM>, as well as an external computing device and/or apparatus. The hardware interface may also enable a processor to exchange data with various components of the surgical robot system <NUM>, as well as with an external computing device and/or apparatus. Examples of a hardware interface include, but are not limited to: a universal serial bus, IEEE <NUM> port, parallel port, IEEE <NUM> port, serial port, RS-<NUM> port, IEEE-<NUM> port, Bluetooth connection, Wireless local area network connection, TCP/IP connection, Ethernet connection, control voltage interface, MIDI interface, analog input interface, and digital input interface.

The user interface <NUM> allows the clinician to interact with surgical robot system <NUM> through a computer (not shown) or computer system (not shown). The user interface <NUM> may comprise, for example, a touch screen, a keyboard, a mouse, a trackball and/or touchpad. Generally, the user interface <NUM> may provide information or data to the clinician and/or receive information or data from the clinician. The user interface <NUM> may be configured to receive input from the clinician to be received by the computer, and may provide output to the user from the computer. In other words, and as will become clearer, the user interface <NUM> may be configured to enable the operator to control or manipulate the computer, and the user interface <NUM> may be configured to allow the computer to indicate the effects of the clinician's control or manipulation. The display of data or information on the display <NUM> or a graphical user interface thereof, is an example of providing information to the clinician. The receiving of data through a touch screen, keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, gear sticks, steering wheel, wired glove, wireless remote control, and accelerometer are all examples of components of the user interface <NUM>, which enable the receiving of information or data from an operator.

As noted above, the control unit <NUM> comprises I/O circuitry <NUM>. Among other functions, the I/O circuitry <NUM> controls communication to elements and devices external to the control unit <NUM>. The I/O circuitry <NUM> acts as an interface including necessary logic to interpret input and output signals or data to/from the processor <NUM>. For example, the I/O circuitry <NUM> may include a first input configured to receive the medical imagery, such as from the image acquisition device <NUM>, related to rigid proximal portion <NUM> and the flexible distal portion <NUM> of the robot <NUM> at or near the surgical site S, and a second input configured to receive the tracking information of the proximal portion <NUM> and the flexible distal portion <NUM> of the robot <NUM> from the tracking system <NUM>. The I/O circuitry <NUM> may include an output configured to provide the medical imagery related to the robot <NUM> to a display of the display <NUM>.

Representative embodiments may also be directed to a non-transitory computer-readable storage medium (e.g., memory <NUM>/CRM <NUM>) having stored therein machine readable instructions configured to be executed by the processor <NUM> to control the surgical robot system <NUM> including the robot <NUM> having the proximal portion <NUM> to be positioned at an entry to a patient's body and the flexible distal portion <NUM> to be positioned at a surgical site S within the patient's body. The machine readable instructions are stored in memory <NUM> and configured to perform a method <NUM> to compensate for motion of the handheld introducer <NUM>.

For purposes of illustration, it may be assumed that the region of interest ROI is the patient's heart, for example. In this case, the position of the end <NUM>' of the flexible distal potion <NUM> may be determined by the clinician by images provided at the display <NUM>. As such, an end-effector disposed at the end <NUM>' may be used to make an incision at a precise location of the heart (region of interest ROI) at the second entry location E2. The clinician can then further guide the end <NUM>' of the flexible distal portion <NUM> to the location of the aorta valve (target T) to be replaced. The valve can then be replaced, again with the precision location of the heart being determined by the control unit <NUM> using the various image acquisition and registration methods described above.

Based on data from the tracking system <NUM> alone, or in combination with data from the second image acquisition device <NUM>, the processor <NUM> can compensate for sporadic movement (e.g., induced by clinician tremor) of the proximal portion <NUM> through commands to the flexible distal portion <NUM> of the robot <NUM> so that substantially nullifying movement of the flexible distal portion <NUM> can negate the tremor at the end <NUM>' of the flexible distal portion <NUM>. In a representative embodiment, in a position compensation mode, an image related to the flexible distal portion <NUM> is taken using the image acquisition device <NUM>. As noted above, the image may be an X-ray image, cone-beam CT image, an ultrasound image, or an endoscopic image. The shape and pose of the flexible distal portion <NUM>, and/or registration within the surgical site S, is thereby determined and may be shown on the display <NUM>. For example, real-time tracking of surgical tools relative to a pre-operative surgical plan and interoperative images involving an image-based registration and tool tracking registration are disclosed in above-referenced U. Patent and Patent Application Publications. Since anatomy is visible in the image, the relative position of the flexible distal portion <NUM> with respect to the anatomy is also known, and the flexible distal portion <NUM> can be used to reach the anatomical target T (in this example the location of the valve to be replaced) using the position computed by the control unit <NUM>.

In order to keep the flexible distal portion <NUM> in the same position for the duration of procedure, such as biopsy or heart ablation, in a manner described above, the control unit <NUM> continuously updates the position of the proximal portion <NUM> of the robot <NUM> from the tracking system <NUM> using tracking information from the tracking system <NUM>, and possibly the image acquisition device <NUM>. In other words, the flexible distal portion <NUM> is controlled to move inside the patient P to compensate for motion of the rigid proximal portion <NUM> on the outside of the patient P.

In a representative embodiment, the control unit <NUM> may compute robot <NUM> motion parameters of joints (e.g., joints <NUM>) of the flexible distal portion <NUM> in response to a defined entry location, a defined second deployment path DP2, and the anatomical target T. Such parameters may align the flexible distal portion <NUM> to the defined second entry location E2 of the heart muscle and the planned second deployment path DP2. The control unit <NUM> may produce control commands in response to the computed joint motion parameters, which align flexible distal portion <NUM> to the planned second entry location E2 and the second deployment path DP2, and communicate the robot control commands to the robot <NUM>.

The processor <NUM> may perform the described functions and operations using a combination of hardware, software and firmware. The processor <NUM> is configured to process images, such as from the image acquisition device <NUM>, related to the rigid proximal portion <NUM> and the flexible distal portion <NUM> at or near the surgical site S to register the flexible distal portion <NUM> with corresponding anatomy at the surgical site S. As described in connection with <FIG>, the processor <NUM> is configured to process the tracking information of the proximal portion <NUM> of the robot <NUM> from the tracking system <NUM> to determine motion around the RCM. The processor <NUM> may be further configured to transmit the images to the display <NUM> via the I/O circuitry <NUM>.

As can be appreciated from the description above, through the coordinated function of the image acquisition device <NUM>, the tracking system <NUM>, the various data and software stored in memory <NUM>/CRM <NUM> and the actions of the processor <NUM>, the control unit <NUM> is configured to provide one or more control commands to control the acquisition and processing of live and preoperative images related to the flexible distal portion <NUM> of the surgical robot <NUM> at the surgical site S, and the target T, and use tracking information related to the proximal portion <NUM> and/or the flexible distal portion <NUM> of the robot <NUM> to further control the flexible distal portion <NUM> relative to the target T. In the illustrative examples described below, various features of the surgical robot system <NUM> of representative embodiments are further described. It is noted that these examples are merely illustrative, and in no way intended to be limiting.

<FIG> are schematic diagrams illustrating details of operating the robot in the surgical robot system shown in <FIG>, in accordance with representative embodiments. In particular, <FIG> illustrates operation of the robot where the RCM of rigid proximal portion <NUM> is located at the surface of the patient's body, and <FIG> illustrates operation of the robot where the RCM of rigid proximal portion <NUM> is located at the surface of the region of interest ROI (e.g., internal organ).

Referring to <FIG>, the rigid proximal portion <NUM> of the robot <NUM> is inserted into the patient P through first entry location E1 (e.g., surgical port), either under control of the control unit <NUM> or manually. The RCM of the rigid proximal portion <NUM> is located at the first entry location E1 on the surface of the patient's body, enabling the rigid proximal portion <NUM> to pivot among various angles (e.g., A1 and A2) relative to the surface of the patient's body, thereby positioning the rigid proximal portion <NUM> to accommodate access by the flexible distal portion <NUM> to the target T in the region of interest ROI (e.g., an internal organ). The rigid proximal portion <NUM> may be inserted into the region of interest ROI through second entry location E2. Once inserted, the flexible distal portion <NUM> is deployed to access the target T.

A processor (e.g., processor <NUM> in <FIG>) receives images from at least one image acquisition device (e.g., image acquisition device <NUM>), generates a first deployment path (e.g., first deployment path DP1) based, at least in part, on the received images for the rigid proximal portion <NUM> to follow from the first entry location E1 to the second entry location E2, and provides instructions for a controller (e.g., control unit <NUM>) to deploy the rigid proximal portion <NUM> along the first deployment path. The processor also generates a second deployment path (e.g., second deployment path DP2) based, at least in part, on the received images for the flexible distal portion <NUM> to follow from the second entry location E2 to the target T, and provides instructions for the controller to deploy the flexible distal portion <NUM> from the end of the rigid proximal portion <NUM> along the second deployment path.

Referring to <FIG>, the operation is similar, except that the rigid proximal portion <NUM> pivots around the second entry location E2, requiring a larger incision fro the first entry location E1 to accommodate the pivotal movements of the rigid proximal portion <NUM> at the surface of the patient's body. That is, the rigid proximal portion <NUM> of the robot <NUM> is inserted into the patient P through first entry location E1, either under control of the control unit <NUM> or manually. The RCM of the rigid proximal portion <NUM> is located at the second entry location E2 on the surface of the region of interest ROI (e.g., an internal organ) within the patient's body, enabling the rigid proximal portion <NUM> to pivot among various angles (e.g., B1 and B2) relative to the surface of the region of interest ROI, thereby positioning the rigid proximal portion <NUM> to accommodate access by the flexible distal portion <NUM> to the target T in the region of interest ROI.

As described above, a processor (e.g., processor <NUM>) receives images from at least one image acquisition device (e.g., image acquisition device <NUM>), generates a first deployment path based, at least in part, on the received images for the rigid proximal portion <NUM> to follow from the first entry location E1 to the second entry location E2, and provides instructions for a controller (e.g., control unit <NUM>) to deploy the rigid proximal portion <NUM> along the first deployment path. The processor also generates a second deployment path based, at least in part, on the received images for the flexible distal portion <NUM> to follow from the second entry location E2 to the target T, and provides instructions for the controller to deploy the flexible distal portion <NUM> from the end of the rigid proximal portion <NUM> along the second deployment path.

<FIG> is a perspective view a surgical robot system for accessing a patient's heart, in accordance with a representative embodiment, in which the surgical robot system is used for an aortic valve replacement. The aortic valve replacement is an example of implementing the surgical robot system <NUM>, which may be used for other types of medical procedures and surgeries without departing from the scope of the present teachings.

Referring to <FIG>, surgical robot system <NUM> includes robot <NUM> with rigid proximal portion <NUM> and flexible distal portion <NUM>. In the depicted embodiment, the RCM of the rigid proximal portion <NUM> is at the surface of the body of patient P. The rigid proximal portion <NUM> is manipulated, e.g., by a controller or manually, through first entry location E1 (e.g., an initial incision is made between two ribs) and enters the heart <NUM> through second entry location E2. The flexible distal portion <NUM> deploys through the rigid proximal portion <NUM>, and is used to deploy a prosthetic valve into ventricle <NUM>.

The surgical robot system <NUM> further includes image acquisition device <NUM>, which is a C-arm imaging system with image detector <NUM>-<NUM> and source <NUM>-<NUM>. The image acquisition device <NUM> provides live and/or preoperative images of the heart <NUM> and the robot <NUM>. The image data, together with tracking data from a tracking system (e.g., tracking system <NUM>) enable a processor (e.g., processor <NUM>) to determine and monitor deployment paths for the rigid proximal portion <NUM> to follow from the first entry location E1 to the second entry location E2, and for the flexible distal portion <NUM> to follow from the second entry location E2 to the ventricle <NUM> (target). In the depicted embodiment, a transesophageal echocardiography (TEE) probe <NUM>, with probe <NUM> and transducer <NUM>, is also included to provide live imaging of the heart <NUM>.

<FIG> is a flowchart illustrating operations of a method <NUM> of control and guidance which may be performed by the surgical robot system <NUM>, in accordance with a representative embodiment. To provide an example in the description below, it will be assumed that method <NUM> is performed by the version of surgical robot system <NUM> is illustrated in <FIG>, thus all or part of the operations depicted in <FIG> may be performed by or under control of the processor <NUM>. However, the method may be implemented using other embodiments of the surgical robot system, without departing from the scope of the present teachings.

In this embodiment, it is assumed that the rigid proximal portion <NUM> is configured such that the RCM is positioned at the first entry point E1 to the patient P (e.g., between the ribs). This configuration minimizes injury to ribs and chest muscles, but somewhat limits motion of the rigid proximal portion <NUM> to be performed before the flexible distal portion <NUM> is deployed. In operation <NUM>, an initial position (on the patient's body) of the rigid proximal portion <NUM> is determined. The rigid proximal portion <NUM> is aligned with the initial position, e.g., by the clinician, using either manual or master/slave control of the rigid proximal portion, to position the rigid proximal portion <NUM> in the body cavity through the first entry point E1.

The rigid proximal portion <NUM> is advanced through the body cavity toward the surgical site S. Once the rigid proximal portion <NUM> is visible in image(s) provided by the image acquisition device <NUM>, such as x-ray images, the rigid proximal portion <NUM> is registered to the region of interest ROI (e.g., the patient's heart, in this example) using any method, including registration methods provided herein. For example, real-time tracking of surgical tools relative to a pre-operative surgical plan and intra-operative images involving an image-based registration and tool tracking registration are disclosed in commonly owned <CIT>).

In operation <NUM>, a first deployment path DP1 is generated for the rigid proximal portion <NUM> from the RCM (e.g., the first entry location E1) to an entry point of the region of interest ROI (e.g., the second entry location E2). The first deployment path DP1 may be generated by the processor <NUM> based on known locations of the first entry location E1 and the location of the boundaries of the region of interest ROI, for example, through images provided by the image acquisition device <NUM>. In an embodiment, the first deployment path DP1 may be determined by the clinician, for example, using data and/or images provided by the processor <NUM> and/or the image acquisition device <NUM>.

The rigid proximal portion <NUM> is moved along the determined first deployment path DP1 by operation of the robot <NUM>, until the rigid proximal portion <NUM> is advanced into the region of interest ROI through the second entry location E2, using automatic deployment under control of the control unit <NUM>, in operation <NUM>. For example, first guidance information may be generated for positioning the rigid proximal portion <NUM> along the first deployment path DP1. Generally, guidance information includes registration of the robot <NUM> to the image acquisition device <NUM>, and may provide data such as coordinates in a three-dimensional space, corresponding to a determined deployment path within the patient P and commands to maneuver the robot to the coordinates. Alternatively, the clinician may manually move the rigid proximal portion <NUM> along the determined first deployment path DP1, e.g., using the live images from the image acquisition device <NUM>.

In operation <NUM>, the flexible distal portion <NUM> is deployed into the region of interest ROI through the rigid proximal portion <NUM>. Since, in the present example, the flexible distal portion <NUM> is introduced in the heart, it is visible under x-ray and ultrasound. The flexible distal portion <NUM> may be registered to x-ray from the image acquisition device <NUM>, as well as to ultrasound from an ultrasound TEE probe (optional), such as TEE probe <NUM> discussed above, using EchoNavigator® available from Philips Electronics, for example, in order to provide close-up live imaging of the heart.

The position of the flexible distal portion is tracked within the region of interest ROI in operation <NUM>. A second deployment path DP2 is generated in operation <NUM> for the flexible distal portion <NUM> from the tracked location of the flexible distal portion <NUM> to the target T (e.g., the heart muscle). The second deployment path DP2 may be generated by the processor <NUM> based on known locations of the second entry location E2 and the location of the target T, for example, through images provided by the image acquisition device <NUM>.

In operation <NUM>, a distal tip of the flexible distal portion <NUM> is moved along the second deployment path DP2 to the target T. For example, second guidance information may be generated for positioning the flexible distal portion <NUM> along the second deployment path DP2 in order to move the flexible distal portion <NUM>. In the present example, the valve annulus of the aortic valve of the heart may be detected in at least one of the x-ray and ultrasound images for generating the second deployment path DP2 and/or the second guidance information, and the flexible distal portion of the robot <NUM> may be controlled using such live images to position the distal tip of the flexible distal portion at or perpendicular to the annulus. Once the position is reached, the therapy device is deployed. As discussed above, therapy device may include a balloon catheter and a prosthetic valve passed the left ventricle of the heart, and using a pusher and sleeve to advance the balloon catheter and prosthetic valve mounted thereon in order to properly position the prosthetic valve within the aortic annulus.

In an alternative embodiment, the rigid proximal portion <NUM> is configured such that the RCM is positioned at the second entry location E2 to the region of interest ROI (e.g., the heart). In this embodiment, the first entry location E1 at the chest cavity has to be a larger incision to allow small motion of the rigid proximal portion <NUM>. This enables repositioning of the flexible distal portion <NUM> once deployed.

The deployment operations are the same as in the embodiment discussed above with reference to <FIG>. However, in the present embodiment, a last operation is added to facilitate repositioning of the flexible distal portion <NUM>. For example, if the plan for the second deployment path DP2 yields an unreachable path, the rigid proximal portion <NUM> of the robot <NUM> may reposition the flexible distal portion device to a reachable position by pivoting around the second entry location E2 to the region of interest ROI.

In certain embodiments, the method <NUM> may further include transmitting the medical imagery to display <NUM>. The display arrangement is broadly defined herein as any device structurally configured for displaying images and tracked surgical tools and other end-effectors under any suitable technique. Examples of a display include a computer monitor, a television screen, a touch screen, a projector, and head-mounted display (HMD).

The present teachings are part of a technological progression towards smart systems and devices. Possible applications include augmented reality of live video with preoperative CT, surgical navigation, especially in minimally invasive surgery where the workspace is obscured from view, and finding anatomical targets and tumors.

Claim 1:
A control unit (<NUM>) for a surgical robot system (<NUM>) including a robot having a rigid proximal portion (<NUM>) having a remote center of motion (RCM), a distal portion (<NUM>) being flexible to permit the maneuvering of the distal portion with one or more degrees of freedom and being deployable through the proximal portion and at least one image acquisition device (<NUM>), the control unit comprising:
a processor (<NUM>) configured to:
receive images from the at least one image acquisition device;
generate a first deployment path to a first target position based on the received images;
generate a second deployment path to a second target position from the first target position based on the received images;
generate first guidance information for positioning the proximal portion along the first deployment path;
track position of the distal portion within a region of interest;
generate second guidance information for positioning the distal portion along the second deployment path; and
deploy the first guidance information to the proximal portion for guiding the proximal portion to the first target position, and subsequently deploy the second guidance information to the distal portion for guiding the distal portion to the second target position.