Patent Description:
Computer-assisted surgical procedures, which may include image guided surgery and robotic surgery, have attracted increased interest in recent years. These procedures include the integration of a "virtual" three-dimensional dataset of the patient's anatomy, typically obtained using pre-operative or intra-operative medical imaging (e.g., x-ray computed tomography (CT) or magnetic resonance (MR) imaging), to the actual position of the patient and/or other objects (e.g., surgical instruments, robotic manipulator(s) or end effector(s) in the surgical area. These procedures may be used to aid the surgeon in planning a surgical procedure and may also provide the surgeon with relevant feedback during the course of surgical procedure. There is a continuing need to improve the safety and ease-of-use of computer-assisted surgical systems.

Document <CIT> discloses a marker System. The system includes: a first marker, and a second marker; wherein the first marker and the second marker are configured to emit light from one or more light sources coupled to the first marker and the second marker, and wherein the first marker and the second marker are configured to emit the light for detection by a camera. A method performed using a marker system includes: generating light using one or more light sources; emitting the light at a plurality of markers that are coupled to the light sources; and detecting the light emitted from the plurality of markers using a camera; wherein the act of detecting comprises using one or more filters to reduce ambient light to a level that corresponds with a noise level of the camera while allowing light emitted from the markers to be imaged by the camera.

Document <CIT> a device to be used in combination with a surgical navigation system for defining the position of a straight line determined by an operating surgeon within a three-dimensional coordinate system in an operating theatre. The device includes a body with a surface and at least three markers which emit waves and the position of which can be determined within a tree-dimensional coordinate system by a position detector belonging to the surgical navigation system. A laser mounted on said body emits a laser beam, directed away from the body, which has a geometrical central beam and a wavelength in the visible range. The laser beam is emitted in a geometrically defined position in relation to the markers so that the position of the central beam in relation to the three dimensional coordinate system can be calculated from the previously measured positions of said markers by a computer, which is also part of the surgical navigation system.

Document <CIT> discloses an imagery system including: a camera having a field of view; a light source including a source of structured light, the light source movable relative to the camera; and at least one processor, wherein the at least one processor is programmed to determine, at least, estimated locations of points on a surface exposed to the structured light according to image data received from the field of view of the camera.

Document <CIT> discloses a sensored surgical tool, and surgical intraoperative tracking and imaging system incorporating same, wherein the surgical tool comprises a rigid elongate tool body having a substantially rigid tool tip to be displaced and tracked within the surgical.

Certain optional features of the invention are defined in the dependent claims. Various embodiments include methods and systems for performing computer assisted surgery, including robot-assisted image-guided surgery.

Embodiments include a marker device for an image guided surgery system that includes an electronics unit having at least one light source, a rigid frame attached to the electronics unit, the rigid frame having at least one channel extending from the electronics unit to at least one opening in the rigid frame, and an optical guide apparatus located within the at least one channel to couple light from the at least one light source of the electronics unit to the at least one opening in the rigid frame.

Further embodiments include a marker device for an image guided surgery system that includes an electronics unit including a flexible circuit having a plurality of peripheral arm regions and a light source located on each of the peripheral arm regions, and a rigid frame attached to the electronics unit, the rigid frame having a plurality of channels terminating in openings in the rigid frame, each of the plurality of peripheral arm regions located within a channel with each of the plurality of light sources configured to direct light from a respective opening in the rigid frame.

Further embodiments include a marker system for tracking a robotic arm using a motion tracking system that includes a light source located within the robotic arm, and a marker comprising an optical diffuser that attaches to an outer surface of the robotic arm to optically couple the light source to the diffuser.

Further embodiments include a marker array having a plurality of markers for tracking a robotic arm that includes multiple axes between a proximal end and a distal end of the robotic arm, and an end effector attached to the distal end of the robotic arm, where the marker array includes at least one first marker that is distal to the most distal axis of the robotic arm, and at least one second marker that is proximal to the most distal axis of the robotic arm.

Further embodiments include a multi-axis robotic arm that includes a first section that comprises at least one axis that provides both pitch and yaw rotation, a second section, distal to the first section, that comprises two mutually orthogonal rotary wrist axes, and an end effector coupled to the second section.

Further embodiments include an image guided surgery system that includes an optical sensor facing in a first direction to detect optical signals from a marker device located in a surgical site, a reference marker device located along a second direction with respect to the optical sensor, and a beam splitter optically coupled to the optical sensor and configured to redirect optical signals from the reference marker device to the optical sensor.

Further embodiments include an optical sensing device for a motion tracking system that includes a support structure, at least one optical sensor mounted to the support structure and configured to generate tracking data of one or more objects within a field-of-view of the optical sensor, and an inertial measurement unit mounted to the support structure and configured to detect a movement of the at least one optical sensor.

Further embodiments include an image guided surgery system that includes a marker device, a least one optical sensor configured to detect optical signals from the marker device, an inertial measurement unit mechanically coupled to the at least one optical sensor and configured to measure a movement of the at least one optical sensor, and a processing system, coupled to the at least one optical sensor and the inertial measurement unit, and including at least one processor configured with processor-executable instructions to perform operations including tracking the position and orientation of the marker device based on the optical signals received at the at least one optical sensor, receiving measurement data from the inertial measurement unit indicating a movement of the at least one optical sensor, and correcting the tracked position and orientation of the marker device based on the measurement data from the inertial measurement unit.

Further embodiments include a method of performing image guided surgery that includes tracking the position and orientation of a marker device based on optical signals from the marker device received by at least one optical sensor, receiving measurement data from an inertial measurement unit indicating a movement of the at least one optical sensor, and correcting the tracked position and orientation of the marker device based on the measurement data from the inertial measurement unit.

Further embodiments include an image guided robotic surgery system that includes a robotic arm, a plurality of marker devices, a sensor array located on the robotic arm and configured to detect optical signals from the plurality of marker devices, and a processing system, coupled to the sensor array, and configured to track the position of the plurality of marker devices in three-dimensional space based on the detected optical signals from the sensor array and the joint coordinates of the robotic arm.

Other features and advantages of the present invention will be apparent from the following detailed description of the invention, taken in conjunction with the accompanying drawings of which:.

<FIG> illustrates a system <NUM> for performing computer assisted surgery, including robotically-assisted image-guided surgery according to various embodiments. The system <NUM> in this embodiment includes an imaging device <NUM>, a motion tracking system <NUM> and a robotic arm <NUM> for performing a robotically-assisted surgical procedure. The robotic arm <NUM> may comprise a multi-joint arm that includes a plurality of linkages connected by joints having actuator(s) and optional encoder(s) to enable the linkages to rotate, bend and/or translate relative to one another in response to control signals from a robot control system. The robotic arm <NUM> may be fixed to a support structure at one end and may have an end effector <NUM> at the other end of the robotic arm <NUM>.

The imaging device <NUM> may be used to obtain diagnostic images of a patient <NUM>, which may be a human or animal patient. In embodiments, the imaging device <NUM> may be an x-ray computed tomography (CT) imaging device. The patient <NUM> may be positioned within a central bore <NUM> of the imaging device <NUM> and an x-ray source and detector may be rotated around the bore <NUM> to obtain x-ray image data (e.g., raw x-ray projection data) of the patient <NUM>. The collected image data may be processed using a suitable processor (e.g., computer) to perform a three-dimensional reconstruction of the object. In other embodiments, the imaging device <NUM> may comprise one or more of an x-ray fluoroscopic imaging device, a magnetic resonance (MR) imaging device, a positron emission tomography (PET) imaging device, a singlephoton emission computed tomography (SPECT), or an ultrasound imaging device. In embodiments, image data may be obtained pre-operatively (i.e., prior to performing a surgical procedure) or intra-operatively (i.e., during a surgical procedure) by positioning the patient <NUM> within the bore <NUM> of the imaging device <NUM>. In the system <NUM> of <FIG>, this may be accomplished by moving the imaging device <NUM> over the patient <NUM> to perform a scan while the patient <NUM> may remain stationary.

Examples of x-ray CT imaging devices that may be used according to various embodiments are described in, for example, D. Patent No. <CIT>, D. Patent Application Publication No. <CIT>, D. Patent Application Publication No. <CIT>, D. Patent Application Publication No. <CIT> and D. Patent Application Publication No. <CIT>. In the embodiment shown in <FIG>, the patient support <NUM> (e.g., surgical table) upon which the patient <NUM> may be located is secured to the imaging device <NUM>, such as via a column <NUM> which is mounted to a base <NUM> of the imaging device <NUM>. A portion of the imaging device <NUM> (e.g., an O-shaped imaging gantry <NUM>) which includes at least one imaging component may translate along the length of the base <NUM> on rails <NUM> to perform an imaging scan of the patient <NUM>, and may translate away from the patient <NUM> to an out-of-the-way positon for performing a surgical procedure on the patient <NUM>.

An example imaging device <NUM> that may be used in various embodiments is the AIRO® intra-operative CT system manufactured by Mobius Imaging, LLC and distributed by Brainlab, AG. Other imaging devices may also be utilized. For example, the imaging device <NUM> may be a mobile CT device that is not attached to the patient support <NUM> and may be wheeled or otherwise moved over the patient <NUM> and the support <NUM> to perform a scan. Examples of mobile CT devices include the BodyTom® CT scanner from Samsung Electronics Co. and the O-arm® surgical imaging system form Medtronic, plc. The imaging device <NUM> may also be a C-arm x-ray fluoroscopy device. In other embodiments, the imaging device <NUM> may be a fixed-bore imaging device, and the patient <NUM> may be moved into the bore of the device, either on a surgical support <NUM> as shown in <FIG>, or on a separate patient table that is configured to slide in and out of the bore. Further, although the imaging device <NUM> shown in <FIG> is located close to the patient <NUM> within the surgical theater, the imaging device <NUM> may be located remote from the surgical theater, such as in another room or building (e.g., in a hospital radiology department).

The motion tracking system <NUM> shown in <FIG> includes a plurality of marker devices <NUM>, <NUM>, <NUM> and an optical sensor device <NUM>. Various systems and technologies exist for tracking the position (including location and/or orientation) of objects as they move within a three-dimensional space. Such systems may include a plurality of active or passive markers fixed to the object(s) to be tracked and a sensing device that detects radiation emitted by or reflected from the markers. A 3D model of the space may be constructed in software based on the signals detected by the sensing device.

The motion tracking system <NUM> in the embodiment of <FIG> includes a plurality of marker devices <NUM>, <NUM> and <NUM> and a stereoscopic optical sensor device <NUM> that includes two or more cameras (e.g., IR cameras). The optical sensor device <NUM> may include one or more radiation sources (e.g., diode ring(s)) that direct radiation (e.g., IR radiation) into the surgical field, where the radiation may be reflected by the marker devices <NUM>, <NUM> and <NUM> and received by the cameras. The marker devices <NUM>, <NUM>, <NUM> may each include three or more (e.g., four) reflecting spheres, which the motion tracking system <NUM> may use to construct a coordinate system for each of the marker devices <NUM>, <NUM> and <NUM>. A computer <NUM> may be coupled to the sensor device <NUM> and may determine the transformations between each of the marker devices <NUM>, <NUM>, <NUM> and the cameras using, for example, triangulation techniques. A 3D model of the surgical space in a common coordinate system may be generated and continually updated using motion tracking software implemented by the computer <NUM>. In embodiments, the computer <NUM> may also receive image data from the imaging device <NUM> and may register the image data to the common coordinate system as the motion tracking system <NUM> using image registration techniques as are known in the art. In embodiments, a reference marker device <NUM> (e.g., reference arc) may be rigidly attached to a landmark in the anatomical region of interest (e.g., clamped or otherwise attached to a bony portion of the patient's anatomy) to enable the anatomical region of interest to be continually tracked by the motion tracking system <NUM>. Additional marker devices <NUM> may be attached to surgical tools <NUM> to enable the tools <NUM> to be tracked within the common coordinate system. Another marker device <NUM> may be rigidly attached to the robotic arm <NUM>, such as on the end effector <NUM> of the robotic arm <NUM>, to enable the position of robotic arm <NUM> and end effector <NUM> to be tracked using the motion tracking system <NUM>. The computer <NUM> may also include software configured to perform a transform between the joint coordinates of the robotic arm <NUM> and the common coordinate system of the motion tracking system <NUM>, which may enable the position and orientation of the end effector <NUM> of the robotic arm <NUM> to be controlled with respect to the patient <NUM>.

In addition to passive marker devices described above, the motion tracking system <NUM> may alternately utilize active marker devices that may include radiation emitters (e.g., LEDs) that may emit radiation that is detected by an optical sensor device <NUM>. Each active marker device or sets of active marker devices attached to a particular object may emit radiation in a pre-determined strobe pattern (e.g., with modulated pulse width, pulse rate, time slot and/or amplitude) and/or wavelength which may enable different objects to be uniquely identified and tracked by the motion tracking system <NUM>. One or more active marker devices may be fixed relative to the patient, such as secured to the patient's skin via an adhesive membrane or mask. Additional active marker devices may be fixed to surgical tools <NUM> and/or to the end effector <NUM> of the robotic arm <NUM> to allow these objects to be tracked relative to the patient.

In further embodiments, the marker devices may be passive maker devices that include moiré patterns that may enable their position and orientation to be tracked in three-dimensional space using a single camera using Moiré Phase Tracking (MPT) technology. Each moiré pattern marker may also include a unique identifier or code that may enable different objects within the camera's field of view to be uniquely identified and tracked. An example of an MPT-based tracking system is available from Metria Innovation Inc. of Milwaukee, Wisconsin. Other tracking technologies, such as computer vision systems and/or magnetic-based tracking systems, may also be utilized.

The system <NUM> may also include a display device <NUM> as schematically illustrated in <FIG>. The display device <NUM> may display image data of the patient's anatomy obtained by the imaging device <NUM>. The display device <NUM> may facilitate planning for a surgical procedure, such as by enabling a surgeon to define one or more target positions in the patient's body and/or a path or trajectory into the patient's body for inserting surgical tool(s) to reach a target position while minimizing damage to other tissue or organs of the patient. The position and/or orientation of one or more objects tracked by the motion tracking system <NUM> may be shown on the display <NUM>, and may be shown overlaying the image data. In the embodiment of <FIG>, the display <NUM> is located on a mobile cart <NUM>. A computer <NUM> for controlling the operation of the display <NUM> may also be housed within the cart <NUM>. In embodiments, the computer <NUM> may be coupled to the optical sensor device <NUM> and may also perform all or a portion of the processing (e.g., tracking calculations) for the motion tracking system <NUM>. Alternatively, one or more separate computers may perform the motion tracking processing, and may send tracking data to computer <NUM> on the cart <NUM> via a wired or wireless communication link. The one or more separate computers for the motion tracking system <NUM> may be located on the imaging system <NUM>, for example.

<FIG> illustrates an alternative embodiment of a system for performing robotically-assisted image-guided surgery according to various embodiments. In this embodiment, the optical sensor device <NUM> includes an array of cameras <NUM> mounted to a rigid support <NUM>. The support <NUM> including the camera array is suspended above the patient surgical area by an arm <NUM>. The arm <NUM> may be mounted to or above the imaging device <NUM>. The position and/or orientation of the rigid support <NUM> may be adjustable with respect to the arm <NUM> to provide the camera array with a clear view into the surgical field while avoiding instructions. In embodiments, the rigid support <NUM> may pivot with respect to the arm <NUM> via a joint <NUM>. In some embodiments, the position of the rigid support <NUM> may be adjustable along the length of the arm <NUM>. A handle <NUM> attached to the rigid support <NUM> may be used to adjust the orientation and/or position of the optical sensor device <NUM>. The optical sensor device <NUM> may be normally locked in place with respect to the arm <NUM> during an imaging scan or surgical procedure. A release mechanism <NUM> on the handle <NUM> may be used to unlock the optical sensor device <NUM> to enable its position and/or orientation to be adjusted by the user. In some embodiments, the arm <NUM> or a portion thereof may be pivotable with respect to the imaging system <NUM>, such as via joint <NUM>. In embodiments, the arm <NUM> may be raised or lowered relative to the top surface of the imaging system <NUM>, and in some embodiments the entire arm <NUM> may reciprocate (e.g., to the left or right in <FIG>) with respect to the imaging system <NUM>.

In some embodiments, the rigid support <NUM> and cameras <NUM> may be removably secured to the arm <NUM> so that the support <NUM> and cameras <NUM> may be detached from the system for storage and/or transport. A docking system between the arm <NUM> and the rigid support <NUM> may provide mechanical coupling between the support <NUM> and the arm <NUM> and may also provide an electrical connection for data and/or power between the arm <NUM> and the array of cameras <NUM> mounted to the support <NUM>.

<FIG> also illustrates a display device that may comprise a handheld display device <NUM>. As used herein, "handheld computing device" and "handheld display device" are used interchangeably to refer to any one or all of tablet computers, smartphones, pendant controllers, cellular telephones, personal digital assistants (PDA's), netbooks, e-readers, laptop computers, palm-top computers, wearable computers, and similar portable electronic devices which include a programmable processor and memory coupled to a display screen and may include hardware and/or software to enable display of information, including patient information and/or images, on the display screen. A handheld computing device typically also includes an antenna coupled to circuitry (e.g., a transceiver) to enable wireless communication over a network. A handheld computing or display device may be characterized by a sufficiently compact and lightweight structure to enable a user to easily grasp, maneuver and operate the device using one or both hands.

A holder for a handheld computing device <NUM> may be in a suitable location to enable the user to easily see and/or interact with the display screen and to grasp and manipulate the handheld computing device <NUM>. The holder may be a separate cart or a mount for the handheld computing device that may be attached to the patient support <NUM> or column <NUM> or to any portion of the imaging system <NUM>, or to any of the wall, ceiling or floor in the operating room. In some embodiments, a handheld computing device <NUM> may be suspended from the arm <NUM> to which the optical sensing device <NUM> is attached. One or more handheld display devices <NUM> may be used in addition to or as an alternative to a conventional display device, such as a cart-mounted monitor display device <NUM> as shown in <FIG>.

As shown in <FIG> and <FIG>, the robotic arm <NUM> may be fixed to the imaging device <NUM>, such as on a support element <NUM> (e.g., a curved rail) that may extend concentrically over the outer surface of the O-shaped gantry <NUM> of the imaging device <NUM>. In embodiments, an arm <NUM> to which the optical sensing device <NUM> is mounted may be mounted to the same or a similar support element <NUM> (e.g., curved rail) as the robotic arm <NUM>. The position of the robotic arm <NUM> and/or the support arm <NUM> may be adjustable along the length of the support element <NUM>. In other embodiments, the robotic arm <NUM> may be secured to any other portion of the imaging device <NUM>, such as directly mounted to the gantry <NUM>. Alternatively, the robotic arm <NUM> may be mounted to the patient support <NUM> or column <NUM>, to any of the wall, ceiling or floor in the operating room, or to a separate cart. In further embodiments, the robotic arm <NUM> and/or the optical sensing device <NUM> may be mounted to a separate mobile shuttle, as described in <CIT>. Although a single robotic arm <NUM> is shown in <FIG> and <FIG>, it will be understood that two or more robotic arms <NUM> may be utilized.

<FIG> is a process flow diagram that illustrates a method <NUM> of registering patient images. Computer-assisted surgery techniques generally utilize a process of correlating a dataset representing a portion of the patient's anatomy that is to be operated on with the position of the patient at the time of the surgical intervention. The position of the patient may be determined based on a second image dataset which may include real-time camera image(s) from a motion tracking system <NUM> as described above. The correlation between these datasets may be accomplished computationally using software, and may be referred to as "patient registration. " The registration method <NUM> of <FIG> may be implemented using one or more computing devices, such as computer <NUM> shown in <FIG>.

In block <NUM> of method <NUM>, a first image dataset of the patient's anatomy may be obtained using an imaging device, such as the imaging device <NUM> shown in <FIG> and <FIG>. The first image dataset may be a three-dimensional dataset (e.g., a 3D CT tomographic reconstruction, a 3D MRI dataset, etc.) representing at least a portion of the patient's anatomy, including the internal anatomy and/or structure(s) that are to be operated on (i.e., a surgically-relevant portion of the patient's anatomy). The first image dataset may be stored electronically in a memory. The first image dataset may be in any suitable format, such as in a file format that conforms to the Digital Imaging and Communications in Medicine (DICOM) standard.

In block <NUM> of method <NUM>, a second image dataset of the patient and the surrounding patient space may be obtained using a motion tracking system, such as the motion tracking system <NUM> shown in <FIG> and <FIG>. The second image dataset may indicate the current position and/or orientation of the patient. The second image dataset may include at least one image of a marker device that may be obtained using an optical sensing device <NUM> (e.g., cameras <NUM>). The marker device (e.g., reference arc <NUM>) detected by the optical sensing device <NUM> may be in a known fixed relationship with the surgically-relevant portion of the patient's anatomy. The motion tracking system <NUM> may determine the transformation between the marker device <NUM> and the optical sensing device <NUM> (e.g., using well-known triangulation techniques), and may thereby determine the transformation between the sensing device <NUM> (e.g., camera <NUM> position) and the surgically-relevant portion of the patient's anatomy. The motion tracking system <NUM> may similarly determine transformations between each of the other marker devices (e.g., marker devices <NUM> and <NUM> in <FIG>) and the optical sensing device <NUM>. Each of the markers <NUM>, <NUM> and <NUM> being tracked may then be placed within a common coordinate system. In embodiments, the common coordinate system may have an origin or zero point that may be considered to be fixed relative to the surgically-relevant portion of the patient's anatomy, and may also be referred to the patient coordinate system.

In block <NUM> of method <NUM>, the first image dataset may be registered to the common coordinate system as the second image dataset (e.g., the patient coordinate system). This may include performing a rigid transformation to map each pixel or voxel of the first image dataset into corresponding 3D coordinates (i.e., x, y, z coordinates) of the common coordinate system. A number of techniques may be utilized for registering multiple image datasets. In one non-limiting example of a registration process for x-ray CT imaging data, a pre-scan calibration process may be used to precisely calculate (e.g., within <NUM>) the transformation between the isocenter of the x-ray gantry <NUM> and the optical sensing device <NUM>. A set of markers <NUM> (e.g., <NUM> or more, such as <NUM>-<NUM> markers) may be provided on the surface of the gantry <NUM>, as shown in <FIG>. The markers <NUM> may be within the field of view of the optical sensing device <NUM> to enable the gantry <NUM> position to be tracked by the motion tracking system <NUM>. A calibration phantom (not shown for clarity) having a marker device (e.g., similar to marker device <NUM> in <FIG>) fixed thereto may be placed on the patient support <NUM> such that the marker device is also within the field of view of the optical sensing device <NUM>. The motion tracking system <NUM> may determine the transformation between the gantry <NUM> coordinate system defined by the markers <NUM> and the optical sensing device <NUM> coordinate system as well as the transformation between the phantom coordinate system defined by the marker device on the phantom and the optical sensing device <NUM> coordinate system. These transformations may be used to determine the gantry-to-phantom transformation. The phantom may then be scanned using the imaging device <NUM>. A set of elements (e.g., x-ray visible beads) that may be easily identified from the imaging data may be located in the phantom, where the geometry of these elements within the phantom coordinate system may be previously-known. An algorithm may be used to analyze the x-ray image data to identify the x-ray visible elements with respect to the center point of the image data, which corresponds to the isocenter of the gantry <NUM>. Thus, the x-ray visible elements may be located in a coordinate system having an origin at the isocenter of the x-ray gantry <NUM>, and the transformations between the isocenter and the phantom and the isocenter and the markers <NUM> on the gantry <NUM> may be calculated.

During a subsequent scan of the patient <NUM>, the position and orientation of the patient <NUM> with respect to the isocenter of the imaging device <NUM> may be determined (i.e., by tracking the positions of the markers <NUM> on the gantry <NUM>, which are known with respect to the isocenter, and the patient reference arc <NUM>, which is known with respect to the surgically-relevant portion of the patient anatomy). This may enable the image data obtained during the scan to be registered into the patient coordinate system.

In an alternative embodiment, the position of the optical sensing device <NUM> may be known relative to the imaging system <NUM> with sufficient accuracy such that the image dataset of the patient's anatomy obtained using the imaging system <NUM> may be registered in the common coordinate system of the patient without the motion tracking system <NUM> needing to track the position or orientation of the imaging system <NUM>. In embodiments, separate markers <NUM> on the gantry <NUM> of the imaging system <NUM> as shown in <FIG> may not be required or used. In some embodiments, the position of the optical sensing device <NUM> (e.g., the position of each of the cameras <NUM> as shown in <FIG> and <FIG>) may be known relative to the isocenter of the gantry <NUM> of the imaging system <NUM>, such as via a calibration process that may be performed at the factory or during installation or pre-calibration of the system. The gantry <NUM> and/or the optical sensing device <NUM> may include keying features (e.g., high-precision bolt patterns) where the optical sensing device <NUM> attaches to the gantry <NUM> to ensure that the position of the sensing device <NUM> on the gantry <NUM> remains accurately fixed. In embodiments where the camera(s) <NUM> may be movable relative to the gantry <NUM>, high-precision encoders may precisely record and correct for any changes in camera position/orientation relative to the isocenter of the gantry <NUM>. During imaging scans, the optical sensing device <NUM> may track the position and orientation of the patient <NUM> with respect to the camera position, which is in a known, fixed geometric relationship with the isocenter of the imaging device <NUM>. The image data obtained during a scan may thus be registered into the common coordinate system of the patient without needing to first perform a calibration scan on a phantom, as described above.

In block <NUM> of method <NUM>, images of the patient's anatomy from the first image dataset may be displayed with an overlay of one or more features derived from the second image dataset in the common coordinate system. The images may be displayed on a suitable display device, such as display <NUM> shown in <FIG>. The images of the patient's anatomy may include 2D slices of a three-dimensional image dataset (e.g., a tomographic reconstruction) and/or a 3D volume rendering of all or a portion of the image dataset. In embodiments, images obtained using multiple imaging devices or imaging modalities may be fused and displayed in a common coordinate system. For example, the first image dataset of the patient's internal anatomy may be an x-ray CT scan. Another image dataset of the patient's internal anatomy, such as an MRI scan, may be combined with the x-ray CT data and displayed on the display <NUM>. The MRI scan data may be registered into the common coordinate system using a similar registration process as described above. Alternately or in addition, an algorithm for matching landmarks or fiducials identifiable from both image datasets may be used to merge the datasets for display.

The one or more features derived from the second image dataset that may be displayed overlaying the images of the patient's anatomy may include graphical depictions of a tool <NUM>, an end effector <NUM> or another object that is tracked by the motion tracking system <NUM>. The graphical depiction may be based on a known geometry of the tool <NUM>, end effector <NUM> or another object. The graphical depiction may be a rendering of the actual size and shape of the object or may be a depiction of select features of the object, such as a location of a tip end of the object and/or an orientation of the object. The graphical depiction may also indicate a trajectory defined by the object (e.g., a ray extending from a tip end of the object into the patient) and/or a target point within the patient's anatomy that may be defined based on the position and/or orientation of one or more objects being tracked. In various embodiments, the tool <NUM> may be a pointer. The tool <NUM> may also be a surgical instrument, such as a needle, a cannula, dilator, a tool for gripping or cutting, an electrode, an implant, a drill bit, a screw, a screw driver, a radiation source, a drug and an endoscope. In embodiments, the end effector <NUM> of the robotic arm <NUM> may include a hollow tube or cannula that may be configured to hold one or more tools, such as a surgical instrument, and may be used to guide an instrument as it is inserted into the patient's body. Alternately, the end effector <NUM> itself may be or may include an instrument that may be inserted into the patient's body.

The motion tracking system <NUM> may repeatedly acquire new images from the optical sensing device <NUM>, and the relative positions and/or orientations of objects within the field of view of the optical sensing device <NUM> may be updated with each acquisition of new images from the optical sensing device <NUM>. The display <NUM> may be updated to reflect any change(s) in the position and/or orientation of the objects within the common coordinate system (e.g., relative to the patient reference arc <NUM>), which may include adding additional graphical elements to depict new objects that are moved within the field of view of the optical sensing device <NUM> and removing graphical depictions of objects when they are no longer within the field of view of the optical sensing device <NUM>. In some embodiments, the optical sensing device <NUM> may include a motorized system to enable the position and/or orientation of the camera(s) <NUM> to move to maintain the surgical area within the center of the field of view of the camera(s) <NUM>.

<FIG> is a component block diagram of an image-guided surgery system <NUM> according to an embodiment. The system <NUM> may be implemented using one or more computing devices, such as computer <NUM> shown in <FIG>. The system <NUM> may be operatively coupled to a first display device <NUM>, which may include a monitor that is fixed to a cart <NUM> or other structure (e.g., wall, ceiling, floor, imaging device, etc.) within the operating suite. The system <NUM> may also be operatively coupled to at least one additional display device <NUM>, which may be a handheld computing device, as described above with reference to <FIG>. The system <NUM> may also include an audio input/output component <NUM>, which may include a speaker or other output component for outputting audible signals (e.g., audio instructions, alerts, etc.) and/or a microphone or other input component for receiving audio inputs (e.g., voice commands) that may be interpreted by the system <NUM>. The system <NUM> may be implemented at least partially in software and may be based on one or more of the Image-Guided Surgery Toolkit (IGSTK), Visualization Toolkit (VTK) and Insight Segmentation and Registration Toolkit (ITK) development frameworks.

The system <NUM> may be configured to receive and store imaging data <NUM> (e.g., DICOM data) collected by an imaging device <NUM>. The imaging data <NUM> may be received directly from the imaging device <NUM> or may be retrieved from another source, such as a remote server. The imaging data <NUM> may be imaging data that is obtained prior to a surgical procedure (e.g., pre-operative image data) and/or imaging data that is obtained during a surgical procedure (e.g., intra-operative image data). In embodiments, the system <NUM> may be configured to display the most-current image data <NUM> collected by the imaging device <NUM>. The image data <NUM> may be registered to a common coordinate system as the tracking data <NUM> from the motion tracking system <NUM> in accordance with a registration method such as method <NUM> described above with reference to <FIG>.

The system <NUM> may also receive tracking data <NUM> from a motion tracking system <NUM>. The system <NUM> may be configured to repeatedly read the tracking data from the motion tracking system <NUM> indicating the current position/orientation of the patient and any other objects tracked by the motion tracking system <NUM>. The system <NUM> may read the tracking data at a frequency (e.g., refresh rate) of greater than <NUM> (e.g., <NUM>). In embodiments, the tracking data from the motion tracking system <NUM> may include data to enable the system <NUM> to identify particular objects from within the tracking data. For example, each marker device (e.g., marker devices <NUM>, <NUM> and <NUM> in <FIG>) may include a unique characteristic (e.g., a unique geometric pattern of reflective markers, a unique flash pattern of active markers, etc.) to enable the marker device to be identified. These unique characteristics of the marker devices may be registered with particular objects or tools (e.g., associated with a particular object or tool in a database) by the system <NUM>. The unique characteristics of the marker devices may be pre-registered in the system <NUM> and/or may be registered to particular objects or tools during the course of a surgical procedure.

In one embodiment, the image guided surgery system <NUM> may include an automatic identification and data capture (AIDC) component <NUM> that may be used during registration of surgical tools or instruments with unique marker devices. The AIDC component <NUM> may include a sensor device, such as an optical scanner, an RF receiver, a camera, etc. that may be configured to analyze a characteristic of the surgical tool (e.g., scan an identifying mark, such as a model or serial number, etched into the tool, scan a barcode, RFID tag or near-field communication (NFC) tag on the tool, analyze a geometric feature of the tool using machine vision, etc.) while the motion tracking system <NUM> identifies the marker device attached to the tool. The AIDC component may search a database to determine whether the surgical tool or instrument has been previously entered into the IGS system <NUM>, and if so, the IGS system <NUM> may automatically register the marker pattern in association with the known surgical tool or instrument. This may improve workflow and patient safety by obviating the need for medical personnel to manually enter data to pre-register tools/instruments. In embodiments, the registration process for surgical tools may occur while the tool is placed within a calibration fixture that may be used to precisely determine one or more geometric characteristics of the tool, such as the location of the tip end of the tool relative to the marker device, that may be registered in association with the tool and the unique marker pattern during a surgical procedure.

The system <NUM> may also include a library of graphical elements that may be associated with particular objects or tools (e.g., in a database). The system <NUM> may display graphical elements associated with the objects or tools being tracked by the motion tracking system <NUM> in the common coordinate system with the image data on the display(s) <NUM>, <NUM>.

The system <NUM> may include a user-interface component that may control the display of system information and/or graphical user interface elements on the display(s) <NUM> and <NUM>. The system <NUM> may further process and implement user commands received from user interface devices. A user interface device, may include, for example, a touchscreen user interface which may be integrated with a display device <NUM>, <NUM>. In embodiments, a user interface device may alternately or additionally include one or more of a button, a keyboard, a joystick, a mouse, a touchpad, etc. which may be located on a display device <NUM>, <NUM> and/or on a workstation (e.g., a workstation located on a cart <NUM>). In embodiments, the user interface device(s) may also include a microphone (e.g., audio input/output component <NUM>) that may receive voice commands that may be interpreted by the system (e.g., using voice recognition software). The user commands received via one or more user input devices may enable a user to control various functions of the system <NUM>, such as changing what is shown on the display(s) <NUM>, <NUM> (e.g., displaying different image datasets, displaying different slice(s) and/or different 3D rendering(s) within an image dataset, zooming in or out of an image, displaying different menu options, returning to a home screen, etc.). In embodiments, the user commands may enable a user to set one or more trajectories and/or target locations within the patient's anatomy. The system <NUM> may store the positions and/or orientations of user-defined trajectories or target locations within the common coordinate system, and may display graphical representations of such trajectories or target locations on the display(s) <NUM>, <NUM>.

The user commands received by the system <NUM> may also include commands for controlling the operation of other components, such as the imaging device <NUM>, the motion tracking system <NUM> and/or a robotic arm <NUM>. For example, for a robotically-assisted surgical procedure, the user command may include an instruction to move a robotic arm <NUM> to a particular position and/or orientation. The instruction to move the robotic arm <NUM> may be based on a user interaction with image data of the patient's anatomy that is displayed on a display device <NUM>, <NUM>. For example, the user may use the display device <NUM>, <NUM> to define a particular trajectory with respect to the patient's anatomy and may send an instruction for the robotic arm <NUM> to move such that that the end effector <NUM> of the robotic arm <NUM> is positioned along the defined trajectory.

A robotic control system <NUM> may control the movement of one or more robotic arms <NUM>. The robotic control system <NUM> may receive sensor data indicating the current parameters of the robotic arm <NUM> (e.g., robot position, joint angles, measured axis forces, motor currents) and may send motor control signals to drive the movement of the arm <NUM>. In embodiments, the motion tracking system <NUM> may track the position of the robotic arm <NUM> (e.g., via marker device <NUM> on or proximate to end effector <NUM> as shown in <FIG>) to determine the position of the end effector <NUM> within the common coordinate system of the patient. A control loop, which may be executed using the image-guided surgery system <NUM>, the motion tracking system <NUM> and/or the robotic control system <NUM>, may continuously read the tracking data and the robot parameter data and may send instructions to the robotic control system <NUM> to cause the robotic arm <NUM> to move to a desired position and orientation.

In various embodiments, display device <NUM> may be a primary display device (e.g., a monitor) that may be connected to the image-guided surgery system <NUM> by a wired or wireless link. In one embodiment, the system <NUM> may stream video data to the display device <NUM> over a suitable video data interface (e.g., an HDMI interface) and may also exchange other signals with the display device over a separate data connection (e.g., a USB connection).

In various embodiments, display device <NUM> may be a handheld computing device. A handheld display device <NUM> may generally be smaller and lighter than the primary display device <NUM> (e.g., monitor), and may in certain embodiments be referred to as a secondary display device. In some embodiments, display device <NUM> may be a mirror of display device <NUM> and may display all or a portion of the same information as is shown on display device <NUM>. Alternately, display device <NUM> may display different information than is shown on display device <NUM>. In some embodiments, display device <NUM> may be omitted, and handheld display device <NUM> may be the only display device operably connected to the image-guided surgery system <NUM>. In such a case, display device <NUM> may be referred to as the primary display device. Further, although a single handheld display device <NUM> (i.e., a tablet computer) is shown in <FIG>, it will be understood that multiple handheld display devices <NUM> may be simultaneously connected to and used with the system <NUM>.

The handheld display device <NUM> may be coupled to the image-guided surgery system <NUM> by a wired or wireless communication link. In one embodiment, the handheld display device <NUM> may communicate with the system <NUM> over a wireless communication interface. The system <NUM> may stream digital video data (e.g., highdefinition video) for display on the handheld display device <NUM>, such as over a wireless local area network (WLAN) connection, including a IEEE <NUM> (e.g., WiFi) connection. The system <NUM> may also exchange other signals with the handheld display device <NUM> (e.g., control signals from the system <NUM> and/or user commands received at a user interface, such as a touchscreen, on the display device <NUM>) over a wireless connection. The system <NUM> and the display device <NUM> may communicate over any suitable wireless protocol or standard, such as over a IEEE <NUM>. 15x (e.g., a BLUETOOTH®) connection.

An image-guided surgical system <NUM> according to various embodiments may provide a plurality of modes for displaying patient information. For example, a first display mode may include displaying a 3D image dataset (e.g., an x-ray CT, MRI, sonogram, PET or SPECT image dataset) in multiple two-dimensional slices corresponding to anatomic planes (e.g., axial, sagittal, coronal planes) transecting the patient. This is illustrated in the screenshot of a display device shown in <FIG>. The display device may be a display device <NUM> (e.g., monitor) as shown in <FIG> or a handheld display device as shown in <FIG> and <FIG>. The display screen <NUM> in this example illustrates four different patient images in four quadrants of the display screen <NUM>. Three of the quadrants (i.e., top left, top right and bottom left quadrants of display screen <NUM>) depict different two-dimensional slices <NUM>, <NUM>, <NUM> of CT image data. A fourth quadrant (i.e., lower left quadrant of display screen <NUM>) includes a 3D volume rendering <NUM> illustrating a "virtual" view of anatomic feature(s) (e.g., bony structures or other discrete internal anatomic features). The two-dimensional slices <NUM>, <NUM>, <NUM> correspond, respectively, to views taken along axial, sagittal and coronal planes through the patient <NUM>.

The display screen <NUM> may also display graphical elements illustrating the relationship of each slice <NUM>, <NUM>, <NUM> relative to the other slices shown on the display screen <NUM>. For example, as shown in <FIG>, the axial slice <NUM> image data may include an overlay of a cross pattern <NUM> showing the intersection of the axial slice <NUM> with the planes corresponding to the sagittal and coronal slices <NUM> and <NUM> shown on the display screen <NUM>. Similar cross patterns <NUM> may be displayed overlaying the display of image data in the sagittal and coronal slices <NUM> and <NUM>. The display screen <NUM> may also include graphical representations or renderings of other objects or tools tracked by the motion tracking system <NUM>. In the example of <FIG>, a graphical representation of a tool <NUM> is shown in the lower right quadrant of the display screen <NUM>. The graphical representation of the tool <NUM> may illustrate the position and orientation of the tool relative to the anatomic features depicted in the 3D volume rendering <NUM>. Similar graphical elements may be displayed in the 2D slice images <NUM>, <NUM> and <NUM> to illustrate the position and/or orientation of one or more objects with respect to the patient.

It will be understood that the four-quadrant view shown in <FIG> is one possible implementation of a display of patient information on a display device <NUM>, <NUM>. Other possible display modes are possible. For example, rather than illustrating multiple different images (e.g., slices) from a patient image dataset (e.g., reconstructed volume), the display screen <NUM> may show only a single image (e.g., a single axial, sagittal or coronal slice <NUM>, <NUM>, <NUM> or a single 3D volume rendering <NUM>). The display screen <NUM> may illustrate only two slices corresponding to different anatomic planes (e.g., axial and sagittal, axial and coronal, or sagittal and coronal slices), or may illustrate a single slice along with a 3D volume rendering. In some embodiments, the display screen <NUM> may illustrate multiple two-dimensional slices corresponding to the same anatomic planes (e.g., multiple axial, sagittal and/or coronal slices taken through different sections of the reconstructed volume) and/or multiple 3D volume renderings viewed from different angles. The display screen <NUM> may also display real-time video images of the surgical area. The real-time video images may be obtained from a camera located at a suitable location, such as a head-mounted camera worn by a surgeon and/or a camera mounted to a structure (e.g., on the optical sensing device <NUM>, arm <NUM> or support element <NUM>, on the imaging device <NUM>, to the patient support <NUM>, to a surgical light apparatus, or to any of to any of the wall, ceiling or floor in the operating room, or to a separate cart). The video images may also be images obtained from within the surgical site, such as from an endoscope inserted into the patient.

The different images and display modes of the display screen <NUM> may be customizable based on user selections, which may be made via a user input device and/or user voice commands. In embodiments, the user may be able to select (e.g., scroll through) different patient images, such as sequentially illustrating multiple axial, sagittal and/or coronal slices taken through different sections of the reconstructed volume, or sequentially illustrating multiple 3D volume renderings viewed from different angles. The display screen <NUM> may also display slices along oblique planes taken through the reconstructed volume. The user may also have the capability to control the magnification of images, such as by zooming into or out from a particular portion of an image shown in the display screen <NUM>. The user may control the selection of patient images for display using a user input device, voice commands and/or via a separate tool, such as a pointer device. In some embodiments, the intersection of the three image planes (i.e., axial, sagittal and coronal) shown on the display panel <NUM> may coincide with a target position within the patient's body. The surgeon may use the display panel <NUM> as a "virtual cutting tool" to move through the various slices/views of the patient image volume and to identify and select a target region for a surgical intervention.

The user (e.g., a surgeon) may be able to set one or more target positions and/or trajectories within the patient <NUM>. There may be a variety of ways to set a trajectory or target location. For example, the surgeon may move through different views of the patient image data by manipulating a tool (e.g., a pointer/stylus device and/or an end effector of a robotic arm) over the patient <NUM>, where the tool may define a unique trajectory into the patient. The tool may be tracked within the patient coordinate system using the motion tracking system <NUM>. In some embodiments, an imaginary ray projected forward from the tip end of the tool may define the unique trajectory into the patient, which may be graphically depicted on the display screen <NUM>. A target location along the unique trajectory may be defined based on a pre-determined offset distance from the tip end of the tool. Alternately, the surgeon may directly manipulate and interact with the displayed image data to identify a particular target or trajectory, such as using a workstation computer. A particular target point or trajectory may be set by the system <NUM> in response to an input event, which may include, for example, a voice command, a touch event on a touchscreen interface, and/or an input on a user interface device (e.g., a keyboard entry, a mouse click, a button push, etc.). In embodiments, the surgeon may set a target position and/or trajectory by interacting with image data displayed on a display device, such as display devices <NUM> and/or <NUM>. For example, the surgeon may define a target point and/or trajectory in the patient <NUM> by selecting one or more points on a display screen <NUM> of a display device <NUM>, <NUM> (e.g., marking the points using a stylus, a cursor or mouse pointer, or a touch on a touchscreen user interface). To define a trajectory, for instance, the user may select two or more points in the image data (e.g., a target point and an entrance point on the skin of the patient). In embodiments, the user may be able to make fine adjustments to a selected target point and/or trajectory using any suitable user interface device. Multiple target points and/or trajectories may be set and saved in a memory (e.g., in an image-guided surgery system <NUM> as illustrated in <FIG>), where each target point and/or trajectory may be saved in association with a unique identifier (e.g., file name).

In embodiments, the display screen <NUM> may display graphical element(s) overlaying the image data corresponding to one or more target locations and/or trajectories that are set by the user. For example, defined target locations may be illustrated as identifiable dots or points in the image data, which may be color coded and/or labeled on the display screen <NUM> to enable easy visualization. Alternately or in addition, defined trajectories may be depicted as identifiable lines or line segments in the image data, which may be similarly color coded and/or labeled. As discussed above, the display screen <NUM> may also display graphical elements associated with particular tools or objects, including invasive surgical tools or instruments that are tracked by the motion tracking system <NUM>. In embodiments, the display screen <NUM> may depict at least a portion (e.g., a tip end) of a surgical instrument as it is inserted into the patient <NUM>, which may enable the surgeon to track the progress of the instrument as it progresses along a defined trajectory and/or towards a defined target location in the patient <NUM>. In some embodiments, the patient images on the display screen <NUM> may be augmented by graphical illustrations of pre-calibrated tools or implants (e.g., screws) that are located within the patient <NUM>.

The at least one robotic arm <NUM> may aid in the performance of a surgical procedure, such as a minimally-invasive spinal surgical procedure or various other types of orthopedic, neurological, cardiothoracic and general surgical procedures. In some embodiments, when the robotic arm <NUM> is pointed along a set trajectory to a target position, the robotic arm <NUM> may maintain a rigid or fixed pose to enable the surgeon to insert an instrument or tool through a cannula or similar guide arranged along a vector that coincides with the predefined trajectory into the body of the patient <NUM>. The cannula may be a portion of the end effector <NUM> of the robotic arm <NUM> or it may be separate component that is held by the end effector <NUM>. The cannula/guide may be positioned by the robotic arm <NUM> such that the central axis of the cannula is collinear with the pre-defined trajectory into the patient <NUM>. The surgeon may insert one or more invasive surgical instrument through the cannula/guide along the trajectory and into the body of the patient to perform a surgical intervention. Alternately, the end effector <NUM> itself may comprise a surgical instrument that may be moved into the body of the patient, such as, without limitation, a needle, a dilator, a tool for gripping, cutting or ablating tissue, an implant, a drill bit, a screw, a screw driver, a radiation source, a drug and/or an endoscope.

Various embodiments include an image guided surgery system that has an optical system that provides a visible indication of a range of a motion tracking system. In various embodiments, the optical system includes at least one light source that directs visible light to indicate a field-of-view of one or more optical sensing devices (e.g., cameras) of a motion tracking system. <FIG> and <FIG> illustrate an example of an optical sensing device <NUM> of a motion tracking system <NUM>, such as described above with reference to <FIG> and <FIG>. The optical sensing device <NUM> includes an array of multiple (e.g., four) cameras <NUM> mounted to a rigid support <NUM> (see <FIG>). The support <NUM> may be mounted above the surgical area, such as on an arm <NUM> as shown in <FIG>. The support <NUM> in this embodiment includes a handle <NUM> that extends from the bottom of the support <NUM> that may be used to adjust the position and/or orientation of the optical sensing device <NUM>.

As discussed above, an optically-based motion tracking system <NUM> may include a stereoscopic camera array that detects optical radiation (typically infrared (IR) radiation) from a plurality of marker devices. The markers may be active markers that include IR emitters or may be passive markers that reflect IR radiation from an external source, which may be co-located with the camera array. In either type of motion tracking system <NUM>, it may be difficult for the user to determine which objects are within the field-of-view of the camera array at a given time (and thus are being tracked) and whether the cameras' line of sight to the surgical area is blocked by an obstruction.

Embodiments include an optical system for providing a visible indication of the range of a motion tracking system <NUM>, including a field-of-view of an optically-based motion tracking system. As shown schematically in <FIG>, an optical sensing device <NUM> includes a visible light source <NUM>, which may be a laser source, mounted to the rigid support <NUM> that supports an array of cameras <NUM>. Although a single visible light source <NUM> is shown in <FIG>, it will be understood that multiple visible light sources may be mounted to the support <NUM>. In this embodiment, the visible light source <NUM> is mounted inside the handle <NUM> of the support <NUM>. The visible light source <NUM> projects a beam <NUM> of radiation in a first (e.g., vertical) direction onto the surface of a rotating mirror <NUM>, as shown in the partial crosssection view of <FIG>. The rotating mirror <NUM> redirects the beam <NUM> along a second (e.g., horizontal) direction that is substantially perpendicular to the first direction. A motor <NUM> mechanically coupled to the rotating mirror <NUM> rotates the mirror within the rigid support <NUM>. The rotation of the mirror <NUM> causes the beam <NUM> to revolve around the support. A second set of mirrors <NUM> spaced radially from the rotating mirror <NUM> redirects the beam <NUM> along a third direction towards the surgical site. As shown in <FIG>, the second set of mirrors <NUM> may be adjacent to and optionally coupled to the cameras <NUM> of the camera array. Each of the mirrors <NUM> may have an angled or contoured surface that redirects the beam in the direction of the camera's <NUM> field of view. As the rotating mirror <NUM> rotates around the support <NUM>, the beam <NUM> repeatedly traces a visible outline <NUM> on whatever surface(s) are located in front of the camera array. At a sufficiently high rate of revolution of the rotating mirror <NUM>, the outline <NUM> may be continuously perceptible to the user. The second set of mirrors <NUM> may be calibrated so that the visible outline <NUM> generally corresponds to a boundary of a field-of-view of the camera array. For example, the visible outline <NUM> may be a circle or arc that encompasses an outer boundary of the camera's <NUM> field of view, as shown in <FIG>. In one embodiment shown schematically in <FIG>, the visible outline <NUM> traced by the beam may encompass a region <NUM> that is within the field of view of at least two cameras <NUM> of the camera array.

The visible light source <NUM> may also be a source of incoherent light, such as a light emitting diode (LED), as an alternative to a laser as described above. An advantage of a laser source is that may be used to create a sharply-delineated boundary. However, a drawback to the use of a laser light beam is that it may reflect off of shiny surfaces, including instruments, and can create safety issues. The visible light source <NUM> may be a high-intensity non-laser light source, such as an LED, which may be configured to reflect off of a <NUM>-degree reflector to produce a disc of light. The <NUM>-degree reflector may be an alternative to the rotating mirror <NUM> as described above. The disc of light from the reflector may be directed to reflect off the aforementioned angled or contoured mirrors <NUM> to project a pattern of illumination which overlaps the field-of-view of the cameras <NUM>.

In embodiments, the optical sensing device <NUM> may be positioned to optimize the view of the cameras <NUM> into the surgical space. However, it may be desirable for the cameras <NUM> to also see marker devices that are located outside of the surgeon's work space, such one or more markers attached to the base end of the robotic arm <NUM> (e.g., to provide a "ground truth" measurement of robot position) and/or on the imaging device <NUM>, such as the markers <NUM> on the gantry <NUM> used for scan registration as shown in <FIG>. The cameras <NUM> may have a large enough field of view to see all of the markers if the cameras <NUM> are positioned far from the patient, but such a camera position may be sub-optimal for viewing and tracking instruments within the surgical area.

<FIG> illustrates an embodiment of an image guided surgery system that includes a motion tracking system <NUM> having an optical sensing device <NUM> as described above. The optical sensing device <NUM> includes an array of cameras <NUM> mounted to a rigid support <NUM>. The support <NUM> may be mounted above the surgical area, with the cameras <NUM> pointed in a first direction to detect optical signals from one or more first marker array(s) 703a, 703b located in the surgical area. In the embodiment of <FIG>, marker arrays 703a, 703b are shown attached to the patient <NUM> and the robotic arm <NUM>, respectively. The first marker array(s) may include active markers that emit their own optical signals (e.g., IR radiation), or passive markers that reflect optical signals from an external source. At least one beam splitter <NUM> is optically coupled to the optical sensing device <NUM>. In the embodiment shown in <FIG>, each camera <NUM> of the optical sensing device <NUM> includes an associated beam splitter <NUM> positioned in front of the camera <NUM>. The beam splitters <NUM> enable optical signals from the first marker array(s) 703a, 703b in the surgical area to pass through the beam splitters <NUM> along a first direction to be detected by the cameras <NUM>. The beam splitters <NUM> are configured redirect optical signals received along a second direction from one or more second marker array(s) <NUM> located outside of the surgical area so that the signals may be received by the cameras <NUM>. The one or more second marker array(s) <NUM> may be located on or proximate to the base end of the robotic arm <NUM> or on the imaging device <NUM>, for example. The one or more second marker array(s) <NUM> may be referred to as reference markers that are located outside of the active surgical site.

In some embodiments, both the first marker array(s) 703a, 703b in the surgical area and the second marker array(s) <NUM> located outside of the surgical area may be active marker arrays. The operation of the marker arrays <NUM>, <NUM> may be synchronized so that the cameras <NUM> receive signals from the first marker array(s) 703a, 703b and the second marker array(s) <NUM> at different times.

Alternately, the first marker arrays 703a, 703b and the second marker arrays <NUM> may be passive maker arrays that reflect IR radiation. The motion tracking system <NUM> may be configured to capture tracking data from the first direction and from the second direction at different times by, for example, projecting IR radiation along the first direction and along the second direction at different times.

In some embodiments, the motion tracking system <NUM> may be a hybrid system that utilizes both active and passive markers. In one example, the first marker array(s) 703a, 703b in the surgical area may be passive makers and the second marker array(s) <NUM> outside of the surgical area (e.g., at the base of the robotic arm <NUM> and/or the imaging system <NUM>) may be active makers. The operation of the active second marker array(s) <NUM> may be synchronized with an IR source that projects IR radiation into the surgical area so that when the cameras <NUM> are receiving reflected radiation from the first marker array(s) 703a, 703b the second marker array(s) <NUM> are not emitting, and when the cameras <NUM> are receiving radiation emitted by the second marker array(s) <NUM> the IR source is not projecting into the surgical area.

As noted above, the optical sensing device <NUM> of the motion tracking system <NUM> may include a plurality of cameras <NUM> mounted to rigid support <NUM>. The rigid support <NUM> may maintain the cameras <NUM> in a fixed relationship relative to one another. However, depending on how the rigid support <NUM> is mounted within the operating room, there can occur small movements (e.g., vibration, shaking, etc.) of the rigid support <NUM> and cameras <NUM> relative to the patient <NUM> and/or robotic arm <NUM>. The optical sensing device <NUM> may also be repositioned by the user. The software of the motion tracking system <NUM> may not be able to distinguish between movements that are actual movements of the objects being tracked (such as marker arrays 703a, 703b within the surgical area) and an apparent movement of the tracked object(s) due to motion of the cameras <NUM> themselves. This may result in decreased accuracy of the surgical navigation and unnecessary movements of the robotic arm <NUM> to compensate for apparent motions of objects (such as the patient <NUM> and/or robotic arm <NUM>) within the surgical field.

<FIG> illustrates an embodiment of an image guided surgery system that includes a motion tracking system <NUM> having an inertial measurement unit <NUM> attached to an optical sensing device <NUM>. The inertial measurement unit <NUM> may be mounted to the rigid support <NUM> on which an array of cameras <NUM> are mounted. The inertial measurement unit <NUM> may detect movements of the optical sensing device <NUM>, such as a shaking or vibration of the rigid support <NUM> holding the cameras <NUM>, or an intentional or accidental repositioning of the rigid support by a user. The measurements detected by the inertial measurement unit <NUM> may be sent to a processing device (e.g., a computer <NUM>, such as shown in <FIG>) along with tracking data from the optical sensing device <NUM>. The measurements from the inertial measurement unit <NUM> may be utilized by motion tracking software implemented on the computer <NUM> to correct for detected movements of the optical sensing device <NUM> when tracking objects within the surgical area, including the patient <NUM>, the robotic arm <NUM> and surgical tools. The inertial measurement unit <NUM> may provide an measurement of movement of the optical sensing device <NUM> that is independent of the tracking data obtained by the optical sensing device <NUM>, which can aid the motion tracking system <NUM> in accurately differentiating between actual movements of the objects being tracked from apparent movements of the objects due to motion of the optical sensing device <NUM> itself.

The inertial measurement unit <NUM> may include a three-axis accelerometer and a three-axis gyroscope. The accelerometer and gyroscope may be fabricated utilizing MEMS technology. The accelerometer and gyroscope may be separate components (e.g., chips) located in the rigid support <NUM> or may be integrated on a single device (e.g., integrated circuit). The inertial measurement unit <NUM> may also include circuitry coupled to the accelerometer and gyroscope that may be configured to read output signals from these components. The accelerometer may output signals measuring the linear acceleration of the rigid support <NUM>, preferably in three-dimensional space. The gyroscope may output signals measuring the angular velocity of the rigid support, preferably also in three-dimensional space. The signals from the accelerometer and gyroscope may be processed using a suitable processor, such as a computer <NUM> shown in <FIG>, to determine the position and orientation of the rigid support <NUM> with respect to an initial inertial reference frame via a dead reckoning technique. In particular, integrating the angular velocity measurements from the gyroscope may enable the current orientation of the rigid support <NUM> to be determined with respect to a known starting orientation. Integrating the linear acceleration measurements from the accelerometer may enable the current velocity of the rigid support <NUM> to be determined with respect to a known starting velocity. A further integration may enable the current position of the rigid support <NUM> to be determined with respect to a known starting position.

In embodiments, measurement data from the inertial measurement unit <NUM> may transmitted from the optical sensing device <NUM> to a separate computing device (e.g., computer <NUM>) via a wired or wireless link. The measurement data from the inertial measurement unit <NUM> may be sent via the same communication link as the tracking data from the cameras <NUM>, or by a different communication link.

Although the embodiment of <FIG> illustrates an inertial measurement unit <NUM> located on the rigid support <NUM> holding the cameras <NUM>, it will be understood that the inertial measurement unit <NUM> may be located on a camera <NUM>. Each camera <NUM> may include an inertial measurement unit <NUM> that measures the motion of the camera <NUM> to correct for camera movement in the motion tracking system <NUM>.

<FIG> illustrates an embodiment of a robotic arm <NUM> for use in roboticassisted surgery. The robotic arm <NUM> includes a base end <NUM> and a distal end <NUM>, with an end effector <NUM> located at the distal end <NUM>. As discussed above, the end effector <NUM> may comprise a cannula or guide that may be used to insert an invasive surgical tool or instrument along a trajectory into the patient, or alternately, the end effector <NUM> itself may comprise an invasive surgical instrument. During a surgical procedure, the robotic arm <NUM> may be covered in a surgical drape to maintain a sterile surgical field. The end effector <NUM> may be a sterile component that may be attached (e.g., snapped into) the distal end <NUM> of the robotic arm <NUM>, optionally over the surgical drape.

The robotic arm <NUM> in this embodiment includes a first portion <NUM> that includes at least one <NUM>-DOF joint <NUM>. As used herein a "<NUM>-DOF joint" is a joint that enables robot articulation in two mutually orthogonal directions (i.e., pitch and yaw rotation). A <NUM>-DOF joint is in contrast to a conventional (i.e., <NUM>-DOF) robotic joint that rotates within a single plane. In the embodiment of <FIG>, the first portion <NUM> of the robotic arm <NUM> includes a chain of five <NUM>-DOF joints <NUM> extending from the base end <NUM> of the robotic arm <NUM>, although it will be understood that the first portion <NUM> may have more or less <NUM>-DOF joints <NUM>. The <NUM>-DOF joints <NUM> may be modular in design, with each joint <NUM> including a central section <NUM> having a generally spherical outer surface between a pair of end sections 810a, 810b. The end sections 810a, 810b of adjacent joints <NUM> may be connected. The central section <NUM> may include a pair of wedge segments 812a, 812b having angled interfacing surfaces so that the rotation of the wedge segments 812a, 812b relative to one another produces pitch and yaw rotation of end section 810b relative to end section 810a over a particular angular range (e.g., ±<NUM> degrees, such as ±<NUM> degrees, ±<NUM> degrees, ±<NUM> degrees and ±<NUM> degrees or more). Motors (not illustrated) mounted to the end sections 810a, 810b may drive the rotation of the wedge segments 812a, 812b. A universal joint (not visible) located inside the central section <NUM> and coupled to the end sections 810a, 810b may inhibit twisting motion of the end sections 810a, 810b.

The robotic arm <NUM> in <FIG> may also include a second portion <NUM> that includes at least two rotary joints 814a, 814b (i.e., <NUM>-DOF joints) that rotate about mutually perpendicular axes. The second portion <NUM> may include a housing <NUM> that includes motors (not visible) for driving the rotation of joints 814a and 814b. The second portion <NUM> may be located between the first portion <NUM> and the end effector <NUM> of the robotic arm <NUM>. Joint 814a may be a theta wrist joint that may rotate the entire housing <NUM> with respect to the end section 810b of the adjacent <NUM>-DOF joint <NUM>. In embodiments, joint 814a may rotate continuously (e.g., > <NUM>°) in either direction. Joint 814b may be a wrist joint located on the side of the housing <NUM>. Joint 814b may rotate the end effector <NUM> at least about ±<NUM> degrees, such as ±<NUM> degrees, ±<NUM> degrees, ±<NUM> degrees or more with respect to the housing <NUM>. In one embodiment illustrated most clearly in <FIG>, a connector <NUM> between the joint 814b and the end effector <NUM> may align the end effector <NUM> with the midline of the housing <NUM> so that the rotation of the end effector <NUM> via joint 814b is in the same plane as the axis of rotation of theta wrist joint 814a. Alternately, the end effector <NUM> may rotate in a plane that is off-set from the axis of joint 814a, as shown in the embodiment of <FIG>.

Further embodiments include marker arrays for tracking the position and/or orientation of an end effector <NUM> of a robotic arm <NUM> using a motion tracking system <NUM>. It may be desirable to provide a marker array that is proximate to the end effector <NUM> of the robotic arm <NUM> to maximize the accuracy of the positioning of the end effector <NUM>. Conventional marker arrays include rigid frames having marker elements affixed thereon. Such arrays may project into the surgeon's workspace and may interfere with a surgical procedure.

In the embodiment shown in <FIG>, a marker array <NUM> includes a plurality of marker elements attached to the robotic arm <NUM>, and in particular, a plurality of marker elements attached to the second portion <NUM> of the robotic arm <NUM> proximate to the end effector <NUM>. The marker array <NUM> includes a first set of one or more markers <NUM> located distal to the most distal joint 814b of the robotic arm <NUM> and a second set of one or more markers <NUM> located proximal to the most distal joint 814b of the robotic arm <NUM>. Together, the first and second sets of markers <NUM>, <NUM> form a marker array <NUM> that may be tracked by a motion tracking system <NUM> as described above.

In the embodiment of <FIG>, the second set of markers includes a plurality of markers <NUM> that may be secured to the housing <NUM> of the second portion <NUM> of the robotic arm <NUM>. The second set of markers <NUM> may be spaced circumferentially around the housing <NUM>. The first set of markers includes a single marker <NUM> that is located distal to the most distal joint of the robotic arm (i.e., joint 814b). Marker <NUM> may be located on the connector <NUM> that connects joint 814b to the end effector <NUM>. Alternately, marker <NUM> may be located on the end effector <NUM> itself. Although the embodiment of <FIG> illustrates the first set of markers as consisting of a single marker <NUM>, it will be understood that a plurality of markers <NUM> may be located distal to joint 814b.

The second set of markers <NUM> may be disposed in a geometric pattern that may be detected by the motion tracking system <NUM> and used to determine both the position of the robotic arm <NUM> in three-dimensional space as well as the rotational position of theta wrist joint 814a. As the end effector <NUM> is rotated on joint 814b, the change in relative position of the first set of marker(s) (i.e., marker <NUM> in <FIG>) to the second set of markers <NUM> detected by the motion tracking system <NUM> may be used to accurately determine the position and orientation of the end effector <NUM>. The markers <NUM>, <NUM> may be relatively unobtrusive so as not to interfere with the patient or the surgeon's work space.

In some embodiments, the markers <NUM>, <NUM> attached to the robotic arm <NUM> may be active (i.e., light emitting) markers. The electrical power for light-emitting elements of the markers <NUM>, <NUM> may be provided through the robotic arm <NUM>. Alternately, the markers <NUM>, <NUM> may be passive markers (e.g., spherical elements having a retroflective coating) that reflects light from an external source.

<FIG> and <FIG> illustrate embodiments of an active marker device <NUM> that is located on a robotic arm <NUM>. Power for the marker device <NUM> may be provided via a conductor <NUM> through the robotic arm <NUM> to a light source <NUM> (e.g., an LED source that emits light in an infrared (IR) wavelength range). In the embodiment of <FIG>, a projection <NUM> comprising an optical guide <NUM> (e.g., a light pipe) projects from the surface <NUM> of the robotic arm <NUM>. Light from the light source <NUM> is coupled into the optical guide <NUM> and is transmitted through the projection <NUM>. An optical diffuser <NUM> is attached to the projection <NUM> over the optical guide <NUM> and scatters the light as it emerges from the guide <NUM>. In the embodiment shown in <FIG>, the robotic arm <NUM> is covered by a surgical drape <NUM>. The surgical drape <NUM> is at least partially light-transmissive. The diffuser <NUM> may be attached to the projection <NUM> over the surgical drape <NUM>. Mating features <NUM> on the projection <NUM> and the diffuser <NUM> may enable the diffuser <NUM> to snap on to the projection <NUM> over the surgical drape <NUM>, which may be held tight against the projection <NUM>. The diffuser <NUM> may be a sterile component, and may be a single-use disposable component.

<FIG> illustrates an alternative embodiment in which the robotic arm <NUM> includes a recessed portion <NUM>, and the optical diffuser <NUM> includes a projection <NUM> that may be inserted into the recessed portion <NUM> to secure the diffuser <NUM> to the robotic arm <NUM>. The diffuser <NUM> may be inserted over a surgical drape <NUM>. Light from the light source <NUM> may be directed through the bottom of the recessed portion <NUM> and coupled into an optical guide <NUM> in the projection <NUM> of the optical diffuser <NUM>. The projection <NUM> and the recessed portion <NUM> may optionally have mating features such as shown in <FIG> that secure the optical diffuser <NUM> to the robotic arm <NUM>. As in the embodiment of <FIG>, the diffuser <NUM> may be a sterile component, and may be a single-use disposable component.

10A-10E illustrate various embodiments of an active marker array <NUM> that may be used to track various objects and tools during image guided surgery. For example, the marker device <NUM> may be a reference marker device that is attached to the patient (e.g., reference marker device <NUM> as shown in <FIG>), it may be a marker device that is attached to a surgical tool or instrument (e.g., marker device <NUM> as shown in <FIG>), and/or it may be a marker device that is attached to a robotic arm <NUM> for robot-assisted surgery (e.g., marker device <NUM> as shown in <FIG>).

In general, an active marker device <NUM> according to various embodiments includes a rigid frame <NUM>, an electronics module <NUM> that includes at least one light source <NUM> (e.g., an LED that emits light in an infrared range), and an optical guide apparatus <NUM> that couples light from the at least one light source <NUM> to an array of emitter locations <NUM> on the rigid frame <NUM>. In embodiments, the rigid frame <NUM> may be made (e.g., machined) to precise dimensions and tolerances out of metal or another suitable structural material. The rigid frame <NUM> may include a network of channels <NUM> extending within the rigid frame <NUM>. The optical guide apparatus <NUM> may be located within the channels <NUM>. The channels <NUM> may terminate in openings <NUM> in the frame <NUM> which may define the emitter locations <NUM> of the marker device <NUM>.

<FIG> is a side cross-sectional view of an active marker device <NUM> according to an embodiment. In this embodiment, the frame <NUM> may include a recess <NUM> or housing within which the electronics module <NUM> may be located. In addition to the at least one light source <NUM>, the electronics module <NUM> may also include a power source (e.g., a battery) and circuitry for controlling the operation of the at least one light source <NUM>. For example, the circuitry may control the at least one light source <NUM> to emit light having a particular pulse pattern or frequency. In embodiments, the circuitry may comprise or include a programmable microprocessor. The electronics module <NUM> may also include communications circuitry, such as an RF receiver and/or transmitter, to enable communication with an external device, such as computer <NUM> in <FIG>. In embodiments, the electronics module may communicate with an external device for synchronization of the pulsing of the light source and/or for setting or programming a particular pulse pattern. In embodiments, the electronics module <NUM> may be located within a sealed housing or package. The power source may be a rechargeable battery, and in embodiments may be recharged wirelessly (e.g., via inductive charging).

The optical guide apparatus <NUM> in this embodiment comprises a plurality of light pipes <NUM>. The light pipes may be made of a thermoplastic material (e.g., polycarbonate) and may be at least partially flexible or deformable. Alternately, the optical guide apparatus <NUM> may comprise a plurality of optical fibers. The optical guide apparatus <NUM> may comprise a unitary component. The optical guide apparatus <NUM> may be separate from the electronics module <NUM> or may be integral with the electronics module <NUM>. For example, the electronics module <NUM> may be formed as a flex circuit such as shown in <FIG>, where a plurality of light pipes <NUM> or similar optical guides may extend from the flex circuit.

<FIG> is an exploded view of an active marker device <NUM>, including a rigid frame <NUM>, an electronics module <NUM>, and an optical guide apparatus <NUM>. In this example, the optical guide apparatus <NUM> includes a plurality of light pipes <NUM> extending from a protective cover <NUM>. The cover <NUM> can attach to (e.g., snap into) the rigid frame <NUM> to enclose the electronics module <NUM> within a recess <NUM> in the rigid frame <NUM>. In some embodiments, the electronics module <NUM> may be attached to the protective cover <NUM> so that the electronics module <NUM> and optical guide apparatus <NUM> form an integral component. Alternatively, the electronics module <NUM> may be a separate component that may be inserted within the recess <NUM> of the rigid frame <NUM>, and the optical guide apparatus <NUM> may be attached over the electronics module <NUM>. <FIG> also illustrates a separate power source <NUM> (e.g., battery) for the electronics module <NUM> that may be housed within the rigid frame <NUM>. Alternately, the power source <NUM> may be integrated with the electronics module <NUM>.

The optical guide apparatus <NUM> may be positioned within the frame <NUM> such that light from the at least one light source <NUM> of the electronics module <NUM> is coupled into the light pipes <NUM> of the optical guide apparatus <NUM>. Each of the light pipes <NUM> may be inserted within a respective channel <NUM> of the frame <NUM>. The light pipes <NUM> may terminate proximate to the respective openings <NUM> of the frame <NUM> corresponding to the emitter locations <NUM>. In embodiments, optical diffusers <NUM>, which may be similar to the diffusers <NUM> described above in connection with <FIG>, may be located at the ends of the light pipes <NUM> and over the openings <NUM> in the rigid frame <NUM>. The diffusers <NUM> may be integrated with the optical guide apparatus <NUM> or may be snap-on components that attach over the ends of the light pipes <NUM> and/or the openings <NUM> in the rigid frame <NUM>.

<FIG> illustrates an alternative embodiment of an electronics module <NUM> for an active marker device <NUM> that is formed as a flex circuit. The electronics module <NUM> includes a flexible substrate <NUM> on which is located circuitry <NUM> for controlling the operation of at least one light source <NUM>. The circuitry <NUM> may contain, for example, a programmable microprocessor, and may also include communications circuitry, such as an RF receiver and/or transmitter, and an integrated power source as described above. The flexible substrate <NUM> includes a plurality of peripheral arm regions <NUM>, each having a light source <NUM> (e.g., LED) attached thereto. Electrical conductors <NUM> may extend along the arm regions <NUM> to connect each of the light sources <NUM> to the rest of the circuitry <NUM>. The light sources <NUM> may be located on or proximate to the distal ends of the arm regions <NUM>. The electronics module <NUM> as shown in <FIG> may be inserted into a rigid frame <NUM>, such as shown in <FIG>. The arm regions <NUM> may extend along channels <NUM> in the rigid frame <NUM> so that each light source <NUM> is aligned with a respective opening <NUM> of the frame <NUM> corresponding to emitter locations <NUM> of the active marker device <NUM>.

<FIG> illustrate an additional embodiment of an active marker device <NUM> in which the electronics module <NUM> is housed within the handle <NUM> of a surgical tool <NUM>. A rigid frame <NUM> as described above may comprise a part or all of the handle <NUM>. The rigid frame <NUM> includes a plurality of arms <NUM> that extend from the handle <NUM> of the tool <NUM>. Each arm <NUM> includes a channel <NUM> that extends along the length of the arm <NUM> to an opening <NUM> at the end of the arm <NUM>. The optical guide apparatus <NUM> includes a plurality of optical fibers <NUM> that are located within respective channels <NUM> of the frame <NUM>. The ends of the optical fibers <NUM> may be positioned so that light is directed out of the respective openings <NUM>. The optical fibers <NUM> direct light from at least one light source <NUM> in the handle <NUM> along the arms <NUM> and out of the respective openings <NUM>. A plurality of diffusers <NUM> may be located over the openings <NUM>.

The electronics module <NUM> may be integrated with the rigid frame <NUM> or may be removable from the frame <NUM>. <FIG> also illustrates a power source <NUM> (e.g., battery) within the frame <NUM> for providing power to the electronics module <NUM>. Alternatively, the electronics module <NUM> may have an integrated power supply <NUM>. The rigid frame <NUM> may be detachable from the rest of the surgical tool <NUM>, or may be integral with the surgical tool <NUM>.

In embodiments of an active marker device <NUM> as described above, the optical guide apparatus <NUM> may optionally be removable from the rigid frame <NUM> and the components may be separately sterilized for reuse or disposed. In embodiments in which the rigid frame <NUM>, optical guide apparatus <NUM> and electronics module <NUM> are comprised of separate components, each of these components may be individually removed and separately sterilized for reuse or disposed. In some embodiments, one or more of the rigid frame <NUM>, optical guide apparatus <NUM> and electronics module <NUM> may be a single-use disposable component.

A plurality of active marker devices <NUM> may utilize an identical design for the rigid frame <NUM>, with the differentiation between markers provided by differences in the pulse patterns produced by the electronics module <NUM>. This may provide an economical marker device <NUM> that may be optimized for ergonomics or other factors.

In some embodiments, the electronics component may include at least one first light source that emits light (e.g., IR light) that is detectable by the motion tracking system <NUM> for tracking purposes as described above, and at least one second light source that emits visible light. The visible light from the at least one second light source may be coupled into the optical guide apparatus <NUM> to provide the user with visual feedback from the marker device <NUM>. The visual feedback may provide feedback on the operation of the marker device <NUM> itself (e.g., an indication of the charge state of the battery, an indication of whether the marker device is on, programmed and actively emitting IR light, etc.). In some embodiments, the electronics module <NUM> may receive feedback data from an external device (such as computer <NUM>) and may control the at least one second light source to provide visual feedback to the user based on the received feedback data. The visual feedback may provide feedback regarding a surgical procedure. For example, the at least one second light source may flash light of a certain color (e.g., green) when the tool to which the marker device <NUM> is attached is determined to be in the correct position (e.g., at a target location or along a pre-set trajectory within the patient) and may flash a different color (e.g., yellow) when the tool is in an incorrect position. In addition, the visual feedback may indicate whether or not a tool is currently being tracked by the motion tracking system <NUM>. The at least one first (IR) light source and the at least one second (visible) light source may be multiplexed so that only one source is emitting at a time.

Alternatively or in addition, a cover <NUM> of the active marker device <NUM> (see <FIG>) may comprise a transparent material over at least a portion of the cover <NUM> to enable the user to view visual feedback from the at least one second light source through the cover <NUM>.

Further embodiments include a motion tracking system <NUM> for roboticassisted image guided surgery that utilize an "inside out" architecture in which the sensors for tracking the marker devices are located on a robotic arm <NUM>. An exemplary embodiment is illustrated in <FIG>. In this embodiment, the robotic arm <NUM> includes a sensor array <NUM> extending around the circumference of the arm <NUM> proximate to the end effector <NUM>. The sensor array <NUM> may include a plurality of outward-facing cameras, which may be similar to the cameras <NUM> described above with reference to <FIG> and <FIG>. The sensor array <NUM> may also include one or more IR light sources for illuminating reflective marker. The sensor array <NUM> may be configured to detect radiation from passive or active marker devices <NUM> within the field of view of the sensor array <NUM>, which may be used to track the positions of the marker devices <NUM> relative to the sensor array <NUM> (e.g., using triangulation techniques) as described above. The position of the sensor array <NUM> and of the end effector <NUM> within a common reference frame may be determined based on the robot joint coordinates and the known geometry of the robotic arm <NUM>.

As shown in <FIG>, the sensor array <NUM> may simultaneously track multiple marker devices <NUM>, including a first marker device 1103a attached to the patient and a second marker device 1103b attached to a tool <NUM> that is inserted in the end effector <NUM> of the robotic arm <NUM>. The field of view of the sensor array <NUM> may extend around the entire circumference of the robotic arm <NUM>, and may encompass the entire surgical area. In embodiments, a separate marker device on the robotic arm <NUM> may not be necessary. Further, the need for an external optical sensing device, such as sensing device <NUM> shown in <FIG> and <FIG>, may be avoided.

In some embodiments, the sensor array <NUM> may also be used for collision avoidance for the robotic arm <NUM>. Optionally, the sensor array <NUM> may include a user interface component, such as a touchscreen display interface, for controlling various operations of the robotic arm <NUM> and/or the image guided surgery system.

<FIG> illustrates the robotic arm <NUM> moved to a second position during an imaging scan using the imaging device <NUM>. In this position, the sensor array <NUM> has a clear view of both the patient marker device 1103a and additional markers <NUM> on the imaging gantry <NUM>, which may facilitate registration of image scan data, as discussed above in <FIG>. The control system for the robotic arm <NUM> may be configured to move the robotic arm <NUM> to optimally position the sensor array <NUM> for scan data registration during an imaging scan.

<FIG> is a system block diagram of a computing device <NUM> useful for performing and implementing the various embodiments described above. The computing device <NUM> may perform the functions of an image guided surgery system <NUM> and/or a robotic control system <NUM>, for example. While the computing device <NUM> is illustrated as a laptop computer, a computing device providing the functional capabilities of the computer device <NUM> may be implemented as a workstation computer, an embedded computer, a desktop computer, a server computer or a handheld computer (e.g., tablet, a smartphone, etc.). A typical computing device <NUM> may include a processor <NUM> coupled to an electronic display <NUM>, a speaker <NUM> and a memory <NUM>, which may be a volatile memory as well as a nonvolatile memory (e.g., a disk drive). When implemented as a laptop computer or desktop computer, the computing device <NUM> may also include a floppy disc drive, compact disc (CD) or DVD disc drive coupled to the processor <NUM>. The computing device <NUM> may include an antenna <NUM>, a multimedia receiver <NUM>, a transceiver <NUM> and/or communications circuitry coupled to the processor <NUM> for sending and receiving electromagnetic radiation, connecting to a wireless data link, and receiving data. Additionally, the computing device <NUM> may include network access ports <NUM> coupled to the processor <NUM> for establishing data connections with a network (e.g., LAN coupled to a service provider network, etc.). A laptop computer or desktop computer <NUM> typically also includes a keyboard <NUM> and a mouse pad <NUM> for receiving user inputs.

The foregoing method descriptions are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. Words such as "thereafter," "then," "next," etc. are not necessarily intended to limit the order of the steps; these words may be used to guide the reader through the description of the methods.

In one or more exemplary aspects, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on as one or more instructions or code on a non-transitory computer-readable medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module executed which may reside on a non-transitory computer-readable medium. Non-transitory computer-readable media includes computer storage media that facilitates transfer of a computer program from one place to another. By way of example, and not limitation, such non-transitory computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to carry or store desired program code in the form of instructions or data structures and that may be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of non-transitory computer-readable storage media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a machine readable medium and/or computer-readable medium, which may be incorporated into a computer program product.

Claim 1:
A marker device (<NUM>, <NUM>, <NUM>, <NUM>) for an image guided surgery system (<NUM>), comprising:
an electronics unit (<NUM>) comprising at least one light source (<NUM>);
a rigid frame (<NUM>) attached to the electronics unit (<NUM>), the rigid frame (<NUM>) having at least one channel (<NUM>) extending from the electronics unit (<NUM>) to at least one opening (<NUM>) in the rigid frame (<NUM>); and
an optical guide apparatus (<NUM>) located within the at least one channel (<NUM>) to couple light from the at least one light source (<NUM>) of the electronics unit (<NUM>) to the at least one opening (<NUM>) in the rigid frame (<NUM>).