Patent Description:
Navigation systems assist users in locating objects. For instance, navigation systems are used in industrial, aerospace, and medical applications. In the medical field, navigation systems assist surgeons in placing surgical tools relative to a patient's anatomy. Surgeries in which navigation systems are used include neurosurgery and orthopedic surgery. Typically, the tool and the anatomy are tracked together with their relative movement shown on a display.

Navigation systems may employ light signals, sound waves, magnetic fields, radio frequency signals, etc. in order to track the position and/or orientation of objects. Often the navigation system includes tracking devices attached to the object being tracked. A localizer cooperates with tracking elements on the tracking devices to determine a position of the tracking devices, and ultimately to determine a position and/or orientation of the object. The navigation system monitors movement of the objects via the tracking devices.

Many navigation systems rely on an unobstructed line-of-sight between the tracking elements and sensors of the localizer that receive tracking signals from the tracking elements. These navigation systems also rely on the tracking elements being positioned within a field-of-view of the localizer. As a result, efforts have been undertaken to reduce the likelihood of obstructing the line-of-sight between the tracking elements and the sensors and to maintain the tracking elements within the field-of-view of the localizer. For example, in some navigation systems, during initial setup of the navigation system, a display graphically represents a field-of-view of the localizer to guide initial placement of the tracking devices so that the tracking elements are located in the field-of-view free from obstructions to the line-of-sight. However, such navigation systems are unable to prevent obstructions to the line-of-sight that may arise during the surgical procedure as a result of the movement of objects into the line-of-sight, e.g., after the initial setup and during treatment of a patient, or to prevent the tracking elements from moving outside of the field-of-view.

When the line-of-sight is obstructed, or when the tracking elements are outside the field-of-view, tracking signals being transmitted from the tracking elements are not received by the localizer. As a result, errors can occur. Typically, in this situation, navigation is discontinued and error messages are conveyed to the user until the tracking signals are again received or the navigation system is reset. This can cause delays in surgical procedures. For instance, manipulators that rely on navigation data to autonomously position a cutting tool relative to the patient's tissue must cease operation should these errors occur. This could significantly increase the surgical procedure time, particularly if difficulty arises in restoring the line-of-sight. This is contrary to the demands of modern medical practice that require reduced surgery times in order to reduce risks of infection and risks associated with prolonged use of anesthesia.

Thus, there is a need in the art for navigation systems and methods that reduce tracking interruptions between tracking devices and a localizer receiving signals from the tracking devices so that surgical procedures are uninterrupted.

<CIT> discloses systems and methods for establishing and tracking virtual boundaries. The virtual boundaries can delineate zones in which an instrument is not permitted during a surgical procedure. The virtual boundaries can also delineate zones in which the surgical instrument is permitted during the surgical procedure. The virtual boundaries can also identify objects or structures to be treated by the instrument or to be avoided by the instrument during the surgical procedure.

Advantageous embodiments are set forth in the dependent claims and hereinbelow.

In one example a navigation system is provided for reducing tracking interruptions caused by an object. The navigation system includes a localizer having a field-of-view. A tracking device is placed within the field-of-view to establish a line-of-sight relationship with the localizer. A virtual boundary generator generates a virtual line-of-sight boundary based on the line-of-sight relationship between the tracking device and the localizer. The virtual boundary generator also updates the virtual line-of-sight boundary to account for relative movement between the tracking device and the localizer during the surgical procedure. The object is defined in virtual space as a virtual object. A collision detector evaluates movement of the virtual object relative to the virtual line-of-sight boundary to detect a collision between the virtual object and the virtual line-of-sight boundary to enable a response to the detection that prevents the object from obstructing the line-of-sight between the tracking device and the localizer.

An exemplary method is also provided for reducing tracking interruptions between a tracking device and a localizer of a navigation system. The method includes detecting the tracking device within a field-of-view of the localizer. A virtual line-of-sight boundary is generated based on a line-of-sight relationship between the tracking device and the localizer. The virtual line-of-sight boundary is updated to account for relative movement between the tracking device and the localizer. A virtual object is associated with an object in the field-of-view of the localizer. A collision is detected between the virtual object and the virtual line-of-sight boundary based on an evaluation of relative movement between the virtual object and the virtual line-of-sight boundary to enable a response to the detection that prevents the object from obstructing the line-of-sight between the tracking device and the localizer.

Another exemplary navigation system is provided for reducing tracking interruptions. The system includes a localizer having a field-of-view. A tracking device is placed within the field-of-view so that the localizer is capable of receiving signals from the tracking device. A virtual object is associated with the tracking device. A virtual boundary generator generates a virtual field-of-view boundary based on the field-of-view of the localizer. A collision detector evaluates movement of the virtual object relative to the virtual field-of-view boundary to detect a collision between the virtual object and the virtual field-of-view boundary and enable a response to the collision that prevents the tracking device from moving outside of the field-of-view of the localizer.

Another exemplary method is also provided for reducing tracking interruptions between a tracking device and a localizer of a navigation system. The method includes detecting the tracking device within a field-of-view of the localizer. A virtual field-of-view boundary is generated based on the field-of-view of the localizer. A virtual object is associated with the tracking device. Movement of the virtual object relative to the virtual field-of-view boundary is tracked in order to detect a collision between the virtual object and the virtual field-of-view boundary and enable a response to the collision that prevents the tracking device from moving outside of the field-of-view of the localizer.

One advantage of these navigation systems and methods is to reduce tracking interruptions between a tracking device and a localizer receiving signals from the tracking device so that interruptions to a surgical procedure can be avoided. Such interruptions can be caused by an object that interferes with the line-of-sight between the tracking device and the localizer or by virtue of the tracking device moving outside the field-of-view of the localizer.

Referring to <FIG> a material removal system <NUM> for removing material from a workpiece is illustrated. The material removal system <NUM> is shown in a surgical setting such as an operating room of a medical facility. In the embodiment shown, the material removal system <NUM> includes a machining station <NUM> and a navigation system <NUM>. The navigation system <NUM> is set up to track movement of various objects in the operating room. Such objects include, for example, a surgical tool <NUM>, a femur F of a patient, and a tibia T of the patient. The navigation system <NUM> tracks these objects for purposes of displaying their relative positions and orientations to the surgeon and, in some cases, for purposes of controlling or constraining movement of the surgical tool <NUM> relative to virtual cutting boundaries (not shown) associated with the femur F and tibia T.

The navigation system <NUM> includes a computer cart assembly <NUM> that houses a navigation computer <NUM>. A navigation interface is in operative communication with the navigation computer <NUM>. The navigation interface includes a first display <NUM> adapted to be situated outside of a sterile field and a second display <NUM> adapted to be situated inside the sterile field. The displays <NUM>, <NUM> are adjustably mounted to the computer cart assembly <NUM>. First and second input devices <NUM>, <NUM> such as a keyboard and mouse can be used to input information into the navigation computer <NUM> or otherwise select/control certain aspects of the navigation computer <NUM>. Other input devices are contemplated including a touch screen (not shown) or voice-activation.

A localizer <NUM> communicates with the navigation computer <NUM>. In the embodiment shown, the localizer <NUM> is an optical localizer and includes a camera unit <NUM>. The camera unit <NUM> has an outer casing <NUM> that houses one or more optical position sensors <NUM>. In some embodiments at least two optical sensors <NUM> are employed, preferably three or four (three shown). The optical sensors <NUM> may be separate charge-coupled devices (CCD). In one embodiment three, one-dimensional CCDs are employed. It should be appreciated that in other embodiments, separate camera units, each with a separate CCD, or two or more CCDs, could also be arranged around the operating room. The CCDs detect infrared (IR) signals.

Camera unit <NUM> is mounted on an adjustable arm to position the optical sensors <NUM> with a field-of-view of the below discussed trackers that, ideally, is free from obstructions. In some embodiments the camera unit <NUM> is adjustable in at least one degree of freedom by rotating about a rotational joint. In other embodiments, the camera unit <NUM> is adjustable about two or more degrees of freedom.

The camera unit <NUM> includes a camera controller <NUM> in communication with the optical sensors <NUM> to receive signals from the optical sensors <NUM>. The camera controller <NUM> communicates with the navigation computer <NUM> through either a wired or wireless connection (not shown). One such connection may be an IEEE <NUM> interface, which is a serial bus interface standard for high-speed communications and isochronous real-time data transfer. The connection could also use a company specific protocol. In other embodiments, the optical sensors <NUM> communicate directly with the navigation computer <NUM>.

Position and orientation signals and/or data are transmitted to the navigation computer <NUM> for purposes of tracking objects. The computer cart assembly <NUM>, display <NUM>, and camera unit <NUM> may be like those described in <CIT>, entitled "Surgery System".

The navigation computer <NUM> can be a personal computer or laptop computer. Navigation computer <NUM> has the display <NUM>, central processing unit (CPU) and/or other processors, memory (not shown), and storage (not shown). The navigation computer <NUM> is loaded with software as described below. The software converts the signals received from the camera unit <NUM> into data representative of the position and orientation of the objects being tracked.

Navigation system <NUM> is operable with a plurality of tracking devices <NUM>, <NUM>, <NUM>, also referred to herein as trackers. In the illustrated embodiment, one tracker <NUM> is firmly affixed to the femur F of the patient and another tracker <NUM> is firmly affixed to the tibia T of the patient. Trackers <NUM>, <NUM> are firmly affixed to sections of bone. Trackers <NUM>, <NUM> may be attached to the femur F and tibia T in the manner shown in <CIT>. Trackers <NUM>, <NUM> could also be mounted like those shown in <CIT>, entitled, "Navigation Systems and Methods for Indicating and Reducing Line-of-Sight Errors,". In additional embodiments, a tracker (not shown) is attached to the patella to track a position and orientation of the patella. In yet further embodiments, the trackers <NUM>, <NUM> could be mounted to other tissue types or parts of the anatomy.

A tool tracker <NUM> is firmly attached to the surgical tool <NUM>. The tool tracker <NUM> may be integrated into the surgical tool <NUM> during manufacture or may be separately mounted to the surgical tool <NUM> in preparation for surgical procedures. The working end of the surgical tool <NUM>, which is being tracked by virtue of the tool tracker <NUM>, may be a rotating bur, electrical ablation device, or the like.

The trackers <NUM>, <NUM>, <NUM> can be battery powered with an internal battery or may have leads to receive power through the navigation computer <NUM>, which, like the camera unit <NUM>, preferably receives external power.

In the embodiment shown, the surgical tool <NUM> is attached to a manipulator <NUM> of the machining station <NUM>. Such an arrangement is shown in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

Referring to <FIG>, the optical sensors <NUM> of the localizer <NUM> receive light signals from the trackers <NUM>, <NUM>, <NUM>. In the illustrated embodiment, the trackers <NUM>, <NUM>, <NUM> are active trackers. In this embodiment, each tracker <NUM>, <NUM>, <NUM> has at least three active tracking elements or markers for transmitting light signals to the optical sensors <NUM>. The active markers can be, for example, light emitting diodes or LEDs <NUM> transmitting light, such as infrared light. The optical sensors <NUM> preferably have sampling rates of <NUM> or more, more preferably <NUM> or more, and most preferably <NUM> or more. In some embodiments, the optical sensors <NUM> have sampling rates of <NUM>. The sampling rate is the rate at which the optical sensors <NUM> receive light signals from sequentially fired LEDs <NUM>. In some embodiments, the light signals from the LEDs <NUM> are fired at different rates for each tracker <NUM>, <NUM>, <NUM>.

Each of the LEDs <NUM> is connected to a tracker controller (not shown) located in a housing of the associated tracker <NUM>, <NUM>, <NUM> that transmits/receives data to/from the navigation computer <NUM>. In one embodiment, the tracker controllers transmit data on the order of several Megabytes/second through wired connections with the navigation computer <NUM>. In other embodiments, a wireless connection may be used. In these embodiments, the navigation computer <NUM> has a transceiver (not shown) to receive the data from the tracker controller.

In other embodiments, the trackers <NUM>, <NUM>, <NUM> may have passive markers (not shown), such as reflectors that reflect light emitted from the camera unit <NUM>. The reflected light is then received by the optical sensors <NUM>. Active and passive arrangements are well known in the art.

In some embodiments, the trackers <NUM>, <NUM>, <NUM> also include a gyroscope sensor and accelerometer, such as the trackers shown in <CIT>, entitled, "Navigation System Including Optical and Non-Optical Sensors,".

The navigation computer <NUM> includes a navigation processor <NUM>. It should be understood that the navigation processor <NUM> could include one or more processors to control operation of the navigation computer <NUM>. The processors can be any type of microprocessor or multi-processor system. The term processor is not intended to limit the scope any embodiment to a single processor.

The camera unit <NUM> receives optical signals from the LEDs <NUM> of the trackers <NUM>, <NUM>, <NUM> and outputs to the processor <NUM> signals relating to the position of the LEDs <NUM> of the trackers <NUM>, <NUM>, <NUM> relative to the localizer <NUM>. Based on the received optical (and non-optical signals in some embodiments), navigation processor <NUM> generates data indicating the relative positions and orientations of the trackers <NUM>, <NUM>, <NUM> relative to the localizer <NUM>. In one version, the navigation processor <NUM> uses well known triangulation methods for determining position data.

Prior to the start of the surgical procedure, additional data are loaded into the navigation processor <NUM>. Based on the position and orientation of the trackers <NUM>, <NUM>, <NUM> and the previously loaded data, navigation processor <NUM> determines the position of the working end of the surgical tool <NUM> (e.g., the centroid of a surgical bur) and the orientation of the surgical tool <NUM> relative to the tissue against which the working end is to be applied. In some embodiments, the navigation processor <NUM> forwards these data to a manipulator controller <NUM>. The manipulator controller <NUM> can then use the data to control the manipulator <NUM> as described in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

In one embodiment, the manipulator <NUM> is controlled to stay within a preoperatively defined virtual boundary set by the surgeon (not shown), which defines the material of the femur F and tibia T to be removed by the surgical tool <NUM>. More specifically, each of the femur F and tibia T has a target volume of material that is to be removed by the working end of the surgical tool <NUM>. The target volumes are defined by one or more virtual cutting boundaries. The virtual cutting boundaries define the surfaces of the bone that should remain after the procedure. The navigation system <NUM> tracks and controls the surgical tool <NUM> to ensure that the working end, e.g., the surgical bur, only removes the target volume of material and does not extend beyond the virtual cutting boundary, as disclosed in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

The virtual cutting boundary may be defined within a virtual model of the femur F and tibia T and be represented as a mesh surface, constructive solid geometry (CSG), voxels, or using other boundary representation techniques. The surgical tool <NUM> cuts away material from the femur F and tibia T to receive an implant. The surgical implants may include unicompartmental, bicompartmental, or total knee implants as shown in <CIT>, entitled, "Prosthetic Implant and Method of Implantation,".

The navigation processor <NUM> also generates image signals that indicate the relative position of the working end to the tissue. These image signals are applied to the displays <NUM>, <NUM>. The displays <NUM>, <NUM>, based on these signals, generate images that allow the surgeon and staff to view the relative position of the working end to the surgical site. The displays, <NUM>, <NUM>, as discussed above, may include a touch screen or other input/output device that allows entry of commands.

Referring to <FIG>, tracking of objects is generally conducted with reference to a localizer coordinate system LCLZ. The localizer coordinate system has an origin and an orientation (a set of x, y, and z axes). During the procedure one goal is to keep the localizer coordinate system LCLZ in a known position. An accelerometer (not shown) mounted to the localizer <NUM> may be used to track sudden or unexpected movement of the localizer coordinate system LCLZ, as may occur when the localizer <NUM> is inadvertently bumped by surgical personnel.

Each tracker <NUM>, <NUM>, <NUM> and object being tracked also has its own coordinate system separate from the localizer coordinate system LCLZ. Components of the navigation system <NUM> that have their own coordinate systems are the bone trackers <NUM>, <NUM> and the tool tracker <NUM>. These coordinate systems are represented as, respectively, bone tracker coordinate systems BTRK1, BTRK2, and tool tracker coordinate system TLTR.

Navigation system <NUM> monitors the positions of the femur F and tibia T of the patient by monitoring the position of bone trackers <NUM>, <NUM> firmly attached to bone. Femur coordinate system is FBONE and tibia coordinate system is TBONE, which are the coordinate systems of the bones to which the bone trackers <NUM>, <NUM> are firmly attached.

Prior to the start of the procedure, pre-operative images of the femur F and tibia T are generated (or of other tissues in other embodiments). These images may be based on MRI scans, radiological scans or computed tomography (CT) scans of the patient's anatomy. These images are mapped to the femur coordinate system FBONE and tibia coordinate system TBONE using well known methods in the art. These images are fixed in the femur coordinate system FBONE and tibia coordinate system TBONE. As an alternative to taking pre-operative images, plans for treatment can be developed in the operating room (OR) from kinematic studies, bone tracing, and other methods.

During an initial phase of the procedure, the bone trackers <NUM>, <NUM> are firmly affixed to the bones of the patient. The pose (position and orientation) of coordinate systems FBONE and TBONE are mapped to coordinate systems BTRK1 and BTRK2, respectively. In one embodiment, a pointer instrument P (see <FIG>), such as disclosed in <CIT>. , having its own tracker PT (see <FIG>), may be used to register the femur coordinate system FBONE and tibia coordinate system TBONE to the bone tracker coordinate systems BTRK1 and BTRK2, respectively. Given the fixed relationship between the bones and their bone trackers <NUM>, <NUM>, positions and orientations of the femur F and tibia T in the femur coordinate system FBONE and tibia coordinate system TBONE can be transformed to the bone tracker coordinate systems BTRK1 and BTRK2 so the camera unit <NUM> is able to track the femur F and tibia T by tracking the bone trackers <NUM>, <NUM>. These pose-describing data are stored in memory integral with both manipulator controller <NUM> and navigation processor <NUM>.

The working end of the surgical tool <NUM> (also referred to as energy applicator distal end) has its own coordinate system EAPP. The origin of the coordinate system EAPP may represent a centroid of a surgical cutting bur, for example. The pose of coordinate system EAPP is fixed to the pose of tool tracker coordinate system TLTR before the procedure begins. Accordingly, the poses of these coordinate systems EAPP, TLTR relative to each other are determined. The pose-describing data are stored in memory integral with both manipulator controller <NUM> and navigation processor <NUM>.

Referring to <FIG>, a localization engine <NUM> is a software module that can be considered part of the navigation system <NUM>. Components of the localization engine <NUM> run on navigation processor <NUM>. In some embodiments, the localization engine <NUM> may run on the manipulator controller <NUM>.

Localization engine <NUM> receives as inputs the optically-based signals from the camera controller <NUM> and, in some embodiments, the non-optically based signals from the tracker controller. Based on these signals, localization engine <NUM> determines the pose of the bone tracker coordinate systems BTRK1 and BTRK2 in the localizer coordinate system LCLZ. Based on the same signals received for the tool tracker <NUM>, the localization engine <NUM> determines the pose of the tool tracker coordinate system TLTR in the localizer coordinate system LCLZ.

The localization engine <NUM> forwards the signals representative of the poses of trackers <NUM>, <NUM>, <NUM> to a coordinate transformer <NUM>. Coordinate transformer <NUM> is a navigation system software module that runs on navigation processor <NUM>. Coordinate transformer <NUM> references the data that defines the relationship between the pre-operative images of the patient and the bone trackers <NUM>, <NUM>. Coordinate transformer <NUM> also stores the data indicating the pose of the working end of the surgical tool <NUM> relative to the tool tracker <NUM>.

During the procedure, the coordinate transformer <NUM> receives the data indicating the relative poses of the trackers <NUM>, <NUM>, <NUM> to the localizer <NUM>. Based on these data and the previously loaded data, the coordinate transformer <NUM> generates data indicating the relative position and orientation of both the coordinate system EAPP, and the bone coordinate systems, FBONE and TBONE to the localizer coordinate system LCLZ.

As a result, coordinate transformer <NUM> generates data indicating the position and orientation of the working end of the surgical tool <NUM> relative to the tissue (e.g., bone) against which the working end is applied. Image signals representative of these data are forwarded to displays <NUM>, <NUM> enabling the surgeon and staff to view this information. In certain embodiments, other signals representative of these data can be forwarded to the manipulator controller <NUM> to guide the manipulator <NUM> and corresponding movement of the surgical tool <NUM>.

In the embodiment shown in <FIG>, the surgical tool <NUM> forms part of an end effector of the manipulator <NUM>. The manipulator <NUM> has a base <NUM>, a plurality of links <NUM> extending from the base <NUM>, and a plurality of active joints (not numbered) for moving the surgical tool <NUM> with respect to the base <NUM>. The manipulator <NUM> has the ability to operate in a manual mode or a semi-autonomous mode in which the surgical tool <NUM> is autonomously moved along a predefined tool path, as described in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

The manipulator controller <NUM> can use the position and orientation data of the surgical tool <NUM> and the patient's anatomy to control the manipulator <NUM> as described in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

The manipulator controller <NUM> may have a central processing unit (CPU) and/or other manipulator processors, memory (not shown), and storage (not shown). The manipulator controller <NUM>, also referred to as a manipulator computer, is loaded with software as described below. The manipulator processors could include one or more processors to control operation of the manipulator <NUM>. The processors can be any type of microprocessor or multi-processor system. The term processor is not intended to limit any embodiment to a single processor.

Referring to <FIG>, a plurality of position sensors <NUM>, <NUM>, <NUM> are associated with the plurality of link <NUM> of the manipulator <NUM>. In one embodiment, the position sensors <NUM>, <NUM>, <NUM> are encoders. The position sensors <NUM>, <NUM>, <NUM> may be any suitable type of encoder, such as rotary encoders. Each position sensor <NUM>, <NUM>, <NUM> is associated with an actuator, such as motor M. Each position sensor <NUM>, <NUM>, <NUM> is a sensor that monitors the angular position of one of three motor driven components of the manipulator <NUM> with which the position sensor is associated. Manipulator <NUM> includes two additional position sensors, <NUM> and <NUM>. Position sensors <NUM> and <NUM> are associated with additional driven links. In some embodiments, the manipulator <NUM> includes two arm structures with six position sensors at six active joints. One such embodiment is described in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

The manipulator <NUM> may be in the form of a conventional robotic system or other conventional machining apparatus, and thus the components thereof shall not be described in detail.

Manipulator controller <NUM> determines the desired location to which the surgical tool <NUM> should be moved. Based on this determination, and information relating to the current location (e.g., pose) of the surgical tool <NUM>, the manipulator controller <NUM> determines the extent to which each of the plurality of links <NUM> needs to be moved in order to reposition the surgical tool <NUM> from the current location to the desired location. The data regarding where the plurality of links <NUM> are to be positioned is forwarded to joint motor controllers <NUM> that control the active joints of the manipulator <NUM> to move the plurality of links <NUM> and thereby move the surgical tool <NUM> from the current location to the desired location.

In order to determine the current location of the surgical tool <NUM>, data from the position sensors <NUM>, <NUM>, <NUM>, <NUM> and <NUM> is used to determine measured joint angles. The measured joint angles of the active joints are forwarded to a forward kinematics module, as known in the art. Also applied to the forward kinematics module are the signals from the position sensors <NUM> and <NUM>. These signals are the measured joint angles for passive joints integral with these encoders. Based on the measured joint angles and preloaded data, the forward kinematics module determines the pose of the surgical tool <NUM> in a manipulator coordinate system MNPL. The preloaded data are data that define the geometry of the plurality of links <NUM> and joints. With this information, the manipulator controller <NUM> and/or navigation processor <NUM> can transform coordinates from the localizer coordinate system LCLZ into the manipulator coordinate system MNPL, or vice versa.

In one embodiment, the manipulator controller <NUM> and joint motor controllers <NUM> collectively form a position controller that operates to move the surgical tool <NUM> to desired positions and/or orientations. The position controller operates in a position control loop. The position control loop may include multiple position control loops in parallel or series for each active joint. The position control loop processes position and orientation information to indicate and direct the pose of the surgical tool <NUM>.

During operation of the manipulator <NUM>, line-of-sight between the trackers <NUM>, <NUM>, <NUM> and the localizer <NUM> should be maintained to ensure accurate movement of the surgical tool <NUM> to the desired positions and/or orientations. Periods of time in which the line-of-sight is blocked or obstructed may require the material removal system <NUM> to display an error message and cease operation of the manipulator <NUM> until the line-of-sight returns or the navigation system <NUM> is reset. This can cause delays in surgical procedures. This could significantly increase the surgical procedure time, particularly if difficulty arises in restoring the line-of-sight.

The navigation computer <NUM> determines that there is an error if any one of the optical sensors <NUM> fails to receive a signal from an LED <NUM>, even though other optical sensors <NUM> may still receive the signal. In other embodiments, the navigation computer <NUM> determines that there is an error if none of the optical sensors <NUM> receive the signal. In either case, when the navigation system <NUM> determines that there is an error based on the failure of one or more optical sensors <NUM> to receive signals from one or more LEDs <NUM>, an error signal is generated by the navigation computer <NUM>. An error message then appears on displays <NUM>, <NUM>. The navigation computer <NUM> also transmits an error signal to the tracker controller.

In some embodiments, the tracker <NUM> may include four or more tracking LEDs <NUM> so that if the tracking signal from one of the LEDs <NUM> is obstructed, position and orientation data can still be obtained using the remaining LEDs <NUM>. In this instance, before any error signals are generated, the navigation computer <NUM> will first run through a complete tracking cycle. The complete tracking cycle includes sequentially activating all the LEDs <NUM> on the tracker <NUM> to determine if the optical sensors <NUM> receive tracking signals from at least three of the LEDs <NUM> in the tracking cycle. The error signal is then generated, and the error message displayed, if an optical sensor <NUM> (or all optical sensors <NUM> in some embodiments) did not receive tracking signals from at least three LEDs <NUM> in the tracking cycle. In some of the embodiments described further below, the navigation system <NUM> reduces the potential for line-of-sight obstructions in order to avoid such error messages.

Line-of-sight obstructions block light signals being sent from the LEDs <NUM> of the trackers <NUM>, <NUM>, <NUM> to the optical sensors <NUM> of the localizer <NUM>. The navigation system <NUM> reduces these line-of-sight obstructions intraoperatively, i.e., during the surgical procedure, by tracking objects that may cause such line-of-sight obstructions and generating feedback to the user should any of the objects pose a risk of blocking or obstructing the line-of-sight between one of the tracking devices <NUM>, <NUM>, <NUM> and the localizer <NUM>.

Objects that can cause line-of-sight obstructions include any physical objects that may be within the field-of-view of the localizer <NUM> during the surgical procedure. Examples of such physical objects include the structures associated with each of the trackers <NUM>, <NUM>, <NUM>, or portions thereof. Other physical objects may include the surgical tool <NUM>, retractors at the surgical site, a limb holder, other tools, surgical personnel, or portions of any of these, that may be within the field-of-view of the localizer <NUM>. If unchecked, these physical objects could move in a way that causes a line-of-sight obstruction. The navigation system <NUM> tracks the positions and orientations of each of these physical objects and generates feedback to the user before a line-of-sight obstruction arises to at least reduce, and ideally prevent, line-of-sight obstructions.

Each of the physical objects that can cause line-of-sight obstructions are modeled in virtual space for purposes of tracking these physical objects. These models are referred to as virtual objects. Virtual objects are maps in the localizer coordinate system LCLZ of each of the physical objects being tracked in the field-of-view of the localizer <NUM> such as the trackers <NUM>, <NUM>, <NUM>, the surgical tool <NUM>, the retractors, the limb holder, other tools, or surgical personnel. The virtual objects could be represented by polygonal surfaces, splines, or algebraic surfaces (including parametric surfaces). In one more specific version, these surfaces are presented as triangular meshes. The corners of each polygon are defined by points in the localizer coordinate system LCLZ. An individual area section that defines a portion of each virtual object boundary or mesh is referred to as a tile. The virtual objects can also be represented by <NUM>-D volumes using voxel-based models or other modeling techniques.

Referring to <FIG>, for purposes of illustration, virtual objects <NUM>', <NUM>', <NUM>', <NUM>' associated with the physical structures of the trackers <NUM>, <NUM>, <NUM> and the surgical tool <NUM> are shown in the localizer coordinate system LCLZ. Note that the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' are modeled as simple shapes for purposes of computational efficiency. Additionally, the tool tracker and tool virtual objects <NUM>' and <NUM>' associated with the tool tracker <NUM> and the surgical tool <NUM> are fixed relative to one another and could alternatively be represented as a single virtual object.

The tool tracker and tool virtual objects <NUM>' and <NUM>' can be tracked by virtue of tracking the tool tracker <NUM>. In particular, the geometric models of the tool tracker and tool virtual objects <NUM>' and <NUM>' are stored in memory and their relationships to the LEDs <NUM> on the tool tracker <NUM> are known. The bone tracker virtual objects <NUM>' and <NUM>' can be tracked by virtue of tracking the bone trackers <NUM>, <NUM>. In particular, the geometric models of the bone tracker virtual objects <NUM>' and <NUM>' are stored in memory and their relationships to the LEDs <NUM> on the bone trackers <NUM>, <NUM> are known. Other tracking devices (not shown) may be attached to other physical objects, such as the retractors, the limb holder, other tools, or surgical personnel present in the field-of-view of the localizer <NUM> in order to track these other physical objects.

Before the surgical procedure begins, each of the trackers <NUM>, <NUM>, <NUM> are placed into the field-of-view of the localizer <NUM>. The displays <NUM>, <NUM> graphically depict the field-of-view of the localizer <NUM> from a top and side perspective, as shown in <FIG> in order to visually confirm that the trackers <NUM>, <NUM>, <NUM> are placed into the field-of-view of the localizer <NUM>. The field-of-view is defined by the spatial relationship of the optical sensors <NUM> and the range of the optical sensors <NUM> for receiving light from the LEDs <NUM> of the trackers <NUM>, <NUM>, <NUM>. The navigation system <NUM> then verifies that each of the trackers <NUM>, <NUM>, <NUM> is visible in the field-of-view. Once verified, the surgical procedure can begin.

Referring to <FIG>, a virtual boundary generator <NUM> (see <FIG>) generates a virtual line-of-sight boundary <NUM>, <NUM>, <NUM> based on the line-of-sight relationship between each of the tracking devices <NUM>, <NUM>, <NUM> and the localizer <NUM>. The virtual line-of-sight boundary <NUM>, <NUM>, <NUM> delineates space in which the physical objects should be restricted from entering so that light from the LEDs <NUM> of each tracking device <NUM>, <NUM>, <NUM> is able to be transmitted, without obstruction or blockage, to the optical sensors <NUM> of the localizer <NUM>.

In some embodiments the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> are cylindrical, spherical, or frustoconical in shape, as shown in <FIG>. Other shapes are also possible. In other embodiments, the virtual line-of-sight boundaries are represented as lines (e.g., lines from each of the LEDs to each of the optical sensors <NUM>). The virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> shown in <FIG> extend from a first end <NUM>, <NUM>, <NUM> defined about the LEDs <NUM> on each of the tracking devices <NUM>, <NUM>, <NUM> to a second end <NUM> defined about the optical sensors <NUM> of the localizer <NUM>. The virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> may be oversized such that the physical objects may penetrate slightly into the virtual line-of-sight boundaries in order to detect collisions, as explained further below, without causing a line-of-sight obstruction.

The virtual boundary generator <NUM> updates the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> to account for relative movement between the tracking devices <NUM>, <NUM>, <NUM> and the localizer <NUM> during the surgical procedure. Updating may occur each time the navigation system <NUM> receives a complete set of signals from the LEDs <NUM> for each of the tracking devices <NUM>, <NUM>, <NUM> (e.g., at least three signals for each tracking device). Updating may occur each time a new commanded position is determined for the surgical tool <NUM>. In embodiments in which the surgical tool <NUM> is controlled by the manipulator <NUM>, the time frame for determining each new commanded position may be every <NUM> to <NUM> milliseconds.

The virtual boundary generator <NUM> is a software module that runs on the navigation processor <NUM> or the manipulator controller <NUM>, or both. The virtual boundary generator <NUM> generates a map that defines the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM>. A first input into the virtual boundary generator <NUM> includes the position and orientation of each of the LEDs <NUM> for each of the tracking devices <NUM>, <NUM>, <NUM> in the localizer coordinate system LCLZ. From this LED pose data, the position and orientation of the first end <NUM>, <NUM>, <NUM> can be defined. A second input into the virtual boundary generator <NUM> includes the position and orientation of each of the optical sensors <NUM> of the localizer <NUM> in the localizer coordinate system LCLZ. From this optical sensor pose data, the position and orientation of the second end <NUM> about the optical sensors <NUM> can be defined. Based on the above data and through instructions, the virtual boundary generator <NUM> generates the map that defines the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> in the localizer coordinate system LCLZ.

In some embodiments, the virtual boundary generator <NUM> generates the virtual line-of-sight boundaries as polygonal surfaces, splines, or algebraic surfaces (including parametric surfaces). In one more specific version, these surfaces are presented as triangular meshes. The corners of each polygon are defined by points in the localizer coordinate system LCLZ. An individual area section that defines a portion of each virtual line-of-sight boundary or mesh is referred to as a tile. The virtual line-of-sight boundaries can also be represented as <NUM>-D volumes using voxel-based models or other modeling techniques.

A collision detector <NUM> (see <FIG>) evaluates movement of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' relative to the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> to detect collisions between the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> (which are effectively virtual objects as well). More specifically, the collision detector <NUM> detects collisions between the geometric models representing the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and the geometric models representing the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM>. Collision detection includes detecting actual virtual collisions or predicting virtual collisions before they occur.

The purpose of the tracking performed by the collision detector <NUM> is to prevent any physical objects from obstructing the line-of-sight between the LEDs <NUM> of the tracking devices <NUM>, <NUM>, <NUM> and the optical sensors <NUM> of the localizer <NUM>. A first input into the collision detector <NUM> is a map of each of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' being tracked in the field-of-view of the localizer <NUM>. A second input into the collision detector <NUM> is the map of each of the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM>.

The collision detector <NUM> is a software module that runs on the navigation processor <NUM> or the manipulator controller <NUM>, or both. The collision detector <NUM> may use any conventional algorithm for detecting collisions between the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM>. For example, suitable techniques for finding the intersection of two parametric surfaces include subdivision methods, lattice methods, tracing methods, and analytic methods. For voxel-based virtual objects, collision detection can be carried out by detecting when any two voxels overlap in the localizer coordinate system LCLZ, as described in <CIT>.

A feedback generator <NUM> (see <FIG>) is in communication with the collision detector <NUM> to respond to the detection of a collision between any of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and any of the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM>. The feedback generator <NUM> is a software module that runs on the navigation processor <NUM> or the manipulator controller <NUM>, or both. The feedback generator <NUM> responds to the detection of a collision by providing the user with one or more forms of feedback, including one or more of audible, visual, vibration, or haptic feedback.

In one embodiment, the feedback generator <NUM> causes activation of a feedback device in the form of an annunciator <NUM> in communication with the navigation processor <NUM> to produce an audible alert to the user in response to a collision.

Referring to <FIG>, the feedback generator <NUM> may also cause the displays <NUM>, <NUM> to display an image representing the collision so that the user can determine how to avoid the collision (in the case that the collision has been predicted) or reverse the collision (in the case that the collision has already occurred). The collision may be represented by showing the virtual line-of-sight boundary <NUM>, <NUM> or <NUM> being affected, along with a graphic representation of where the physical object involved has collided with or is about to collide with the virtual line-of-sight boundary <NUM>, <NUM>, <NUM>. A text description of the particular tracker <NUM>, <NUM>, or <NUM> involved, i.e., the tracker that is about to be obstructed, such as "femur tracker," may also be displayed on the displays <NUM>, <NUM>.

In some embodiments, every physical object in the field-of-view of the localizer <NUM> that is tracked using virtual objects could be represented on the displays <NUM>, <NUM>. In this case, the collision may be illustrated using color coding. For instance, the color red could be shown surrounding the portion of the physical object (associated by virtue of its virtual object) colliding with the virtual line-of-sight boundary <NUM>, <NUM>, or <NUM>. The tracker <NUM>, <NUM>, or <NUM> being affected could also be color coded (possibly the same or a different color) so that visually the user immediately sees which physical object is going to obstruct which tracker line-of-sight, and intuitively the user can avoid the obstruction. In addition, arrows could be graphically depicted on the display to show the direction in which the physical object should be moved to avoid the collision or reverse the collision. These arrows could be generated based on the direction of a feedback force determined by the collision detector <NUM>, as described further below.

Referring to <FIG>, the feedback generator <NUM>, in response to detecting the collision, may also cause the displays <NUM>, <NUM> to display a message to the user including instructions to reposition particular anatomy of the patient. The particular anatomy may include the anatomy to which the bone tracker <NUM>, <NUM> about to be obstructed is attached. For instance, if the tool virtual object <NUM>' representing the surgical tool <NUM> was found to have collided with the virtual line-of-sight boundary <NUM> associated with the bone tracker <NUM> on the tibia T, the navigation processor <NUM> may cause the displays <NUM>, <NUM> to display a message to the user to "move the tibia. " The particular message may be stored in a look-up table of messages that are associated with particular scenarios of possible collisions. In this example, this message is located in the look-up table with the scenario in which the tool virtual object <NUM>' has collided with the virtual line-of-sight boundary <NUM>. More detailed instructions are also possible based on an avoidance or repulsion vector that defines the direction to be taken to avoid or reverse the collision. The instructions may be to "move the tibia" with an arrow A further displayed or flashing on the displays <NUM>, <NUM>, as shown in <FIG>, wherein the arrow A is in the direction of the avoidance or repulsion vector.

The feedback generator <NUM> may also cause the displays <NUM>, <NUM> to display a message to the user including instructions to reposition the localizer <NUM> in response to detecting the collision. For instance, if the tool virtual object <NUM>' representing the surgical tool <NUM> was found to have collided with the virtual line-of-sight boundary <NUM> associated with the bone tracker <NUM> on the tibia T, the navigation processor <NUM> may cause the displays <NUM>, <NUM> to display a message to the user to "move the camera unit. " The particular message may be stored in a look-up table of messages that are associated with particular scenarios of possible collisions. In this example, this message is located in the look-up table with the scenario in which the tool virtual object <NUM>' has collided with the virtual line-of-sight boundary <NUM>.

The feedback generator <NUM> may also cause the displays <NUM>, <NUM> to display a message to the user including instructions to reposition the manipulator <NUM> in response to detecting the collision. For instance, if the tool virtual object <NUM>' representing the surgical tool <NUM> was found to have collided with the virtual line-of-sight boundary <NUM> associated with the bone tracker <NUM> on the tibia T, the navigation processor <NUM> may cause the displays <NUM>, <NUM> to display a message to the user to "move the manipulator. " The particular message may be stored in a look-up table of messages that are associated with particular scenarios of possible collisions. In this example, this message is located in the look-up table with the scenario in which the tool virtual object <NUM>' has collided with the virtual line-of-sight boundary <NUM>. One reason this feedback may be used is in situations in which the surgical tool <NUM> or tibia T cannot otherwise be manipulated to avoid the collision. Additionally, the manipulator <NUM> has a limited range of motion and if the manipulator <NUM> is within a predefined threshold of that limited range, this message may be needed to regain additional range of motion during the surgical procedure to avoid collisions.

In addition, the feedback generator <NUM> may cause the user to experience vibration feedback in the form of vibrations to the physical object associated with the virtual object <NUM>', <NUM>', <NUM>', <NUM>' that is colliding with or about to collide with a virtual line-of-sight boundary <NUM>, <NUM>, <NUM>. For instance, when the user is positioning the surgical tool <NUM> in a manual mode in which the user is grasping a handle of the surgical tool <NUM>, a vibration device <NUM>, such as an eccentric motor, may be actuated if the tool virtual object <NUM>' is colliding with or about to collide with a virtual line-of-sight boundary <NUM>, <NUM>, <NUM>. The vibration device <NUM> is mounted to the surgical tool <NUM> such that vibrations from the vibration device <NUM> can be transmitted to the handle. The vibration feedback indicates to the user that the intended position may cause a line-of-sight obstruction thereby allowing the user to cease further motion and prevent the line-of-sight obstruction. The user can then determine an alternate course that will avoid a line-of-sight obstruction.

In one embodiment, the feedback generator <NUM> provides haptic feedback to the user by responding to a collision with a feedback force that avoids or repulses the collision. The feedback force is determined by the collision detector <NUM>. The feedback force may have force and/or torque components including up to three components of force along x, y, and z axes, and three components of torque about these axes.

In one example, the feedback generator <NUM> provides the haptic feedback to the user through the surgical tool <NUM> when the manipulator <NUM> is operated in the manual mode. This prevents the manipulator <NUM> from positioning the tool virtual object <NUM>' associated with the surgical tool <NUM> into the virtual line-of-sight boundaries <NUM>, <NUM> associated with the bone trackers <NUM>, <NUM> thereby avoiding any line-of-sight obstruction. In one embodiment, the collision detector <NUM> detects the collision by predicting whether a virtual collision will occur if the manipulator <NUM> moves the surgical tool <NUM> to a commanded pose, but before the manipulator controller <NUM> actually moves the surgical tool <NUM> to the commanded pose. If a virtual collision is predicted, then the manipulator <NUM> is controlled to move the surgical tool <NUM> to an altered commanded pose to avoid the collision.

In some embodiments, the manipulator <NUM> is a passive manipulator. In this case, the haptic feedback provides feedback to the user after a virtual collision occurs to prevent any further penetration of the virtual object <NUM>', <NUM>', <NUM>', <NUM>' into the affected virtual line-of-sight boundary <NUM>, <NUM>, <NUM> or to reverse the collision. Thus, the collision detection may be responsive to an actual virtual collision or a predicted virtual collision. The feedback generator <NUM> thus ensures that the manual mode positioning of the surgical tool <NUM> is controlled so that the tool virtual object <NUM>' stays outside of, or only penetrates so far into, the virtual line-of-sight boundaries <NUM>, <NUM> to prevent the surgical tool <NUM> from causing line-of-sight obstructions between the bone trackers <NUM>, <NUM> and the localizer <NUM>.

When the virtual line-of-sight boundaries <NUM>, <NUM> are represented by a polygonal surface such as a mesh, the collision detector <NUM> identifies any boundary-defining tiles that the tool virtual object <NUM>' could cross during a time frame. This step is often described as a broad phase search. This step is performed by identifying the set or sets of tiles that are within a defined distance (d) of the tool virtual object <NUM>'. This defined distance (d) is a function of: the dimensions of the tool virtual object <NUM>'; the velocity of the tool virtual object <NUM>' relative to the tiles (the velocity of advancement during the past frame is acceptable); the time period of the frame; a scalar defining a characteristic size of the boundary defining sections; and a rounding factor.

As a result of the execution of the broad phase search, the collision detector <NUM> may determine that, in the frame for which this analysis is being performed, all of the tiles are outside of the defined distance (d). This means that, by the end of the frame for which this analysis is being performed, the tool virtual object <NUM>' will not have advanced to a location beyond either of the virtual line-of-sight boundaries <NUM>, <NUM>. This is illustrated by <FIG> where the tool virtual object <NUM>', is spaced well away from the virtual line-of-sight boundary <NUM>. It should be appreciated that this analysis may be conducted for a set of points defining the tool virtual object <NUM>', such as points 128a-<NUM> defining an outer surface of the tool virtual object <NUM>', with each point being analyzed to detect whether that particular point will cross the virtual line-of-sight boundary <NUM>.

Since the continued advancement of the surgical tool <NUM> will not cause any line-of-sight obstructions, the collision detector <NUM> does not modify either the commanded pose or the commanded velocity of the surgical tool <NUM> originally commanded by the manipulator controller <NUM>. The collision detector <NUM> thus outputs a final commanded pose and a final commanded velocity for the surgical tool <NUM> that is the same as that originally determined by the manipulator controller <NUM>.

The collision detector <NUM> may alternatively identify a broad set of boundary-defining tiles that are within the defined distance (d) of the tool virtual object <NUM>' or the points 128a-<NUM>. The collision detector <NUM> then identifies a narrow set of boundary-defining tiles that are within the broad set of tiles that the tool virtual object <NUM>', or any of points 128a-<NUM> on the tool virtual object <NUM>' could cross. This step is referred to as the narrow phase search. This narrow phase search can be performed by initially defining a bounding volume. This bounding volume extends between what are considered to be initial and final poses of the tool virtual object <NUM>'. If this is the first execution, the initial pose of the tool virtual object <NUM>' is based on the previous commanded pose of the surgical tool <NUM>; the final pose of the tool virtual object <NUM>' is based on the current commanded pose of the surgical tool <NUM>, i.e., the pose generated by the manipulator controller <NUM> to which the surgical tool <NUM> should be moved in this frame if the collision detector <NUM> does not detect any collisions.

In its most elemental form, the bounding volume may be lines extending from the points 128a-<NUM> in the initial pose to the points 128a-<NUM> in the final pose. Once the bounding volume is defined, as part of the narrow phase search, the collision detector <NUM> determines which, if any, of the broad set of tiles are intersected by this bounding volume. The tiles intersected by the bounding volume are the narrow set tiles.

It may be determined that none of the broad set of tiles are intersected by the bounding volume; the narrow set is an empty set. If this evaluation tests true, the collision detector <NUM> interprets this condition as indicating that the final pose of the tool virtual object <NUM>' is outside the volumes defined by the virtual line-of-sight boundaries <NUM>, <NUM>. If the tool virtual object <NUM>' is so located, the original commanded pose and commanded velocity are unaltered by the collision detector <NUM> and are output by the collision detector <NUM> as the final commanded pose and final commanded velocity.

Alternatively, it may be determined that the bounding volume crosses one or more tiles; the narrow set contains one or more tiles. If so, the collision detector <NUM> interprets this condition as indicating that the final pose of the tool virtual object <NUM>' is penetrating a boundary. This condition is illustrated by <FIG>. Here the initial pose of the tool virtual object <NUM>' is represented by solid lines and the final pose is represented by phantom lines.

If the condition of intruding on a virtual line-of-sight boundary <NUM>, <NUM> exists, the next step is to determine which of the narrow set of tiles the tool virtual object <NUM>' (and by extension the surgical tool <NUM>) would cross first. If the bounding volume comprises lines, the collision detector <NUM>, for each tile, and for each line, determines the percentage of distance the surgical tool virtual object <NUM>' will advance during the frame prior to crossing the tile (see note of seventy percent in <FIG>). The tile crossed at the lowest percentage of distance is the tile understood to be crossed first.

The boundary defining tiles closest to the tool virtual object <NUM>' may not be the tiles that the tool virtual object <NUM>' could cross. As shown in <FIG>, it was initially determined that tiles T1-T5 of the virtual line-of-sight boundary <NUM> are within the defined distance (d), the distance the tool virtual object <NUM>' could potentially move within the time frame. The closest tile to the tool virtual object <NUM>' is tile T4. However, the tool virtual object <NUM>' is moving along a trajectory that is, for purposes of illustration, downward and to the left towards tile T3. Therefore, the collision detector <NUM> determines that tile T3 is the tile the bounding volume would intersect.

Once the collision detector <NUM> generally determines which boundary-defining tile the tool virtual object <NUM>' will cross if the manipulator controller <NUM> moves the surgical tool <NUM> to the originally commanded pose, the collision detector <NUM> determines a time (t) and a point P. Time (t) is the time period relative to the start of the frame, when the tool virtual object <NUM>' will cross the virtual line-of-sight boundary <NUM>. This time (t) is determined based on the percentage of distance the tool virtual object <NUM>' will advance during the frame prior to contacting the virtual line-of-sight boundary <NUM>, which in this case is seventy percent of the distance, as shown in <FIG>. This determination is made based on the assumption that, during any given frame, the velocity of the surgical tool <NUM>, and thus the tool virtual object <NUM>' is constant. Point P is the point in the localizer coordinate system LCLZ where the tool virtual object <NUM>' will cross the tile. This point P is determined by calculating where the path of advancement of the tool virtual object <NUM>' crosses the tile. Both calculations use as input variables the initial and final poses of the particular point 128a-<NUM> that crosses a tile first and data defining the perimeter of the boundary tile.

In some embodiments, in this situation, the original commanded pose is altered by the collision detector <NUM> to be the position and orientation that the surgical tool <NUM> reaches before contacting the virtual line-of-sight boundary <NUM>, e.g., the position and orientation reached at seventy percent of the distance/time. The user, by virtue of grasping the surgical tool <NUM> with an expectation of moving the surgical tool <NUM> the entire one hundred percent of movement would experience haptic feedback similar to encountering a physical wall when movement ceased at seventy percent, i.e., only to the altered position and orientation. Thus, the manipulator <NUM> to which the surgical tool <NUM> is attached is considered to be a haptic device that transmits haptic feedback to the user.

In another embodiment, the feedback generator <NUM> determines a feedback force to be applied to the surgical tool <NUM> (modeled as a virtual rigid body) to stop the unwanted progression of the surgical tool <NUM> beyond the virtual line-of-sight boundary <NUM>. The feedback generator <NUM> determines the feedback force as a boundary constraining force applied to the surgical tool <NUM>. More specifically, the feedback generator determines a scalar feedback force FBNDR that, if applied to the surgical tool <NUM> at time (t), would stop the advancement of the surgical tool <NUM> in the direction normal to and towards the virtual line-of-sight boundary <NUM>. The feedback generator <NUM> may use any one of a number of different methods to determine the magnitude of force FBNDR. For instance, an impulse method may be used, as described in <CIT>, entitled, "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,".

The final commanded pose and commanded velocity are then calculated to account for the force FBNDR. As opposed to merely ceasing movement of the surgical tool <NUM> at seventy percent to prevent contacting the virtual line-of-sight boundary <NUM>, this method only ceases the component of movement that is normal to the virtual line-of-sight boundary <NUM>, by virtue of the impulse force. Thus, movement along the virtual line-of-sight boundary <NUM> may continue the entire time frame to provide a more natural haptic feedback to the user, as opposed to an abrupt stop.

Ultimately, the final commanded pose from the collision detector <NUM> is applied to an inverse kinematics module (not shown) of the manipulator controller <NUM>. The inverse kinematics module is a motion control module executed by the manipulator controller <NUM>. Based on the commanded pose and preloaded data, the inverse kinematics module determines the desired joint angle of the joints of the manipulator <NUM>. The preloaded data are data that define the geometry of the links <NUM> and joints. In some versions, these data are in the form of Denavit-Hartenberg parameters.

As previously discussed, before the surgical procedure begins, each of the trackers <NUM>, <NUM>, <NUM> are placed into the field-of-view of the localizer <NUM>. The navigation system <NUM>, which operates to reduce line-of-sight obstructions, also operates to maintain the trackers <NUM>, <NUM>, <NUM> within the field-of-view. In particular, the navigation system <NUM> operates to maintain the trackers <NUM>, <NUM>, <NUM> within the field-of-view intraoperatively, i.e., during the surgical procedure, by tracking movement of the trackers <NUM>, <NUM>, <NUM> during the surgical procedure and generating feedback to the user should any of the trackers <NUM>, <NUM>, <NUM> pose a risk of moving outside of the field-of-view of the localizer <NUM>.

The field-of-view of the localizer <NUM> is shown from a top and side view in <FIG>. The virtual boundary generator <NUM> also generates a virtual field-of-view boundary <NUM> based on the field-of-view of the localizer <NUM>. The virtual field-of-view boundary <NUM> delineates a volume of space in which signals from the trackers <NUM>, <NUM>, <NUM> can be received by the localizer <NUM> for purposes of determining the position and/or orientation of the trackers <NUM>, <NUM>, <NUM>. In other words, signals from at least three LEDs <NUM> of each tracker <NUM>, <NUM>, <NUM> can be received by each of the optical sensors <NUM> of the localizer <NUM>.

In some embodiments the virtual field-of-view boundary <NUM> is frustoconical in shape, as shown in <FIG>. In other embodiments, the virtual field-of-view boundary <NUM> is cylindrical or spherical in shape. Other shapes are also possible. The virtual field-of-view boundary <NUM> shown in <FIG> extends divergently outward from the localizer <NUM> to a distal end. The virtual field-of-view boundary <NUM> may be oversized such that the virtual objects <NUM>', <NUM>', <NUM>' representing the trackers <NUM>, <NUM>, <NUM> may penetrate slightly into the virtual field-of-view boundary <NUM> in order to detect collisions, as explained further below, without moving beyond the actual field-of-view of the localizer <NUM>.

The virtual field-of-view boundary <NUM> is intended to remain static during the surgical procedure, but may require adjustment should the localizer <NUM> be moved during the surgical procedure. In this case, the virtual boundary generator <NUM> updates the virtual field-of-view boundary <NUM> to account for such movement during the surgical procedure.

The virtual boundary generator <NUM> generates a map that defines the virtual field-of-view boundary <NUM>. An input into the virtual boundary generator <NUM> includes the position and orientation of the localizer <NUM> in the localizer coordinate system LCLZ, i.e., the locations/arrangement of the optical position sensors <NUM> in the localizer coordinate system LCLZ, which is established during manufacturing (e.g., measured by a CMM) and stored in memory in the camera unit <NUM> or the navigation computer <NUM>. From this localizer pose data, the position and orientation of the virtual field-of-view boundary <NUM> can be established. The virtual field-of-view boundary <NUM> can also be established during manufacturing and stored in the memory of the camera unit <NUM> or the navigation computer <NUM>. The size and shape of the virtual field-of-view boundary <NUM> is predetermined before the surgical procedure and is fixed in position with respect to the localizer <NUM>. Data associated with the size and shape of the virtual field-of-view boundary <NUM> is stored in the memory on the camera unit <NUM> and/or navigation computer <NUM> for retrieval by the navigation processor <NUM>. Based on the above data and through instructions, the virtual boundary generator <NUM> generates the map that defines the virtual field-of-view boundary <NUM> in the localizer coordinate system LCLZ.

In some embodiments, the virtual boundary generator <NUM> generates the virtual field-of-view boundary <NUM> as a polygonal surface, splines, or algebraic surface (including parametric surface). In one more specific version, the surface is presented as triangular meshes. The corners of each polygon are defined by points in the localizer coordinate system LCLZ. An individual area section that defines a portion of the mesh is referred to as a tile. The virtual field-of-view boundary <NUM> can also be represented as a <NUM>-D volume using voxel-based models.

The collision detector <NUM> evaluates movement of the bone tracker and tool tracker virtual objects <NUM>', <NUM>', <NUM>' relative to the virtual field-of-view boundary <NUM> to detect collisions between the virtual objects <NUM>', <NUM>', <NUM>' and the virtual field-of-view boundary <NUM> (which is effectively a virtual object as well). More specifically, the collision detector <NUM> detects collisions between the geometric models representing the virtual objects <NUM>', <NUM>', <NUM>', and the geometric model representing the virtual field-of-view boundary <NUM>. Collision detection includes detecting actual virtual collisions or predicting virtual collisions before they occur.

The purpose of the tracking performed by the collision detector <NUM> is to prevent the trackers <NUM>, <NUM>, <NUM> from moving outside of the field-of-view of the localizer <NUM>. A first input into the collision detector <NUM> is a map of each of the virtual objects <NUM>', <NUM>', <NUM>' being tracked in the field-of-view of the localizer <NUM>. A second input into the collision detector <NUM> is the map of the virtual field-of-view boundary <NUM>.

The collision detector <NUM> may use any conventional algorithm for detecting collisions between the virtual objects <NUM>', <NUM>', <NUM>' and the virtual field-of-view boundary <NUM>. For example, suitable techniques for finding the intersection of two parametric surfaces include subdivision methods, lattice methods, tracing methods, and analytic methods. For voxel-based virtual objects, collision detection can be carried out by detecting when any two voxels overlap in the localizer coordinate system LCLZ, as described in <CIT>.

The feedback generator <NUM> responds to the detection of a collision between any of the virtual objects <NUM>', <NUM>', <NUM>' and the virtual field-of-view boundary <NUM>. The feedback generator <NUM> responds to the detection of a collision by providing the user with one or more forms of feedback, including one or more of audible, visual, vibration, or haptic feedback.

In one embodiment, the feedback generator <NUM> causes activation of the annunciator <NUM> to produce an audible alert to the user in response to a collision.

The feedback generator <NUM> may also cause the displays <NUM>, <NUM> to display an image representing the collision so that the user can determine how to avoid the collision (in the case that the collision has been predicted) or reverse the collision (in the case that the collision has already occurred). The collision may be represented by showing a graphic representation of where the tracker involved has collided with or is about to collide with the virtual field-of-view boundary <NUM>. A text description of the particular tracker <NUM>, <NUM>, or <NUM> involved, such as "femur tracker," may also be displayed on the displays <NUM>, <NUM>.

In some embodiments, every tracker <NUM>, <NUM>, <NUM> in the field-of-view of the localizer <NUM> that is tracked using virtual objects could be represented on the displays <NUM>, <NUM>. In this case, the collision may be illustrated using color coding. For instance, the tracker <NUM>, <NUM>, or <NUM> being affected could be color coded so that visually the user immediately sees which tracker is going to move outside the field-of-view, and intuitively the user can avoid such movement. In addition, arrows could be graphically depicted on the display to show the direction in which the tracker should be moved to stay within the field-of-view. These arrows could be generated based on the direction of a feedback force determined by the collision detector <NUM> in the manner previously described.

Referring to <FIG>, the feedback generator <NUM>, in response to detecting the collision, may also cause the displays <NUM>, <NUM> to display a message to the user including instructions to reposition particular anatomy of the patient. The particular anatomy may include the anatomy to which the bone tracker <NUM>, <NUM> about to move outside the field-of-view is attached. For instance, if the bone tracker <NUM> on the tibia T is about to move outside the field-of-view of the localizer <NUM>, the navigation processor <NUM> may cause the displays <NUM>, <NUM> to display a message to the user to "move the tibia. " The particular message may be stored in a look-up table of messages that are associated with particular scenarios of possible collisions. In this example, this message is located in the look-up table with the scenario in which the bone tracker virtual object <NUM>' has collided with the virtual field-of-view boundary <NUM>. More detailed instructions are also possible based on an avoidance or repulsion vector that defines the direction to be taken to avoid or reverse the collision. The instructions may be to "move the tibia" with an arrow B further displayed or flashing on the displays <NUM>, <NUM> wherein the arrow B is in the direction of the avoidance or repulsion vector.

The feedback generator <NUM> may also cause the displays <NUM>, <NUM> to display a message to the user including instructions to reposition the localizer <NUM> in response to detecting the collision. For instance, if one of the bone tracker or tool tracker virtual objects <NUM>', <NUM>', <NUM>' was found to have collided with the virtual field-of-view boundary <NUM>, the navigation processor <NUM> may cause the displays <NUM>, <NUM> to display a message to the user to "move the camera unit. " The particular message may be stored in a look-up table of messages that are associated with particular scenarios of possible collisions. In this example, this message is located in the look-up table with the scenario in which one of the bone tracker or tool tracker virtual objects <NUM>', <NUM>', <NUM>' has collided with the virtual field-of-view boundary <NUM>.

The feedback generator <NUM> may also cause the displays <NUM>, <NUM> to display a message to the user including instructions to reposition the manipulator <NUM> in response to detecting the collision. For instance, if the tool tracker virtual object <NUM>' was found to have collided with the virtual field-of-view boundary <NUM>, the navigation processor <NUM> may cause the displays <NUM>, <NUM> to display a message to the user to "move the manipulator. " The particular message may be stored in a look-up table of messages that are associated with particular scenarios of possible collisions. In this example, this message is located in the look-up table with the scenario in which the tool tracker virtual object <NUM>' has collided with the virtual field-of-view boundary <NUM>. One reason this feedback may be used is in situations in which the surgical tool <NUM> cannot otherwise be manipulated to avoid the collision. Additionally, the manipulator <NUM> has a limited range of motion and if the manipulator <NUM> is within a predefined threshold of that limited range, this message may be needed to regain additional range of motion during the surgical procedure to avoid collisions.

In addition, the feedback generator <NUM> may cause the user to experience vibration feedback in the form of vibrations. For instance, when the user is positioning the surgical tool <NUM> in a manual mode in which the user is grasping a handle of the surgical tool <NUM>, the vibration device <NUM> may be actuated if the tool tracker virtual object <NUM>' is colliding with or about to collide with the virtual field-of-view boundary <NUM>. The vibration feedback indicates to the user that the tool tracker <NUM> may be close to moving out of the field-of-view of the localizer <NUM> thereby allowing the user to cease further motion and prevent the tool tracker <NUM> from traveling outside the field-of-view. The user can then determine an alternate course.

In one example, the feedback generator <NUM> provides the haptic feedback to the user through the surgical tool <NUM> when the manipulator <NUM> is operated in the manual mode. This prevents the manipulator <NUM> from positioning the tool tracker virtual object <NUM>' into the virtual field-of-view boundary <NUM> thereby avoiding movement of the tool tracker <NUM> outside of the field-of-view. In one embodiment, the collision detector <NUM> detects the collision by predicting whether a virtual collision will occur if the manipulator <NUM> moves the surgical tool <NUM> to a commanded pose, but before the manipulator controller <NUM> actually moves the surgical tool <NUM> to the commanded pose. If a virtual collision is predicted, then the manipulator <NUM> is controlled to move the surgical tool <NUM> to an altered commanded pose to avoid the collision.

In some embodiments, the manipulator <NUM> is a passive manipulator. In this case, the haptic feedback provides feedback to the user after a virtual collision occurs to prevent any further penetration of the tool tracker virtual object <NUM>' into the virtual field-of-view boundary <NUM> or to reverse the collision. Thus, the collision detection may be responsive to an actual virtual collision or a predicted virtual collision. The feedback generator <NUM> thus ensures that the manual mode positioning of the surgical tool <NUM> is controlled so that the tool tracker virtual object <NUM>' stays within, or only penetrates so far into, the virtual field-of-view boundary <NUM> to prevent the tool tracker <NUM> from moving outside of the field-of-view of the localizer <NUM>.

When the virtual field-of-view boundary <NUM> is represented by a polygonal surface such as a mesh, the collision detector <NUM> can detect collisions in the same manner described above with respect to the tool virtual object <NUM>' and <FIG>.

The feedback generator <NUM> can also determine a feedback force to be applied to the surgical tool <NUM> to stop the unwanted progression of the tool tracker <NUM> beyond the virtual field-of-view boundary <NUM> in the same manner described above. In this case, the tool tracker virtual boundary <NUM>' is fixed in relation to the tool virtual boundary <NUM>'. Thus, movement of the tool tracker virtual boundary <NUM>' is controlled by controlling movement of the surgical tool <NUM> and its virtual boundary <NUM>' as previously described.

During operation of the material removal system <NUM> in a surgical procedure, the navigation system <NUM> continuously tracks the position and orientation of each of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' for purposes of determining whether any of the physical objects associated with these virtual objects <NUM>', <NUM>', <NUM>', <NUM>' pose a risk of causing a line-of-sight obstruction between one of the trackers <NUM>, <NUM>, <NUM> and the localizer <NUM>. The navigation system <NUM> also continuously trackers the position and orientation of each of the virtual objects <NUM>', <NUM>', <NUM>' for purposes of determining whether any of the trackers <NUM>, <NUM>, <NUM> associated with these virtual objects <NUM>', <NUM>', <NUM>' pose a risk of moving outside of the field-of-view of the localizer <NUM>. The purpose being to reduce tracking interruptions so that operation of the manipulator <NUM> can continue without unnecessary delays caused by losing line-of-sight or by moving outside of the field-of-view. One exemplary method is outlined below.

Referring to the flow chart of <FIG>, in a step <NUM>, the navigation system <NUM> first detects each of the tracking devices <NUM>, <NUM>, <NUM> within the field-of-view of the localizer <NUM>. Once the tracking devices <NUM>, <NUM>, <NUM> are detected, in step <NUM> the virtual boundary generator <NUM> generates the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> based on the line-of-sight relationships between the tracking devices <NUM>, <NUM>, <NUM> and the localizer <NUM>. The virtual boundary generator <NUM> also generates the virtual field-of-view boundary <NUM> based on the field-of-view of the localizer <NUM>.

The surgical procedure begins in step <NUM> once the initial virtual boundaries <NUM>, <NUM>, <NUM>, <NUM> have been generated.

In step <NUM>, the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> are updated to account for relative movement between the trackers <NUM>, <NUM>, <NUM> and the localizer <NUM> during the surgical procedure.

The virtual objects <NUM>', <NUM>', <NUM>', <NUM>' are preoperatively associated with the physical objects being tracked in the field-of-view of the localizer <NUM>. These are the physical objects that pose a threat of creating a line-of-sight obstruction. Additionally, the bone tracker and tool tracker virtual objects <NUM>', <NUM>', <NUM>' are associated with the trackers <NUM>, <NUM>, <NUM> that are to be kept in the field-of-view of the localizer <NUM>.

The virtual objects <NUM>', <NUM>', <NUM>', <NUM>' are created and then stored in memory in the navigation computer <NUM> or the manipulator controller <NUM>, or both, with their parameters being defined relative to the particular coordinate system of their associated tracker <NUM>, <NUM>, <NUM>. For instance, the bone tracker virtual object <NUM>' which represents the structure of the bone tracker <NUM> attached to the femur F, is created preoperatively and mapped to the bone tracker coordinate system BTRK1 so that the localizer <NUM> is able to track the bone tracker virtual object <NUM>' by tracking the bone tracker <NUM>, and then transform the parameters defining the bone tracker virtual object <NUM>' into the localizer coordinate system LCLZ.

The collision detector <NUM> evaluates the relative movement between the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and the virtual boundaries <NUM>, <NUM>, <NUM>, <NUM> in step <NUM>. Evaluating movement of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' may include tracking the position and orientation of each of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' with respect to a position and orientation of the virtual boundaries <NUM>, <NUM>, <NUM>, <NUM> to facilitate detection of collisions between the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and the virtual boundaries <NUM>, <NUM>, <NUM>, <NUM>.

Decision block <NUM> determines whether the collision detector <NUM> detected a collision between one or more of the virtual objects <NUM>', <NUM>', <NUM>', <NUM>' and one or more of the virtual boundaries <NUM>, <NUM>, <NUM>, <NUM> (either an actual virtual collision or a predicted virtual collision). If a collision is not detected, then the process flows to decision block <NUM> to determine whether the surgical procedure is complete. If the surgical procedure is not yet complete, then the process loops back to step <NUM> and the position and/or orientation of the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> is updated (and the virtual field-of-view boundary <NUM> is updated if the localizer <NUM> has been moved). If the surgical procedure is complete, then collision detection ends.

Referring back to decision block <NUM>, if a collision is detected, then feedback is generated in step <NUM>. The feedback is in the form of one or more of the audible feedback, visual feedback, vibration feedback, or haptic feedback, as previously described. In particular, the feedback generator <NUM> instructs the navigation processor <NUM> or the manipulator controller <NUM> to activate the annunciator <NUM>, manipulate the displays <NUM>, <NUM>, activate the vibration device <NUM>, and/or generate haptic feedback through the manipulator <NUM>.

Once the feedback is generated, the navigation processor <NUM> or manipulator controller <NUM> determines if the surgical procedure is complete in decision block <NUM>. If so, the procedure ends. If not, the process loops again to step <NUM> to repeat until the surgical procedure is complete. The process loop between subsequent updates to the virtual line-of-sight boundaries <NUM>, <NUM>, <NUM> in step <NUM> may occur every time frame in which a commanded position is generated for the manipulator <NUM> or each time the localizer <NUM> detects a new position and/or orientation of the trackers <NUM>, <NUM>, <NUM>.

Claim 1:
A method of reducing tracking interruptions between a tracking device (<NUM>, <NUM>, <NUM>) and a localizer (<NUM>) of a navigation system (<NUM>), the navigation system (<NUM>) further comprising a virtual boundary generator and a collision detector, the method comprising the steps of:
detecting, by the localizer (<NUM>), the tracking device (<NUM>, <NUM>, <NUM>) within a field-of-view of the localizer (<NUM>);
generating, by the virtual boundary generator, a virtual field-of-view boundary (<NUM>) based on the field-of-view of the localizer (<NUM>);
associating, by the virtual boundary generator, a virtual object (<NUM>', <NUM>', <NUM>') with the tracking device (<NUM>, <NUM>, <NUM>);
tracking, by the collision detector, movement of the virtual object (<NUM>', <NUM>', <NUM>') relative to the virtual field-of-view boundary (<NUM>); and
detecting, by the collision detector, a collision between the virtual object (<NUM>', <NUM>', <NUM>') and the virtual field-of-view boundary (<NUM>) while tracking to enable a response that prevents the tracking device (<NUM>, <NUM>, <NUM>) from moving outside of the field-of-view of the localizer (<NUM>).