Patent Description:
Low invasive techniques for accessing various body parts has become common practice for diagnostic purposes, therapeutic purposes and other surgical purposes. For example, health care providers can percutaneously access the gastrointestinal tract, respiratory tract, urinary tract and vasculature. In some cases, the objects being inserted into the patient may be directly visible, but in other situations no direct line of sight may exist.

For the example of endovascular surgery or other procedures where no direct line of sight exists, x-ray fluoroscopy is often utilized to obtain images to assist introduction and guidance of objects through patient anatomy. The increased use of x-ray c-arm fluoroscopy for guiding endovascular and other devices has resulted in escalating concerns on the risks of radiation exposure to patients and operating room staff.

<CIT> discloses a reference tracker, system and method including a tracker portion having a coordinate system encoded therein against which a medical device is tracked. A base portion is associated with the tracker portion. The base portion has a geometry configured to coincide with two axes of the coordinate system to provide a visible indicator of the coordinate system in an anatomical image to permit registration between the anatomical image and the coordinate system.

According to a first aspect of the present invention there comprises one or more non-transitory computer-readable media having data and machine readable instructions executable by a processor, the machine readable instructions causing the processor to at least: compute a registration matrix based on applying a tracking-marker coordinate system for at least one combination marker device to a radiographic transformation matrix, such that the registration matrix provides a transformation from a tracking coordinate system of a tracking system to a common three-dimensional coordinate system, the radiographic transformation matrix being generated based on an image space transformation matrix and an image-marker transformation matrix, the image space transformation matrix being generated based on pre-operative image data and intraoperative image data for a patient, in which the intraoperative image data comprises one or more images acquired by an imaging modality intraoperatively of an anatomical structure of the patient and the at least one combination marker device positioned on the patient, the image-marker transformation matrix being generated based on the intraoperative image data and the tracking-marker coordinate system; storing in memory a patient-specific implicit model defining a geometry of the anatomical structure of the patient that is computed from the pre-operative image data and spatially registered in the common three-dimensional coordinate system; register tracking data based on the registration matrix in the common three-dimensional coordinate system, in which the patient-specific implicit model defining a geometry of the anatomical structure of the patient is also registered in the common three-dimensional coordinate system, the tracking data being generated by the tracking system and representing a location of a sensor attached to an object within the patient in the tracking coordinate system of the tracking system; and generate an output visualization representing a location of the object relative to geometry of the anatomical structure of the patient in the common three-dimensional coordinate system based on the registered tracking data and the patient-specific implicit model.

According to a second aspect of the present invention there comprises a system comprising: a non-transitory memory configured to store machine-readable instructions and data; and a processing unit configured to access the memory and execute the machine-readable instructions to: compute registration matrix based on applying a tracking-marker coordinate system for at least one combination marker device to a radiographic transformation matrix, the registration matrix providing a transformation from a tracking coordinate system of a tracking system to a common three-dimensional coordinate system, the radiographic transformation matrix being generated based on an image space transformation matrix and an image-marker transformation matrix, the image space transformation matrix being generated based on the pre-operative image data and intraoperative image data, the image-marker transformation matrix being generated based on the intraoperative image data and the tracking-marker coordinate system, the intraoperative image data comprising one or more images captured intraoperatively of the anatomical structure of the patient and the at least one combination marker device positioned on the patient; generate a patient-specific implicit model defining a geometry of an anatomical structure of the patient based on pre-operative image data of the patient; apply the registration matrix to tracking data to provide registered tracking data to transform the tracking data from the tracking coordinate system of the tracking system to the common three-dimensional coordinate system, the tracking data representing a location of a sensor attached to an object within the patient in the tracking coordinate system of the tracking system; and generate a three-dimensional graphical visualization representing a location of the object relative to the patient-specific implicit model of the anatomical structure of the patient in the common three-dimensional coordinate system based on the registered tracking data.

This disclosure relates to a method and system to facilitate intraoperative positioning and guidance of an object.

The approach disclosed herein receives and stores tracking data in memory. The tracking data can represent a location of an object that is being moved within an anatomical structure (e.g., a tubular structure) of a patient's body. The tracking data can represent a location of the object without the use of ionizing radiation. For example, one or more sensors can be coupled to the object being tracked and provide sensing signals in response to a field provided by a tracking system. The tracking system can determine the tracking data to indicate a three-dimensional position and orientation of the object in a coordinate system of the tracking system. The tracking data can be registered into a three-dimensional coordinate system in which a patient-specific implicit model is also registered. The patient-specific implicit model can be generated based on image data acquired for the patient to define geometry of the anatomical structure of the patient. The image data used to generate the patient-specific implicit model can be acquired before the intraoperative procedure that is being tracked by the tracking system. For example, the image data can be acquired as part of a preoperative planning stage.

An output visualization can be generated based on the registered tracking data and the implicit model to render a corresponding three-dimensional graphical representation of the object at a position relative to the anatomical structure of the patient. For example, the position, direction and shape of the object can be tracked, represented and visualized in an intuitive 3D shaded surface model of the patient's anatomy. The implicit model of the patient anatomy enables the visualization to be rendered and updated such as to provide a substantially real time dynamic visualization of the intraoperative procedure in the absence of any direct line of sight. Moreover, the approach can eliminate or at least significantly reduce ionizing radiation that is typically used intraoperatively during many procedures.

<FIG> depicts an example of a system <NUM> to facilitate intraoperative guidance and positioning. The system <NUM> includes a registration engine <NUM> that is configured to compute registered tracking data <NUM> based on input tracking data <NUM>. The input tracking data <NUM> can be provided by a tracking system, such as in response to non-ionizing radiation that is provided to a patient to track the position of one or more sensors that can be integrated into an object, such as an instrument or an implantable device. As used herein, non-ionizing radiation can refer to any type of electromagnetic radiation that does not carry enough energy per quantum to ionize atoms or molecules-that is, to completely remove an electron from an atom or molecule. Instead of producing charged ions when passing through matter, the electromagnetic radiation provided by the tracking system can have sufficient energy only for excitation, the movement of an electron to a higher energy state. Other types of tracking systems, such as ultrasonic sensors or the like, can also be employed to provide the tracking data. The tracking data <NUM> can include a position and orientation (e.g., a vector) for each sensor that can be detected by the tracking system.

The registration engine <NUM> is programmed to compute a registration matrix <NUM> that can convert the tracking data <NUM> from a coordinate system of the tracking system into a coordinate system that is common to anatomical model data <NUM>. For example, the anatomical model data <NUM> can represent geometry for one or more anatomical structure of a patient by an implicit model. As used herein, an implicit model can represent a geometric structure by a small number of parameters. For example, the implicit model data <NUM> can represent parameters that define the geometry of a physical anatomical structure of a patient that can be generated based on imaging data. In the example of a tubular anatomical structure, the implicit model can include parameters that define the geometry of a centerline and surface of the tubular anatomical structure. As an example, the implicit model can be implemented as a lofted basis (b-) spline.

The imaging data used to generate the implicit model can be acquired by an imaging modality, such as computed tomography (CT), magnetic residence imaging, multi-plane x-ray or the like, which can be configured to provide a three-dimensional image of patient anatomy in a coordinate system of the imaging modality. Since the tracking data <NUM> is generated in a coordinate system of a tracking system that is different from the anatomical model data <NUM>, the registration engine <NUM> can be configured to convert the tracking data into the coordinate system in which the anatomical model data resides or another common coordinate system.

As an example, the anatomical model can be generated based on pre-operative imaging data whereas the tracking data <NUM> can be generated by the tracking system intraoperatively such as to provide real time tracking data corresponding to a position and orientation of each sensor that is monitored by the tracking system. For example, each of the one or more sensors can be attached to an object that is moveable relative to the patient. For example, a sensor detectable by the tracking system can be attached to a guide wire, a catheter, a stent, or other device that may be transluminally moved and positioned within the patient's body. In some examples, each sensor can be detectable by the tracking system to enable tracking in five or six degrees of freedom. Examples of sensors that can be detected by an electromagnetic type of tracking system are commercially available from Northern Digital, Inc. , of Ontario, Canada. Other types of sensors can be used depending on the type of tracking system.

The registration engine <NUM> can include a transformation calculator <NUM> that is programmed to compute a transform to which the tracking data can be applied for generating a corresponding registration matrix <NUM>. For example, the transformation calculator <NUM> can employ a radiographic transform <NUM>, such as can be generated based upon the imaging data that is utilized to construct the anatomical model data <NUM> and the intraoperative imaging data corresponding to the position of the patient during a procedure. The radiographic transform <NUM> thus can be a fixed transform that can be applied to one or more frames to which the tracking data <NUM> is captured over time. The transformation calculator <NUM> can accommodate movement of the patient and/or sensors being monitored by the tracking system. For example, the registration matrix <NUM> can recompute the registration matrix to provide a corresponding transformation for each frame of tracking data to convert the position and orientation of a predetermined location on the object being tracked (e.g., at each sensor location) into the registered tracking data <NUM> that is in the same coordinate system as the anatomical model <NUM>. As disclosed herein, the registered tracking data <NUM> can represent position and orientation for any number of one or more objects in such common coordinate system.

An output generator <NUM> can be configured to generate a graphical visualization based on the registered tracking data and the anatomical model data. The graphical visualization can be provided to an output display <NUM> for viewing by one or more users. The output generator <NUM> further can be programmed to render a graphical visualization of the anatomical structure that is represented by the anatomical model data <NUM>. The one or more points in orientation of the object represented by the registered tracking data can also be rendered in the graphical visualization that is generated.

The output generator <NUM> can also render graphical representation of an object to which the sensor is associated. For example, the sensor can be attached to a catheter or guide wire having one or more parts that can be movable relative to the patient. By attaching the sensor to a predetermined location of such object, the registered tracking data <NUM> can correspond to an identifiable point for the object that can provide an origin for graphically rendering a representation of the object in the output visualization superimposed in conjunction with a rendering of the anatomical structure.

As a further example, the output generator <NUM> can employ other data <NUM> in generating the output visualization. The other data <NUM> can represent a model or other representation form that can drive rendering based on the registered tracking data <NUM>. The other data <NUM> can be stored in memory and accessed by the output generator <NUM> according to a specification of the object to which the sensor is attached. This can be set in response to a user input or can be determined automatically based on data acquired by the system <NUM> or otherwise. For example, the other data <NUM> can include a library of objects (e.g., instruments and/or implantable devices) that can be selectively used. Each object in the library can include a respective model for rendering the object and location of one or more sensors attached at a predetermined location to the object. Such a library of models thus can be constructed for each of the different types and available devices that are being utilized in the system <NUM>. The modeling of each object further can correspond to an implicit object model, such that small number of parameters can define the entire geometry of the corresponding device structure as well as its behavior characteristics.

By way of further example, the other data <NUM> can represent the geometry and behavioral characteristics and a variety of devices. For instance, such geometry and behavioral characteristics can be derived based on CAD modeling that may be provided by a manufacturer of an instrument or implantable device or be determined based upon structural analysis of a given device.

<FIG> depicts an example of a system <NUM> that can be utilized for generating an implicit model such as for a tubular anatomical structure. The system <NUM> includes an anatomical model generator <NUM> that is programmed to generate an implicit model data <NUM> based on anatomical image data <NUM>. The anatomical image data <NUM>, for example, can be acquired by preoperatively for a given patient by an image modality. As an example, the preoperative image data <NUM> can correspond to a preoperative arterial CT scan for a region of interest of the patient, such as can be acquired weeks or months prior to a corresponding operation. Other imaging modalities can be used to provide three dimensional image data <NUM>, such as MRI, ultrasonography, positron emission tomography or the like. Such scans are common part of preoperative planning in a surgical workflow to help size prostheses and to plan surgery or other interventions.

The corresponding image data <NUM> can be stored in memory that can be accessed by or transferred to an intraoperative positioning system (e.g., the system <NUM> of <FIG>). The image data <NUM> can include image data representing pixels (two-dimensional slices) or voxels (three-dimensional image information) corresponding to patient anatomy. The image data <NUM> can also include one or more transformations that can specify a translation of the pixels or voxels from an image space to a corresponding three-dimensional coordinate system. Such transformation can be provided as metadata, for example, as part of the image data <NUM>. Thus the preoperative image data <NUM> contains information sufficient to convert coordinates of points (pixels) or volumes (voxels) of image data in the image space to the corresponding three-dimensional coordinate system corresponding to the imaging modality.

The anatomical model generator <NUM> is programmed to generate the implicit model data <NUM> based on processing the input image data <NUM>. As an example, the anatomical model generator <NUM> can implement image pre-processing <NUM>. The image preprocessing can include automated and/or manual processes, such as to perform background correction, segmentation and thresholding for identification of a corresponding anatomic structure of interest. For example, the anatomical structure can correspond to a major blood vessel as well as one or more branches that may extend from such vessel. For instance, the vessel can correspond to a patient's descending aorta and associated renal arteries as well as other branches thereof. In other examples, the anatomical structure can correspond to an intestinal tract, a portion of a respiratory tract, the patient's digestive tract or other anatomical structures in which objects may be positioned transluminally for a variety of diagnostic and surgical purposes.

The model generator <NUM> can also include a centerline calculator <NUM>. The centerline calculator can be programmed to compute a corresponding centerline for the elongated tubular anatomical structure. As one example, the centerline can be computed as a pixel or voxel thickness extending longitudinally along the central axis of the structure. A corresponding surface boundary of the tubular structure can be computed by a lumen calculator <NUM>. The tubular structure can correspond to a surface of the anatomical structure having a corresponding functional relationship relative to the centerline along the length of the structure as computed by the centerline calculator <NUM>.

A parameter estimator <NUM> can compute a set of model parameters corresponding to the centerline and surface of the lumen structure, which parameter can correspond to the implicit model data <NUM>. The set of parameters can be a small set of parameters such as corresponding to a lofted b-spline (basis spline) function for the elongated anatomical structure. As one example, the anatomical model generator <NUM> can be programmed to compute the implicit model data according to the disclosure of <CIT> entitled Automated Centerline Extraction Method and Generation of Corresponding Analytical Expression and Use Thereof, which is incorporated herein by reference. Another example of generating an implicit model for tubular anatomical structures is disclosed in<NPL>), which is incorporated herein by reference. Other approaches for generating the implicit model data can also be utilized. Other types of geometric representations can also be utilized to provide the implicit model data <NUM>. For example, parameters representing lofted ellipses or triangular meshes can be generated to provide the anatomical model data <NUM> representing the patient's anatomical structure of interest.

<FIG> depicts an example of a subsystem <NUM> that can be utilized for generating a radiographic transformation matrix <NUM>. The radiographic transformation matrix <NUM> can provide a transformation from a preoperative coordinate system to a corresponding coordinate system of a combination marker system <NUM> that can be attached to a patient's body <NUM>. The combination marker system <NUM> can include a plurality of radio opaque objects, such as fiduciary markers, arranged in a pre determined relationship relative to each other. As used herein, radio opaque refers to the inability of ionizing electromagnetic radiation to pass through sufficient to make the objects visible in a corresponding image obtained by an imaging modality <NUM>. Thus, the radio opaque objects can be radiodense materials with respect to the imaging modality <NUM>.

The combination marker system <NUM> can also include one or more sensors having a predetermined position relative to the radio opaque fiduciary markers. The combination marker system can include any number of one or more combination markers that may be attached to a patient's body, such as to a patient's torso (e.g., to a patient's back) at a location that is close to the anatomical structure of interest. By way of example, the marker system <NUM> can include two or more (e.g., three) markers that can be placed to the region of tracking interest. In some examples, the marker system <NUM> can include a plurality of spaced apart combination markers. For an example of a procedure in which an endovascular device is to be positioned or tracked within a descending aorta, the tracking system can be placed close to where the renal artery is attached to the aorta. Other locations could be utilized depending upon the region of tracking interest.

An example of a combination marker system <NUM> is demonstrated in the examples of <FIG>. In <FIG>, a single combination marker system <NUM> is demonstrated. In this example, the combination marker system <NUM> includes a plurality of radio opaque fiduciary structures <NUM> having a predetermined geometry and a range with a predetermined geometric relationship relative to each other. For example, the radio opaque objects <NUM> can be implemented as spheres such as having a predetermined angular orientation and spatial arrangement (e.g., configured as a scalene right triangle). Thus, each of the radio opaque objects <NUM> can be identified in a corresponding radiograph (e.g., obtained intraprocedurally via a CT scan, bi-plane x-ray or the like). As mentioned, the type of material utilized for the respective objects <NUM> can vary depending upon the imaging modality <NUM> being utilized. The combination marker <NUM> also includes one or more sensors <NUM> detectable by the tracking system. Each sensor <NUM> can be dimensioned and configured to have a predetermined spatial relationship (e.g., distance and angle) relative to the geometry of the respective radio opaque objects <NUM>. For example, the sensor <NUM> can include an elongated sensor that is positioned at the origin of a pair of axes that can be computed based on the geometric relationship of the objects <NUM>. Additionally, the sensor itself <NUM> can extend along an axis <NUM> or be parallel to an axis defined by the respective radio opaque objects <NUM>.

<FIG> demonstrates example geometry for a corresponding coordinate system <NUM> that can be determined for the combination marker <NUM> (<FIG>). In the example of <FIG>, the coordinate system <NUM> includes X and Z axis lying in the plane of the triangle (of the page) with the corresponding Y axis extending perpendicular to the plane (e.g., the page in which the figure is demonstrated). As demonstrated in <FIG>, a sensor body <NUM>' is shown to extend along the Z axis of the coordinate system <NUM>. A center of a body of the sensor <NUM>, demonstrated at <NUM>, is at the origin of the X, Y, and Z axes. As disclosed herein, the sensor <NUM> can be configured as an elongated coil that extends axially along a length of the Z axis, and is detectable by the tracking system. For example, the sensor <NUM> can be implemented as a coil of the electrically conductive material within the combination marker system <NUM> with a center of the sensor coil located an origin of a corresponding coordinate system <NUM>.

<FIG> demonstrates a marker pad device <NUM> that can help protect the patient's skin from the hard surface of the combination markers. One or more of the combination marker systems <NUM> (<FIG>) can be implemented within the pad device <NUM> to enable co-registration between the domain of the tracking system and domain of the intraoperative image data, such as disclosed with respect to <FIG>. For example, the pad <NUM> can contain a gel or other soft flexible material to provide a cushion around each combination marker.

In the example of <FIG>, the pad device <NUM> includes three combination markers <NUM> distributed in a spaced apart arrangement with respect to each other. The pad device <NUM> can be configured to hold each of the combination markers in a substantially fixed spatial relationship while allowing flexibility to accommodate patient movement. Each of the combination markers <NUM> also includes a corresponding connection <NUM> that can be connected to the tracking system. For example, the tracking system can be implemented as an electromagnetic tracking system, such as disclosed herein, and each of the connections <NUM> thus can provide an electrical signal to the tracking system representing induced current in response to an electromagnetic field that is generated by a transmitter of the tracking system and detected by the respective sensing coil. In other examples, the connections can be wireless and the sensors can communicate via RF or other wireless technology. The tracking system can convert the sensor signals into corresponding tracking system data, which can be analyzed as disclosed herein. For example, the tracking data can include a position and orientation of a point in a three-dimensional coordinate space with respect to the transmitter of the tracking system for each combination marker <NUM>.

Returning to <FIG>, an intraoperative imaging modality <NUM> can be utilized to generate intraoperative image data <NUM> corresponding to the patient geometry for at least the region of tracking interest in the patient's body as well as the combination marker system <NUM> that has been attached to the patient's body. As mentioned above, the system <NUM> can also utilize the preoperative anatomical image data <NUM>, which can be obtained using the same or a different type of imaging modality that is utilized for the intraoperative imaging modality <NUM>.

An image space transformation calculator <NUM> can be configured to register the intraoperative image data <NUM> to a corresponding coordinate system of the preoperative image data <NUM>. The computation by the transformation calculator <NUM> can be facilitated based on transform metadata provided by the imaging modality for converting the image pixels or voxels to points or volumes within a coordinate system of the imaging modality <NUM>. The image space transformation calculator <NUM> thus can generate a corresponding image space transformation matrix <NUM> that provides a transformation of the intraoperative image data <NUM> into the preoperative image data <NUM> (<FIG>). The image space transformation matrix <NUM> can also be utilized to enable the markings or other information from a preoperative scan to be overlaid on corresponding intraoperative fluoroscopy images obtained during a corresponding procedure. The intraoperative image data <NUM> and the preoperative image data <NUM> can be stored in memory such as in one or more non-transitory computer readable media. In some examples the memory can be accessible via a network or the memory can be a portable storage device such as a solid state memory device (e.g., flash drive or the like).

The subsystem <NUM> also includes a marker registration engine <NUM> programmed to generate an image-marker transformation matrix <NUM> based on the intraoperative imaging data <NUM>. The image-marker transformation matrix encodes the location of the combination marker system <NUM> (or at least a portion thereof) along with an orientation of the marker provided therein based on the intraoperative image data <NUM>. The marker registration engine <NUM> can include an image processing component <NUM> that is programmed to process the intraoperative image data <NUM> for identifying the radio opaque fiduciary markers (e.g., markers <NUM> of <FIG>). The image preprocessing <NUM> can include thresholding and image segmentation to produce an edge detected data set of pixels and/or voxels representing boundaries of each radio opaque fiduciary marker <NUM>.

A marker locator <NUM> can compute a corresponding distance transformation of the edge detected data set. As an example, the marker locator <NUM> can compute a distance for each pixel or voxel in the image data <NUM> relative to the edge detected data set. For instance, a partial transformation can be utilized such that the computations only compute the distance for voxels at the edge or within the boundary of each fiduciary marker. A corresponding computed distance value can be stored for each voxel. For example, the distance transformation can provide a voxel set that contains for each respective voxel a distance to a nearest edge of the radio opaque fiduciary marker such as can correspond to a sphere. The marker locator <NUM> can analyze the distance transform data set along a corresponding axis (e.g., extending from feet to head or anterior to posterior or left to right for the presence of a sphere at each respective location.

A marker evaluator can be programmed to evaluate whether a particular location in a voxel data set is the center of a sphere such as by calculating a surface integral of the distance transformation over the surface of a respective sphere in such center. If the sphere is determined to be present for the voxels being evaluated, the surface interval should approximate zero. Accordingly, a threshold can be set (e.g., a tunable threshold) to compare relative to the surface integral to identify whether or not the location should be recorded as a sphere. The marker evaluator thus can test all potential locations for radio opaque markers and compare the points to ascertain the marker does in fact exist at such location. Each cluster having a value that is below the tunable threshold can be identified as a radio opaque marker and a mean of such locations can in turn be utilized for identifying a corresponding radio opaque marker.

With reference back to <FIG>, after all such spheres have been identified based on intraoperative image data, each possible grouping of the radio-opaque markers <NUM> can be evaluated to locate each composite marker. For example, the distances between each of the respective markers <NUM> (distance between centroids thereof) can be tested to determine if the distances match the predetermined lengths of legs for the triangle formed by the radio opaque markers in the physical design of the combination marker. In the example of <FIG>, each of the markers <NUM> is identified as markers 108a, 108b, and 108c.

In the example of <FIG>, the marker registration engine <NUM> includes a transformation matrix calculator <NUM>. The transformation matrix calculator <NUM> is programmed to compute a coordinate system <NUM> for each combination marker. For example, an origin <NUM> of the coordinate system can reside at a centroid of a triangle formed between markers 108a, 108b and 108c. As mentioned above, a center of the sensor (detectable by the tracking system) <NUM> can be located at the origin of the coordinate system <NUM> or have another predetermined spatial relationship with respect to the markers 108a, 108b and 108c.

The transformation matrix calculator <NUM> can also be programmed to compute the corresponding image-marker transformation matrix <NUM>. The transformation matrix calculator <NUM> can compute the transformation matrix <NUM> as a translation component to encode the location of the centroid of the triangle and include a rotation component that encodes the orientation of the respective marker <NUM>. The rotation can correspond to a change of basis function, for example. For the coordinate system <NUM>, an X basis vector can represent the normalized vector for the sphere by to the marker 108a. The Z basis vector can correspond to the normalized vector marker extending from 108b to marker 108c. The Y basis vector can correspond to the process product of the Z and X basis vectors. After the transformation matrix calculator <NUM> computes the Y vector, each of the respective X and Z vectors can be adjusted, if necessary, to ensure that all vectors are mutually orthogonal. The output of the corresponding coordinate system can be provided as the image-marker transformation matrix <NUM>.

The system <NUM> further includes a transform calculator <NUM> that is programmed to generate the radiographic transformation matrix <NUM> based on the image-marker transformation matrix <NUM> and the image space transformation matrix <NUM>. The transform calculator <NUM>, for example, can compute the radiographic transformation matrix by concatenating the image-marker transformation matrix <NUM> with an inverse of the image space transformation matrix <NUM>. As a result, the radiographic transformation matrix <NUM> can represent a transformation from the origin of the preoperative image scan to the position and orientation of the combination marker.

<FIG> depicts an example of a system <NUM> for translating tracking data <NUM> that is acquired from a tracking system <NUM> into corresponding registered tracking data <NUM>, which is registered in a common coordinate system with an implicit anatomical model (e.g., as defined by model data <NUM> of <FIG>). As disclosed herein, the common coordinate system can represent a coordinate system for image data that has been acquired preoperatively relative to the tracking data that is generated intraoperatively by the tracking system <NUM>. As an example, the tracking system <NUM> can generate the tracking data <NUM> to represent a position and orientation of one or more sensors <NUM> being positioned within a patient's body <NUM>.

A combination marker system <NUM> (e.g., one or more combination marker <NUM> of <FIG>) can be attached to the patient's body <NUM>. In the example of <FIG>, the combination marker system <NUM> can include one or more sensors that provide respective signals to the tracking system indicative of a location of the combination marker within the coordinate system of the tracking system <NUM>. One or more other sensors can be affixed relative to an object that is movable within the patient's body <NUM> for identifying a location of such sensor in the coordinate system of the tracking system. Each such sensor <NUM> thus can also provide a signal to the tracking system based on which the tracking system can compute corresponding tracking data for such sensor. As mentioned, the tracking data <NUM> represents a position and orientation of each respective object sensor <NUM> as well as marker sensors within the combination marker system <NUM>.

The tracking system <NUM> can provide the tracking data with an output sample rate to enable computation of real time positioning and visualization of the object to which the sensor is attached as well as the combination marker system. Since the combination marker system <NUM> is attached to the patient's body <NUM>, the coordinate system of the tracking system <NUM>, the registered tracking data <NUM> is consistently computed to accommodate for movement in the patient's body <NUM>. For example, the tracking system <NUM> can include a transmitter (e.g., an electromagnetic field generator) that provides a non-ionizing field, demonstrated at <NUM>, which is detected by each sensor <NUM> to provide a corresponding sensor signal to the tracking system. An example tracking system <NUM> is commercially available from Northern Digital, Inc. , of Ontario, Canada. The tracking system <NUM> can provide the tracking data <NUM> at an output sample rate (e.g., sixty samples per second) for each sensor sufficient to enable substantially real time determination of sensor location (e.g., to provide a vector describing sensor position and orientation). The tracking processing subsystem thus can process each frame of tracking data such that the registered tracking data can likewise represent real time tracking data acquired by the tracking system that can be registered into the coordinate system of the anatomical model and rendered as a graphical representation, as disclosed herein.

The marker identification function <NUM> can be configured to identify each composite marker (e.g., the marker <NUM>). For instance, the marker identification function <NUM> is programmed to associate a tracking system sensor with a respective combination marker. For example, the marker identification function <NUM> can include a match calculator <NUM> programmed to compute a distance between the respective markers in the coordinate space (e.g., electromagnetic space) of the tracking system and in the intraoperative image coordinate system, as represented by intraoperative image data <NUM>. For example, the match calculator <NUM> can be programmed to compute a difference between the distance between two markers in tracking system coordinate system and in the intraoperative image coordinate system. The match calculator <NUM> can also compute a difference between angles between two markers Z axis in both the tracking system coordinate system and the intraoperative image coordinate system.

Based upon the computations by the match calculator, a scoring function <NUM> can assign a score to represent the quality and results of the matching calculation. For example, the scoring function <NUM> can assign a score to each combination marker can be the sum of scores computed based upon each calculation performed by the match calculator <NUM>. The marker identification function <NUM> can in turn identify which tracking system marker corresponds to which radio opaque marker from the image operative image data <NUM>. The results of the scoring and analysis by the marker identification function <NUM> can be utilized to generate a corresponding tracking system transformation.

A composite transformation calculator <NUM> can compute a corresponding composite transformation matrix for each combination marker in the combination marker system <NUM>. As mentioned, the combination marker system can include one or more composite markers each of which can result in a corresponding composite transformation matrix <NUM>. The composite transformation calculator <NUM> can compute the composite transformation matrix <NUM> based upon the pre-computed radiographic transformation matrix <NUM> (e.g., corresponding to the radiographic transformation matrix <NUM> of <FIG>) and the tracking system transformation information provided by the marker identification function <NUM>. For example, the calculator <NUM> can multiply the tracking system transformation information by the inverse of the radiographic transformation matrix <NUM> to generate the corresponding composite transformation matrix from the coordinate system of the tracking system <NUM> to the coordinate system in which the anatomical model resides.

As disclosed herein, in some examples the coordinate system of the anatomical model can correspond to the coordinate system of the preoperative image data. In examples where multiple combination markers are utilized in the combination marker system <NUM>, a corresponding composite transformation matrix can be computed for each combination marker. An aggregation function <NUM> can in turn compute a corresponding registration matrix <NUM> such as corresponding to the mean or average of the all combination marker composite transformation matrixes <NUM>. The corresponding tracking data for a given frame for which the registration matrix <NUM> has been computed can in turn be multiplied by the registration matrix <NUM> to provide the corresponding registered tracking data for the given frame of such tracking data. As mentioned, a corresponding registration matrix <NUM> can be computed for each frame of tracking data such that the registered tracking data can be generated on a frame by frame basis, such as for tracking data acquired over one or more sequences of frames.

<FIG> depicts an example of position correction function <NUM> that can be implemented (e.g., by output generator <NUM> of <FIG> or output generator of <FIG>) to correct a position of the anatomical structure represented by the implicit anatomical model (e.g., based on corresponding to anatomical model data <NUM> of <FIG>). The position correction function <NUM> can be programmed to implement deformation corrections that can occur in response to insertion of an object (e.g., an instrument such as a catheter or guide wire) into an elongated anatomical structure such as a patient's vasculature. As just described with respect to <FIG>, the position of sensors attached to an intravascular instrument can be constantly monitored and updated as part of the real time registered tracking data.

As the instrument or other object is moved within anatomical region of interest, the anatomical structure may deform and such deformation can be identified and utilized to modify the anatomical model data that is utilized to generate a visualization of the anatomic structure. For example, the position correction function <NUM> can include a position anomaly detector <NUM> that is programmed to detect a condition when an adjustment to the anatomical model is necessary to provide a visually accurately representation of the instrument within the anatomical structure. The position correction function <NUM> thus can employ a deformation evaluator <NUM> to analyze the registered tracking data for the instrument relative to the anatomical model data to determine whether or not a deformation condition exists that requires correction.

For example, the registered tracking data <NUM> can be utilized to construct a visualization of an instrument or other object carrying one or more sensors (e.g., sensors <NUM> detectable by tracking system <NUM> of <FIG>) that are moving within the anatomic structure of the patient's body. In this example, the anatomical model data <NUM> can correspond to a fixed representation of the patient's anatomic structure that can be adjusted spatially according to a deformation parameter <NUM> of a deformation model <NUM>. Thus, by adjusting the deformation parameter a desired amount of deformation can be imposed on the model such that the object resides within the patient's anatomic structure, a resulting output visualization.

By way of example, the position anomaly detector <NUM> can detect if the deformation evaluator determines that the object represented by the tracking data <NUM> is outside a volume of the anatomic structure provided by the anatomical model data <NUM>. If the position of the object represented by the tracking data resides within the volume, the position anomaly detector can determine that the vessel is not deforming such that the anatomical model data <NUM> can remain unchanged by the position correction function <NUM>. If the position anomaly detector <NUM> determines that the object represented by the registered tracking data <NUM> is outside the volume of the anatomical structure represented by the anatomical model data <NUM> the position anomaly detector can instruct the deformation model <NUM> that the anatomical structure is deforming.

The deformation model <NUM> can include one or more parameters <NUM> that can be modified by a parameter adjust function <NUM> to adapt the shape of the elongated anatomical structure based on a corresponding amount of deformation that is determined. The deformation model <NUM> can include a deformation calculator <NUM> that is programmed to determine compute positions of the object and the boundary (e.g., surface) of the anatomical structure. A deformation evaluator <NUM> can compare the computed positions and determine if the computed position information indicates that the object represented by the registered tracking data is outside the structure represented by the anatomical model. In response to the deformation evaluator <NUM> determining that deformation exists, the parameter adjust function <NUM> can adjust the deformation parameter <NUM>. The deformation parameter <NUM> can be applied to the anatomical model data to implement a corresponding adjustment to the anatomical model <NUM>. For example, a model adjustment function <NUM> can include a translation component <NUM> and a rotation component <NUM> for adjusting different components of the anatomical model according to a value of the deformation parameter.

By way of example, <FIG> demonstrate operations that can be performed by the model adjustment function <NUM> to implement deformation of an implicit model, which is demonstrated as a vessel model <NUM>. While the examples of <FIG> are described in the context of a vessel model <NUM>, other types of anatomical structures could be deformed in a similar manner. In the example of <FIG>, the deformation can be implemented by perturbing the vessel model to more closely follow a straight path corresponding to an elongated instrument. As demonstrated in <FIG>, the straight path can be defined by an axis <NUM>, such as a straight line segment whose endpoints are the centroids of the first and last slices of the vessel model <NUM>. Other shapes of paths, such as a curved path or a computed shape of the object could also be utilized as the goal towards which the model <NUM> is deformed.

For the example where the implicit model is a lofted b-spline, the model adjust function <NUM> can perform the deformation operation on the geometric knots which define each cross-sectional slice of the implicit model. As discussed with respect to <FIG>, when the implicit model is computed, the knots correspond to the actual geometry of the surface derived from the image data. By transforming these knots, the model adjustment function <NUM> of the correction method <NUM> can adjust the shape of the vessel model <NUM>. For any given slice, all knots are transformed together to retain the correct cross-sectional shape, and each slice can undergo both translation and rotation.

<FIG> demonstrates an example of how translation can be computed (e.g., by translation function <NUM>) for a given slice <NUM>. The translation function <NUM> can compute the slice's centroid C. The translation function <NUM> can also compute a point P on the axis <NUM> that is nearest C. The translation function <NUM> can compute a vector CP which is multiplied by the deformation parameter <NUM>. This vector is then added to each geometric knot in the slice. The effect is that slice is translated along the vector from C to P by an amount commensurate with the computed deformation parameter <NUM>.

With reference to <FIG> and <FIG>, the rotational adjustment function <NUM> is programmed to compute a rotational deformation for each slice in the model <NUM> based on the deformation parameter <NUM>. The rotational adjustment function <NUM> can rotate the slice <NUM> so that its plane lies more perpendicularly to the axis <NUM>, with the relative perpendicularity depending on (e.g., being proportional to) the deformation parameter <NUM>. The rotational adjustment function <NUM> can compute a unit normal vector N that is perpendicular to the plane of the slice <NUM>. The rotational adjustment function <NUM> can also compute unit vector T parallel to the axis <NUM> and extending through the centroid of such slice.

The rotational adjustment function <NUM> can compute a cross product N x T and the direction of the cross product yields an axis of rotation. The rotational adjustment function <NUM> can compute the arc-cosine of the magnitude of the cross product to determine an angle of rotation. If each point on the slice were rotated about this axis by this angle, the slice would become perpendicular to the axis. For each slice in the model <NUM>, the rotational adjustment function <NUM> thus is programmed to multiply the computed angle of rotation by the deformation parameter <NUM> and then perform the rotation as a fractional part of the computed angle of rotation based on deformation parameter. <FIG> demonstrates an example of the vessel model <NUM> after both translation and rotation have been performed on the model.

<FIG> depicts examples of a vessel model demonstrating different amounts of deformation, demonstrated at <NUM>, <NUM>, <NUM>, <NUM> and <NUM>. For instance, each example model <NUM>, <NUM>, <NUM>, <NUM> and <NUM> can be generated based on applying different values of deformation parameters that have been computed (e.g., by parameter adjust function <NUM>), such as disclosed herein. <FIG> demonstrates vessel models <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> computed for different deformation parameter and including a surface <NUM> that has been rendered for each respective model such as by connecting each respective slice, such as by connecting each slice with a cylindrical segment.

<FIG> depicts an example of a visualization system <NUM> that includes an output generator <NUM> programmed to generate visualization data <NUM>, which can be provided to a display to render a corresponding graphical representation. The output generator <NUM> can generate the visualization data <NUM> based on registered tracking data <NUM>, anatomical model data <NUM> and object model data <NUM>. The registered tracking data <NUM> can correspond to the registered tracking data <NUM> of <FIG> as well as the registered tracking data <NUM> of <FIG>. The anatomical model data <NUM> can correspond to the anatomical model data <NUM> of <FIG> as well as the anatomical model data <NUM> of <FIG>. The object model data <NUM> can correspond to another implicit model that has been generated as corresponding to the object. As disclosed herein, one or more sensors can be affixed to the object to enable tracking of its location by a tracking system, such as tracking system <NUM> of <FIG> that generates the tracking data from which the registered tracking data <NUM> is computed.

By way of example, the object model data <NUM> can correspond to an analytical or parametric representation of a surgical instrument, which may be a generally rigid surgical instrument or an articulated instrument that includes a flexible tip such as wires, catheters and the like. Accordingly the complexity of the model data <NUM> and the corresponding implicit model that it defines can vary according to the type of instrument or other object that is being tracked within the patient's anatomy. In addition to parameterizing the geometry of the object, the object model data <NUM> can also be configured to model other properties of the object (e.g., resilience and/or ductility).

The output generator <NUM> includes a rendering method <NUM> programmed to produce a three-dimensional plot corresponding to the visualization data <NUM> based on the input data <NUM>, <NUM> and <NUM>. Various types of rendering software (e.g., commercially available or proprietary) can be utilized and implemented as the rendering method <NUM> and can vary according to the type of models generated for use by the output system <NUM>.

The output generator <NUM> can also include display controls <NUM> that can control the output that is provided intraoperatively. The display controls <NUM> can be configured to selectively generate any number of one or more displays concurrently on one or more screens, each of which can include a different view of the object and the anatomic structure. The respective views can be selected automatically such as by default parameters or it can be adjusted in response to the user input just as can be provided a user interface <NUM>. This display controls <NUM> can further control a viewing angle for each of the visualizations of the anatomical model and the object that are presented to the user. Since the structures in each visualization are virtual renderings based on implicit models, the output visualization is not constrained to any particular viewing angle or type of visualization.

In some examples, the display controls <NUM> can compute and display task-specific visualizations, such as may include an optimal view for a particular task (for example, cannulating a renal artery). For example, when cannulating a vessel, it is useful to visualize the vessel and wire without distractions or obstructions. The output generator is able to create this visualization since each vessel and each device are virtual renderings. Additionally, because each the models <NUM> and <NUM> are easily separated into its constituent parts, other items can be effectively removed from the display and only show the clinician the pertinent geometry and telemetry for the task at hand. Thus, the display controls can request the rendering method <NUM> to produce nearly any visualization in two- or three-dimensional space, which can be rendered rapidly.

The output generator <NUM> can also include a position correction function <NUM> such as corresponding to the position correction function <NUM> disclosed with respect to <FIG>. Thus, the anatomical model data <NUM> that is utilized by the rendering method <NUM> can operate on a position-corrected version of the anatomical model data that includes a corresponding deformation model for adjusting the position (e.g., translational and rotational position) of the model according to a computed deformation parameter.

In the example of <FIG>, the rendering method <NUM> includes a centerline render <NUM> programmed to plot a centerline of the anatomic structure based on the anatomical model <NUM> which includes parameters to define the geometry of the anatomical structure (e.g., an elongated tubular structure such as a vessel or intestine). For example, the anatomical model data <NUM> can be stored as a spline curve corresponding to a series of geometric knots. The centerline render <NUM> can evaluate the curve by calculating spline control points from the geometric knots. The centerline render <NUM> can in turn evaluate the spline equation using the computed control points for a given parameter value such that the centerline is a function of a single parameter.

As an example, the centerline render function <NUM> can compute the centerline as a function of a single parameter (u) that goes from zero to one and varies along a path corresponding to the axis of the tubular structure taken by selecting a spacing of the parameter u. The value can be computed at each respective spacing and the rendering method <NUM> can plot the curve as a series of corresponding line segments drawn between the values of the parameter u. For example, if a spacing of <NUM> is selected, the curve corresponding to the centerline can be evaluated at u = <NUM>, u = <NUM>, u = <NUM>, etc. and the corresponding points for each value of u can be connected to provide a plot corresponding to the centerline of the anatomical model.

The rendering method <NUM> can also include a surface render function <NUM> that can produce a plot for a surface of the anatomical structure based on the implicit model defined by the model data <NUM>. As an example, the surface render function <NUM> can compute the surface as a function of two variables, such as the variable u, which extends along the axis of the tubular structure and another parameter (v) which varies as one travels tangentially around the surface. As disclosed herein, the anatomical model data <NUM> can store the surface information as a series of slices in which each slice can be represented by a series of geometric knots of an elongated tubular structure.

As a further example, the surface render function <NUM> can compute a location of a surface point for any given (u, v) parameter tuple as follows. Each slice of the spline can be evaluated a given v parameter using the same technique as for the centerline. The result can be a series of points all on the same tangential location of the surface. Such points serve as a series of geometric knots for a new one-dimensional spline, which can then be evaluated at the given u parameter. The surface render function <NUM> can visualize the surface by evaluating the parameters to generate triangles that tessellate the surface. Such triangles can be rendered efficiently with various computer graphics, hardware and software. The surface render function <NUM>, for example, can employ two spacings Su and Sv which corresponds to one spacing in the u direction and one in the v direction, respectively. For example, the surface render function <NUM> can iterate of over the surface, plotting triangles such as follows:.

While triangles are demonstrated above, other polygonal shapes could be used.

The rendering method <NUM> can also include an object render function <NUM> to render a graphical representation of the object based on the object model data <NUM> and the registered tracking data <NUM>. As disclosed herein, there can be one or more different objects that can be rendered concurrently with respect to the patient geometry, and each object has its own model provided by the model data <NUM>. The registered tracking data <NUM> represents a point of one or more sensors in three-dimensional space corresponding to the same coordinate system in which the anatomical model has been registered. The object render function <NUM> thus can be programmed to generate a graphical representation for each object depending on the location of the object defined by the registered tracking data.

By way of example, the object render function <NUM> can plot rigid objects (and parts of objects that are rigid) by applying their transformation matrix multiplied by the overall registration matrix. For the case of articulated objects (e.g., instruments with a flexible tip, such as wires and catheters), the object render function can be programmed to plot different parts of the structure separately.

As an example, the object render function <NUM> can render an elongated instrument in discrete parts. <FIG> depicts an example of a rendering of an elongated instrument (e.g., a catheter) <NUM> that can be generated by object render function <NUM> based on tracking data provided for multiple object sensors. In the example of <FIG>, the rendering of the instrument <NUM> includes a tip portion <NUM>, a distal body portion <NUM>, a connection portion <NUM>, a proximal body portion <NUM>, a tail portion <NUM>. The rending <NUM> of such discrete portions, based on tracking data for two or more sensors can thus represent the shape of the instrument. For example, one sensor can be affixed to an instrument in its distal body and another sensor can be affixed to the instrument in its proximal body.

As a further example, the object render function <NUM> can render the tip <NUM> as a rigid cylinder translated along the +Z axis of the distal sensor such that it resides just distal of the distal body. The distal body <NUM> can be rendered as a rigid cylinder at the location of the distal sensor (e.g., based on tracking data for such sensor). The object render function <NUM> can also render the proximal body <NUM> as a rigid cylinder at the location of the proximal sensor (e.g., based on tracking data for such sensor). The connection <NUM> can be rendered as two circles (e.g., one at the proximal tip of the distal body and one at the distal tip of the proximal body), which can be lofted by interpolating between the respective circles, such as by lofting with Bezier curves. The object render function <NUM> can render the tail as a rigid cylinder translated along the -Z axis of the proximal sensor such that it resides just proximal of the proximal body. The lengths, radii, and colors of each part can be selected according to the objects actual physical appearance. In some situations, non-cylindrical shapes could also be used by the object render, such as when appropriate to further match the geometry of the object being rendered.

The output generator <NUM> can also include a guidance generator <NUM> programmed to generate user perceptible guidance that can be based on the registered tracking data <NUM>, corresponding to the location of the object, and the patient's anatomy. Some guidance can be static whereas other guidance can be dynamic. For example, the guidance generator <NUM> can include an object position evaluator <NUM> that is programmed to evaluate the position of the object based upon the registered tracking data <NUM> relative to the position of one or more anatomical feature that can be specified in or determined from the anatomical model data <NUM>. Such features, for example can include bifurcations in a tubular structure or other anatomical landmarks (e.g., a target anatomical site). The guidance provided relative to such anatomical features can include a position of the feature or trajectory path along which an object may be advanced to arrive at such position.

As an example, the object position evaluator <NUM> thus can compute a distance between a selected feature and a point along the object (e.g., corresponding to a distal tip of an instrument or other predetermined location along the object). The object position evaluator <NUM> can utilize the distance to ascertain the relative proximity between the object and the anatomical feature of interest. Based upon the evaluation, the guidance generator <NUM> can provide a visual indicator, an audible indicator or a combination of audible and visual indicators. For example, an audible indicator can provide a series of beeps or tones that increase in frequency as a function of decreasing distance between the object and the location of the target feature. The guidance generator <NUM> can specify a color code to be applied to a selected feature of the output visualization, such as green to indicate that the position is on target, yellow to indicate a deviation within a predetermined parameter or red to indicate that the position is outside of expected parameters.

As a further example, the guidance generator <NUM> can also include a directional indicator <NUM> that can produce virtualized graphical indictor showing a direction that a distal tip of the object (e.g., a catheter or wire) is oriented. The graphical indicator can be rendered as a series of short lines translated along a given sensors positive Z axis. The visual indicator thus can provide an easy way to determine whether the object is aligned with a given part of the anatomical structure to facilitate advancing the object through or into a target branch vessel. The appearance of the guidance further will vary depending on the particular viewing angle that is being produced.

The guidance generated at <NUM> can also provide information to graphically differentiate anatomical locations or other target sites, such as by using different color codes for different structures of interest. For example, the guidance generated at <NUM> can render a perimeter of the ostium of each branch vessel as a thick annular line that appears surrounding the entrance to a corresponding branch. Those skilled in the art will understand and appreciate for the guidance generator <NUM> can provide additional feedback to the user. For example, when the tip of the object gets within a predetermined distance of an ostium, which has been indicated by a graphically differentiated ring at the branch, the ring can change colors as the tip gets within a predetermined distance.

<FIG> depict examples of visualizations that can be generated by the output generator <NUM> (also corresponding to output generator <NUM> of <FIG>). While the examples of <FIG> are demonstrated in the context of a major blood vessel, namely the descending aorta, the systems and methods disclosed herein are equally applicable to generate visualizations for other anatomical structures and objects that can be positioned in the body relative to such structures.

<FIG> demonstrates a virtualized output visualization <NUM> that can be generated by an output generator. The visualization <NUM> includes a centerline <NUM> and a surface <NUM> of the vessel that can be rendered based on an implicit anatomical model of patient geometry as disclosed therein. The visualization <NUM> also can include a branch extending laterally from the main branch. In the example of <FIG>, the object is demonstrated as a rendering of flexible catheter <NUM> having a tip that is approaching an ostium <NUM>. As demonstrated in <FIG>, the shape of the catheter <NUM> substantially matches the shape of the physical tool since the shape in the virtual rendering is provided (e.g., by object render function <NUM> of <FIG>) based on tracking data generated for two or more sensors disposed on the physical tool. The visualization of <FIG> also illustrates an axial guidance feature <NUM> extending from the tip of the catheter, such as can be rendered (e.g., by guidance generator <NUM> of <FIG>) as projecting outwardly from the Z axis of the object. As shown, the axial guidance feature <NUM> demonstrates the tip is heading in a correct direction for insertion through the ostium <NUM> of the adjacent branch.

<FIG> demonstrates another example of an output visualization of a vessel <NUM> that includes a centerline <NUM> and surface <NUM> that can be rendered from an anatomical model data. The example of <FIG> demonstrates a plurality of branches <NUM>, <NUM> and <NUM> that extend outwardly from the main branch. A flexible elongated object, such as a catheter (or other instrument) <NUM> is also rendered in the visualization based on registered tracking data and a corresponding object model. The catheter <NUM> is demonstrated as advancing towards an ostium <NUM> of the branch <NUM>, and the ostium changes color providing guidance that the catheter is on target, for example.

<FIG> demonstrates an output visualization <NUM> that includes a plurality of different windows from different orthogonal views in a corresponding coordinate system. In each of the view in the example of <FIG>, the patient anatomy and object is generated concurrently based on anatomical model data and based on a corresponding sample or frame of registered tracking data. As disclosed herein, each of the views can be user selectable views in response to a user input. The size of views and the information presented can be varied automatically and/or in response to user input. <FIG> depicts an example of another set of output visualizations that can be generated in a multiport format concurrently based upon anatomical model data and registered tracking data.

In view of the foregoing structural and functional features described above, methods that can be implemented will be better appreciated with reference to <FIG>. While, for purposes of simplicity of explanation, the method of <FIG> is shown and described as executing serially, it is to be understood and appreciated that the present disclosure is not limited by the illustrated order, as some aspects could, in accordance with the present disclosure, occur in different orders and/or concurrently with other aspects from that shown and described herein. Moreover, not all illustrated features may be required to implement a method in accordance with an aspect of the present disclosure. The methods or portions thereof can be implemented as instructions stored in a non-transitory storage medium as well as be executed by a processor of a computer device or special purpose computer device (e.g., a dedicated computer or workstation) to access data sources and perform the functions disclosed herein, for example.

<FIG> depicts an example of a method <NUM> that can be implemented to facilitate intraoperative positioning and guidance. At <NUM> an implicit model for the geometry of a patient's anatomy can be generated (e.g., by model generator <NUM> of <FIG>) based on image data as disclosed herein. The anatomical model and corresponding image data can be stored in memory.

At <NUM>, tracking data can be generated (e.g., by tracking system <NUM> of <FIG>) to provide an indication of location for an object. The tracking data can be stored in memory for processing. As disclosed herein, the object can be an instrument or other device that is moveable intraoperatively within a patient. For example, the tracking data can be generated in response to signals provided by one or more sensors carried on the object that is being positioned intraoperatively in the patient (e.g., transluminally or endovascularly). As disclosed herein, the tracking data can be generated for the object in the absence of ionizing radiation, which is in contrast to conventional x-ray fluoroscopy.

At <NUM>, the tracking data and the patient implicit specific model can be registered (e.g., by registration engine <NUM> of <FIG>) in a common three-dimensional coordinate system. For example, the coordinate system can be the coordinate system of the preoperative image based on which the implicit model for the anatomical structure of the patient has been generated.

At <NUM>, an output visualization can be generated (e.g., by output generator <NUM> of <FIG> or generator <NUM> of <FIG>) to represent a location of the object relative to the geometry of the anatomical structure. For example, the output visualization can represent a position, orientation and shape of the object in three-dimensional space based on multi-sensor tracking data that has been registered into a common coordinate system with the implicit anatomical model (e.g., corresponding to a preoperative image space). The generating tracking data, registration of the tracking data and implicit model and generating of the output visualization can be repeated over time in response to tracking data that is generated. In this way the output visualization can be dynamically updated (e.g., in real time), such as according to an output sample rate at which the tracking data is generated. Accordingly, the output visualization can be generated in substantially real time to facilitate positioning and guidance of the object. The output visualization can include any number of concurrently generated views, which can be modified (e.g., in response to a user input or deformation correction), such as disclosed herein.

In view of the foregoing structural and functional description, those skilled in the art will appreciate that portions of the systems and method disclosed herein may be embodied as a method, data processing system, or computer program product such as a non-transitory computer readable medium. Accordingly, these portions of the approach disclosed herein may take the form of an entirely hardware embodiment, an entirely software embodiment (e.g., in a non-transitory machine readable medium), or an embodiment combining software and hardware. Furthermore, portions of the systems and method disclosed herein may be a computer program product on a computer-usable storage medium having computer readable program code on the medium. Any suitable computer-readable medium may be utilized including, but not limited to, static and dynamic storage devices, hard disks, optical storage devices, and magnetic storage devices.

Certain embodiments have also been described herein with reference to block illustrations of methods, systems, and computer program products. It will be understood that blocks of the illustrations, and combinations of blocks in the illustrations, can be implemented by computer-executable instructions. These computer-executable instructions may be provided to one or more processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus (or a combination of devices and circuits) to produce a machine, such that the instructions, which execute via the processor, implement the functions specified in the block or blocks.

Claim 1:
One or more non-transitory computer-readable media having data and machine readable instructions executable by a processor, the machine readable instructions causing the processor to at least:
compute a registration matrix (<NUM>) based on applying a tracking-marker coordinate system for at least one combination marker device (<NUM>) to a radiographic transformation matrix (<NUM>), such that the registration matrix provides a transformation from a tracking coordinate system of a tracking system (<NUM>) to a common three-dimensional coordinate system, the radiographic transformation matrix (<NUM>) being generated based on an image space transformation matrix (<NUM>) and an image-marker transformation matrix (<NUM>), the image space transformation matrix (<NUM>) being generated based on pre-operative image data (<NUM>) and intraoperative image data (<NUM>) for a patient, in which the intraoperative image data (<NUM>) comprises one or more images acquired by an imaging modality (<NUM>) intraoperatively of an anatomical structure of the patient and the at least one combination marker device (<NUM>) positioned on the patient, and the image-marker transformation matrix (<NUM>) is generated based on the intraoperative image data (<NUM>) and the tracking-marker coordinate system;
storing in memory a patient-specific implicit model (<NUM>) defining a geometry of the anatomical structure of the patient that is computed from the pre-operative image data (<NUM>) and spatially registered in the common three-dimensional coordinate system;
register tracking data (<NUM>) based on the registration matrix (<NUM>) in the common three-dimensional coordinate system, in which the patient-specific implicit model (<NUM>) defining the geometry of the anatomical structure of the patient is also registered in the common three-dimensional coordinate system, the tracking data (<NUM>) is generated by the tracking system (<NUM>) to represent a location of a sensor (<NUM>) attached to an object within the patient in the tracking coordinate system of the tracking system (<NUM>); and
generate an output visualization (<NUM>) representing a location of the object relative to geometry of the anatomical structure of the patient in the common three-dimensional coordinate system based on the registered tracking data (<NUM>) and the patient-specific implicit model (<NUM>).