Patent Description:
<CIT>, discloses an ultrasonic diagnostic imaging system in which anatomical images are stabilized in the presence of probe motion, anatomical motion, or both. A decision processor analyzes motional effects and determines whether to inhibit or allow image stabilization. Images are anatomically aligned on the basis of probe motion sensing, image analysis, or both. The stabilization system can be activated either manually or automatically and adaptively.

In medical procedures (hereinafter procedures for the sake of clarity), real-time information about the spatial position and orientation (i.e., the "pose") of a medical device is often required. Typically, such information is obtained using optical, electro-magnetic, mechanical or ultrasound-based tracking systems. Such systems are expensive and can require significant setup time and effort. Typically, motion-based inertial tracking devices experience bias which can lead to tracking inaccuracies. For example, bias in acceleration and rotational velocity estimates can lead to inaccurate velocity, position, and rotational position estimates when using motion-based inertial tracking systems.

When using motion-based inertial tracking system for medical device tracking, the pose of the medical device may not always be tracked accurately over extended periods of time. The requirements for accuracy may vary by procedure and/or medical device. For example, depending upon procedure, when the device to be tracked is an imaging device (e.g. an ultrasound probe), the requirements for the accuracy of the device position tracking may be less stringent. In this case, it may be desirable to provide position tracking while implementing a less complex position-tracking system which may conserve system resources and reduce cost. Thus, embodiments of the present system may overcome these and other disadvantages of conventional tracking-systems and methods.

The present invention is explained in further detail in the following exemplary embodiments and with reference to the figures, where identical or similar elements are partly indicated by the same or similar reference numerals, and the features of various exemplary embodiments being combinable. In the drawings:.

The following are descriptions of illustrative embodiments that when taken in conjunction with the following drawings will demonstrate the above noted features and advantages, as well as further ones. In the following description, for purposes of explanation rather than limitation, illustrative details are set forth such as architecture, interfaces, techniques, element attributes, etc. However, it will be apparent to those of ordinary skill in the art that other embodiments that depart from these details would still be understood to be within the scope of the appended claims. Moreover, for the purpose of clarity, detailed descriptions of well-known devices, circuits, tools, techniques, and methods are omitted so as not to obscure the description of the present system. It should be expressly understood that the drawings are included for illustrative purposes and do not represent the entire scope of the present system. In the accompanying drawings, like reference numbers in different drawings may designate similar elements. The term and/or and formatives thereof should be understood to mean that only one or more of the recited elements may need to be suitably present (e.g., only one recited element is present, two of the recited elements may be present, etc., up to all of the recited elements may be present) in a system in accordance with the claims recitation and in accordance with one or more embodiments of the present system.

<FIG> shows a schematic block diagram of a portion of a tracking system <NUM> (hereinafter system <NUM> for the sake of clarity) operating in accordance with embodiments of the present system. The system <NUM> includes one or more of a medical imaging device (MID) such as an ultrasound probe <NUM> (hereinafter probe for the sake of clarity unless the context indicates otherwise), an ultrasound scanner control <NUM>, an inertial measurement unit (IMU) <NUM>, and a tracking corrector <NUM> communicatively coupled to each other via any suitable wired and/or wireless methods. Further, it is envisioned that one or more of the imaging probes <NUM>, the ultrasound scanner control <NUM>, and the tracking corrector <NUM> may be formed integrally with another of the imaging probe <NUM>, the ultrasound scanner control <NUM>, and the tracking corrector <NUM>, as may be desired.

The IMU <NUM> is coupled to the MID such as the probe <NUM> and may include sensors <NUM>. For example, the IMU <NUM> may be releasably or fixedly coupled to the probe <NUM> as may be desired. The IMU may include a coupler or may be shaped to couple to the probe <NUM>.

The sensors <NUM> may include one or more inertial measurement sensors, such as accelerometers and/or gyroscopes, to determine linear acceleration or rotational velocity, respectively, of the probe <NUM>, to form corresponding sensor information, and to provide this sensor information as an inertial data stream (InDS) to the tracking corrector <NUM> for further processing. It is envisioned that the sensors <NUM> may include any suitable sensors such as miniature electro-mechanical (MEMS) inertial tracking sensors, and/or the like. The sensors <NUM> may have accuracy levels on the order of milli-g (i.e., <NUM>/s<NUM>) for linear acceleration, and a few degrees per hour bias stability for rotation, and may be low-cost sensors. A workflow and algorithm for using low-cost-sensors combined with image-based position estimates, along with determined reliability values, are described in <CIT> entitled "System and Method for Medical Device Tracking".

The sensor information may include one or more of linear acceleration and rotational velocity information generated by the accelerometers and the gyroscopes, respectively, and may be formed into corresponding tracking frames in time. More particularly, the accelerometers may sense linear acceleration along one or more axes (e.g., x, y, and z axes as shown by arrow <NUM>) and generate corresponding linear acceleration information, and the gyroscopes may sense rotation about one or more axes (e.g., x, y, and z axes) and generate corresponding rotational velocity information. The sensor information may be included within the InDS and the InDS may be provided to a pose estimator <NUM> of the tracking corrector <NUM> for further processing. Time information (e.g., acquisition time) may be included within the sensor information or may be separate from the sensor information and included within the InDS. Thus, the InDS may include time information indicating an acquisition time which may be used for synchronization with ultrasound information obtained from the image data stream as may be desired.

The sensors <NUM> may be situated at a single location or may be distributed throughout the IMU <NUM> as may be desired. For example, the sensors <NUM> may be located integrally with each other or may be located separately from each other. Further, the sensors <NUM> may be integrated with, e.g., located inside the probe <NUM>, and/or may be outside, e.g., attached to the probe <NUM>. In one embodiment, the sensors <NUM> are positioned as close to the image plane (i.e., as close to the tip of the probe <NUM>) as possible, in order to minimize any errors introduced by extrapolating the motion/rotation (which was measured at the sensor position) to the image plane position. Thus, according to an embodiment, the sensors <NUM> may be positioned in the probe <NUM> within <NUM> of the transducer array <NUM>.

The probe <NUM> may include one or more of a transducer array <NUM> and a body <NUM>. The transducer array <NUM> may include one or more ultrasound transducers configured in a desired order so as to transmit ultrasound waves and acquire ultrasound information of at least one ultrasound image plane such as an ultrasound image plane <NUM>. Although the MID is shown to include the probe <NUM>, it is envisioned that other types of MIDs may be used to acquire medical imaging data as may be desired. For example, other types of MIDs may include X-ray imaging devices or nuclear imaging devices such as Gamma-cameras.

The ultrasound information may include in any suitable format such as an analog and /or digital format and may be transmitted to the ultrasound scanner control <NUM> for further processing. In accordance with embodiments of the present system, the ultrasound information may include corresponding time information indicating an acquisition time which may be used for synchronization as may be desired.

The probe <NUM> may include any suitable ultrasound probe for a procedure being performed. For example, in the present embodiments, the probe <NUM> may be assumed to include at least one of an endo-cavity probe, endo-bronchial probe, cardiac probe, laparoscopic ultrasound probe, abdominal probe, small-parts or general imaging probe, and the like, including a multi-channel ultrasound probe, which may obtain ultrasound image information for constructing images of a region-of-interest (ROI) which may include one or more ultrasound image planes, such as the ultrasound image plane <NUM>. The probe may be capable of <NUM>-dimensional (1D), 2D, and/or 3D imaging.

The ultrasound scanner control <NUM> may receive the ultrasound information and process this information to form corresponding ultrasound image information which may include ultrasound image frame information (UIFIs) for a plurality of image frames. This ultrasound image information may then be provided within the image data stream to the pose estimator unit <NUM> for further processing. The ultrasound image information may include time information indicating an acquisition time as may be desired.

The tracking corrector (TC) <NUM> may include one or more of a controller <NUM>, the pose estimation unit <NUM>, an application unit <NUM>, a memory <NUM>, a user interface (Ul) <NUM> with which a user may interface with, and a display <NUM>, one or more of which may be formed integrally with, or separately from, each other.

The controller <NUM> may control the overall operation of the system <NUM> and may communicate with one or more of the probe <NUM>, the IMU <NUM>, the ultrasound scanner control <NUM>, the pose estimator <NUM>, the application unit <NUM>, the memory <NUM>, the UI <NUM>, and/or the display <NUM> using any suitable wired or wireless communication methods. The functions of one or more of the pose estimator <NUM> and the applicator <NUM> may be integrated within controller <NUM> as may be desired. Further, the controller <NUM> may control operation of the ultrasound scanner control <NUM> and/or the probe <NUM>.

The memory <NUM> may include any suitable non-volatile memory in which information such as operating instructions, information generated by the system, user inputs and/or settings, historical information, operating settings and/or parameters, identification information, user information, patient information, etc., may be stored.

The Ul <NUM> may include any suitable user interface which may allow for input (e.g., user input) and output such as rendering information for the convenience of the user, such as the display <NUM> which may display graphical user interfaces (GUIs) generated by the system and/or other image information. Accordingly, the UI may include a speaker (SPK), the display <NUM> (e.g., a touch-screen display, etc.), a haptic device (e.g., vibrators, etc.) with which to render information for the convenience of the user.

The pose estimator <NUM> may obtain the inertial data stream (InDS) from the IMU <NUM> and the image data stream from the ultrasound scanner <NUM>, and process information within these streams in accordance with embodiments of the present system to determine pose estimates which may be provided to the applicator <NUM> for further processing.

Generally, the pose estimates may be determined by integrating the acceleration information from the sensors <NUM> (e.g., obtained from the inertial data stream) to obtain velocity, and integrating again to obtain spatial position; further angular velocity data from the gyroscopes are integrated once to obtain heading (rotational position). However, any small error, bias or drift in acceleration and angular velocity may be accumulative over time and, thus, in the pose estimate, leading to deteriorating pose estimates over time. Accordingly, the pose estimator <NUM> may determine a type of displacement (e.g., large, small, no, etc.) and thereafter determine the pose estimate in accordance with settings for the type of displacement.

A method to analyze the inertial data stream and the image data stream as performed by the pose estimator <NUM> and the applicator <NUM> under the control of the controller <NUM> of the tracking corrector <NUM> in accordance with embodiments of the present system may be described with reference to <FIG> below. However, in brief, the pose estimator <NUM> may receive one or more of the image data stream and the inertial data stream, and determine a pose information related to the determined pose of the probe <NUM>.

Referring back to <FIG>, the applicator <NUM> may receive the determined pose information from the pose estimator <NUM> and process this pose information in accordance with system settings and output results. The system settings may be set and/or reset by the system and/or user, and may be stored in a memory of the system for later use. For example, the system settings may be set such that the applicator <NUM> may generate a graphical user interface (GUI) including a graphical depiction of the MID such as the ultrasound probe <NUM>, and provide this content to the UI <NUM> which may then render this information on a suitable rendering device of the system, such as on the display <NUM> of the system <NUM> for the convenience of the user. Similarly, the content may include other information such as audible information which may be rendered on a suitable rendering device of the system such as on a speaker of the system. Likewise haptic information may be rendered using a vibrator of the system. Thus, for example, the system may determine whether a current pose of the probe <NUM> indicates that the probe <NUM> is at a desired location, and when it is determined that the current pose of the probe <NUM> indicates that the probe <NUM> is at a desired location, the controller <NUM> may output a signal to drive a haptic device of the system to inform a user (e.g., via a vibratory feedback) that the probe <NUM> is at the desired location. The controller <NUM> may further determine guidance information to place the probe <NUM> in the desired position and render this information on a rendering device of the system such as the display <NUM>. Thus, the applicator <NUM> may obtain the pose information and generate corresponding guidance information to, for example, control a location of the ultrasound probe <NUM> and/or inform a user of an actual and/or desired location of the ultrasound probe <NUM> so that the user may move the ultrasound to the desired location. For example, the guidance information may include a color-coded bar which indicates the distance (length of bar) and out-of-plane direction (toward front/back of current image plane, indicated by the bar having different colors, such as green/blue) from the current image plane to the desired image plane. It is further envisioned that the applicator <NUM> may obtain the pose information and fuse ultrasound image information obtained from the ultrasound probe <NUM> with previously-obtained ultrasound image information using any suitable method such as a fusing method provided by, for example, the UroNav™ (Philips Invivo, Gainesville, FL) fusion biopsy system or the like.

<FIG> shows a functional flow diagram performed by a process <NUM> in accordance with embodiments of the present system. The process <NUM> may be performed using one or more processors, computers, controllers, etc., communicating over a network and may obtain information from, and/or store information to one or more memories which may be local and/or remote from each other. The process <NUM> includes the following acts. In accordance with embodiments of the present system, the acts of process <NUM> may be performed using one or more suitable coordinate registration systems operating in accordance with embodiments of the present system. Further, one or more of these acts may be combined and/or separated into sub-acts, as desired. Further, one or more of these acts may be skipped depending upon settings. For the sake of clarity, the process may be described with reference to a single probe such as an ultrasound probe. However, without limitation, it should be understood that the process may employ a plurality of probes each of which may be include a separate workflow. Also for the sake of clarity, in the illustrative embodiment shown in <FIG>, process <NUM>, correlation (Cframe) between at least two frames is used as a similarity metric, however, without limitation, it should be understood that other measures of similarity may be used instead of, or in addition to correlation, such as mutual information, normalized mutual information, sum of squared differences, correlation ratio, correlation coefficient, and/or other measures of similarity. Similarly, instead of, or in addition to computing correlation for at least last two frames (Cframe), other similarity metrics may be computed for the at least last two frames (Cframe) and compared with a similarity metric threshold (Cthresh).

In operation, the process <NUM> starts during act <NUM> and then proceeds to act <NUM>. During act <NUM>, the system obtains ultrasound image information acquired by the probe in real time. The ultrasound information includes a plurality of image frames acquired by the probe and may be included within an image data stream. Accordingly, these image frames may be obtained through an analysis of the image data stream. Simultaneously with the ultrasound image acquisition, the system acquires inertial information which may include acceleration information (e.g., linear and rotational) from an IMU of the system that may be coupled to, or otherwise part of, the probe. After completing act <NUM>, the process continues to act <NUM>.

During act <NUM>, the system determines a frame-to-frame similarity metric, such as a frame-to-frame correlation (Cframe) for two or more adjacent image frames of the ultrasound image information. These frames may be adjacent to each other in time or may be situated over a certain interval of time (e.g., a running window of time). After completing act <NUM>, the process continues to act <NUM>.

During act <NUM>, the process determines whether the Cframe is less than a correlation threshold Cthresh (i.e., whether Cframe < Cthresh). Accordingly, the process compares Cframe to Cthresh and if it is determined that Cframe is less than Cthresh (e.g., indicative of a large displacement of the probe) the process continues to act <NUM>. However, if it is determined that Cframe is not less than Cthresh (e.g., Cframe is greater than or equal to Cthresh, which is indicative of a small displacement), the process continues to act <NUM>. With regard to Cthresh, this value may be set to any suitable value such as <NUM> in the present embodiments. However, other values or ranges of values are also envisioned.

During act <NUM>, the system determines a pose estimate based at least upon integrated IMU data of tracking information obtained from the IMU (e.g., tracking data). Accordingly, the system may perform an IMU data integration process to integrate the IMU data (between previous and present bias correction time points) and determine a pose estimate (which may be defined as a current pose or pose estimate) based at least upon this integrated IMU data and a previous pose. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the process determines a pose estimate based at least upon image-based processing methods as may be applied to the ultrasound image information acquired by the ultrasound probe. During this act, the system may employ image-based processing (such as frame-to-frame correlation and out-of-plane correlation) to estimate a current velocity of the ultrasound probe. Then, the system may compare the change since the last bias correction (e.g., between previous and present bias correction time points) in this current velocity estimate (obtained by the image-based processing) with a change in velocity estimate provided by the integrated IMU data between previous and present bias correction time points, and use the difference to estimate an acceleration bias of the IMU as indicated by the tracking data. The system may then apply the bias correction retrospectively to all time points since the last bias correction (e.g., from m to (m-<NUM>)) and update the current pose estimate accordingly. The current pose estimate may then be provided to the application unit as a current pose.

A special subset of small displacements may occur when the probe is stationary. If this is the case, then the frame-to-frame correlations are highest (between identical images except micron level tissue movements and ultrasound scanner system noise) and the resulting velocity estimation (e.g., determined via image analysis) is essentially zero. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the process may transmit the pose estimate to the applicator (e.g., the applicator <NUM>) for further processing as may be described elsewhere in this application and may thereafter continue to act <NUM> where it may end.

With regard to bias correction, the bias correction may be triggered to occur automatically when the ultrasound probe is determined to be stopped or slowed (e.g., below a certain velocity threshold), or may be triggered manually (e.g., by a user as will be discussed below, etc.). For example, bias correction may be triggered manually by a user when the ultrasound probe is stationary and/or immediately prior to an ultrasound probe motion sequence that requires high tracking accuracy. Additionally, the applicator may monitor a period of time since a last motion correction (e.g., a bias correction) has been applied, and when it is determined that the period of time since the last motion correction is greater than a motion correction time threshold (Tthesh), the applicator may generate and render a message requesting the user to slow down or stop the motion so that motion correction may be applied. In other words, when it is determined that a predetermined period of time since a previous motion correction has elapsed (e.g., motion correction has not been applied for a predetermined period of time (e.g., <NUM> seconds, etc.)), the system may inform a user (e.g., by rendering the message requesting the user to slow down or stop the motion so that motion correction may be applied) to slow down probe motion and thereafter preform motion correction once the frame-to-frame correlation Cframe increases to reach and/or become greater than the correlation threshold Cthresh. When the user follows the request slow down or stop and indeed slows down or stops the ultrasound probe motion, then the frame-frame correlation Cframe increases to reach and/or become greater Cthresh. It should be noted that act <NUM> is again performed where it is checked that Cframe increased to reach and/or exceed Cthresh, and act <NUM> is performed to determine pose estimates based on the actually acquired imaging data following the request to user and the user act of slowing down or stopping the motion of the ultrasound probe.

A method for performing image-based bias correction in the pose estimation unit will now be discussed with reference to <FIG> which shows a functional flow diagram performed by a process <NUM> in accordance with embodiments of the present system. The process <NUM> may be performed using one or more processors, computers, controllers, etc., communicating over a network and may obtain information from, and/or store information to one or more memories which may be local and/or remote from each other. The process <NUM> may include one of more of the following acts. In accordance with embodiments of the present system, the acts of process <NUM> may be performed using one or more suitable coordinate registration systems operating in accordance with embodiments of the present system. Further, one or more of these acts may be combined and/or separated into sub-acts, as desired. Further, one or more of these acts may be skipped depending upon settings. For the sake of clarity, the process may be described with reference to a single probe such as an ultrasound probe. However, without limitation, it should be understood that the process may employ a plurality of probes each of which may be include a separate workflow such as a sub-workflow. In operation, the process may start during act <NUM> and then proceed to act <NUM>. Further, information determined by the system may be stored in a memory of the system for later use.

During act <NUM>, the system may perform a baseline acquisition to simultaneously acquire baseline tracking and image frames from each of the IMU and the probe (e.g., an ultrasound probe). For example, the system may acquire a baseline tracking frame (Fn, n =<NUM>, m=<NUM>) from an IMU during sub-act 303F and may simultaneously capture a baseline image frame (In, n =<NUM>, m=<NUM>) from the probe (e.g., directly or via an ultrasound scanner) during act 303I, where Fn denotes a tracking frame (e.g., from IMU data), In denotes an image frame (e.g., an ultrasound image frame from the ultrasound probe <NUM>), wherein n denotes an index (which may start at <NUM> for the initial frame), and m denotes a bias correction iteration and may be initially set to the value of n, which is <NUM> during the current act. Thus, the baseline tracking frame may be known as F<NUM> and the baseline image frame (e.g., ultrasound image frame) may be known as I<NUM>. The absolute baseline probe velocity is assumed to be <NUM> (probe stationary) and the probe pose at baseline is assumed to be at the origin of the coordinate system (i.e. position x = <NUM> in a <NUM>-dimensional example, or x = (<NUM>,<NUM>,<NUM>) in a <NUM>-dimensional coordinate system). All subsequent position and velocity estimates are estimates of position/velocity change relative to this baseline, or relative to the last bias correction point as detailed below. After the baseline acquisitions, the system may advance a count of index n such that n=n+<NUM>.

Simultaneous acquisition of tracking frames and imaging frames may be initiated from the application unit at a known initial pose (e.g., at origin of a selected coordinate system) when the probe is stationary. Accordingly, the system may render information to inform a user to place the ultrasound probe in a desired location (e.g., in a holder) for initialization. However, it is also envisioned that the system may detect when the probe is placed in a desired position (e.g., using sensors in a base, etc.) which may provide information to indicate that the probe is positioned correctly within the base and that an initialization process may begin. It is further envisioned that the user may initiate the initialization process manually when the probe is place in a desired pose, such as when it is placed in a stand, or when it is manually held in a stationary position at the beginning of a desired scan sequence. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the system may perform an acquisition to simultaneously acquire current (e.g., new) frames from each of the IMU and the ultrasound probe in real time. For example, the system may acquire a current baseline tracking frame (Fn) from the IMU during sub-act 305F and may simultaneously capture a current image frame (In) from the ultrasound scanner during sub-act 305I. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the system may save the frames acquired during acts <NUM> and <NUM> in a memory of the system for later use. For example, the system may save the acquired tracking frames (Fn) in a tracking frame memory during sub-act 307F and may save the acquired imaging frames (In) during sub-act 307I. The system may further store meta information, such as time of acquisition, probe and/or acquisition parameters, etc., with corresponding tracking and imaging frames. After completing act <NUM>, the system may continue to act <NUM>.

During act <NUM>, the system may estimate a current tracking-based velocity of the probe. Thus, the system may determine an inertial-tracking-based velocity vTracking (Fn, Fn-<NUM>,. Fm) of the probe for each tracking frame Fn over m frames. Accordingly, the system may obtain the acquired tracking frames (Fn) from the tracking frame memory and process this information to determine a current velocity (e.g., a tracking-based velocity) of the probe, by integrating the inertial acceleration data over time for all frames since the last correction, resulting in the tracking-based velocity change since the last correction. The absolute velocity estimate is obtained by adding the velocity at the last correction time to the velocity change estimates. Thus, if n = <NUM>, and m = <NUM>, the system may determine vTracking (F<NUM>, F<NUM>). After completing act <NUM>, the system may continue to act <NUM>.

During act <NUM>, the system may compute a frame-to-frame correlation (Cframe) for the acquired imaging frames (In). Accordingly, the system may determine a correlation between adjacent imaging frames such as a correlation between a current nth imaging frame (In) and a previous imaging frame (In-<NUM>) which correspond with index (e.g., n) of the image frames that were used to determine the current tracking-based velocity during act <NUM>. Illustratively, up to k=<NUM> frames are correlated to increase the robustness of the image-based velocity estimate. Further, the frames used for correlation may be adjacent in time. In an embodiment, an image quality metric may be employed to select which frames have sufficient image information to allow image-based velocity estimates. For example, ultrasound shadowing can obscure parts of the image. If too much of an image is "dark" (i.e., shadowed), then this image should not be used to attempt calculation of probe velocity. The image quality metric could thus measure average image brightness and only use frames that show sufficient brightness, such as a brightness above a preselected/predetermined brightness threshold. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the process may determine whether Cframe is less than a correlation threshold value (Cthresh). Accordingly, if it is determined that Cframe is less than Cthresh, the process may continue to act <NUM> and determine a current tracking-based pose based upon an IMU information integration method as will be discussed below. However, if it is determined that Cframe is not less than Cthresh (e.g., Cframe is equal to or greater than Cthresh), the process may continue to act <NUM> and determine current pose based upon image-based processing method of image frames (In) from the imaging probe <NUM> corresponding to and/or obtained in real-time simultaneously with tracking frames (Fn) from the IMU, as will be discussed below. This image-based processing method may employ a bias correction mode as may be discussed elsewhere.

It is envisioned that Cthresh may be set by the system and/or user and may be stored in a memory of the system and/or obtained when desired. In accordance with embodiments of the present system, Cthresh may be set equal to <NUM>. However, other values are also envisioned. Further, a plurality of correlation threshold values (Cthresh(x)), where x is an integer, also envisioned. For example, a first correlation threshold value (Cthresh(<NUM>)) may be set equal to Cthresh =. <NUM>, and a second correlation threshold value (Cthresh(<NUM>)) may be set equal to <NUM>. These two values may then be used for comparison purposes as may be described elsewhere.

During act <NUM>, the process may determine a tracking-based pose TTracking(Fn, Fn-<NUM>,. Fm) for the probe based upon an integration of the estimated inertial tracking-based velocity vTracking (Fn, Fn-<NUM>,. Fm) determined during act <NUM>. More particularly, the process may determine the tracking-based pose of the probe based upon an IMU data integration method which may integrate over the estimated inertial tracking-based velocity vTracking (Fn, Fn-<NUM>,. Fm) to determine the tracking-based pose TTracking(Fn, Fn-<NUM>,. In this method, bias correction may not be applied because the low correlation (Cframe < Cthresh) suggest that any image-based correction attempt would be less accurate than estimating velocity and pose directly from the inertial data. The pose change (since the last correction time) estimates are obtained by integrating over (adding) all velocity estimates vTracking (Fn, Fn-<NUM>,. Fm) to get the pose change since the last correction time, and adding the pose at the last correction time to get the absolute pose (i.e., pose change since baseline).

Thus, tracking data may be integrated to estimate the probe velocity, position and orientation, where vTracking and TTracking are the (inertial) tracking-based velocity and pose changes respectively, since a last bias correction (e.g., which may be a previous correction instant), which may be denoted by subscript m as will be discussed elsewhere. After completing act <NUM>, the system may continue to act <NUM>. As described in connection with acts <NUM> and <NUM>, all estimates are estimates of the pose/velocity change relative to the last correction point. By adding the pose/velocity at the last correction point, the overall velocity/pose change since baseline is obtained, which is the absolute pose/velocity estimate since the pose/velocity were both zero by definition at baseline, e.g., at origin of the coordinate system.

During acts <NUM> through <NUM>, the system may employ bias correction to correct the tracking pose to determine a corrected tracking pose which may then be set as the current pose.

More particularly, during act <NUM>, the system may determine an (estimated) image-based velocity vImaging(In, In-<NUM>,. In-k) using up to k number (where k is an integer <=<NUM>, for example,) of retrospective imaging frames (In, In-<NUM>,. In particular, the system may only use the <NUM> most recent image frames (In and In-<NUM>) to determine the current velocity. This act may be performed using any suitable imaging-based processing methods such as digital signal processing (DSP) methods or the like. During this act, the system may determine velocity based upon a difference of adjacent (in time) image frames. For example, the system may further identify landmarks within the image frames and may thereafter measure a shift of these landmarks to determine velocity, if any. After completing act <NUM>, the system may continue to act <NUM>.

During act <NUM>, the system may compute a measurement bias (B). Accordingly, the process may determine a change between estimated tracking-based velocity changes since the last correction, and image-based velocity changes since the last correction (at frame m). For example, the change in the imaging-based velocity may be represented as Δvimaging, with Δ vimaging = vimaging,n - vimaging,m, and the change in tracking-based velocity may be represented as Δvtracking = vtracking,n - vtracking,m. The process may then determine the bias (B) based upon a difference between Δvimaging to Δvtracking. Thus, the bias (B) may be based upon the difference of image- and tracking-based velocity changes. If probe motion in d dimensions is considered (d = <NUM>, <NUM> or <NUM>), the bias can be described as a d-dimensional vector. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the process may determine corrected tracking-based velocities for all estimates since the last correction vTracking, correct for the probe. Accordingly, the process may determine the tracking-based velocities vtracking, correct by applying the measurement bias (B) or a fraction thereof to the previously determined tracking-based velocity vTracking (Fn, Fn-<NUM>,. Fm) information which may have been previously stored in the memory. For example, the process may use linear interpolation to apply a fraction of the bias to each previously stored velocity estimate since the last correction, such that zero bias is applied (added to) the velocity estimate at the last correction point (at frame m), the full bias B is applied to the current velocity estimate (at frame n), and a fraction of the bias (j-m)/(n-m)*B is applied to all previously stored velocity estimates for frames Fj with j = m. n, i.e. for all frames between the last correction point and the current frame. The determined corrected tracking-based velocities vTracking, correct for the probe may be referred to as estimated corrected tracking-based velocities. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the process may determine a corrected tracking-based pose TTracking, correct for the probe. The process may determine the corrected tracking-based probe TTracking, correct based upon an integration of determined corrected tracking-based velocities vTracking, correct for the probe determined during act <NUM>. More particularly, the process may determine the corrected tracking-based pose TTracking, correct based upon an IMU data integration method which may integrate over the corrected tracking-based velocities vTracking, correct to determine the corrected tracking-based pose TTracking, correct. This integration may be similar to the integration performed during act <NUM> above. The corrected tracking-based pose TTracking, correct for the probe may then be set as the tracking-based pose TTrackin. After completing act <NUM>, the process may continue to act <NUM>. Thus, during acts <NUM>-<NUM>, the system may determine pose based upon an image-based bias correction of the integrated tracking information.

During act <NUM>, the process may set m = n, thus identifying the current frame n as the frame with the last bias correction. After completing act <NUM>, the process may continue to act <NUM>.

During act <NUM>, the system may increment a value of n (e.g., n = n+<NUM>) and may continue to act <NUM>.

During act <NUM>, the system may send the updated pose information to an application unit for further processing and rendering, such as aiding a user to determine location and/or orientation of the probe, for surgical guidance (e.g., move probe to left, right, etc.), and/or to fuse information from the ultrasound probe with other medical imaging information (e.g., magnetic-resonance imaging (MRI) scan information, X-ray computed tomography (CT) scan information, etc.) an/or to render this fused information, as may be desired and/or determined by a corresponding use and/or settings. After completing act <NUM>, the system may repeat act <NUM>. Further, it is envisioned that the process may end or otherwise be suspended when desired such as when a user or the system requests to end or suspend the process, respectively.

During the process <NUM>, it is envisioned that once the correction is applied, data stored in the memory can be deleted to clear space for incoming frames. However, it is envisioned that this data may be stored in the memory for a plurality of correction cycles or longer for a more complex estimation of the bias (linear, higher order polynomial, etc.. ) as may be desired. Thus, the correction cycle may employ data from a plurality of correction cycles (e.g., acts <NUM> through <NUM>) as may be desired.

A method to determine bias (B) as illustrated with respect to act <NUM> will now be discussed in further detail with respect to a one-dimensional (1D) translational case (e.g. an ultrasound probe translating in the lateral direction only, with an IMU tracking sensor measuring lateral acceleration) will now be discussed in further detail. A frame-to-frame image correlation may be performed to detect instances where the ultrasound probe may be stationary, which may result in very high correlation (e.g. greater than <NUM>). An image-based velocity estimate of vimaging = <NUM> m/s may be assumed for these stationary instances.

From one stationary instant to the next one, the change in the linear velocity of the probe may be written as <MAT> where t<NUM> and t<NUM> are two time points with image-based zero velocity estimates, and ax,actual is the acceleration of the probe. In the absence of IMU tracking measurement bias, the above expression is the same and equal for measured tracking data <MAT>.

If the measurement bias is written as ax,measured = ax,actual + ax,bias, then the expression in Eq. (<NUM>) may become <MAT>.

Therefore, it can be concluded that when the IMU acceleration data is integrated from t<NUM> to t<NUM> to obtain tracking based velocity estimates, any non-zero velocity estimation may be due to the tracking measurement bias.

It may be possible to estimate abias assuming that it is constant between t<NUM> and t<NUM>, and correct for it in the retrospective measurements such that the tracker based velocity change is equal to the image based estimate of the velocity change between t<NUM> and t<NUM>. Thus, <MAT>.

IMU acceleration measurements and results for a 1D demonstration case for an imaging probe with and without bias correction performed in accordance with embodiments of the present system will now be described with reference to <FIG> where:.

With reference to graph 400B, an effect of bias upon uncorrected velocity estimates is seen as a drift relative to expected velocities (line <NUM>). However, this drift may be eliminated using bias correction as illustrated by line <NUM>. With reference to graph 400C, without correction, the displacement estimates quickly move to unrealistically large values (line <NUM>), which may be generally unacceptable for application in medical imaging and image guidance. However, with the correction these displacement estimates (line <NUM>) may remain near the expected values.

Although the above-described methods are shown for 1D and zero velocity, the above-described methods may be generalized to three-dimensional (3D) and/or non-zero velocity estimates, i.e. Δvimaging = vimaging(t<NUM>) - vimaging(t<NUM>), which is compared with Δvtraking = <MAT>. Additionally, the constant bias assumption can be replaced with any other (e.g. linear, parabolic, etc.. ) time variant bias characteristics and similar corrections can be applied over the course of multiple image based velocity estimation points.

During an initial setup and/or when desired, the system may render information to prompt the user to scan slowly and/or keep the probe stationary if no such slow/no motion is detected after some scan time (e.g. <NUM> seconds, however other periods of time are also envisioned). This may provide for calibration when desired. It is further envisioned that when the motion of the probe motion is stationary and/or below a velocity threshold for a period of time (e.g. <NUM> seconds, however other periods of time are also envisioned), image-based velocity estimates may be used to calculate and update the pose.

It is further envisioned that image-based velocity estimates may be based on multiple recent frames stored in a memory of the system.

<FIG> shows a portion of a system <NUM> in accordance with embodiments of the present system. For example, a portion of the present system may include a processor <NUM> (e.g., a controller) operationally coupled to a memory <NUM>, a user interface (Ul) including a rendering device such as a display <NUM>, sensors <NUM>, and a user input device <NUM>. The memory <NUM> may be any type of device for storing application data as well as other data related to the described operation. The application data and other data are received by the processor <NUM> for configuring (e.g., programming) the processor <NUM> to perform operation acts in accordance with the present system. The processor <NUM> so configured becomes a special purpose machine particularly suited for performing in accordance with embodiments of the present system.

The operation acts may include configuring a system by, for example, an inertial tracking system in accordance with system settings.

The processor <NUM> may determine velocity and/or pose (e.g., position and/or orientation) information received from a medical imaging probe such as an ultrasound probe including an IMU coupled thereto. The processor <NUM> may further determine guidance information based upon difference information based upon a difference of a desired location and/or pose and the actual location and/or pose of the ultrasound probe. The processor <NUM>, thereof may process received signals such as sensor information, transform these signals to determine velocity, location, and/or orientation information (e.g., related to the probe), and may generate content which may include image information (e.g., still or video images (e.g., video ultrasound information)), data, and/or graphs that may be rendered on, for example, a UI of the system such as on the display <NUM>, a speaker, etc. The content may include image information as may be generated by a medical imaging system of the present system and/or may include guidance information (e.g., move right, left, arrows, etc.) to guide a user during a procedure. Further, the content may then be stored in a memory of the system such as the memory <NUM> for later use. The processor <NUM> may further register a location of the probe and/or fuse the content obtained from the probe (e.g., the ultrasound information) with information obtained from other medical imaging systems such as MRI and/or computer-aided tomography (CAT), X-ray, etc. systems. Thus, operation acts may include requesting, providing, and/or rendering of content. The processor <NUM> may render the content such as video information on a UI of the system such as a display of the system.

The user input <NUM> may include a keyboard, a mouse, a trackball, or other device, such as a touch-sensitive display, which may be stand alone or part of a system, such as part of a personal computer, a personal digital assistant (PDA), a mobile phone (e.g., a smart phone), a monitor, a smart or dumb terminal or other device for communicating with the processor <NUM> via any operable link such as a wired and/or wireless communication link. The user input device <NUM> may be operable for interacting with the processor <NUM> including enabling interaction within a UI as described herein. Clearly the processor <NUM>, the memory <NUM>, display <NUM>, and/or user input device <NUM> may all or partly be a portion of a computer system or other device such as a client and/or server.

The methods of the present system are particularly suited to be carried out by a computer software program, such program containing modules corresponding to one or more of the individual steps or acts described and/or envisioned by the present system. Such program may of course be embodied in a non-transitory computer-readable medium, such as an integrated chip, a peripheral device or memory, such as the memory <NUM> or other memory coupled to the processor <NUM>.

The program and/or program portions contained in the memory <NUM> may configure the processor <NUM> to implement the methods, operational acts, and functions disclosed herein. The memories may be distributed, for example between the clients and/or servers, or local, and the processor <NUM>, where additional processors may be provided, may also be distributed or may be singular. The memories may be implemented as electrical, magnetic or optical memory, or any combination of these or other types of storage devices. Moreover, the term "memory" should be construed broadly enough to encompass any information able to be read from or written to an address in an addressable space accessible by the processor <NUM>. With this definition, information accessible through a network is still within the memory, for instance, because the processor <NUM> may retrieve the information from the network for operation in accordance with the present system.

The processor <NUM> is operable for providing control signals and/or performing operations in response to input signals from the user input device <NUM> as well as in response to other devices of a network and executing instructions stored in the memory <NUM>. The processor <NUM> may include one or more of a microprocessor, an application-specific or general-use integrated circuit(s), a logic device, etc. Further, the processor <NUM> may be a dedicated processor for performing in accordance with the present system or may be a general-purpose processor wherein only one of many functions operates for performing in accordance with the present system. The processor <NUM> may operate utilizing a program portion, multiple program segments, or may be a hardware device utilizing a dedicated or multi-purpose integrated circuit. Embodiments of the present system may provide imaging methods to acquire and/or reconstruct images. Suitable applications may include imaging systems such as ultrasound. However, without limitation it should be understood that embodiments of the present system may further include imaging systems such as MRI, computer-aided tomography (CAT), optical, X-ray, and/or combinations thereof. Further, embodiments of the present system may be ideally suited for surgical interventional techniques which may generate and render image and/or sensor information from one or more imaging systems (e.g., ultrasound, CAT scans, MRI, X-ray etc.) having different coordinate systems in real-time with a unified coordinate system. The system may determine pose of the probe and may register the probe and/or image information obtained from the probe with these other systems. Accordingly, the system may determine velocity and/or pose of the probe for registration with these other systems.

Accordingly, embodiments of the present system process image-based information to correct bias errors in inertial sensors that may be attached to imaging devices for position tracking. This may reduce the need for highly-complex and expensive inertial or non-inertial sensors (such as electro-magnetic tracking sensors) and may allow the implementation of simple, low-cost inertial sensors. It is envisioned that embodiments of the present system may be ideal for various image-based inertial guidance systems. For example, and without limitation, it is envisioned that embodiments of the present system may be employed with ultrasound image guidance systems for prostate biopsy and the like.

Without limitation, it is envisioned that embodiments of the present system may be used for tracking poses of various medical probes such as for tracking a three-dimensional (3D) pose of an imaging probe. In particular, embodiments of the present system may be used for tracking the 3D pose of an ultrasound imaging probe during procedures that may require the fusion of ultrasound image data with pre-acquired data. Such systems may include a US/MRI-fusion-guided prostate biopsy system (e.g., see, UroNav™ system by Philips Invivo, Gainesville, FL). It is also envisioned that embodiments of the present system may be used for the general fusion of US with prior 3D imaging (CT, MR, cone-beam CT etc.), or diagnostic exams in which an extended field-of-view (FOV) is to be covered or reconstructed.

Further variations of the present system would readily occur to a person of ordinary skill in the art and may be encompassed by the following claims.

Finally, the above-discussion is intended to be merely illustrative of the present system and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the present system has been described with reference to exemplary embodiments, it should also be appreciated that numerous modifications and alternative embodiments may be devised by those having ordinary skill in the art without departing from the broader and intended scope of the present system as set forth in the claims that follow. In addition, any section headings included herein are intended to facilitate a review but are not intended to limit the scope of the present system. Accordingly, the specification and drawings are to be regarded in an illustrative manner and are not intended to limit the scope of the appended claims.

Claim 1:
A tracking system (<NUM>, <NUM>) for ultrasound imaging, the tracking system comprising:
an imaging probe (<NUM>) configured to acquire ultrasound image information including a plurality of ultrasound image frames;
an inertial measurement unit (IMU, <NUM>, <NUM>) coupled to the imaging probe and configured to synchronously acquire tracking information including a plurality of tracking frames indicative of motion of the imaging probe; and
a controller (<NUM>, <NUM>) configured to:
obtain the ultrasound image information for at least two of the plurality of ultrasound image frames from the plurality of ultrasound imaging frames,
determine a similarity value based upon a comparison of the at least two ultrasound image frames,
compute whether the similarity value (Cframe) is less than a similarity threshold value (Cthresh), and
select first or second pose estimation methods for the imaging probe each different from each other based upon the results of the computation of whether the similarity value (Cframe) is less than the similarity threshold value (Cthresh);
wherein the controller is configured to select the first pose estimating method when it is determined that the similarity value (Cframe) is less than the similarity threshold value (Cthresh) and wherein the first pose estimating method determines pose by integrating the tracking information from the IMU and storing the result in a memory of the system; and
wherein the controller is configured to select the second pose estimating method when it is determined that the similarity value (Cframe) is not less than the similarity threshold value (Cthresh) ; characterised in that the second pose estimating method determines pose based upon an image-based bias correction of integrated tracking information.