Patent Description:
In image-guided biopsy procedures, physicians rely on real-time imaging to guide an insertion of a biopsy gun to targets, which may be visible directly in a live image, or may be transferred from a prior image and superimposed on the live image using image fusion. Most biopsy guns have a spring-loaded mechanism that, when fired, shoots forward to obtain a tissue sample from a location that is offset from a needle tip by a fixed distance (i.e., the "throw" of the needle). Operators need to estimate that distance and position the biopsy needle proximal to the intended target, offset by the "throw", and with the needle trajectory intersecting the target. When the operator considers the needle to be positioned correctly, the biopsy gun is "fired" to obtain the tissue sample. If the distance to the target or the trajectory of the biopsy needle is not estimated correctly, the target will not be sampled accurately.

In addition, by using hardware tracking technology and fusing pre-procedural diagnostic images with intra-procedural live imaging, e.g., ultrasound, pre-identified targets from diagnostic images can be mapped to the space of live imaging. An example of such system is the Philips® UroNav™ system, which is used for image fusion guided prostate cancer biopsy. In this system, magnetic resonance (MR) images are fused with the transrectal ultrasound (TRUS) images in real time. The suspicious prostate cancer lesions are identified as biopsy targets by radiologists on MR images. Those targets are usually not visible from TRUS images. During a prostate biopsy procedure, MR images are fused with TRUS images via electromagnetic (EM) tracking. In this way, the MR targets can be mapped to the TRUS images and thus can be superimposed over TRUS images when they are in view. With such guidance, users can aim for a target in TRUS with a biopsy gun.

However, even with the targets displayed over TRUS, there is no guarantee that a user can hit the target accurately with the biopsy needle for several reasons. For example, a biopsy needle is usually spring loaded, and the core-taking part will be fired out when a button is released. The user has to insert the needle to a certain depth but stay proximal to the target, accounting for the throw of the needle when fired. This mental estimation of the insertion depth may be error prone, and inaccurate insertion depth estimation may result in the sampled location being either too deep or too shallow and thus missing the target. Another cause for missing a target is due to the biopsy guide bending and/or shifting. When this happens, the needle will deviate from a biopsy guide line displayed on a screen. Thus, when a user uses the displayed guide line to aim for the target, the actual sample will be taken from an area other than the targeted area. Yet another factor can be the motion due to either patient movement or TRUS probe movement. During needle firing, if there is such motion, the needle may be directed away from the target.

<CIT> describes a method and apparatus for enabling a biopsy needle to be observed in a three-dimensional ultrasound diagnostic system using an interventional ultrasound system.

<CIT> relates to tools to improve a <NUM>-D image aided biopsy or treatment procedure for prostate gland. The tools include i) the identification of various parts of prostate to classify as per regular classification in pathological reports, ii) Computing and displaying the insertion depth of needle with respect to a selected target point during the procedure, iii) Computing and displaying the distance from needle tip to prostate surface following a procedure and, iv) Calibration for misalignment of a <NUM>-D imaging transducer when used under tracked motion for a procedure.

<CIT> describes a method of guiding an interventional instrument within a patient anatomy. The method comprises processing a target location within the patient anatomy and receiving a position for a tip portion of an interventional instrument at a first location within the patient anatomy. The method also comprises determining a three-dimensional distance between the first location and the target location and displaying a symbol representing the target location and a symbol representing the tip portion of the interventional instrument.

These and other objects, features and advantages of the present disclosure will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.

This disclosure will present in detail the following description of arrangements with reference to the following figures wherein:.

In accordance with the present principles, systems and methods are provided that produce feedback to a user before and after a biopsy gun firing by determining a spatial relationship between the biopsy needle and a target. Users will be able to take corresponding actions when needed to improve the accuracy and in turn achieve higher success rates. Inan arrangement, a system in accordance with the present principles uses image processing and analysis methods to detect a needle in real-time imaging. The system can provide visual feedback superimposed on images to show a desired position of biopsy gun before firing. This can also be used together with hardware-based device tracking (e.g., electromagnetic (EM) tracking, optical shape sensing (OSS), etc.).

Under such settings, a detected needle can be mapped into a same space with pre-identified targets. Thus, the spatial relationship between the needle and the target can be computed, based on which specific visual or auditory feedback is provided to the users both before and after needle firing. The present system and method provide intra-procedural-accuracy feedback to users immediately before and after the biopsy gun firing. With the feedback, a user can either adjust the needle to be in a better position for firing or take another sample if the target has already been missed. The feedback can be visual, auditory, or other feedback signals may be employed.

It should be understood that the present disclosure will be described in terms of medical instruments; however, the teachings are much broader and are applicable to any image based technology that involve alignment or activities. In somearrangements, the present principles are employed in tracking or analyzing complex biological or mechanical systems. In particular, the present principles are applicable to internal tracking procedures of biological systems and procedures in all areas of the body such as the lungs, gastro-intestinal tract, excretory organs, blood vessels, etc. The elements depicted in the FIGS. may be implemented in various combinations of hardware and software and provide functions which may be combined in a single element or multiple elements.

The functions of the various elements shown in the FIGS. can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor ("DSP") hardware, read-only memory ("ROM") for storing software, random access memory ("RAM"), non-volatile storage, etc..

Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure). Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative system components and/or circuitry embodying the principles of the disclosure.

Similarly, it will be appreciated that any flow charts, flow diagrams and the like represent various processes which may be substantially represented in computer readable storage media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

Furthermore, arrangements can take the form of a computer.

program product accessible from a computer-usable or computer-readable storage medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that may include, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read/write (CD-R/W), Blu-Ray™ and DVD.

Reference in the specification to "one embodiment" or "an embodiment" of the present principles, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment", as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

It is to be appreciated that the use of any of the following "/", "and/or", and "at least one of", for example, in the cases of "A/B", "A and/or B" and "at least one of A and B", is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of "A, B, and/or C" and "at least one of A, B, and C", such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as readily apparent by one of ordinary skill in this and related arts, for as many items listed.

It will also be understood that when an element such as a layer, image, region or material is referred to as being "on" or "over" another element, it can be directly on the other element or intervening elements may also be present. In contrast, when an element is referred to as being "directly on" or "directly over" another element, there are no intervening elements present.

Referring now to the drawings in which like numerals represent the same or similar elements and initially to <FIG>, a system <NUM> for instrument guidance using feedback before and after a trigger event is illustratively shown in accordance with an arrangement. System <NUM> may include a workstation or console <NUM> from which a procedure is supervised and/or managed. Workstation <NUM> preferably includes one or more processors <NUM> and memory <NUM> for storing programs and applications. The workstation <NUM> may include an imaging system <NUM> integrated therein or have the imaging system <NUM> independent from the workstation <NUM>. The workstation <NUM> may be configured to provide other functions instead of or in addition to those described herein.

Memory <NUM> may store a feedback guidance application or module <NUM> configured to identify and track objects in an image. The feedback guidance application <NUM> can be run to detect an instrument <NUM> in an image, e.g., to assist in needle insertion and needle firing using live imaging. The feedback guidance application <NUM> includes a detection module or algorithm <NUM> configured to determine a position and orientation of the instrument <NUM> within an image. The live imaging may be collected using the imaging device or system <NUM>. The imaging system <NUM> may include an ultrasound system, although other imaging modalities may be employed, e.g., fluoroscopy, etc..

The feedback guidance application <NUM> provides for live imaging only guided procedures and well as procedures with fused images (e.g., live images with stored static/preoperative images). The feedback guidance application <NUM> provides visual or audible signal warnings if the instrument <NUM> is deviating from a guideline (path determined for a biopsy needle or the like) or other stored criteria. In one example, the instrument <NUM> may include a biopsy gun. Before firing the biopsy gun, a display <NUM> can display a projected location of a biopsy core-taking portion based on a current position and orientation of a biopsy needle attached to the gun. The feedback guidance application <NUM> generates an actual tissue sample-taking area graphic before and/or after needle firing that can be displayed in an image shown on the display <NUM>. Such graphics are employed as feedback for the proper alignment or positioning of the instrument <NUM> and/or the realignment or repositioning for a next task.

For image fusion guided procedures, a device tracking system <NUM> (with a sensor <NUM> (e.g., EM sensor, optical shape sensor, etc.) and an image registration module <NUM> may be employed to spatially map the detected instrument <NUM> (e.g., a needle) in 3D space (images <NUM>). The feedback guidance application <NUM> computes the distance between a biopsy core location and a target according to the detected instrument <NUM>. The feedback guidance application <NUM> provides signals for visual or audible feedback to users based on the distances between the biopsy core and the target. In the example of a biopsy, before firing, feedback on whether the target is on the needle insertion path and whether the needle is inserted to the correct depth to sample the target is provided. After firing, feedback on whether the biopsy core was actually taken from the targeted area is provided.

The instrument <NUM> may include a fired biopsy needle, other needles, a catheter, a guidewire, a probe, an endoscope, a robot, an electrode, a balloon device or other medical component, etc..

In an arrangement, an image generation module <NUM> is configured to generate objects to assist in the planning of a biopsy. The image generation module <NUM> generates overlays on displayed images to provide visual feedback to the user. In an arrangement, a biopsy guideline is generated by the image generation module <NUM> and projected in a real-time image, which is displayed on display <NUM>.

The feedback guidance application <NUM> may use the geometric dimensions of the biopsy needle (specifically the distance between the needle tip and the biopsy-core taking part of the needle), and the estimate position and trajectory of a needle to determine a location of a core taken after a biopsy needle has fired. These features may be generated by the image generation module <NUM> and projected in the real-time image as feedback for the user. In another arrangement, acoustic information may be generated instead of or in addition to the visual feedback. A speaker or speakers <NUM> may be provided that receive audio signals from the feedback guidance application <NUM> to provide different forms of audio feedback. For example, the amplitude of an audio signal or its tone may be employed to indicate that a throw region is being approached or that the throw region has been exceeded. In another arrangement, textual information (on display <NUM>) or audio commands (on speakers <NUM>) may provide the same function by informing the user of the position based on measurements, etc. performed by the feedback guidance application <NUM>.

Workstation <NUM> includes the display <NUM> for viewing internal images of a subject (patient) or volume <NUM> and may include images as an overlay or other rendering as generated by the image generation module <NUM>. Display <NUM> may also permit a user to interact with the workstation <NUM> and its components and functions, or any other element within the system <NUM>. This is further facilitated by an interface <NUM> which may include a keyboard, mouse, a joystick, a haptic device, or any other peripheral or control to permit user feedback from and interaction with the workstation <NUM>.

Referring to <FIG>, a block/flow diagram is illustratively shown for a system/method using feedback in accordance with the feedback guidance application <NUM>. An exemplary arrangement will be described using a needle for a biopsy procedure. It should be understood that other instruments may be employed for other procedures as well. For image fusion based targeted biopsy, a user picks a target (Tcurrent) from a list of targets for biopsy in block <NUM>. In block <NUM>, once a biopsy needle enters a viewing field of a live imaging viewing pane or image, the needle object (Oneedle) can be detected by using an image based detection method. Other methods may be employed as well, e.g., EM tracking, optical shapes sensing, etc. The detection can be triggered either automatically by continuously or intermittently monitoring imaging or manually by a user. An example of detecting a needle from an ultrasound image is described with reference to <FIG>.

Referring to <FIG>, a detection process (e.g., for detection module <NUM>) may include the following system/method. An image <NUM> (e.g., ultrasound) containing a needle is given to a needle detection algorithm. The image <NUM> is filtered by a needle shape filter <NUM>, which enhances tubular structures (like a needle) and suppresses other structures to provide a filtered image <NUM>. An edge detection module <NUM> may perform edge detection for the filtered image <NUM> to extract the main enhanced areas in an edge detected image <NUM>. Morphological image processing operations are then applied in block <NUM> to a binary edge image <NUM> for further processing. In block <NUM>, a Hough transform is then employed to extract all the line segments from a processed binary image <NUM>. In block <NUM>, a line with highest possibility (highest needle score) to be the needle is picked as a final detection result. The needle tip is labeled in an image <NUM>. The detection process described here may be employed as well as other image processing techniques, and the process can be generalized for needle detection with other imaging modalities (e.g., fluoroscopy, computed tomography, magnetic resonance, etc.).

Referring again to <FIG>, in block <NUM>, for fusion guided targeted procedures, to correctly compute a distance to a target, the target and the detected needle need to be mapped into a common space (e.g., Tcurrent 3D space). This can be achieved by using device tracking and image registration techniques. For example, electromagnetic (EM) tracking may be employed for tracking the ultrasound probe. By registering a reconstructed 3D ultrasound volume with a 3D MR volume, 2D ultrasound images can be mapped to the 3D MR space in real-time. Since the targets are identified from MR images, the transformation chain will bring the needle detected from 2D ultrasound images into the same MR imaging space as the targets. In addition, depending on how a biopsy guide is attached to an imaging device, biopsy guide bending or shifting may occur. In the presence of such an event, the needle will deviate from a biopsy guide line shown by the imaging equipment. This is illustratively depicted in <FIG>.

In block <NUM>, the system (<NUM>) checks whether the target falls on the pointing direction of the detected needle. Feedback is provided to the user, e.g., visual or audio feedback can be provided. For example, the biopsy guide line can be turned into a highlighted color when the current target falls on the line, or a sound can be played to confirm that user is pointing the needle in the right direction.

In block <NUM>, once the needle is in the right direction, the system will continue to check whether the core taking part of the needle will cover the biopsy target once being fired. The 3D position of the core taking part, which is usually in the shape of a cylinder, is computed based on the anatomy of the needle, the needle tip location, and also the needle pointing direction. This is illustratively depicted in <FIG>. For visual feedback, a marker can be put at the desired location for the needle tip along the needle. The user needs to insert the needle to that marked point for firing. Alternatively, a beeping sound can be played when the needle is getting close to the firing point. The frequency of the beeping may be used for denoting the distance between the needle tip and its desired location. In block <NUM>, if feedback is received that the core taking is off target, the process is repeated by returning to block <NUM>.

In block <NUM>, when the user inserts the needle to the desired location, the needle can be fired to acquire a tissue sample. In block <NUM>, the needle firing can be automatically detected or manually indicated. Automatic detection can be achieved by looking for the sudden increase of the needle length, since the firing is very fast, which can be captured by <NUM> to <NUM> frames of the live imaging depending on the frame rate of the system. With the firing detected, the distance between the actual biopsy core and the target can be computed in the same way as described above. If the system detects that the biopsy core is actually away from target but not covering it, a warning signal may be displayed on a screen or a warning sound may be played. Then, the user will have a chance to check the biopsy and redo it if the user determines that is necessary.

Referring to <FIG>, an illustration is shown for providing feedback when a path of a needle <NUM> deviates from a projected biopsy guideline <NUM>. The methods in accordance with the present principles can detect whether the actual needle is deviating from the projected biopsy guideline <NUM>. Based on that information, either visual or audible feedback will be provided. A projected core taking region <NUM> is shown and is coupled with an orientation of the needle <NUM> at a distance from a tip of the needle.

Referring to <FIG>, an estimated final location of a needle <NUM> for a firing based on the detected needle location can be superimposed over live or static images. With this feedback, users can know accurately where the needle <NUM> will end up after firing. This provides more accurate guidance than just virtual estimation based on a users' own knowledge and experience. An estimated or projected core taking region <NUM> is shown on the projected biopsy guideline <NUM> at an appropriate distance to account for firing of the needle <NUM>.

Referring to <FIG>, an example of visual feedback on biopsy needle insertion depth is shown in accordance with the present principles. A needle <NUM> is detected and visual feedback is provided as to a distance from the needle where a core sample will be taken. In instance one <NUM>, a core projection <NUM> coincides well with a target <NUM> to be biopsied. In instance two <NUM>, the core projection <NUM> is too shallow. In instance three <NUM>, the core projection <NUM> is too deep. The present systems and methods provide feedback to users before and after the biopsy gun firing by determining the spatial relationship between the biopsy needle and the target. This enables users to take appropriate actions to improve the accuracy and to achieve a higher success rate.

Referring to <FIG>, an example of visual feedback on biopsy needle insertion depth is shown in accordance with another embodiment. The needle <NUM> includes a representation of a core-taking portion <NUM> as visual feedback for instances <NUM>, <NUM> and <NUM>.

Referring to <FIG>, a method for instrument guidance is illustratively shown. In block <NUM>, an instrument is inserted and detected in a real-time image. The real-time image may include an ultrasound image, although other real-time image may be employed, e.g., fluoroscopic images, etc. The instrument may be detected using a detection algorithm to determine instrument position in the image.

In block <NUM>, a projected guideline is generated in the image to indicate a path for the instrument in a subject. In block <NUM>, the instrument is advanced further or aimed in the subject, if needed, to attempt to follow the guideline. The instrument may be partially or completely inserted in block <NUM>. If partially inserted, the instrument may be further advanced here.

In block <NUM>, feedback for aligning or guiding the instrument is generated in accordance with a detected event. The feedback may include audio and/or visual feedback to provide guidance to a user in positioning the instrument in the area of interest relative to the projected guideline in real-time. The audio and visual feedback may include, e.g., an alert on a display, a sound, a change in frequency of a sound, a change in the projected guideline, etc. The detected event may include misalignment of the instrument from a projected path, proximity to a target, a display of a projected core taking region in the area of interest and/or a display of a region of an actual core taken from the area of interest after firing a biopsy needle.

In block <NUM>, a trajectory of the instrument is altered in accordance with the feedback. The feedback and altering the trajectory continues until alignment is achieved.

In block <NUM>, procedure specific projections are generated in the image. For example, the instrument may include a biopsy needle having a throw for a core taking portion. In block <NUM>, a projected region is generated representing the throw in the displayed image as visual feedback). In block <NUM>, the projected region is positioned in the image.

In block <NUM>, an instrument event is performed. For example, the needle is fired to collect a sample.

In block <NUM>, post event projections are generated. In block <NUM>, a core projection region is generated representing a core taken from the subject. In block <NUM>, the core projection region is positioned on the target in the image.

In block <NUM>, a decision as to the adequacy of the procedure (e.g., core taken) is determined. If adequate, stop; otherwise return to block <NUM>.

As described, before firing the needle, the (un-fired) needle tip is detected and the throw is added to estimate the location of the biopsy core if the needle were fired at that moment. Feedback is provided on whether the needle is positioned correctly to sample the target. After firing, the (fired) needle tip is detected and a dead zone (between core-taking portion and needle tip) is subtracted to estimate where the tissue was actually sampled. Feedback is provided on whether the target was correctly sampled, or whether it may need re-sampling.

In interpreting the appended claims, it should be understood that:.

Claim 1:
A feedback system for biopsy gun guidance the biopsy gun comprising a fireable biopsy needle having a needle tip, the biopsy needle, when fired, shoots forward to obtain a tissue sample from a location that is offset from the needle tip by a fixed distance, the system comprising:
a feedback guidance module (<NUM>) configured to detect the biopsy needle in a real-time image and generate feedback including proximity to a target for aligning or guiding the instrument; and
an image generation module (<NUM>) configured to generate a projected guideline in the image and a projected location of a biopsy core-taking portion in the form of a graphic of an actual tissue sample-taking area based on a detected current position and orientation of the biopsy needle and the fixed distance offset from the needle tip;
the feedback guidance module configured to generate at least one of audio and visual feedback based on distances between the detected biopsy needle and
(i) the projected guideline;
(ii) the graphic of an actual tissue sample-taking area and the target;
to provide guidance to a user in positioning the biopsy needle relative to the target in real-time.