Patent Description:
Acoustic (e.g., ultrasound) imaging systems are increasingly being employed in a variety of applications and contexts. For example, ultrasound imaging is being increasingly employed in the context of ultrasound-guided medical procedures.

Intervention devices such as catheters and needles are known which have a sensor disposed on a surface thereof at a location near the tip for identification of the intervention instrument in acoustic images produced by an acoustic imaging system. For example, <CIT> describes an arrangement in which a medical needle includes an ultrasound transducer which is responsive to the ultrasound signals emitted by an ultrasound imaging system. Upon detecting an ultrasound pulse from the ultrasound imaging system, a circuit connected to the transducer triggers the insertion of an indication of the transducer position into the ultrasound image through either the generation of a return ultrasound pulse from the transducer, or through the simulation of such a return pulse using a time of flight delay. <CIT> discloses an apparatus for tracking a position of an interventional device relative to an image plane. A plurality of transducer-to-distal-end lengths is stored for different types of interventional device. A reconstructed image is generated based on the type of device being tracked.

In order to successfully perform an interventional procedure using one or more needles, the tip(s) of the needle(s) is/are the focus of attention. Accordingly, in ultrasound-guided medical procedures the physician wants to be able to visually locate the current position of the needle tip (or catheter tip) in acoustic images which are displayed on a display screen or monitor. In order to ensure safety, the displayed needle tip position must never lag the actual tip position.

However, for many intervention devices such as needles, it is not possible or practical to place a sensor or tracking device right at the tip of the device. The mechanical constraints of such instruments limit the ability to position the sensor at-will, for example at the tip of a needle where it might interfere with insertion.

Due to the inability to place the sensor on the tip of an intervention device such as a needle, the location of the sensor on the needle shaft is offset from the tip by a distance, D, which may vary from need to device. For some typical instruments, the distance D may be about1. <NUM>-<NUM>. The position of the sensor may be visually communicated to the end user by plotting a circle on the ultrasound image such that the tip lies on or inside the displayed circle. Clinicians typically use their judgment to determine the location of the tip within the circle. The ability to reduce this region of uncertainty is very beneficial in ensuring safe execution of interventional procedures. Using two or more sensors, as disclosed by <CIT> may solve this problem, but this adds to the cost of the intervention device and the acoustic imaging system, which can be crucial in low margin procedures. <CIT> discloses a system and a method according to the preambles of claims <NUM> and <NUM>.

Accordingly, it would be desirable to provide a system and a method which can provide more accurate estimates of the location of a tip of an intervention device such as a surgical needle, in acoustic images. In particular it would be desirable to provide such a system and method which can operate with a single sensor disposed on a surface of the intervention device.

In one aspect of the invention, a system according to claim <NUM> is provided.

In another aspect of the invention, a method according to claim <NUM> is provided.

The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided as teaching examples of the invention. Herein, when something is said to be "approximately" or "about" a certain value, it means within <NUM>% of that value.

In order to illustrate the principles of the present invention, various systems are described in which the position of the tip of an intervention device, exemplified by a surgical needle, is determined within the image plane of an acoustic field defined by the beams emitted by a 2D acoustic imaging probe. It is however to be appreciated that the invention also finds application in determining the positon of other intervention devices such as a catheter, a guidewire, a probe, an endoscope, an electrode, a robot, a filter device, a balloon device, a stent, a mitral clip, a left atrial appendage closure device, an aortic valve, a pacemaker, an intravenous line, a drainage line, a surgical tool such as a tissue sealing device or a tissue cutting device.

It is also to be appreciated that the invention may find application in beamforming acoustic imaging systems having other types of imaging probes and other types of acoustic arrays which are arranged to provide a planar image, such as the linear array of a 2D acoustic imaging probe, a transrectal ultrasonography probe, an intravascular acoustic probe, a transesophageal probe, a transthoracic probe, a transnasal probe, an intracardiac probe, etc..

<FIG> shows one example of an acoustic imaging system <NUM> which includes an acoustic imaging instrument <NUM> and an acoustic probe <NUM>. Acoustic imaging instrument <NUM> include a processor (and associated memory) <NUM>, a transmit unit <NUM>, a user interface <NUM>, a receive unit <NUM>, a display device <NUM> and a receiver interface <NUM>, connected as shown in <FIG>.

In various embodiments, processor <NUM> may include various combinations of a microprocessor (and associated memory), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), digital circuits and/or analog circuits. Memory (e.g., nonvolatile memory) associated with processor <NUM> may store therein computer-readable instructions which cause a microprocessor of processor <NUM> to execute an algorithm to control acoustic imaging system <NUM> to perform one or more operations or methods which are described in greater detail below. The instructions may be stored on a computer program product. The computer program product may be provided by dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor "DSP" hardware, read only memory "ROM" for storing software, random access memory "RAM", non-volatile storage, etc. Furthermore, embodiments of the present invention can take the form of a computer program product accessible from a computer-usable or computer-readable storage medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that may include, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or apparatus or device, or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory "RAM", a read-only memory "ROM", a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk - read only memory "CD-ROM", compact disk - read/write "CD-R/W", Blu-Ray™ and DVD.

In some embodiments, a microprocessor may execute an operating system. In some embodiments, a microprocessor may execute instructions which present a user of acoustic imaging system <NUM> with a graphical user interface (GYI) via user interface <NUM> and display device <NUM>.

In various embodiments, user interface <NUM> may include any combination of a keyboard, keypad, mouse, trackball, stylus /touch pen, joystick, microphone, speaker, touchscreen, one or more switches, one or more knobs, one or more buttons, one or more lights, etc. In some embodiments, a microprocessor of processor <NUM> may execute a software algorithm which provides voice recognition of a user's commands via a microphone of user interface <NUM>.

Display device <NUM> may comprise a display screen of any convenient technology (e.g., liquid crystal display). In some embodiments the display screen may be a touchscreen device, also forming part of user interface <NUM>.

Transmit unit <NUM> may generate one or more electrical transmit signals under control of processor <NUM> and supply the electrical transmit signals to acoustic probe <NUM>. Transmit unit <NUM> may include various circuits as are known in the art, such as a clock generator circuit, a delay circuit and a pulse generator circuit, for example. The clock generator circuit may be a circuit for generating a clock signal for setting the transmission timing and the transmission frequency of a drive signal. The delay circuit may be a circuit for setting delay times in transmission timings of drive signals for individual paths corresponding to the transducer elements of acoustic probe <NUM> and may delay the transmission of the drive signals for the set delay times to concentrate the acoustic beams to produce acoustic probe signal <NUM> having a desired profile for insonifying a desired image plane. The pulse generator circuit may be a circuit for generating a pulse signal as a drive signal in a predetermined cycle.

Beneficially, acoustic probe <NUM> may include an array of acoustic transducer elements <NUM> (see <FIG>), for example a 2D array. For example, in some embodiments, transducer elements <NUM> may comprise piezoelectric elements. In operation, at least some of acoustic transducer elements <NUM> receive electrical transmit signals from transmit unit <NUM> of acoustic imaging instrument <NUM> and convert the electrical transmit signals to acoustic beams to cause the array of acoustic transducer elements <NUM> to transmit an acoustic probe signal <NUM> to an area of interest <NUM>. Acoustic probe <NUM> may insonify an image plane in area of interest <NUM> and a relatively small region on either side of the image plane (i.e., it expands to a shallow field of view).

Also, at least some of acoustic transducer elements <NUM> of acoustic probe <NUM> receive acoustic echoes from area of interest <NUM> in response to acoustic probe signal <NUM> and convert the acoustic echoes to electrical receive signals which are communicated to receive unit <NUM>.

Receive unit <NUM> is configured to receive the electrical receive signals from transducer elements <NUM> of acoustic probe <NUM> and to process the electrical receive signals to produce to produce acoustic data. Receive unit <NUM> may include various circuits as are known in the art, such as one or more amplifiers, one or more A/D conversion circuits, and a phasing addition circuit, for example. The amplifiers may be circuits for amplifying the electrical receive signals at amplification factors for the individual paths corresponding to the transducer elements <NUM>. The A/D conversion circuits may be circuits for performing analog/digital conversion (A/D conversion) on the amplified electrical receive signals. The phasing addition circuit is a circuit for adjusting time phases of the amplified electrical receive signals to which A/D conversion is performed by applying the delay times to the individual paths respectively corresponding to the transducer elements <NUM> and generating acoustic data by adding the adjusted received signals (phase addition).

Processor <NUM> may reconstruct acoustic data received from receiver unit <NUM> into an acoustic image corresponding to the image plane which intercepts area of interest <NUM>, and subsequently causes display device <NUM> to display this image. The reconstructed image may for example be an ultrasound Brightness-mode "B-mode" image, otherwise known as a "2D mode" image, a "C-mode" image or a Doppler mode image, or indeed any ultrasound planar image.

Acoustic imaging instrument <NUM> may also include receiver interface <NUM> which is configured to receive one or more electrical signals (sensor signals) from an external passive acoustic sensor, for example an acoustic receiver disposed near a distal end (tip) <NUM> of an intervention device, as will be described in greater detail below, particularly with respect to <FIG>.

Of course it is understood that acoustic imaging instrument <NUM> may include a number of other elements not shown in <FIG>, for example a power system for receiving power from AC Mains, an input/output port for communications between processor <NUM> and acoustic probe <NUM>, a communication subsystem for communicating with other eternal devices and systems (e.g., via a wireless, Ethernet and/or Internet connection), etc..

<FIG> illustrates one example embodiment of an intervention device <NUM> having an acoustic sensor (e.g., a passive acoustic sensor) <NUM> disposed near a distal end or tip <NUM> thereof. Acoustic sensor <NUM> is adapted for detecting ultrasound signals emitted by an ultrasound transducer array of a beamforming acoustic imaging system, such as acoustic imaging system <NUM> of <FIG>. Acoustic sensor <NUM> is provided on a surface of shaft <NUM> of intervention device <NUM> at a predetermined distance D from tip <NUM> of intervention device <NUM>.

Optionally, intervention device <NUM> also may include a data carrier <NUM> disposed on shaft <NUM> of intervention device <NUM>. Data carrier <NUM> may include data indicative of a type T of intervention device <NUM>. Moreover, this data, when received by processor <NUM> of acoustic imaging instrument <NUM>, causes processor <NUM> to: i) ascertain a distance D between acoustic sensor <NUM> and tip <NUM> of intervention device <NUM> for the intervention device type T; and to ii) use the knowledge of the distance D to indicate in the reconstructed ultrasound image the estimated location(s) of tip <NUM> of intervention device <NUM> within the acoustic image.

In <FIG>, data carrier <NUM> may for example be a barcode such as a linear or matrix barcode or a QR code, an RFID chip, a memory, or indeed any machine-readable data carrier. The data carrier may be attached to the intervention device by various known means including adhesives, or it may be applied by printing, etching, and the like. Also, whilst illustrated as being disposed on intervention device <NUM>, data carrier <NUM> may alternatively be positioned on the packaging of intervention device <NUM>, for example for sterility reasons. Thus, when the intervention device <NUM> is used with acoustic imaging system <NUM>, the data received by processor <NUM> enables the technical effect of improved determination of the position of tip <NUM> of intervention device <NUM> respective the image plane of acoustic imaging system <NUM>.

In some embodiments, data carrier <NUM> may be omitted. In that case, the distance D between acoustic sensor <NUM> and tip <NUM> of intervention device <NUM> may be communication to processor <NUM> of acoustic imaging system <NUM> by other techniques. For example, in some embodiments a clinician or user of acoustic imaging system <NUM> may enter the distance D directly via user interface <NUM>. In other embodiments, memory (e.g., nonvolatile memory) associated with processor <NUM> may store a look up table containing entries for a plurality of different types of models of intervention device <NUM>, where each entry includes a device type or model number and a corresponding distance D. In that case, the clinician or user of acoustic imaging system <NUM> may enter the type or model number via user interface <NUM> and processor <NUM> may ascertain the corresponding distance D from the look-up table. In still other embodiments where acoustic imaging instrument <NUM> includes a communication port for accessing a remote computer network, such as the Internet, processor <NUM> may access a remote database to ascertain the distance D between acoustic sensor <NUM> and tip <NUM> of intervention device <NUM>. Other arrangements are contemplated.

Beneficially acoustic sensor <NUM> illustrated in <FIG> and <FIG> may be a passive sensor and may be provided by a number of piezoelectric materials, both hard and soft piezoelectric materials being suitable. In some embodiments, acoustic sensor <NUM> may comprise polyvinylidene fluoride, otherwise known as PVDF whose mechanical properties and manufacturing processes lend themselves to attachment to curved surfaces such as needles. Alternative materials include a PVDF copolymer such as polyvinylidene fluoride trifluoroethylene, a PVDF ter-polymer such as <NUM> P(VDF-TrFE-CTFE). Acoustic sensor <NUM> may include various wires or a wireless communication module that are not shown in <FIG> for communicating a sensor signal generated in response to detected acoustic signals with receiver interface <NUM>. Beneficially there may be a single, i.e. one and only one, such acoustic sensor <NUM> disposed on intervention device <NUM>. Advantageously this simplifies the form factor of intervention device <NUM>, any electrical interconnect that may be present, and the processing of any detected acoustic signals.

As described in greater detail below, in some embodiments processor <NUM> of acoustic imaging instrument <NUM> may use one or more sensor signals received by receiver interface <NUM> from passive acoustic sensor <NUM> disposed on intervention device <NUM> to track the location of intervention device <NUM>, and particularly tip <NUM> thereof, in acoustic images generated from acoustic data produced by echoes received by acoustic probe <NUM>.

In various embodiments, intervention device <NUM> may comprise a needle, a catheter, a guidewire, a probe, an endoscope, an intravenous line, a drainage line, a surgical tool such as a tissue sealing device or a tissue cutting device, etc..

<FIG> illustrates example embodiment of a process of overlaying imaging produced from one or more sensor signals received from a sensor, such as passive acoustic sensor <NUM>, with an acoustic image produced from acoustic echoes received by an acoustic probe such as acoustic probe <NUM>.

As illustrated in <FIG>, acoustic probe <NUM> illuminates area of interest <NUM> with acoustic probe signal <NUM> and receives acoustic echoes from area of interest <NUM> in response to acoustic probe signal <NUM>. An acoustic imaging instrument (e.g., acoustic imaging instrument <NUM>) produces acoustic images <NUM> of area of interest <NUM> in response to acoustic echoes received from area of interest <NUM> in response to acoustic probe signal <NUM>. In particular, acoustic probe <NUM> may communicate one or more electrical receive signals (electrical signals) to acoustic imaging instrument <NUM> in response to acoustic echoes received from area of interest <NUM> in response to acoustic probe signal <NUM>, and acoustic imaging instrument <NUM> may produce acoustic images <NUM> from the electrical receive signal(s).

Meanwhile, a receiver interface (e.g., receiver interface <NUM>) receives one or more electrical sensor signals from a passive acoustic sensor (e.g., passive acoustic sensor <NUM>) disposed on a surface of an intervention device (e.g., device <NUM>) disposed in area of interest <NUM>, the one or more sensor signals being produced in response to acoustic probe signal <NUM>.

A processor (e.g., processor <NUM>) executes an algorithm to ascertain or determine, from the one or more sensor signals from passive acoustic sensor <NUM> an estimated range of locations of acoustic sensor <NUM> in area of interest <NUM>. Image <NUM> illustrates sensor data obtained by processor <NUM>, showing a marker <NUM>-<NUM> which identifies an estimated range of locations of passive acoustic sensor <NUM>. For example, processor <NUM> may employ an algorithm to detect a maximum value or intensity peak in sensor data produced from the one or more sensor signals from passive acoustic sensor <NUM>, and may determine or ascertain an estimated range of locations of passive acoustic sensor <NUM> corresponding to the location of an intensity peak in the sensor data. Here it should be understood that, in general, tracking of passive acoustic sensor <NUM> may not be precisely accurate due to noise and other factors, and so marker <NUM>-<NUM> may accommodate a predetermined tracking error or uncertainty, which in some embodiments may typically be in a range of <NUM> - <NUM> by showing an estimated range of locations of passive acoustic sensor <NUM> as a circular marker with a center at the most likely location and the radius corresponding to the expected uncertainty (e.g., <NUM> - <NUM>). Then acoustic imaging instrument <NUM> may overlay the sensor data illustrated in image <NUM> with acoustic image <NUM> to produce an overlaid acoustic image <NUM> which includes marker <NUM>-<NUM> to indicate the an estimated range of locations of passive acoustic sensor <NUM>.

The process described above with respect to <FIG> allows a display of a marker indicating an estimated range of locations of a sensor (e.g., passive acoustic sensor <NUM>) in an acoustic image.

However, as explained above, in ultrasound-guided medical procedures the physician wants to be able to visually locate the current position of the needle tip (or catheter tip) <NUM> in acoustic images which are displayed on a display screen or monitor.

Described below are systems and methods of utilizing a known location in an acoustic image of a passive sensor which is disposed at a known distance from the tip of an intervention device to provide improved estimates for the location of the tip of the intervention device in the acoustic image.

In particular, for in-plane or near in-plane imaging, acoustic imaging system <NUM>, and in particular processor <NUM> may be configured to ascertain an approximate angular orientation of the intervention device (e.g., intervention device <NUM>) with respect to the image plane of the acoustic image, and may use this information to provide an improves estimate of the location of tip <NUM> in the acoustic image.

<FIG> illustrates in-plane and out-of-plane imaging using an acoustic imaging instrument and an acoustic probe and a sensor signal received from an acoustic sensor.

In particular, <FIG> shows at the top three illustrations for explaining in-plane imaging, and shows below that, at the bottom, three corresponding illustrations for explaining out-plane imaging.

Beginning at the top, <FIG> shows an illustration 410a which shows acoustic probe <NUM> insonifying an image plane 412a, where intervention device (needle) <NUM> is disposed in or at a slight angle with respect to image plane 412a, as an example of in-plane imaging. To the right of illustration 410a is shown an illustration 420a of a tip tracking display for tracking the location of tip <NUM> in image plane 412a for the case of in-plane imaging. Here the tip tracking display includes a marker 422a indicating an estimated range of locations of tip <NUM> of needle <NUM> in the image plane. Finally, to the right of illustration 420a is an illustration of an acoustic image 430a of the anatomical features of image plane 412a and needle <NUM> for the case of in-plane imaging. Superimposed on acoustic image 430a is a marker <NUM>-4a indicating an estimated range of locations of tip <NUM> of needle <NUM>.

Proceeding below, <FIG> also shows an illustration 410b which shows acoustic probe <NUM> insonifying an image plane 412a where three intervention devices, labeled <NUM>, <NUM> and <NUM>, are disposed perpendicular out of an image plane 412b, as an example of out-of-plane imaging. To the right of illustration 410b is shown an illustration 420b of a tip tracking display for tracking the locations of the tips of intervention devices <NUM>, <NUM> and <NUM> in the image plane for the case of out-of-plane imaging. Here the tip tracking display includes markers indicating estimated ranges of locations for the tip of each of intervention devices <NUM>, <NUM> and <NUM> in the image plane. For intervention device <NUM>, whose tip is in or very near image plane 412b, the marker indicates a relatively small estimated range of locations for the tip, essentially corresponding to the distance D from the acoustic sensor (e.g., acoustic sensor <NUM>) and the top (e.g. tip <NUM>). For intervention devices <NUM> and <NUM>, whose tips are relatively further from image plane 412b, two markers are shown: a smaller one indicating what the estimated range of locations for the tip would be if the tip was in imaging plane 412b; and a larger one which factors in the additional uncertainly in the tip location due to the distance of the tip from image plane 412b. Finally, to the right of illustration 420b is an illustration of an acoustic image 430a of the anatomical features of image plane 412a and intervention device <NUM> for the case of out-of-plane imaging. Superimposed on acoustic image 430a are markers <NUM>-4b1 and <NUM>-4b2 indicating the estimated range of locations of tip <NUM> of needle <NUM> as described above.

The inventors have appreciated that an improvement in the estimated range of locations of the tip of an intervention device in an acoustic image for the case of in-plane imaging may be obtained by estimating the orientation angle of the intervention device with respect to the image plane and using the estimated orientation angle to determine an effective distance from the passive sensor to the tip in the image plane, where this effective distance may be less than the actual physical distance between the passive sensor and the tip. An explanation will now be provided with respect to <FIG>.

<FIG> illustrates the relationships between an image plane, the tip of an intervention device, and a sensor disposed on the surface of the intervention device at a distance D from the tip. Here the image plane is shown as the XZ plane. θ is the angle between the shaft of intervention device (e.g., needle) <NUM> and a projection in the XY plane. ψ is the angle between the projection in the XY plane and the X axis. D is the actual physical distance between acoustic sensor <NUM> and tip <NUM> of intervention device <NUM> and d is the effective distance in the image plane (XZ plane) between acoustic sensor <NUM> and tip <NUM> of intervention device <NUM>. Furthermore, x, y, and z shown in <FIG> are found to be: <MAT> <MAT> <MAT>.

In that case, it can be shown that: <MAT> <MAT>.

For the case where intervention device <NUM> is purely in-plane with respect to the image plane, then: ψ = <NUM>, x = <NUM>; y = <NUM>; z = D*sinθ; and the effective acoustic sensor-to-tip distance d = D.

For the case where intervention device <NUM> is purely out-of-plane with respect to the image plane, then: ψ = <NUM>°, x = <NUM>; y = D*cosθ; z = D*sinθ; and the effective acoustic sensor-to-tip distance d = D*sinθ < D.

It can further be shown that for an "average" out-of-plane condition where ψ = <NUM>° and θ = <NUM>°, then d ≈ <NUM>D. Furthermore, for a nearly average out-of-plane case where ψ = <NUM>° and θ = <NUM>°, then d ≈ <NUM>D.

<FIG> illustrates a ratio R between an effective acoustic sensor-to-tip distance d in an image plane and an actual acoustic sensor-to-tip distance D, as a function of angles involving the image plane and a line extending from the acoustic sensor <NUM> to the tip <NUM>. In particular, <FIG> illustrates the effective acoustic sensor-to-tip distance d in an image plane as a ratio R of the actual acoustic sensor-to-tip distance D for all possible in-plane and out-of-plane angles used in the computation where ψ and θ are allowed to range independently between <NUM> and <NUM> degrees. In most procedures the ratio R varies between <NUM> and <NUM>, which presents an ability to provide an improved estimated range of locations of tip <NUM> in the acoustic image based on the determined location of acoustic sensor and the known actual acoustic sensor-to-tip distance D.

The inventors have also appreciated that an improvement in the estimated range of locations of the tip of an intervention device in an acoustic image for the case of in-plane imaging may be obtained by estimating the orientation angle of the intervention device with respect to the image plane and using the estimated orientation angle to identify an arc of less than <NUM>°, and beneficially less than <NUM>°, on a circle around the acoustic sensor having a radius equal to the effective acoustic sensor-to-tip distance d as the estimated range of locations of tip <NUM>.

<FIG> illustrates one example embodiment of a method of improving an estimated location for the tip of an intervention device in acoustic images. The method illustrated in <FIG> may be performed by an acoustic imaging instrument (e.g., acoustic imaging instrument <NUM>) under control of a processor (e.g., processor <NUM>) executing instructions stored in memory.

Operation <NUM> estimates a rough orientation of intervention device (e.g., a needle) <NUM> with respect to the image plane (see image plane 412a in <FIG> above) for which an acoustic image is being generated. In particular, processor <NUM> of acoustic imaging instrument <NUM> may be configured, based on instructions stored in memory, to estimate the rough orientation of intervention device <NUM> with respect to the image plane.

In one embodiment, processor <NUM> is configured to ascertain the approximate angular orientation (e.g., ψ and θ - see <FIG> above) of intervention device <NUM> with respect to the image plane by using a history of previous positions of intervention device <NUM> and a prediction of a future path of intervention device <NUM> by employing a Kalman model.

In another embodiment, processor <NUM> is configured to ascertain the approximate angular orientation of intervention device <NUM> with respect to the image plane by monitoring an insertion point of intervention device <NUM> within a patient to estimate a trajectory of intervention device <NUM>. This may include visually monitoring the insertion point to coarsely estimate the trajectory of intervention device <NUM> to estimate both in plane and out of plane orientation (see <FIG> above) and/or using a camera-based method to monitor the insertion point to estimate the in plane and out of plane orientation.

In still another embodiment, processor <NUM> is configured to ascertain the approximate angular orientation of intervention device <NUM> with respect to the image plane by segmenting shaft <NUM> of intervention device <NUM> in the acoustic images.

In yet another embodiment, processor <NUM> may be configured to ascertain the approximate angular orientation of intervention device <NUM> with respect to the image plane by employing a combination of two or more of the techniques described above, for example by finding a best fit match to: the history of previous positions of intervention device <NUM> and prediction of a future path of intervention device <NUM> by employing a Kalman model; a monitored insertion point of intervention device <NUM> within a patient; and a segmented shaft <NUM> of intervention device <NUM> in the acoustic images.

Operation <NUM> predicts a distribution of likely locations of tip <NUM> in the acoustic image. In particular, processor <NUM> may be configured to ascertain an estimated range of locations of tip <NUM> of intervention device <NUM> in the image plane using: the estimated location of passive sensor <NUM> obtained by acoustic imaging instrument <NUM> from the sensor signal as described above; the approximate angular orientation of intervention device <NUM> with respect to the image plane; and the known distance D from passive sensor <NUM> to tip <NUM> of intervention device <NUM>, as described above with respect to <FIG>.

<FIG> shows an acoustic image <NUM> with a coarse estimate <NUM> of the angular orientation of intervention device <NUM> with respect to the image plane, here for example based on a history of previous positions of tip intervention device <NUM>, shown as markers <NUM>-<NUM>.

In some embodiments, processor <NUM> of acoustic imaging instrument <NUM> may ascertain the approximate angular orientation of intervention device <NUM> to within <NUM>° of the actual angular orientation using the techniques described above. This may allow a reduction in the region or locations where tip <NUM> may lie relative to sensor location <NUM>.

Operation <NUM> places one or more markers on the acoustic image to indicate the estimated range of locations of tip <NUM>.

<FIG> illustrates various visualization options for showing the estimated locations of tip <NUM> of intervention device <NUM> in an acoustic image <NUM>.

For example, without the improvements in the estimated locations of tip <NUM> of intervention device <NUM>, acoustic imaging instrument <NUM> may display, via display device <NUM>, a circular marker <NUM>-8a designating a rather large region in the acoustic image for the estimated range of locations of tip <NUM>.

In contrast, with the techniques described above the estimated range of locations of tip <NUM> may be reduced and accordingly acoustic imaging instrument <NUM> may display a circular marker <NUM>-8a designating a considerably smaller region in the acoustic image for the estimated range of locations of tip <NUM>.

In other embodiments, acoustic imaging instrument <NUM> may display an arc-shaped marker <NUM>-8c designating a region in the acoustic image for the estimated range of locations of tip <NUM>.

In still other embodiments, a combination of techniques may be employed. Acoustic imaging instrument <NUM> may display a circular marker <NUM>-8d1 having a first color (e.g., green) designating a large region in the acoustic image for the estimated range of locations of tip <NUM> before the techniques above are applied, and an arc-shaped marker <NUM>-8d2 having a second color (e.g., red) which designates a considerably smaller region in the acoustic image for the estimated range of locations of tip <NUM> by employing the techniques described above.

Here, <FIG> shows a circular marker <NUM>-7b having a first color (e.g., red) designating a large region in the acoustic image for the estimated range of locations of tip <NUM> before the techniques above are applied, and an arc-shaped marker <NUM>-7a having a second color (e.g., green) which designates a considerably smaller region in the acoustic image for the estimated range of locations of tip <NUM> by employing the techniques described above.

Operation <NUM> fine tunes the estimated range of locations of tip <NUM> as time progresses based on the history of previous estimates. <FIG> shows that at a later time acoustic imaging instrument <NUM> may display a circular marker <NUM>-7b2 having the first color (e.g., red) designating a large region in the acoustic image for the estimated range of locations of tip <NUM> before the techniques above are applied, and an arc-shaped marker <NUM>-7a2 having a second color (e.g., red) which designates a considerably smaller region in the acoustic image for the estimated range of locations of tip <NUM> by employing the techniques described above. Here it is seen that the size of arc-shaped marker <NUM>-7a2 may be reduced over time, providing a clinician with an improving estimate for the location of tip <NUM> as time progresses.

<FIG> illustrates a flowchart of an example embodiment of a method of improving estimates of the location of a tip of an intervention device in acoustic images.

An operation <NUM> includes providing transmit signals to least some of the acoustic transducer elements of an acoustic probe (e.g., acoustic probe <NUM>) to cause the array of acoustic transducer elements to transmit an acoustic probe signal to an area of interest (e.g., area of interest <NUM>).

An operation <NUM> includes producing acoustic images of interest <NUM> in response to acoustic echoes received from area of interest <NUM> in response to the acoustic probe signal.

An operation <NUM> includes receiving one or more sensor signals from a sensor (e.g. sensor <NUM>) disposed on a surface of an intervention device (e.g., intervention device <NUM>) disposed in the area of interest, the one or more sensor signals being produced in response to the acoustic probe signal. Here, sensor <NUM> is located at a fixed distance D from tip <NUM> of intervention device <NUM>.

An operation <NUM> includes identifying an estimated location of sensor <NUM> in the acoustic image, for example based on a localized intensity peak in sensor data.

An operation <NUM> includes ascertaining an approximate angular orientation of intervention device <NUM> with respect to the image plane, for example using techniques described above with respect to <FIG>.

An operation <NUM> includes ascertaining an estimated range of locations of tip <NUM> of intervention device <NUM> in the image plane using the estimated location of sensor <NUM>, the approximate angular orientation of intervention device <NUM> with respect to the image plane, and the known distance D from sensor <NUM> to tip <NUM> of intervention device <NUM>.

An operation <NUM> includes displaying acoustic images on a display device (e.g., display <NUM>).

An operation <NUM> includes displaying on display device <NUM> one or more markers to indicate on the acoustic images the estimated range of locations of tip <NUM> of intervention device <NUM> in the image plane.

It should be understood that the order of various operations in <FIG> may be changed or rearranged, and indeed some operations may actually be performed in parallel with one or more other operations. In that sense, <FIG> may be better viewed as a numbered list of operations rather than an ordered sequence.

Claim 1:
A system (<NUM>), comprising:
an acoustic probe (<NUM>) having an array of acoustic transducer elements (<NUM>); and
an acoustic imaging instrument (<NUM>) connected to the acoustic probe and configured to provide transmit signals to least some of the acoustic transducer elements to cause the array of acoustic transducer elements to transmit an acoustic probe signal (<NUM>) to an area of interest (<NUM>), and further configured to produce acoustic images (430a, <NUM>, <NUM>) of an image plane (412a) of the area of interest in response to acoustic echoes received from the area of interest in response to the acoustic probe signal, the acoustic imaging instrument including:
a display device (<NUM>) configured to display the acoustic images of the image plane in the area of interest;
a receiver interface (<NUM>) configured to receive a sensor signal from a passive sensor (<NUM>) disposed on a surface (<NUM>) of an intervention device (<NUM>) disposed in the area of interest, the sensor signal being produced in response to the acoustic probe signal, wherein the passive sensor is located at a fixed distance (D) from a tip (<NUM>) of the intervention device; and
a processor (<NUM>) configured to:
identify an estimated location of the passive sensor from the sensor signal,
wherein the system is characterized in that the processor is further configured to:
ascertain an approximate angular orientation (<NUM>) of the intervention device with respect to the image plane, and
ascertain an estimated range of locations of a projection of the tip location of the intervention device with respect to the image plane, using the estimated location of the passive sensor, the approximate angular orientation of the intervention device with respect to the image plane, and the fixed distance from the passive sensor to the tip of the intervention device,
wherein the display device is further configured to display a marker (<NUM>-7a, <NUM>-7a2, <NUM>-8b, <NUM>-8c, <NUM>-8d2) to indicate on the acoustic images the estimated range of locations of the projection of the tip location of the intervention device with respect to the image plane.