Patent ID: 12186070

DETAILED DESCRIPTION

Reference will now be made to figures wherein like structures will be provided with like reference designations. It is understood that the drawings are diagrammatic and schematic representations of exemplary embodiments of the invention, and are neither limiting nor necessarily drawn to scale.

Regarding terms used herein, it should be understood the terms are for the purpose of describing some particular embodiments, and the terms do not limit the scope of the concepts provided herein. Ordinal numbers (e.g., first, second, third, etc.) are sometimes used to distinguish or identify different components or operations, and do not supply a serial or numerical limitation. For example, “first,” “second,” and “third” components or operations need not necessarily appear in that order, and the particular embodiments including such components or operations need not necessarily be limited or restricted to the three components or operations. Similarly, labels such as “left,” “right,” “top,” “bottom,” “front,” “back,” and the like are used for convenience and are not intended to imply, for example, any particular fixed location, orientation, or direction. Instead, such labels are used to reflect, for example, relative location, orientation, or directions. Singular forms of “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.

In the following description, the terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. As an example, “A, B or C” or “A, B and/or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, components, functions, steps or acts are in some way inherently mutually exclusive.

The term “logic” is representative of hardware and/or software that is configured to perform one or more functions. As hardware, logic may include circuitry having data processing and/or storage functionality. Examples of such circuitry may include, but are not limited or restricted to a processor, a programmable gate array, a microcontroller, an application specific integrated circuit, combinatorial circuitry, or the like. Alternatively, or in combination with the hardware circuitry described above, the logic may be software in the form of one or more software modules, which may be configured to operate as its counterpart circuitry. The software modules may include, for example, an executable application, a daemon application, an application programming interface (API), a subroutine, a function, a procedure, a routine, source code, or even one or more instructions. The software module(s) may be stored in any type of a suitable non-transitory storage medium, such as a programmable circuit, a semiconductor memory, non-persistent storage such as volatile memory (e.g., any type of random access memory “RAM”), persistent storage such as non-volatile memory (e.g., read-only memory “ROM”, power-backed RAM, flash memory, phase-change memory, etc.), a solid-state drive, hard disk drive, an optical disc drive, or a portable memory device.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art.

General Guidance System Architecture

Referring toFIG.1, an illustrative embodiment of a guidance system100is shown, wherein a needle or other medical component110may be located and guided, during an ultrasound procedure, magnetic field imaging procedure, or other suitable guidance procedure for example, to access a subcutaneous vasculature120of a patient130. According to this embodiment of the disclosure, the guidance system100may be configured to operate as an ultrasound imaging system, although the guidance system100may be deployed in accordance with a technology type other than ultrasound.

As illustrated inFIG.1, the guidance system100features a console140, a monitor150and a probe160. The console140may be configured with sensor analytic logic (SAL)170that operates in cooperation with a sensor array (seeFIGS.2-3) within the probe160. The sensor array may capture the position and/or orientation of the medical component110through sound waves (ultrasound), where the medical component110may include a needle, catheter or an introducer for example. Hereinafter, for simplicity, the medical component110may be generally referred to as a “needle”110.

Additionally, or in the alternative to the console140, the monitor150and the probe160, a tip location subsystem “TLS” (not shown, seeFIG.2) may be deployed to conduct a real-time determination of a position and/or orientation of the needle110through magnetic field analyses. As yet another embodiment, in addition to or in lieu of the sensor analytic logic170and/or the TLS, the console140may be configured with artificial intelligence (AI) to control operability of the guidance system100, namely AI-based visualization controls (AVC)180and AI-based guidance assistance logic (AGAL)190to render imaging enhancements that assist in guiding the clinician in the advancement of the needle110, as described below.

As shown inFIG.1, the guidance system100, operating as an ultrasound imaging system for example, is configured to generate an image of a targeted internal portion of a body of a patient immediately prior and subsequent to percutaneous insertion of the needle110. For one embodiment, such as guided needle advancement for example, the guidance system100may be employed to image a region surrounding the vasculature120during and/or subsequent to percutaneous insertion of the needle110to access the targeted vasculature120. As described below, insertion of the needle110may be performed prior to insertion of a catheter into a portion of the vasculature120of the patient130. It is appreciated, however, that insertion of the needle110into the patient130can be performed for a variety of medical purposes.

According to one embodiment of the disclosure, the sensor analytic logic170receives orientation metrics from the sensor array, and in response to entry of the needle110into a region of interest (ROI) defined by the vasculature120itself, the sensor analytic logic170is configured to generate a notification regarding entry into the vasculature. More specifically, the sensor analytic logic170is configured to monitor for a change of state of the needle110(intra-vasculature vs. extra-vasculature). In response to the change in state, the sensor analytic logic170may be configured to generate signaling that would cause the monitor150to render a visual notification of the state change for display or activation of a light on the probe160or console140to identifying whether the needle is entering or exiting the vasculature120. Additionally, or in the alternative, the sensor analytic logic170may be configured to generate signaling that would cause an audio device (e.g., speaker155) to emit an audible notification in response to any state change, such as a first type of audible notification (e.g., beep, tone, etc.) in response to a transition from an extra-vasculature state to an intra-vasculature state and a second type of audible notification in response to a transition an intra-vasculature state to an extra-vasculature state.

Besides visual or audio notifications, again in addition to or in the alternative of the visual and/or audible notification, the sensor analytic logic170may be configured to generate signaling that would cause a haptic feedback device within the probe160to provide a physical warning (e.g., vibration and/or another force-based feedback) as a haptic notification. The haptic feedback device as described below and illustrated inFIG.3.

According to another embodiment of the disclosure, operating separately or in combination with the sensory analytic logic170, the AI-based visualization controls180are configured to receive the orientation metrics of the needle110from the probe160and generate and locate a visualization area195based on these orientation metrics. As shown, the visualization area195corresponds to a portion of a total imaging area192rendered by the guidance system100. For this embodiment, the visualization area195is substantially lesser in size (e.g., less than a 1/10thof size) than the total imaging area192, which may include the image captured from a sound beam162emitted from the probe160and rendered for display on the monitor150of the guidance system100. The sizing of the visualization area195may be static or dynamic (e.g., based on needle gauge, medical component type, etc.). Also, the location of the visualization area195may be altered based on the position, orientation and advancement of the needle110so as to intercept advancement of the needle110prior to contact with an outer wall surface of the targeted vasculature120.

In particular, the guidance system100enables the position, orientation, and advancement of the medical component110(e.g., needle) to be superimposed, in real-time, on the ultrasound image of the vasculature120, thus enabling a clinician to accurately guide the needle110to the vasculature120, as shown in more detail inFIGS.8A-8E. Furthermore, for some embodiments, imaging enhancements may be applied to a portion of the imaged data, such as a color overlay that may be static in color change or dynamic based on its presence and/or location of the medical component within the visualization area. The imaging enhancement may include, additionally or in the alternative, an image overlay of at least a portion of an image of the needle110(e.g., thicker outline, enlarged sizing of certain portions of the needle such as the portions of the needle within the visualization area) to provide better clarity as to its position. Another imaging enhancement may include generation of a virtual representation of the visualization area (with needle110and vasculature120). Other imaging enhancements may be deployed where the enhancements are designed to assist the clinician in the advancement of the needle110.

Furthermore, for this embodiment, the guidance system100tracks the position of the needle110in accordance with numerous degrees of motion during advancement as illustrated by dashed lines inFIG.1. The position of the needle110, from at least the insertion to subsequent upward or downward angular adjustment of the needle110, may be represented by the following information: xyz spatial coordinate space, pitch of the needle110, and/or yaw (e.g., orientation) of the needle110. Herein, for some embodiments, this information may constitute the orientation metrics provided to the AI-based visualization controls180. In other embodiments, the orientation metrics may be based, at least in part, on this information or a portion thereof.

Such tracking enables a distal end (tip)112of the needle110to be guided and placed with relatively high accuracy despite movement of the needle110. Also, based on the information gathered during the tracking operations (e.g., needle reflection, etc.), the location of the needle110may be determined, where the AI-based visualization controls180are configured to generate and reposition the visualization area195to intercept the distal end112of the needle110moving towards the vasculature120.

According to one embodiment of the disclosure, the AI-based guidance assistance logic190may be configured to monitor the position of the needle110for entry into or exit from the visualization area195. As shown, the visualization area190may be represented as a “bounding box,” namely any shaped region located proximate to and potentially inclusive of the vasculature120along a computed path for advancement of the needle110. Upon entry into the visualization area195, the AI-based guidance assistance logic190notifies the clinician of the proximity of the needle110to a targeted vasculature. The notification may be accomplished through visual, audible and/or haptic (e.g., tactile) feedback signaling provided to the monitor150, the audio device155and/or the sensors within the probe160.

Additionally, upon entry into the visualization area195, the AI-based guidance assistance logic190generates additional imaging enhancements to assist in the visualization of the needle110in close proximity to the vasculature120. For example, one of the imaging enhancements may include a color and/or image overlay of an image of the needle110(as described above) to provide better clarity as to its position. Another imaging enhancement may include activation of a light source (e.g., light emitting diode, etc.) accompanying the needle110, installed on the probe160(light161A) or installed on the console140(light161B). Yet another imaging enhancement may include the generation of a secondary (virtualized) image with appropriate magnification to provide a split-screen view of the needle110proximate to and engaging with the vasculature120.

As mentioned, placement of a medical component (e.g., needle)110into the patient vasculature120at the insertion site125may be performed out-of-plane, where the needle110enters the skin away from the probe160, and is aimed at a plane of an ultrasound beam162emitted from the probe160. With this approach, just the distal tip112of the needle110would be visualized and a remainder of the needle110may be off screen at the time of detection.

Enhanced Guidance System with Sensor-Based Feedback

Embodiments of the present invention described herein are generally directed to the guidance system100for locating and guiding the medical component (e.g., needle, etc.) during ultrasound-based or other suitable procedures in accessing the subcutaneous vasculature of a patient. In one embodiment, the guidance system100tracks the position of the needle in five degrees of motion: x, y, and z spatial coordinate space, pitch, and yaw (e.g., orientation). Such tracking enables the needle to be guided and placed with relatively high accuracy.

As shown inFIG.2, various components of the ultrasound-based guidance system100is shown configured in accordance with one embodiment of the present invention. As shown, the guidance system100generally includes the console140, the monitor150, a TLS sensor290and the probe160, each of which is described in further detail below. Herein, the console140may include a housing200to protect a variety of components of the system100from environmental conditions and may be adapted in accordance with one of a variety of forms. For example, a processor, including a memory such as a non-volatile memory (e.g., electrically erasable programmable read only memory (flash), battery-backed random access memory, etc.)205, is included in the console140for controlling functionality of the guidance system110, thus acting as a control processor. A digital controller/analog interface210is also included within the console140and is in communication with both the processor205and certain system components that govern interfacing between the probe160and other system components.

The guidance system100further includes ports215for connection with additional components such as optional components217including a printer, storage media, keyboard, etc. The ports215, according to one embodiment of the disclosure, may include Universal Serial Bus (USB) ports, though other port types or a combination of port types can be used for this and the other interfaces connections described herein. A power connection220is included with the console140to enable operable connection to an external power supply222. An internal power supply (e.g., battery)224can also be employed, either with or exclusive of the external power supply222. Power management logic230is included within the digital controller/analog interface205of the console140to regulate power use and distribution.

According to one embodiment of the disclosure, the monitor150may be integrated into the console140and may be used to display information to the clinician during the placement procedure, such as an ultrasound image of a region of interest of the patient that is captured by the probe160. In another embodiment, however, the monitor150may be separate from the console140. For this embodiment of the disclosure, the monitor150is a liquid crystal display (LCD) device.

In one embodiment, a console interface logic240and/or probe interface logic250is(are) included on the probe160, which can be used to enable the clinician to select a desired mode to the monitor150for assisting in the needle placement procedure. Signaling from the probe160are routed to the console140via the probe interface logic250. The probe interface250includes a piezoelectric (“piezo”) input/output (I/O) component260and a control I/O component265. The piezo I/O component260interfaces with a sensor (piezo) array270that captures an image through sound waves, including needle reflection that identifies the position, orientation, and movement of the needle110during ultrasound imaging procedures. The control I/O component265interfaces with a control button and memory controller275to receive operational commands therefrom.

The TLS sensor290is employed by the guidance system100to detect a magnetic field produced by the magnetic elements of the medical component such as the needle110or a stylet within the needle110. The TLS sensor290may be placed in proximity to the region of interest for the patient during needle insertion. During advancement of the needle, the TLS sensor290detects the magnetic field associated with the magnetic elements, which provides information to the clinician as to the position and orientation of the medical component (needle)110during its advancement.

The TLS sensor290is operably connected to the console140of the guidance system100via one or more of the ports215. Detection by the TLS sensor290of the magnetic elements within the needle110is graphically displayed on the monitor150of the console140during a modality (TLS mode) set by the probe160. In this way, clinician for controlling the guidance system100may activate the TLS sensor290or establish communications between the TLS sensor290and the monitor150.

Referring now toFIG.3, an exemplary embodiment of the probe160ofFIGS.1-2is shown. The probe160is employed in connection with ultrasound-based visualization of a vasculature, such as a vein, in preparation for insertion of the medical component110(e.g., needle, catheter, etc.) into the vasculature. Such visualization gives real time ultrasound guidance and assists in reducing complications typically associated with such introduction of the needle110, including inadvertent arterial puncture, or the like.

In particular, according to one embodiment of the disclosure, the probe160includes a housing300that features an interface310with externally accessible control buttons315to enable a clinician to control operability of the guidance system100without the reach outside of the sterile field. As shown, the housing300encapsulates the sensor array270(inclusive of a piezoelectric array340and/or magnetic field sensors350) a controller360, and a haptic feedback device380. However, it is contemplated that the sensor array270may be positioned outside of the housing300and attached to the probe160.

As shown, located proximate to a head section320of the housing300, the piezoelectric array340may be configured to produce ultrasonic pulses and to receive echoes thereof after reflection by the patient's body when the head section320is placed against the patient's skin proximate the prospective insertion site125as shown inFIG.1.

As further shown inFIG.3, including non-volatile memory such as flash memory for example, the controller360is for governing button and probe operations. The controller360is in operable communication with the probe interface250of the console200, which includes the piezo I/O component260and the control I/O component265. The piezo I/O component260is configured for interfacing with the piezoelectric array340of the sensor array270while the control I/O component265is configured for interfacing with the controller360.

As further seen inFIG.3, the sensor array270is configured to detect the position, orientation, and movement of the needle110during ultrasound imaging procedures, such as those described above. As will be described in further detail below, the sensor array270includes magnetic field sensors350, namely a plurality of magnetic sensors3501-350N(N≥2) embedded within the housing300of the probe160. According to one embodiment of the disclosure, the sensors3501-350Nare configured to detect a magnetic field associated with the needle110and enable the system100to track the needle110. Though configured here as magnetic sensors, it is appreciated that the sensors3501-350Ncan be sensors of other types and configurations, as will be described. Also, the sensors3501-350Nmay be deployed in a component separate from the probe160, such as a separate handheld device. In the present embodiment, the sensors3501-350Nare disposed in a planar configuration below a top face370of the probe160, though it is appreciated that the sensors3501-350Ncan be arranged in other configurations, such as in an arched or semi-circular arrangement.

In the present embodiment, each of the sensors3501-350Nmay correspond to a three-dimensional sensor such as three orthogonal sensor coils for enabling detection of a magnetic field in three spatial dimensions. Additionally, or in the alternative, the sensors3501-350Nmay be configured as Hall-effect sensors, though other types of magnetic sensors could be employed. Additionally, or in the alternative, a plurality of one-dimensional magnetic sensors can be included and arranged as desired to achieve 1-, 2-, or 3-D detection capability

For this embodiment of the disclosure, five sensors3501-3505are included as part of the sensor array270so as to enable detection of the needle110in not only the three spatial dimensions (i.e., X, Y, Z coordinate space), but also the pitch and yaw orientation of the needle110itself. Note that in one embodiment, orthogonal sensing components of two or more of the sensors3501-350Nenable the pitch and yaw attitude of the medical component110, and thus the needle110, to be determined. However, in other embodiments, fewer or more sensors can be employed in the sensor array270. More generally, it is appreciated that the number, size, type, and placement of the sensors3501-350Ncan vary from what is explicitly shown here.

The haptic feedback device380includes refers to one or more devices that, when activated, creates an experience of touch by applying forces, vibrations or motions to the clinician handling the probe160. According to one embodiment of the disclosure, the haptic feedback device380may be configured to support a haptic feedback corresponding to a vibration and/or a force feedback. The use of vibration may rely on an actuator including an unbalanced weight attacked to an axially rotatable shaft such as an eccentric rotating mass (ERM) actuator. When the actuator is activated, as the shaft rotates, the rotation of this irregular mass causes the actuator and the attached device to shake. Similarly, force feedback relies on motors to manipulate the movement of an item held by the user in order to simulate forces applied on that item.

FIGS.4A and4Bshow details of one example of the needle110that can be used in connection with the guidance system100in accessing a targeted internal body portion of the patient, as shown inFIG.1, according to one embodiment. In particular, the needle110includes a hollow cannula400, which defines a proximal end405and a distal end410. A hub415is attached to the proximal end405of the cannula400and includes an open end420that is configured as a connector for connecting with various devices, in the present embodiment. Indeed, the open end420of the hub415is in communication with the hollow cannula400such that a guide wire, stylet, or other component may be passed through the hub415into the cannula400.

As shown inFIGS.4A and4B, a magnetic element450is included with the hub415. As shown in detail inFIG.4B, the magnetic element450in the present embodiment is a permanent magnet, including a ferromagnetic substance for instance, and is ring-shaped so as to define hole460that is aligned with the hollow cannula400. So configured, the magnetic element450produces a magnetic field that is detectable by the sensor array270of the ultrasound probe160(or perhaps the TLS290) so as to enable the location, orientation, and movement of the needle110to be tracked by the system100, as described further below.

In other embodiments, it is appreciated that many other types, numbers, and sizes of magnetic elements can be employed with the needle110or other medical component to enable tracking thereof by the present guidance system.

Reference is now made toFIGS.5A and5B, which show the ultrasound probe160of the system100and the needle110in position and ready for insertion thereof through a skin surface500of a patient to access a targeted internal body portion. In particular, the probe160is shown with its head section320placed against the patient skin and producing the ultrasound beam162so as to ultrasonically image a portion of the vasculature120beneath the patient skin surface500. The ultrasonic image of the vasculature120can be depicted on the monitor150of the system100(FIG.1).

As mentioned above, the system100in the present embodiment is configured to detect the position, orientation, and movement of the needle110described above. In particular, the sensor array270of the probe160is configured to detect a magnetic field of the magnetic element450included with the needle110. Each of the sensors3501-350Nof the sensor array270is configured to spatially detect the magnetic element450in three-dimensional space. Thus, during operation of the system100, magnetic field strength data of the needle's magnetic element450sensed by each of the sensors3501-350Nis forwarded to a processor, such as the processor205of the console140(FIG.1), which computes in real-time the position and/or orientation of the magnetic element450.

Specifically, and as shown inFIGS.5A and5B, the position of the magnetic element450in X, Y, and Z coordinate space with respect to the sensor array270can be determined by the system100using the magnetic field strength data sensed by the sensors3501-350N. Moreover,FIG.5Ashows that the pitch of the magnetic element450can also be determined, whileFIG.5Bshows that the yaw of the magnetic element450can be determined. Suitable circuitry of the probe160, the console140, or other component of the system can provide the calculations necessary for such position/orientation. In one embodiment, the magnetic element450can be tracked using the teachings of one or more of the following U.S. Pat. Nos. 5,775,322; 5,879,297; 6,129,668; 6,216,028; and 6,263,230. The contents of the afore-mentioned U.S. patents are incorporated herein by reference in their entireties.

The above position and orientation information determined by the system100, together with the length of the cannula400and position of the magnetic element450with respect to the distal needle tip112as known by or input into the system, enable the system to accurately determine the location and orientation of the entire length of the needle110with respect to the sensor array270. Optionally, the distance between the magnetic element450and the distal needle tip112is known by or input into the system100. This in turn enables the system100to superimpose an image of the needle110on to an image produced by the ultrasound beam162of the probe160.FIGS.6A and6Bshow examples of such a superimposition of the needle110onto an ultrasound image600.

Herein,FIGS.6A and6Beach show a screenshot600that can be depicted on the monitor150(FIG.1), for instance. InFIG.6A, an ultrasound image610is shown, including depiction of the patient skin surface620, and the subcutaneous vessel630. The ultrasound image610corresponds to an image acquired by the ultrasound beam162shown inFIGS.5A and5B, for instance.

The screenshot600further shows a needle image640representing the position and orientation of the actual needle110as determined by the guidance system100as described above. Because the system100is able to determine the location and orientation of the needle110with respect to the sensor array270, the guidance system100is able to accurately determine the position and orientation of the needle with respect to the ultrasound image610and superimpose it thereon for depiction as the needle image640on the monitor150. Coordination of the positioning of the needle image640on the ultrasound image610is performed by suitable algorithms executed by the processor205or other suitable component of the guidance system100.

The sensors3501-350Nare configured to continuously detect the magnetic field of the magnetic element450of the needle110during operation of the guidance system100. This enables the guidance system100to continuously update the position and orientation of the needle image640for depiction on the monitor150. Thus, advancement or other movement of the needle110is depicted in real-time by the needle image640on the monitor150. Note that the guidance system100is capable of continuously updating both the ultrasound image610and the needle image640on the monitor150as movements of the probe160and the needle110occur during a placement procedure or other activity.

FIG.6Afurther shows that in one embodiment the system100can depict a projected path650based on the current position and orientation of the needle110as depicted by the needle image640. The projected path650assists a clinician in determining whether the current orientation of the needle110, as depicted by the needle image640on the monitor150, will result in arriving at the desired internal body portion target, such as the vasculature120shown here. Again, as the orientation and/or position of the needle image640changes, the projected path650is correspondingly modified by the system100. A target660, indicating an intended destination for the needle. As shown inFIG.6A, in the present example, the target660is located within the vasculature120depicted in the ultrasound image610. Note that the position of the target660on the monitor150can also be modified as the needle110and/or the ultrasound image610are adjusted.

FIG.6Bshows that, in one embodiment, the screenshot600can be configured such that the ultrasound image610and the needle image640are oriented so as to be displayed in a three-dimensional aspect670. This enables the angle and orientation of the needle110, as depicted by the needle image640, to be ascertained and compared with the intended target imaged by the ultrasound image610. It should be noted that the screenshots600are merely examples of possible depictions produced by the system100for display; indeed, other visual depictions can be used. Note further that the particular area of the body being imaged is merely an example; the system can be used to ultrasonically image a variety of body portions, and should not be limited to what is explicitly depicted in the accompanying figures. Further, the system as depicted and described herein can be included as a component of a larger system, if desired, or can be configured as a stand-alone device. Also, it is appreciated that, in addition to providing visual information rendered on the monitor150, aural information, such as beeps, tones, etc., can also be employed by the system100to assist the clinician during positioning and insertion of the needle into the patient.

As mentioned above, in one embodiment it is necessary for the system100to know the total length of the needle110and the location of the magnetic element450thereon in order to enable an accurate depiction of the needle image640and other features of the screenshots600ofFIGS.6A and6Bto be made. The system100can be informed these and/or other pertinent parameters in various ways, including scanning by the system of a barcode included on or with the needle, the inclusion of a radiofrequency identification (“RFID”) chip with the needle for scanning by the system, color coding of the needle, manual entry of the parameters by the clinician into the system, etc.

In one embodiment, a length of the needle110(or other aspect of a medical component) can be determined by measurement by the probe/system160/100of a characteristic of the magnetic element450ofFIG.4A, such as its field strength. For instance, in one embodiment, the magnetic element450of the needle110can be positioned at a predetermined distance from the probe160or at a predetermined location with respect to the probe160. With the magnetic element450so positioned, the sensor array270of the probe160detects and measures the field strength of the magnetic element450. The system100can compare the measured field strength with a stored list of possible field strengths corresponding to different lengths of needles. The system100can match the two strengths and determine the needle length. The needle location and subsequent needle insertion can then proceed as described herein. In another embodiment, instead of holding the magnetic element450stationary at a predetermined location, the magnetic element450can be moved about the probe160such that multiple field strength readings are taken by the probe160. Aspects that can be modified so as to impart different field strengths to a set of magnetic element include size, shape, and composition of the magnetic element, etc.

Further details are given here regarding use of the system100in guiding a needle or other medical device in connection with ultrasonic imaging of a targeted internal body portion (“target”) of a patient, according to one embodiment. With the magnetic element-equipped needle110positioned a suitable distance (e.g., two or more feet) away from the ultrasound probe160including the sensor array270, the probe160is employed to ultrasonically image, for depiction on the monitor150of the system100, the target within the patient that the needle110is intended to intersect via percutaneous insertion. A calibration of the system100is then initiated, in which algorithms are executed by the processor205of the console140to determine a baseline for any ambient magnetic fields in the vicinity of where the procedure will be performed. The system100is also informed of the total length of the needle110, and/or position of the magnetic element with respect to the distal needle tip112such as by user input, automatic detection, or in another suitable manner, as has been discussed above.

The needle110is then brought into the range of the sensors3501-350Nof the sensor array270of the probe160. Each of the sensors3501-350Ndetects the magnetic field strength associated with the magnetic element450of the needle110, which data is forwarded to the processor205. In one embodiment, such data can be stored in memory until needed by the processor205. As the sensors3501-350Ndetect the magnetic field, suitable algorithms are performed by the processor205to calculate a magnetic field strength of the magnetic element450of the needle110at predicted points in space in relationship to the probe. The processor205then compares the actual magnetic field strength data detected by the sensors3501-350Nto the calculated field strength values. Note that this process is further described by the U.S. patents identified above. This process can be iteratively performed until the calculated value for a predicted point matches the measured data. Once this match occurs, the magnetic element450has been positioned in three-dimensional space. Using the magnetic field strength data as detected by the sensors3501-350N, the pitch and yaw (i.e., orientation) of the magnetic element450can also be determined. Together with the known length of the needle110and the position of the distal tip112of the needle110with respect to the magnetic element450, this enables an accurate representation of the position and orientation of the needle can be made by the system100and depicted as a virtual model, i.e., the needle image640. The representation may be provided as orientation metrics provided to the sensor analytic logic170ofFIG.2or the AI-based visualization logic180ofFIG.8described below.

Depiction of the virtual needle image640of the needle110as described above is performed in the present embodiment by overlaying the needle image on the ultrasound image610of the monitor150ofFIG.1. Suitable algorithms of the system100as executed by the processor205or other suitable component further enable the projected path650and the target660to be determined and depicted on the monitor150atop the ultrasound image610of the target660. The above prediction, detection, comparison, and depiction process is iteratively performed to continue tracking the movement of the needle110in real-time.

Enhanced Guidance System with AI-Based Feedback

Referring toFIG.7, an exemplary embodiment of the architecture of the guidance system100ofFIG.1with the AI-based visualization controls180and the AI-based guidance assistance logic190(hereinafter, “AI-based guidance system700”) is shown. Herein, the AI-based guidance system700features the console140, the monitor150and the probe160as illustrated inFIGS.1-3. More specifically, according to this embodiment, the console140features one or more sets of components within its housing710, which may be oriented in a variety of forms.

Similar to the embodiment illustrated inFIG.1and described above, the AI-based guidance system100may include a first set of components such as one or more ports215for connection with additional components such as optional components217(e.g., printer, storage media, keyboard, etc.). The ports215, according to one embodiment of the disclosure, may include USB ports, although other port types or a combination of port types can be used. The power connection220is included with the console140to enable operable connection to the external power supply222. The internal power supply (e.g., battery)224can also be employed either with or exclusive from the external power supply222.

As a second set of components, the AI-based guidance system700may further include console interface logic240and/or probe interface logic250, which may be included on the probe160and may be deployed within the console140itself. The console interface logic240enables a clinician to alter operability of the console140via a physical interface. The probe interface logic250allows a clinician with control in selecting a desired mode to the monitor150for assisting in the needle placement procedure, as described above. Signaling from the probe160are routed to the console140via the probe interface logic250. The probe interface250includes the piezo I/O component260operating as an interface for the sensor array270and the control I/O component265operating as an interface for the control button and memory controller275.

A third set of components within the console140may include the digital controller/analog interface210, which communicates with both the processor205and other system components that govern the interfacing between the probe160as well as other system components. As shown inFIG.7, the digital controller/analog interface210includes the power management logic230regulates power use and distribution within the console140.

A fourth set of components may include the processor with memory205, namely a processor712with access to logic within a memory715, such as a non-volatile memory (e.g., electrically erasable programmable read only memory (flash), battery-backed random access memory, etc.) for example. The processor712is configured to control functionality of the AI-based guidance system700, thus acting as a control processor. The memory715features the AI-based visualization controls180and the AI-based guidance assistance logic190. In particular, the AI-based visualization controls180include visualization area generating logic720and visualization area positioning logic725. The AI-based guidance assistance logic190includes visualization area monitoring logic740, notification logic745and image enhancement and overlay logic750, where the operability of these logics740-750is described below.

More specifically, upon execution by the processor712, the visualization area generation logic720and the visualization area positioning logic725of the AI-based visualization controls180are configured to generate and re-position, as needed, a sub-region (visualization area195) of the total imaging area captured by the ultrasound beam162based on information received from the medical component tracking subsystem (e.g., sensor array270and piezo I/O component260).

In particular, the probe160is employed in connection with ultrasound-based visualization of a vasculature, such as a vein or artery for example, in preparation for insertion of the medical component110(e.g., needle, catheter, etc.) into the vasculature120. Such visualization gives real time ultrasound guidance and assists in reducing complications typically associated with such introduction. As shown, according to one embodiment of the disclosure, the probe160includes a housing760that features the button/memory controller interface275, the sensor array270, which includes the piezoelectric array340, the controller360and/or the haptic feedback device380, where the operations of the same are described above.

Herein, the processor712is configured to receive an image of a region of the patient (e.g., skin surface and the subcutaneous tissue) captured by the ultrasound beam162emitted from the probe160. Additionally, the AI-based visualization controls180are configured to receive orientation metrics765of the needle110from the probe160, and based on these orientation metrics765, the visualization area generation logic720, when executed by the processor712, generates the visualization area195, which may correspond to a portion of the total imaging area192rendered by the guidance system100(seeFIGS.8A-8E). For this embodiment, the visualization area is substantially lesser in size (e.g., less than a 1/10thof size) than the total imaging area in order to (i) reduce complexity in analysis of the visualization area by the visualization area monitoring logic740(in determining a presence of the needle upon detecting needle reflection) and (ii) reduce the latency in reporting the analysis results. The sizing of the visualization area may be static or dynamic (e.g., based on needle gauge, medical component type, etc.).

Also, based on the orientation metrics765, the visualization area positioning logic725, when executed by the processor712, may alter the location of the visualization area based, at least in part on, the position, orientation and the current path of advancement of the needle110. More specifically, the visualization area positioning logic725may alter the location of the visualization area in order to intercept advancement of the needle110prior to contact with an outer wall surface of the targeted vasculature120.

The visualization area monitoring logic740, when executed by the processor712, is configured to monitor the visualization area for needle reflection, namely artifacts that identify the needle110has entered into and/or exited from the visualization area195. Upon detecting the needle110entering the visualization area, the visualization area monitoring logic740signals the notification logic745to initiate a visual notification to the monitor150upon detection of the needle crossing into or existing the visualization area. Additionally, or in the alterative, the notification logic745may be configured to initiate an audible notification to the audio device (e.g., speaker)775associated with the monitor150or a haptic notification directed to the haptic feedback device380of the probe160upon detection of the needle crossing into or existing the visualization area.

The audio/visual/tactile notification logic745may include hardware and/or software that signals an associated device (e.g., a monitor, computer, audio device, and/or other display) to provide the user with an audio, visual, and/or tactile indication/notification of the proximity of the medical component (needle)110to the predetermined location. The audio, visual, and/or tactile notification may take a variety of forms, including as a graphical or numerical display, a graphical or numerical display of distance between the needle110and the vasculature, a graphical representation of the needle110moving relative to a graphical representation of the vasculature, a sound (e.g., a beep) that changes frequency depending on the location of the needle110relative to the desired location, display colors may change depending on the location of the needle (e.g., a red color may be displayed if the tip is incorrectly positioned), a vibration of one or more of the components of the system (e.g., haptic feedback), a change in temperature of one or more of the components of the system, etc., and combinations thereof.

The image enhancement and overlay logic750is configured to overlay the visualization area over the ultrasonically captured an image of a targeted vasculature. Additionally, the logic750may be configured to determine the image associated with the needle110and apply imaging enhancements to the needle image to better identify the presence, location and/or orientation of the needle.

Referring toFIGS.8A-8E, exemplary embodiments of an embodiment of the visualization area that is generated and monitored by the AI-based guidance system ofFIG.7for a presence of needle reflection within the visualization area195, which triggers enhanced display of a medical component (e.g., needle) and audio/visual/tactile notification.

As shown inFIG.8A, a first screenshot800of an ultrasound image (or series of ultrasound images) captured by the sensor array located within the probe and rendered on the monitor150is shown. Herein, the first screenshot800illustrates the needle110advancing through subcutaneous tissue805toward the vasculature120. Generated by the visualization area generation logic720ofFIG.7, the visualization area195is superimposed, in real-time, over an ultrasound image810that displays the targeted vasculature120and the subcutaneous tissue805surrounding the vasculature120. Represented as a dashed box, the visualization area195defines an area within which needle reflection is monitored.

Furthermore, for this embodiment, the AI-based guidance system tracks the positioning and advancement of the needle110in accordance with numerous degrees of motion as illustrated by a second screenshot820inFIG.8B. Herein, the position of the visualization area195is adjusted to accommodate an angular adjustment825of the needle110. The re-positioning is determined from the orientation metrics that corresponds or is related to the xyz spatial coordinate space, pitch, and/or yaw (e.g., orientation) of the needle110.

Such tracking of the needle110and re-positioning of the visualization area195enables the distal end112of the needle110to be guided and placed to intersect a border840of the visualization area195, as shown by a third screenshot830ofFIG.8C. Upon detecting entry of the distal end112of the needle110into the visualization area195(e.g., based on detected needle reflection associated with the distal end112), the visualization area monitoring logic740(seeFIG.7) signals the notification logic745to supply a notification (e.g., signaling associated with visual notification, audible notification, haptic notification or any combination thereof) to a device associated with the console140(e.g., monitor150, speaker155, probe160).

Additionally, upon detecting needle reflection within the visualization area195by the visualization area monitoring logic740ofFIG.7, the image enhancement and overlay logic750applies one or more imaging enhancements845to items within or forming the visualization area195such as the needle110, the border840of the visualization area195or the like, as shown in a fourth screenshot850inFIG.8D. The imaging enhancements may include, but are not limited or restricted to altering the displayed image of the needle (e.g., change color, highlight, lighten or darker from the rest of the ultrasound image, etc.), altering the resolution of the visualization area195by generating a higher resolution virtualization that corresponds to the visualization area, or the like.

As now shown in a fifth screenshot860ofFIG.8E, upon removal of the needle110from the visualization area195, the visualization area monitoring logic740signals the AI-based image enhancement and overlay logic750to cease generation of the imaging enhancements. Optionally, upon removal of the needle110from the visualization area195, the visualization area monitoring logic740signals the AI-based notification logic745to generate a notification (visual, audible, and/or haptic) to notify the clinician that the needle110has exited the visualization area195.

Referring now toFIG.9A-9B, an exemplary embodiment of a method of operation conducted by the AI-based guidance assistance logic ofFIG.7is shown. Herein, as shown inFIG.9A, the visualization area, sized substantially less than the total imaging area, is generated and rendered on the monitor of the AI-based guidance system (operation900). According to one embodiment, the visualization area overlays a portion of the total imaging area proximate to a targeted vasculature. Thereafter, based on communications between the probe interface logic and the AI-based visualization controls, the positioning of the visualization area is adjusted. The adjustment is made by the AI-based visualization controls to position the visualization area around the vasculature so as to intercept the advancement of a medical component (e.g., needle) directed to that vasculature (operation905).

In response to failing to detect needle reflection, the AI-based guidance assistance logic continues to monitor for artifacts (e.g., needle reflection, etc.) that identify a presence of the needle within the visualization area (operations910,912). However, upon detecting needle reflection, the AI-based guidance assistance logic identifies entry of the needle into the visualization area and generates a notification to alert a clinician of the positioning of the needle (operations914,915). Certain images within the visualization area are selected to undergo imaging enhancements by the AI-based guidance assistance logic, such as altering the displayed appearance of the needle (e.g., change in color, highlight, or outline for example (operation920). The imaging enhancements continue while the needle is within the visualization area (operations925,927).

Responsive to the needle being removed from the visualization area as detected by the AI-based guidance assistance logic, as shown inFIG.9B, the imaging enhancements are halted (operations929,930,935). Optionally, the AI-based guidance assistance logic may generate a secondary notification to alert the clinician of movement of the needle away from the targeted vasculature (operation940). Communications between the probe interface logic and the AI-based visualization controls continue (in case of a secondary needle insertion, etc.) until the guidance session (ultrasound session) is terminated (operations945,950).

Embodiments of the invention may be embodied in other specific forms without departing from the spirit of the present disclosure. The described embodiments are to be considered in all respects only as illustrative, not restrictive. The scope of the embodiments is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.