Abstract:
A system for providing navigational guidance to a sonographer acquiring images is disclosed. The system may providehaptic feedback to the sonographer. The haptic feedback may be provided through an ultrasonic probe or a separate device. Haptic feedback may include vibrations or other sensations provided to the sonographer. The system may analyze acquired images and determine the location of acquisition and compare it to a desired image and a location for obtaining the desired image. The system may calculate the location for obtaining the desired image based, at least in part, on the acquired image. The system may then provide the haptic feedback to guide the sonographer to move the ultrasonic probe to the location to acquire the desired image.

Description:
TECHNICAL FIELD 
       [0001]    This invention relates to medical ultrasonic imaging systems and, in particular, to haptic feedback devices to aid sonographers acquiring ultrasound images. 
       BACKGROUND 
       [0002]    A difficulty plaguing ultrasound image acquisitions is that of ultrasound probe navigation. As the acquired ultrasound images do not have a fixed reference frame, it may be difficult for a sonographer to know at any given time where to move the probe to acquire images of an object of interest. Often the knowledge of how and where to move the probe to acquire an image of a particular object of interest is gained from significant experience. There is increasing demand for imaging applications to provide not only qualitative assessments, but also quantitative measurements. These quantitative measurements may be manual, semi-automatic, or fully automatic computer analysis of acquired images. Image quality and correct field of view are even more critical in these automatic analysis applications. These increased demands for image acquisition may be challenging to meet, even for skilled sonographers. 
         [0003]    As real-time segmentation algorithms are becoming more prevalent, the possibility of providing real-time navigational feedback to the sonographer to indicate regions in which the sonographer may want to acquire more image data is becoming a greater possibility. However, translating this navigational information to the sonographer in a meaningful way is not trivial. While information about where to move the probe could be shown on the images displayed on a monitor, it may not be obvious to the sonographer how to translate or rotate the probe so as to acquire images in the desired regions. Furthermore, as it is desirable for the sonographer to continuously observe the acquired images displayed on the monitor, any navigational information about where to move the probe needs to be translated to the sonographer without the use of visual cues solely on the displayed images or on the probe itself. 
       SUMMARY OF THE INVENTION 
       [0004]    Through the use of a haptic information system, real-time navigational feedback may be provided to the sonographer in an intuitive fashion. 
         [0005]    According to one illustrative embodiment of the invention disclosure, a system for providing navigational guidance to a sonographer may include an ultrasound probe that may transmit and receive an echo signal, an acquisition system that may receive a signal from the ultrasound probe corresponding to the echo signal received by the ultrasound probe and produce an image, a display system that may receive the image from the acquisition system, the display system may include an anatomical analytical model that may analyze the image and transmit data to a tracking processor that may calculate a movement of the ultrasound probe to acquire an image based at least in part, on data received from the anatomical model, and a navigation instruction generator that may convert the movement calculated by the tracking processor into navigational instructions that may be sent to a haptic apparatus included with the ultrasound probe which may be operate a haptic feedback device based at least in part on the navigational instructions, and the haptic feedback device may provide haptic navigational instructions to the sonographer. The haptic apparatus may comprise a plurality of haptic feedback devices distributed across an inner surface of the haptic apparatus. The haptic apparatus may operate the plurality of haptic feedback devices in accordance with a navigational instruction set where a combination of haptic feedback devices operated simultaneously may correspond to a navigational instruction. The haptic feedback device may be a motor that may provide vibration. The haptic apparatus may further include a force sensor. The system may receive data from the force sensor and calculate a movement of the ultrasound probe to acquire an image based at least in part on the data received from the force sensor. The system may operate continually to provide the sonographer with navigational guidance. The tracking processor may receive physiological data and calculate a movement of the ultrasound probe to acquire an image based, at least in part on the physiological data. 
         [0006]    According to another disclosed embodiment of the present invention, a method of providing navigational guidance to a sonographer may include analyzing an image acquired by an ultrasound prove with an anatomical analytical model; calculating movement of the ultrasound probe based at least in part on the analysis of the image; and providing haptic feedback through the ultrasound probe to navigate the ultrasound probe. The method may further include analyzing a second image acquired by the ultrasound probe with the anatomical model to determine sufficiency of the image. The method may further include providing a signal to the sonographer when the sufficient image has been acquired. The signal may be a visual signal. The method may further include transmitting instructions to a haptic apparatus attached to the ultrasound probe and activating haptic feedback devices in the haptic apparatus to provide the haptic feedback to the sonographer. 
         [0007]    According to a further embodiment according to the principles of the invention, a non-transitory computer-readable medium with instructions for navigational guidance in acquiring an ultrasound image stored thereon to be executed by one or more processors, which instructions when executed may cause the one or more processors to emit ultrasound waves from an ultrasound probe, generate an image from an echo signal received by the ultrasound probe, analyze the image to determine if the image is sufficient, calculate a required movement of the ultrasound probe to obtain a sufficient image, generate navigational instructions based on the required movement, and transmit navigational instructions to a haptic apparatus. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]      FIG. 1  is a block diagram of a medical ultrasound system according to an illustrative embodiment of the invention. 
           [0009]      FIG. 2  is a block diagram of a haptic apparatus according to an illustrative embodiment of the invention. 
           [0010]      FIG. 3  is a block diagram of a navigational instruction set according to an illustrative embodiment of the invention. 
           [0011]      FIG. 4  is a flow chart of the operation of an illustrative embodiment of the invention. 
       
    
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
       [0012]    In the following detailed description, for purposes of explanation and not limitation, illustrative embodiments disclosing specific details are set forth in order to provide a thorough understanding of an embodiment according to the present teachings. However, it will be apparent to one having ordinary skill in the art having had the benefit of the present disclosure that other embodiments according to the present teachings that depart from the specific details disclosed herein remain within the scope of the appended claims. Moreover, descriptions of well-known apparatus and methods may be omitted so as to not obscure the description of the illustrative embodiments. Such methods and apparatus are within the scope of the present teachings. 
         [0013]    Referring to  FIG. 1 , an ultrasonic imaging system according to an embodiment of the present invention is shown in block diagram form. The ultrasound system is configured by two subsystems, a front end acquisition subsystem  10 A and a display subsystem  10 B. An ultrasound probe  60  is coupled to the acquisition subsystem which includes a two-dimensional matrix array transducer  70  and a micro-beamformer  72 . The micro-beamformer contains circuitry which control the signals applied to groups of elements (“patches”) of the array transducer  70  and does some processing of the echo signals received by elements of each group. 
         [0014]    The acquisition subsystem  10 A includes a beamform controller  74  which is responsive to a user control  36  and provides control signals to the microbeamformer  72 , for example, instructing the probe as to the timing, frequency, direction and focusing of transmit beams. The beamform controller also controls the beamforming of echo signals received by the acquisition subsystem by its control of analog-to-digital (A/D) converters  18  and a system beamformer  20 . Echo signals received by the probe are amplified by preamplifier and TGC (time gain control) circuitry  16  in the acquisition subsystem, then digitized by the A/D converters  18 . The digitized echo signals are then formed into fully steered and focused beams by the system beamformer  20 . The echo signals are then processed by a signal processor  22  which performs digital filtering, B mode and M mode detection, and Doppler processing, and can also perform other signal processing such as harmonic separation, speckle reduction, and other desired image signal processing. 
         [0015]    The echo signals produced by the acquisition subsystem  10 A are coupled to the display subsystem  10 B, which processes the echo signals for display in the desired image format. The echo signals are processed by an image line processor  24 , which is capable of sampling the echo signals, splicing segments of beams into complete line signals, and averaging line signals for signal-to-noise improvement or flow persistence. The image lines for a 2D image are scan converted into the desired image format by a scan converter  26  which performs R-theta conversion as is known in the art. The image is then stored in an image buffer or memory  28  from which it can be displayed on a display  38 . The image in memory  28  is also overlaid with graphics to be displayed with the image, which are generated by a graphics generator (not shown) which is responsive to the user control  36 . Individual images or image sequences can be stored in a cine memory (not shown) during capture of image loops or sequences. 
         [0016]    For real-time volumetric imaging the display subsystem  10 B also includes a 3D image rendering processor  32  which receives image lines from the image line processor  24  for the rendering of real-time three dimensional images. The 3D images can be displayed as live (real time) 3D images on the display  38  or coupled to the image memory  28  for storage of the 3D data sets for later review and diagnosis. 
         [0017]    In accordance with the principles of the present invention the display subsystem may also include an automated anatomical analytical model stored in memory  40 . An example of such an anatomical analytical model is the Heart Model technology described in U.S. patent application Ser. No. 13/884,617 “Identifying individual sub-regions of the cardiovascular system for calcium scoring.” This technology may be able to rapidly segment a majority of the cardiac anatomy (chambers, vasculature, etc.) from 3D ultrasound volumes using a model-based approach, and in doing so, may determine quickly those areas where sufficient or insufficient image data was found. A second example of an anatomical analytical model is a model to predict the deformation of a biopsy needle to aid sonographers in keeping the tip of the needle in the field of view of the transducer  70  during a biopsy procedure. In non-medical applications, the anatomical model may be replaced with any appropriate model for the object to be imaged for determining areas where sufficient or insufficient image data are found. 
         [0018]    Data from the analytical model  40  may be transmitted to the tracking processor  42 . The tracking processor  42  may predict where the ultrasound probe  60  should move relative to its current position to obtain the desired image based at least in part on data provided from the analytical model  40  and transmit the required probe movement to the navigation instruction generator  44 , which generates navigation instructions that are transmitted to a haptic apparatus  200 , described in more detail below. The tracking processor  42  could indicate on the display  38  where more image data is needed and indicate how to move the probe  60  relative to its current position. However, due to the symmetry of the probe  60 , the sonographer may not always know exactly what movements of the probe coincide with the necessary translations and/or rotations required. While visual cues on the ultrasound probe  60  (for example, LEDs) could be used to indicate to the sonographer how to move the probe  60 , it is desirable to have the sonographer maintain constant observation of the displayed images, especially for interventional cases where the relative position of anatomy and tools/instruments/devices are being maneuvered within the field of view. 
         [0019]      FIG. 2  illustrates an embodiment of a haptic apparatus  200  that may be attached to the exterior of the ultrasound probe  60  or integrated inside the enclosure of ultrasound probe  60 . The haptic apparatus  200  provides an intuitive and non-intrusive way to communicate to the sonographer the information from the tracking processor  42 . The haptic apparatus  200  may be configured to provide physical sensations to the sonographer&#39;s hand holding the ultrasound probe  60 . These physical sensations for conveying where the ultrasound probe  60  should be moved as calculated by the tracking processor  42  are haptic navigational instructions. The haptic apparatus  200  comprises a plurality of haptic feedback devices  201 - 208 . Eight haptic feedback devices are pictured in this illustrative embodiment, but more or less could be used. The haptic feedback devices  201 - 208  may be motors that generate a vibration that can be felt by a sonographer holding the ultrasound probe  60 . Power and navigation instructions from the navigation instruction generator  44  are delivered by a cord  215 . 
         [0020]    The desired movement of the ultrasound probe  60  calculated by the tracking processor  42  may be translated into a sequence of vibrational pulses sent to one or more haptic feedback devices  201 - 208  by the navigation instruction generator  44 . The instructions may be translated into causing vibration at one or more haptic feedback devices  201 - 208  and/or different vibrational strengths at one or more haptic feedback devices  201 - 208 . The pattern or sequence of activating the haptic feedback devices  201 - 208  may be determined by a pre-determined navigation instruction set. 
         [0021]    An example of a navigation instruction set  300  is shown in  FIG. 3 . Other navigation instruction sets may be possible. All directions described below are from the perspective of the reader, not the haptic apparatus  200  or a sonographer. Instructions (a)-(f) describe how to move the ultrasound probe  60  in a 3D space. In (a), two haptic feedback devices  207 ,  208  on the right of the haptic apparatus  200  vibrate to indicate to the sonographer to move the probe  60  in direction  305  along the x-axis to the right. In (b), two haptic feedback devices  203 ,  204  on the left side of the haptic apparatus  200  vibrate to indicate to the sonographer to move the probe  60  in direction  310  along the x-axis to the left. In (c), two haptic feedback devices  201 ,  202  on the front of the haptic apparatus  200  vibrate to indicate to the sonographer to move the probe  60  in direction  315  along the y-axis out of the page. In (d), two haptic feedback devices  205 ,  206  on the back of the haptic apparatus  200  vibrate to indicate to the sonographer to move the probe  60  in direction  320  along the y-axis into the page. In (e), four haptic feedback devices  202 ,  204 ,  206 ,  208  on the lower portion of the haptic apparatus  200  vibrate to indicate to the sonographer to move the probe downward in direction  325  along the z-axis. In (f), four haptic feedback devices  201 ,  203 ,  207  on the upper portion of the haptic apparatus  200  vibrate to indicate to the sonographer to move the probe  60  upward in direction  330  along the z-axis. 
         [0022]    Instructions (g)-(l) describe how to rotate the ultrasound probe  60  to adjust the angle at which the transducer  70  is incident to the object being imaged. In (g), the front lower haptic feedback device  202  and the upper back haptic feedback device  205  vibrate to indicate to the sonographer to rotate the probe  60  in direction  335  counterclockwise around the x-axis. In (h) the front upper haptic feedback device  201  and the lower back haptic feedback device  206  vibrate to indicate to the sonographer to rotate the probe  60  in direction  340  clockwise around the x-axis. In (i) the lower left haptic feedback device  204  and the upper right haptic feedback device  207  vibrate to indicate to the sonographer to rotate the probe  60  in direction  345  counterclockwise around the y-axis. In (j), the upper left haptic feedback device  203  and the lower right haptic feedback device  208  vibrate to indicate to the sonographer to rotate the probe  60  in direction  350  clockwise around the y-axis. In (k) the upper front haptic feedback device  201  and the lower left haptic feedback device  204  vibrate to indicate to the sonographer to rotate the probe  60  in direction  355  clockwise around the z-axis. Finally, in (l) the upper front haptic feedback device  201  and the lower right haptic feedback device  208  vibrate to indicate to the sonographer to rotate the probe  60  in direction  360  counter clockwise around the z-axis. 
         [0023]    In another embodiment of the invention, the haptic apparatus  200  may also include one or more force sensors (not shown) adjacent to the transducer  70 . Data from the force sensors may be sent to the tracking processor  42 , and the navigation instruction generator  44  may provide instructions to the sonographer via the haptic apparatus  200  to increase or decrease pressure applied with the probe. Other physiological data that could be collected and provided to the tracking processor  42  to provide haptic feedback to the sonographer include respiration rate and ECG signals. This data could be collected by additional sensors integrated into the haptic apparatus  200  or may be separate devices configured to transmit data to the tracking processor  42 . 
         [0024]      FIG. 4  is a flow diagram of an example process of acquiring an image with an embodiment of the invention. A sonographer acquires an image  405  with the ultrasound probe  60 . The anatomical analytical model  40  analyzes the image to determine if the image is sufficient  410 . If the image is determined to be sufficient, the process terminates at  445 . For an image to be sufficient, the image may be of the desired quality and at the correct field of view. The sonographer may be alerted to the sufficiency of the image by a visual signal on the display  38  or other signal. If the anatomical analytical model  40  determines that the image is insufficient, the tracking processor  42  calculates the required movement of the ultrasound probe  60  to acquire the desired image  420 . The required movement of the ultrasound probe  60  is transmitted to the navigation instruction generator  44 , and the required movement is translated into instructions to be provided to the sonographer  425 . The navigation instruction generator  44  transmits instructions to the haptic apparatus  200  at step  430 . The haptic apparatus transmits the navigation instructions to the sonographer  435  using the haptic feedback devices  201 - 208  utilizing an instruction set such as the one illustrated in  FIG. 3 . The sonographer may move the ultrasound probe  60  based, at least in part, on the instructions provided by the haptic apparatus  200 , and acquires a new ultrasound image  440 . This new image is then transmitted to the anatomical analytical model for analysis  410 . The process repeats until a sufficient image is acquired by the sonographer. 
         [0025]    In various embodiments where the above-described systems and/or methods are implemented using a programmable device, such as a computer-based system or programmable logic, it should be appreciated that the above-described systems and methods can be implemented using any of various known or later developed programming languages, such as “C”, “C++”, “FORTRAN”, Pascal”, “VHDL” and the like. 
         [0026]    Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memories and the like, can be prepared that can contain information that can direct a device, such as a computer, to implement the above-described systems and/or methods. Once an appropriate device has access to the information and programs contained on the storage media, the storage media can provide the information and programs to the device, thus enabling the device to perform the above-described systems and/or methods. 
         [0027]    For example, if a computer disk containing appropriate materials, such as a source file, an object file, an executable file or the like, were provided to a computer, the computer could receive the information, appropriately configure itself and perform the functions of the various systems and methods outlined in the diagrams and flowcharts above to implement the various functions. That is, the computer could receive various portions of information from the disk relating to different elements of the above-described systems and/or methods, implement the individual systems and/or methods and coordinate the functions of the individual systems and/or methods described above. 
         [0028]    In view of this disclosure it is noted that the various methods and devices described herein can be implemented in hardware, software and firmware. Further, the various methods and parameters are included by way of example only and not in any limiting sense. In view of this disclosure, those of ordinary skill in the art can implement the present teachings in determining their own techniques and needed equipment to affect these techniques, while remaining within the scope of the invention.