Patent ID: 12213830

DETAILED DESCRIPTION

Certain embodiments may be found in a method and system for automatically detecting an ultrasound image view and focus to provide feedback on measurement suitability. Aspects of the present disclosure have the technical effect of automatically detecting an ultrasound image view and focus of an acquired ultrasound image. Various embodiments have the technical effect of presenting a list of measurements associated with anatomy present in a detected ultrasound image view. Certain embodiments have the technical effect of automatically grading a suitability of the acquired ultrasound image for performing measurements associated with the anatomy present in the detected ultrasound image view. Aspects of the present disclosure have the technical effect of presenting the ultrasound image suitability grades of the measurements associated with the anatomy present in the detected ultrasound image view in substantially real-time such that an ultrasound operator may manipulate the ultrasound probe to acquire a view with improved measurement grades and/or select to manually or automatically perform one or more of the listed measurements. Certain embodiments have the technical effect of providing immediate user feedback of ultrasound image suitability for performing measurements.

The foregoing summary, as well as the following detailed description of certain embodiments will be better understood when read in conjunction with the appended drawings. To the extent that the figures illustrate diagrams of the functional blocks of various embodiments, the functional blocks are not necessarily indicative of the division between hardware circuitry. Thus, for example, one or more of the functional blocks (e.g., processors or memories) may be implemented in a single piece of hardware (e.g., a general-purpose signal processor or a block of random access memory, hard disk, or the like) or multiple pieces of hardware. Similarly, the programs may be stand alone programs, may be incorporated as subroutines in an operating system, may be functions in an installed software package, and the like. It should be understood that the various embodiments are not limited to the arrangements and instrumentality shown in the drawings. It should also be understood that the embodiments may be combined, or that other embodiments may be utilized, and that structural, logical and electrical changes may be made without departing from the scope of the various embodiments. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and their equivalents.

As used herein, an element or step recited in the singular and preceded with the word “a” or “an” should be understood as not excluding plural of said elements or steps, unless such exclusion is explicitly stated. Furthermore, references to “an exemplary embodiment,” “various embodiments,” “certain embodiments,” “a representative embodiment,” and the like are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. Moreover, unless explicitly stated to the contrary, embodiments “comprising”, “including”, or “having” an element or a plurality of elements having a particular property may include additional elements not having that property.

Also as used herein, the term “image” broadly refers to both viewable images and data representing a viewable image. However, many embodiments generate (or are configured to generate) at least one viewable image. In addition, as used herein, the phrase “image” is used to refer to an ultrasound mode such as B-mode (2D mode), M-mode, three-dimensional (3D) mode, CF-mode, PW Doppler, CW Doppler, Contrast Enhanced Ultrasound (CEUS), and/or sub-modes of B-mode and/or CF such as Harmonic Imaging, Shear Wave Elasticity Imaging (SWEI), Strain Elastography, TVI, PDI, B-flow, MVI, UGAP, and in some cases also MM, CM, TVD where the “image” and/or “plane” includes a single beam or multiple beams.

Furthermore, the term processor or processing unit, as used herein, refers to any type of processing unit that can carry out the required calculations needed for the various embodiments, such as single or multi-core: CPU, Accelerated Processing Unit (APU), Graphic Processing Unit (GPU), DSP, FPGA, ASIC or a combination thereof.

It should be noted that various embodiments described herein that generate or form images may include processing for forming images that in some embodiments includes beamforming and in other embodiments does not include beamforming. For example, an image can be formed without beamforming, such as by multiplying the matrix of demodulated data by a matrix of coefficients so that the product is the image, and wherein the process does not form any “beams”. Also, forming of images may be performed using channel combinations that may originate from more than one transmit event (e.g., synthetic aperture techniques).

In various embodiments, ultrasound processing to form images is performed, for example, including ultrasound beamforming, such as receive beamforming, in software, firmware, hardware, or a combination thereof. One implementation of an ultrasound system having a software beamformer architecture formed in accordance with various embodiments is illustrated inFIG.1.

FIG.1is a block diagram of an exemplary ultrasound system100that is operable to automatically detect an ultrasound image view and focus to provide feedback on measurement suitability, in accordance with various embodiments. Referring toFIG.1, there is shown an ultrasound system100and a training system200. The ultrasound system100comprises a transmitter102, an ultrasound probe104, a transmit beamformer110, a receiver118, a receive beamformer120, A/D converters122, a RF processor124, a RF/IQ buffer126, a user input device130, a signal processor132, an image buffer136, a display system134, and an archive138.

The transmitter102may comprise suitable logic, circuitry, interfaces and/or code that may be operable to drive an ultrasound probe104. The ultrasound probe104may comprise a two dimensional (2D) array of piezoelectric elements. The ultrasound probe104may comprise a group of transmit transducer elements106and a group of receive transducer elements108, that normally constitute the same elements. In certain embodiment, the ultrasound probe104may be operable to acquire ultrasound image data covering at least a substantial portion of an anatomy, such as the heart, a blood vessel, or any suitable anatomical structure.

The transmit beamformer110may comprise suitable logic, circuitry, interfaces and/or code that may be operable to control the transmitter102which, through a transmit sub-aperture beamformer114, drives the group of transmit transducer elements106to emit ultrasonic transmit signals into a region of interest (e.g., human, animal, underground cavity, physical structure and the like). The transmitted ultrasonic signals may be back-scattered from structures in the object of interest, like blood cells or tissue, to produce echoes. The echoes are received by the receive transducer elements108.

The group of receive transducer elements108in the ultrasound probe104may be operable to convert the received echoes into analog signals, undergo sub-aperture beamforming by a receive sub-aperture beamformer116and are then communicated to a receiver118. The receiver118may comprise suitable logic, circuitry, interfaces and/or code that may be operable to receive the signals from the receive sub-aperture beamformer116. The analog signals may be communicated to one or more of the plurality of A/D converters122.

The plurality of A/D converters122may comprise suitable logic, circuitry, interfaces and/or code that may be operable to convert the analog signals from the receiver118to corresponding digital signals. The plurality of A/D converters122are disposed between the receiver118and the RF processor124. Notwithstanding, the disclosure is not limited in this regard. Accordingly, in some embodiments, the plurality of A/D converters122may be integrated within the receiver118.

The RF processor124may comprise suitable logic, circuitry, interfaces and/or code that may be operable to demodulate the digital signals output by the plurality of A/D converters122. In accordance with an embodiment, the RF processor124may comprise a complex demodulator (not shown) that is operable to demodulate the digital signals to form I/Q data pairs that are representative of the corresponding echo signals. The RF or I/Q signal data may then be communicated to an RF/IQ buffer126. The RF/IQ buffer126may comprise suitable logic, circuitry, interfaces and/or code that may be operable to provide temporary storage of the RF or I/Q signal data, which is generated by the RF processor124.

The receive beamformer120may comprise suitable logic, circuitry, interfaces and/or code that may be operable to perform digital beamforming processing to, for example, sum the delayed channel signals received from RF processor124via the RF/IQ buffer126and output a beam summed signal. The resulting processed information may be the beam summed signal that is output from the receive beamformer120and communicated to the signal processor132. In accordance with some embodiments, the receiver118, the plurality of A/D converters122, the RF processor124, and the beamformer120may be integrated into a single beamformer, which may be digital. In various embodiments, the ultrasound system100comprises a plurality of receive beamformers120.

The user input device130may be utilized to input patient data, scan parameters, settings, select protocols and/or templates, select measurements, and the like. In an exemplary embodiment, the user input device130may be operable to configure, manage and/or control operation of one or more components and/or modules in the ultrasound system100. In this regard, the user input device130may be operable to configure, manage and/or control operation of the transmitter102, the ultrasound probe104, the transmit beamformer110, the receiver118, the receive beamformer120, the RF processor124, the RF/IQ buffer126, the user input device130, the signal processor132, the image buffer136, the display system134, and/or the archive138. The user input device130may include button(s), rotary encoder(s), a touchscreen, motion tracking, voice recognition, a mousing device, keyboard, camera and/or any other device capable of receiving a user directive. In certain embodiments, one or more of the user input devices130may be integrated into other components, such as the display system134or the ultrasound probe104, for example. As an example, user input device130may include a touchscreen display.

The signal processor132may comprise suitable logic, circuitry, interfaces and/or code that may be operable to process ultrasound scan data (i.e., summed IQ signal) for generating ultrasound images for presentation on a display system134. The signal processor132is operable to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the acquired ultrasound scan data. In an exemplary embodiment, the signal processor132may be operable to perform display processing and/or control processing, among other things. Acquired ultrasound scan data may be processed in real-time during a scanning session as the echo signals are received. Additionally or alternatively, the ultrasound scan data may be stored temporarily in the RF/IQ buffer126during a scanning session and processed in less than real-time in a live or off-line operation. In various embodiments, the processed image data can be presented at the display system134and/or may be stored at the archive138. The archive138may be a local archive, a Picture Archiving and Communication System (PACS), or any suitable device for storing images and related information.

The signal processor132may be one or more central processing units, microprocessors, microcontrollers, and/or the like. The signal processor132may be an integrated component, or may be distributed across various locations, for example. In an exemplary embodiment, the signal processor132may comprise an image view detection processor140, a measurement identification processor150, and a measurement grading processor160. The signal processor132may be capable of receiving input information from a user input device130and/or archive138, generating an output displayable by a display system134, and manipulating the output in response to input information from a user input device130, among other things. The signal processor132, image view detection processor140, measurement identification processor150, and measurement grading processor160may be capable of executing any of the method(s) and/or set(s) of instructions discussed herein in accordance with the various embodiments, for example.

The ultrasound system100may be operable to continuously acquire ultrasound scan data at a frame rate that is suitable for the imaging situation in question. Typical frame rates range from 20-120 but may be lower or higher. The acquired ultrasound scan data may be displayed on the display system134at a display-rate that can be the same as the frame rate, or slower or faster. An image buffer136is included for storing processed frames of acquired ultrasound scan data that are not scheduled to be displayed immediately. Preferably, the image buffer136is of sufficient capacity to store at least several minutes' worth of frames of ultrasound scan data. The frames of ultrasound scan data are stored in a manner to facilitate retrieval thereof according to its order or time of acquisition. The image buffer136may be embodied as any known data storage medium.

The signal processor132may include an image view detection processor140that comprises suitable logic, circuitry, interfaces and/or code that may be operable to analyze acquired ultrasound images to automatically identify an ultrasound image view and focus based on detected anatomical structures in the ultrasound images. The image view detection processor140may include image analysis algorithms, artificial intelligence algorithms, one or more deep neural networks (e.g., a convolutional neural network) and/or may utilize any suitable form of image analysis techniques or machine learning processing functionality configured to analyze acquired ultrasound images to automatically identify an ultrasound image view and focus. For example, the image view detection processor140may identify a four chamber (4CH) or parasternal long-axis (PLAX) ultrasound image view based on a detected presence of anatomical structures in the particular view, such as by detecting a left ventricle, right ventricle, left atrium, right atrium, aorta, left ventricle outflow tract, and/or any suitable anatomical structures. As another example, the image view detection processor140may identify anatomical structures in focus based on an amount of image detail for each of the anatomical structures. The image view detection processor140may be configured to provide the identified ultrasound image views and focus to the measurement identification processor150and/or the measurement grading processor160. The image view detection processor140may additionally and/or alternatively store the identified ultrasound image views and focus at archive138and/or any suitable data storage medium.

The image view detection processor140may comprise suitable logic, circuitry, interfaces and/or code that may be operable to automatically identify an ultrasound image view and focus based on detected anatomical structures in acquired ultrasound images. In various embodiments, the image view detection processor140may be provided as a deep neural network that may be made up of, for example, an input layer, an output layer, and one or more hidden layers in between the input and output layers. Each of the layers may be made up of a plurality of processing nodes that may be referred to as neurons. For example, the image view detection processor140may include an input layer having a neuron for each pixel or a group of pixels from a scan plane of a heart. The output layer may have neurons corresponding to a left ventricle, right ventricle, left atrium, right atrium, aorta, left ventricle outflow tract, and/or any suitable anatomical structures of the heart. Each neuron of each layer may perform a processing function and pass the processed ultrasound image information to one of a plurality of neurons of a downstream layer for further processing. As an example, neurons of a first layer may learn to recognize edges of structure in the ultrasound image data. The neurons of a second layer may learn to recognize shapes based on the detected edges from the first layer. The neurons of a third layer may learn positions of the recognized shapes relative to landmarks in the ultrasound image data. The processing performed by the image view detection processor140deep neural network (e.g., convolutional neural network) may identify an ultrasound image view and focus based on detected anatomical structures in acquired ultrasound images with a high degree of probability.

The signal processor132may include measurement identification processor150that comprises suitable logic, circuitry, interfaces and/or code that may be operable to automatically select one or more measurements associated with anatomical structures in the ultrasound image view identified by the image view detection processor140. For example, ultrasound image views may each be associated with one or more potentially relevant measurements. The one or more potentially relevant measurements may be associated with one or more ultrasound image views and stored at archive138and/or any suitable data storage medium. For example, potentially relevant measurements associated with a 4CH view of a heart may include a left ventricle (LV) area measurement, right ventricle (RV) area measurement, left atrium (LA) area measurement, right atrium (RA) area measurement, RV length measurement, RV mid diameter measurement, RV base diameter measurement, and/or any suitable 4CH view measurement. As another example, potentially relevant measurements associated with a PLAX view of a heart may include an interventricular septum (IVS) measurement, left ventricle internal dimension (LVID) measurement, left ventricle posterior wall (LVPW) measurement, right ventricle internal dimension (RVID) measurement, left ventricle outflow tract (LVOT) measurement, left atrium (LA) measurement, aorta (Ao) measurement, and/or any suitable PLAX view measurement. The measurement identification processor150may be configured to select the one or more potentially relevant measurements from the archive138and/or any suitable data storage medium based on the association and the ultrasound image view identified by the image view detection processor140. The measurement identification processor150may be configured to present the selected one or more potentially relevant measurements at the display system134and/or provide the selected one or more potentially relevant measurements to the measurement grading processor160.

The signal processor132may include a measurement grading processor160that comprises suitable logic, circuitry, interfaces and/or code that may be operable to automatically analyze the ultrasound image and assign a grade for each of the selected measurements and/or associated anatomical structures based on an ability to accurately perform each measurement in the ultrasound image. The measurement grading processor160may include image analysis algorithms, artificial intelligence algorithms, one or more deep neural networks (e.g., a convolutional neural network) and/or may utilize any suitable form of image analysis techniques or machine learning processing functionality configured to analyze acquired ultrasound images and assign grades to selected potentially relevant measurements and/or associated anatomical structures. The measurement grading processor160may assign the grade based on a level of completeness of the anatomical structure to be measured, a focus level of the anatomical structure to be measured, and/or any suitable criterion related to the image quality of the anatomical structure to be measured.

The measurement grading processor160may comprise suitable logic, circuitry, interfaces and/or code that may be operable to assign a grade for each of the selected measurements and/or anatomical structures. In various embodiments, the measurement grading processor160may be provided as a deep neural network that may be made up of, for example, an input layer, an output layer, and one or more hidden layers in between the input and output layers. Each of the layers may be made up of a plurality of processing nodes that may be referred to as neurons. For example, the measurement grading processor160may include an input layer having a neuron for each pixel or a group of pixels from a scan plane of a heart. The output layer may have neurons corresponding to image qualities of anatomical structures of selected measurements, such as a completeness quality and a focus quality of a left ventricle, right ventricle, left atrium, right atrium, aorta, left ventricle outflow tract, and/or any suitable image qualities of anatomical structures of the heart. Each neuron of each layer may perform a processing function and pass the processed ultrasound image information to one of a plurality of neurons of a downstream layer for further processing. As an example, neurons of a first layer may learn to recognize edges of structure in the ultrasound image data. The neurons of a second layer may learn to recognize shapes based on the detected edges from the first layer. The neurons of a third layer may learn positions of the recognized shapes relative to landmarks in the ultrasound image data. The processing performed by the measurement grading processor160deep neural network (e.g., convolutional neural network) may identify image qualities of anatomical structures in acquired ultrasound images with a high degree of probability.

The measurement grading processor160may be configured to cause the display system134to present the grade derived from the image quality criterion for each of the selected measurements and/or anatomical structures. The grades may include a plurality of grade levels, such as ideal for measurement, suitable for measurement, and not suitable for measurement, or any suitable number of grade levels. The grade levels may be represented by symbols (e.g., check mark for suitable and “X” for not suitable), color-coding (e.g., green for ideal, orange for suitable, and red for not suitable), numerical grade levels, letter grade levels, text describing suitability, and/or any suitable grade level indicator. In various embodiments, the grade levels may correspond with whether the measurement may be performed automatically by the signal processor132, whether the measurement may be performed manually by a user via the user input device130, and/or whether the measurement may not be performed.

FIG.2is a display300of an exemplary four chamber (4CH) ultrasound image view310of a heart having measurement suitability feedback320-336, in accordance with various embodiments. Referring toFIG.2, the display300may comprise a 4CH ultrasound image view310and measurement suitability feedback320-336. The measurement identification processor150and/or the measurement grading processor160may be configured to present one or more automatically selected measurements320associated with detected anatomical structures330in the identified 4CH ultrasound image view310at a display system134. For example, the measurement identification processor150and/or the measurement grading processor160may present 4CH measurements320, such as an LV area measurement, an LA area measurement, an RV area measurement, an RA area measurement, an RV mid, base, length measurement, and/or any suitable measurement. The measurement grading processor160may present the grade322,324,326for each of the selected measurements320at the display system134. The grade levels may be represented by symbols (e.g., check mark for suitable and “X” for not suitable), color-coding (e.g., green for ideal, orange for suitable, and red for not suitable), numerical grade levels, letter grade levels, text describing suitability, and/or any suitable grade level indicator. Referring toFIG.2, the grade symbols may include a green check mark322for a highest grade (.e.g., ideal), an orange check mark324for a middle grade (e.g., suitable), and a red “X”326for the lowest grade (e.g., not suitable).

Still referring toFIG.2, the measurement grading processor160may provide a selectable option328to perform a corresponding measurement320. For example, the selectable option328may indicate whether the measurement320may be performed automatically or manually. In various embodiments, an ability to automatically perform the measurement320by the signal processor132may be based on the measurement suitability grade322,324,326. As an example, ideal measurement grades may correspond with an option to automatically perform the measurement. Suitable measurement grades may correspond with options to automatically and/or manually perform the measurement320. Unsuitable measurement grades may correspond with options to manually perform the measurement and/or may not include a measurement option328. In various embodiments, selection of the automatic measurement option328initiates an automatic measurement performed by the signal processor132. In an exemplary embodiment, selection of the manual measurement option328initiates a manual measurement mode where measurement tools are presented such that an operator may manually perform the selected measurement320. The selectable measurement option328may be buttons, a drop down menu, and/or any suitable selectable option. The measurement suitability feedback related to the particular measurements320-328may provide information to an ultrasound operator related to improvements in imaging the anatomical structures such that an ultrasound operator may manipulate the ultrasound probe104to acquire images310suitable for performing the desired measurements320.

As shown inFIG.2, the measurement suitability feedback320-336may further comprise feedback related to the anatomical structures330presented in the detected ultrasound image view310. For example, the image view detection processor140and/or the measurement grading processor160may further present the detected ultrasound image view310and a grade332-336of the anatomical structures330provided in the ultrasound image view310. Referring toFIG.2, the image view detection processor140and/or the measurement grading processor160presents the identity of the detected ultrasound image view310and anatomical structures330therein. For example, the image view detection processor140and/or the measurement grading processor160may present the 4CH view identification and anatomical structures330present in the 4CH view, such as the LV, the LA, the RV, the RA, and/or any suitable anatomical structures present in the 4CH view. The measurement grading processor160may present the grade332,334,336for each of the anatomical structures330associated with the selected measurements320at the display system134. The grade levels may be represented by symbols (e.g., check mark for suitable and “X” for not suitable), color-coding (e.g., green for ideal, orange for suitable, and red for not suitable), numerical grade levels, letter grade levels, text describing suitability (e.g., with respect to completeness and/or focus), and/or any suitable grade level indicator. Referring toFIG.2, the grade symbols may include a green check mark332for a highest grade (.e.g., ideal), an orange check mark334for a middle grade (e.g., suitable), and a red “X”336for the lowest grade (e.g., not suitable). The measurement suitability feedback related to the anatomical structures330-336may provide information to an ultrasound operator related to improvements in imaging the anatomical structures such that an ultrasound operator may manipulate the ultrasound probe104to acquire images310suitable for performing the desired measurements320.

FIG.3is a display of an exemplary parasternal long-axis (PLAX) ultrasound image view310of a heart of having measurement suitability feedback, in accordance with various embodiments.FIG.3shares various characteristics withFIG.2as described above. Referring toFIG.3, the display300may comprise a PLAX ultrasound image view310and measurement suitability feedback320-336. The measurement identification processor150and/or the measurement grading processor160may be configured to present one or more automatically selected measurements320associated with detected anatomical structures330in the identified PLAX ultrasound image view310at a display system134. For example, the measurement identification processor150and/or the measurement grading processor160may present PLAX measurements320, such as an IVS measurement, an LVID measurement, an LVPW measurement, an RVID measurement, an LVOT measurement, an LA measurement, an Ao measurement, and/or any suitable PLAX measurement. The measurement grading processor160may present the grade322,324,326for each of the selected measurements at the display system134. Referring toFIG.3, the grade symbols may include a green check mark322for a highest grade (.e.g., ideal) and a red “X”326for the lowest grade (e.g., not suitable). The measurement grading processor160may provide a selectable option328to perform a corresponding measurement320. For example, the selectable option328may include options for performing measurement320automatically or manually. The measurement suitability feedback related to the particular measurements320-328may provide information to an ultrasound operator related to improvements in imaging the anatomical structures such that an ultrasound operator may manipulate the ultrasound probe104to acquire images310suitable for performing the desired measurements320.

As shown inFIG.3, the measurement suitability feedback320-336may further comprise feedback related to the anatomical structures330presented in the detected ultrasound image view310. For example, the image view detection processor140and/or the measurement grading processor160may present the PLAX view identification and anatomical structures330present in the PLAX view, such as the LV, the LA, the LVOT, the aorta, the RV, the RA, and/or any suitable anatomical structures present in the PLAX view. The measurement grading processor160may present the grade332,334,336for each of the anatomical structures330associated with the selected measurements320at the display system134. Referring toFIG.3, the grade symbols may include a green check mark332for a highest grade (.e.g., ideal), an orange check mark334for a middle grade (e.g., suitable), and a red “X”336for the lowest grade (e.g., not suitable). The measurement suitability feedback related to the anatomical structures330-336may provide information to an ultrasound operator related to improvements in imaging the anatomical structures such that an ultrasound operator may manipulate the ultrasound probe104to acquire images310suitable for performing the desired measurements.

Referring again toFIG.1, the display system134may be any device capable of communicating visual information to a user. For example, a display system134may include a liquid crystal display, a light emitting diode display, and/or any suitable display or displays. The display system134can be operable to present ultrasound image views310, the associated measurements320and detected anatomical structures330, the identification of the detected view, the measurement and/or anatomical structure grades322,324,326,332,334,336, the selectable measurement options328, and/or any suitable information.

The archive138may be one or more computer-readable memories integrated with the ultrasound system100and/or communicatively coupled (e.g., over a network) to the ultrasound system100, such as a Picture Archiving and Communication System (PACS), a server, a hard disk, floppy disk, CD, CD-ROM, DVD, compact storage, flash memory, random access memory, read-only memory, electrically erasable and programmable read-only memory and/or any suitable memory. The archive138may include databases, libraries, sets of information, or other storage accessed by and/or incorporated with the signal processor132, for example. The archive138may be able to store data temporarily or permanently, for example. The archive138may be capable of storing medical image data, data generated by the signal processor132, and/or instructions readable by the signal processor132, among other things. In various embodiments, the archive138stores ultrasound image data310, ultrasound image view detection instructions, measurements320and anatomical structures330associated with ultrasound image views310, and measurement grading instructions, for example.

Components of the ultrasound system100may be implemented in software, hardware, firmware, and/or the like. The various components of the ultrasound system100may be communicatively linked. Components of the ultrasound system100may be implemented separately and/or integrated in various forms. For example, the display system134and the user input device130may be integrated as a touchscreen display.

Still referring toFIG.1, the training system200may comprise a training engine210and a training database220. The training engine160may comprise suitable logic, circuitry, interfaces and/or code that may be operable to train the neurons of the deep neural network(s) (e.g., artificial intelligence model(s)) inferenced (i.e., deployed) by the image view detection processor140and/or the measurement grading processor160. For example, the artificial intelligence model inferenced by the image view detection processor140may be trained to automatically identify anatomical features in ultrasound images310. As an example, the training engine210may train the deep neural networks deployed by the image view detection processor140using database(s)220of different classified ultrasound image views. The artificial intelligence model inferenced by the measurement grading processor160may be trained to automatically identify image qualities (e.g., completeness, focus, and the like) of anatomical features in ultrasound images310. As an example, the training engine210may train the deep neural networks deployed by the image view detection processor140using database(s)220of classified ultrasound images310having different image qualities of anatomical structures to be measured.

In various embodiments, the databases220of training images may be a Picture Archiving and Communication System (PACS), or any suitable data storage medium. In certain embodiments, the training engine210and/or training image databases220may be remote system(s) communicatively coupled via a wired or wireless connection to the ultrasound system100as shown inFIG.1. Additionally and/or alternatively, components or all of the training system200may be integrated with the ultrasound system100in various forms.

FIG.4is a flow chart400illustrating exemplary steps402-416that may be utilized for automatically detecting an ultrasound image view310and focus to provide feedback320-336on measurement suitability, in accordance with various embodiments. Referring toFIG.4, there is shown a flow chart400comprising exemplary steps402through416. Certain embodiments may omit one or more of the steps, and/or perform the steps in a different order than the order listed, and/or combine certain of the steps discussed below. For example, some steps may not be performed in certain embodiments. As a further example, certain steps may be performed in a different temporal order, including simultaneously, than listed below.

At step402, an ultrasound system100acquires ultrasound image310of a target. For example, the ultrasound system100may acquire ultrasound image views310, such as a four chamber (4CH) or parasternal long-axis (PLAX) view, with an ultrasound probe104positioned at a scan position over a heart.

At step404, a signal processor132of the ultrasound system100automatically identifies an ultrasound image view310and focus based on detected anatomical structures in the ultrasound image310. For example, an image view detection processor140of the signal processor132may be configured to apply image analysis algorithms, artificial intelligence algorithms, one or more deep neural networks (e.g., a convolutional neural network) and/or any suitable form of image analysis techniques or machine learning processing functionality to ultrasound images310acquired at step402to automatically identify an ultrasound image view310and focus.

At step406, the signal processor132of the ultrasound system100may automatically select one or more measurements320associated with the detected anatomical structures330in the identified ultrasound image view310. For example, a measurement identification processor150of the signal processor132may be configured to automatically select one or more measurements320associated with anatomical structures330in the ultrasound image view310identified by the image view detection processor140at step404. For example, ultrasound image views may each be associated with one or more potentially relevant measurements320. The one or more potentially relevant measurements320associated with one or more ultrasound image views310may be stored at archive138and/or any suitable data storage medium. The measurement identification processor150may be configured to select the one or more potentially relevant measurements320from the archive138and/or any suitable data storage medium based on the association and the ultrasound image view310identified by the image view detection processor140. The measurement identification processor150may be configured to present the selected one or more potentially relevant measurements320at the display system134.

At step408, the signal processor132of the ultrasound system100may automatically analyze the ultrasound image310and assign a grade322,324,326for each of the selected measurements320based on an ability to accurately perform each measurement320in the ultrasound image310. For example, a measurement grading processor160of the signal processor132may apply image analysis algorithms, artificial intelligence algorithms, one or more deep neural networks (e.g., a convolutional neural network) and/or any suitable form of image analysis techniques or machine learning processing functionality to the ultrasound images acquired310at step402and assign grades322,324,326to the measurements320selected at step406. The measurement grading processor160may further assign grades332,334,336to anatomical structures330associated with the measurements320selected at step406. The measurement grading processor160may assign the grades322,324,326,332,334,336based on a level of completeness of the anatomical structure to be measured, a focus level of the anatomical structure to be measured, and/or any suitable criterion related to the image quality of the anatomical structure to be measured.

At step410, the signal processor132of the ultrasound system100may present the grade322,324,326for each of the selected measurements320at a display system134. For example, the measurement grading processor160of the signal processor132may be configured to cause the display system134to present the grade322,324,326,332,334,336derived from the image quality criterion for each of the selected measurements320and/or anatomical structures330. The grades322,324,326,332,334,336may include a plurality of grade levels, such as ideal for measurement, suitable for measurement, and not suitable for measurement, or any suitable number of grade levels. The grade levels may be represented by symbols (e.g., check mark for suitable and “X” for not suitable), color-coding (e.g., green for ideal, orange for suitable, and red for not suitable), numerical grade levels, letter grade levels, text describing suitability, and/or any suitable grade level indicator. In various embodiments, the grade levels may correspond with whether the measurement may be performed automatically by the signal processor132, whether the measurement may be performed manually by a user via the user input device130, and/or whether the measurement may not be performed.

At step412, the process400may return to stop402if the ultrasound probe104is moved. Additionally and/or alternatively, the process400may proceed to step414if the ultrasound probe104is not moved. For example, an ultrasound operator may reposition the ultrasound probe104of the ultrasound system100if the operator is not satisfied with the detected ultrasound image view, the feedback320-336on measurement suitability, such as the grades322,326,328,332,334,336related to the selected measurements320and/or anatomical structures330, or the feedback related to the ability to perform automated and/or manual measurements328. The ultrasound operator may maintain the position of the ultrasound probe104based on favorable measurement suitability feedback320-336.

At step414, the signal processor132of the ultrasound system100may receive a selection328to automatically or manually perform one or more of the selected measurements320. For example, the measurement grading processor160may provide a selectable option328to perform a corresponding measurement320. For example, the selectable option328may indicate whether the measurement320may be performed automatically or manually. In various embodiments, an ability to automatically perform the measurement320by the signal processor132may be based on the measurement suitability grade322,324,326. The selectable measurement option328may be buttons, a drop down menu, and/or any suitable selectable option. The signal processor132may receive a selection to automatically or manually perform one or more of the selected measurements320via user input device130.

At step416, the signal processor132of the ultrasound system100may perform and present the one or more selected measurements320at the display system134. For example, a selection of the automatic measurement option328of a particular measurement320at step414may initiate an automatic measurement performed by the signal processor132and presented at the display system134. As another example, selection of the manual measurement option328of a particular measurement320at step414may initiate a manual measurement mode where measurement tools are presented such that an operator may manually perform the selected measurement320for presentation at the display system134.

Aspects of the present disclosure provide a method400and system100for automatically detecting an ultrasound image view310and focus to provide feedback320-336on measurement suitability. In accordance with various embodiments, the method400may comprise acquiring402, by an ultrasound system100, an ultrasound image310of a target. The method400may comprise automatically identifying404, by at least one processor132,140of the ultrasound system100, an ultrasound image view310and focus based on detected anatomical structures in the ultrasound image310. The method400may comprise automatically selecting406, by the at least one processor132,150, at least one measurement320associated with the ultrasound image view310. The method400may comprise assigning408, by the at least one processor132,150, a measurement grade322,324,326for each of the at least one measurements320based on an ability to accurately perform each of the at least one measurement320in the ultrasound image310. The method400may comprise causing410, by the at least one processor132,140,150,160, a display system134to present the measurement grade322,324,326for each of the at least one measurement320.

In an exemplary embodiment, the method400may comprise receiving414, by the at least one processor132, a selection328to perform one of the at least one measurement320. In a representative embodiment, the method400may comprise automatically performing416, by the at least one processor132, the one of the at least one measurement320in response to receiving the selection328. The method400may comprise causing416, by the at least one processor132, the display system134to present results of the one of the at least one measurement320. In various embodiments, the method400may comprise causing416, by the at least one processor132, the display system134to present measurement tools for manually performing the one of the at least one measurement320in response to receiving the selection328. In certain embodiments, the ability to accurately perform each of the at least one measurement320in the ultrasound image310may be based on a completeness of the detected anatomical structures in the ultrasound image310and a focus level of the detected anatomical structures in the ultrasound image310. In an exemplary embodiment, the measurement grade322,324,326may be one of a plurality of grade levels, each of the plurality of grade levels represented by one or more of a symbol, color-coding, a numerical grade level, a letter grade level, or a text description of the grade level. In a representative embodiment, the method400may comprise assigning408, by the at least one processor132,160, an anatomical structure grade332,334,336for each of the detected anatomical structures330in the ultrasound image310. The method400may comprise causing410, by the at least one processor132,140,150,160, a display system134to present the anatomical structure grade332,334,336for each of the detected anatomical structures330.

Various embodiments provide a system100for automatically detecting an ultrasound image view310and focus to provide feedback320-336on measurement suitability. The system100may comprise an ultrasound system100, at least one processor132,140,150,160, and a display system134. The ultrasound system100may be configured to acquire an ultrasound image310of a target. The at least one processor132,140may be configured to automatically identify an ultrasound image view310and focus based on detected anatomical structures in the ultrasound image310. The at least one processor132,150may be configured to automatically select at least one measurement320associated with the ultrasound image view310. The at least one processor132,160may be configured to assign a measurement grade322,324,326for each of the at least one measurements320based on an ability to accurately perform each of the at least one measurement320in the ultrasound image310. The display system134may be configured to present the measurement grade322,324,326for each of the at least one measurement320.

In a representative embodiment, the at least one processor132may be configured to receive a selection328to perform one of the at least one measurement320. In various embodiments, the at least one processor132may be configured to automatically perform the one of the at least one measurement320in response to receiving the selection328. The display system134may be configured to present results of the one of the at least one measurement320. In certain embodiments, the display system134may be configured to present measurement tools for manually performing the one of the at least one measurement320in response to the at least one processor132receiving the selection328. In an exemplary embodiment, the ability to accurately perform each of the at least one measurement320in the ultrasound image310may be based on a completeness of the detected anatomical structures in the ultrasound image310and a focus level of the detected anatomical structures in the ultrasound image310. In a representative embodiment, the measurement grade322,324,326may be one of a plurality of grade levels. Each of the plurality of grade levels represented by one or more of a symbol, color-coding, a numerical grade level, a letter grade level, or a text description of the grade level. In various embodiments, the at least one processor132,160may be configured to assign an anatomical structure grade332,334,336for each of the detected anatomical structures330in the ultrasound image310. The display system134may be configured to present the anatomical structure grade332,334,336for each of the detected anatomical structures330.

Certain embodiments provide a non-transitory computer readable medium having stored thereon, a computer program having at least one code section. The at least one code section is executable by a machine for causing the machine to perform steps400. The steps400may comprise receiving402an ultrasound image310of a target. The steps400may comprise automatically identifying404an ultrasound image view310and focus based on detected anatomical structures in the ultrasound image310. The steps400may comprise automatically selecting406at least one measurement320associated with the ultrasound image view310. The steps400may comprise assigning408a measurement grade322,324,326for each of the at least one measurements320based on an ability to accurately perform each of the at least one measurement320in the ultrasound image310. The steps400may comprise causing410a display system134to present the measurement grade322,324,326for each of the at least one measurement320.

In various embodiments, the steps400may comprise receiving414a selection328to perform one of the at least one measurement320. The steps400may comprise automatically performing416the one of the at least one measurement320in response to receiving the selection328. The steps400may comprise causing416the display system134to present results of the one of the at least one measurement320. In certain embodiments, the steps400may comprise receiving414a selection328to perform one of the at least one measurement320. The steps400may comprise causing416the display system134to present measurement tools for manually performing the one of the at least one measurement320in response to receiving the selection328. In an exemplary embodiment, the ability to accurately perform each of the at least one measurement320in the ultrasound image310may be based on a completeness of the detected anatomical structures in the ultrasound image310and a focus level of the detected anatomical structures in the ultrasound image310. In a representative embodiment, the measurement grade322,324,326may be one of a plurality of grade levels, each of the plurality of grade levels represented by one or more of a symbol, color-coding, a numerical grade level, a letter grade level, or a text description of the grade level. In various embodiments, the steps400may comprise assigning408an anatomical structure grade332,334,336for each of the detected anatomical structures330in the ultrasound image310. The steps400may comprise causing410a display system134to present the anatomical structure grade332,334,336for each of the detected anatomical structures330.

As utilized herein the term “circuitry” refers to physical electronic components (i.e. hardware) and any software and/or firmware (“code”) which may configure the hardware, be executed by the hardware, and or otherwise be associated with the hardware. As used herein, for example, a particular processor and memory may comprise a first “circuit” when executing a first one or more lines of code and may comprise a second “circuit” when executing a second one or more lines of code. As utilized herein, “and/or” means any one or more of the items in the list joined by “and/or”. As an example, “x and/or y” means any element of the three-element set {(x), (y), (x, y)}. As another example, “x, y, and/or z” means any element of the seven-element set {(x), (y), (z), (x, y), (x, z), (y, z), (x, y, z)}. As utilized herein, the term “exemplary” means serving as a non-limiting example, instance, or illustration. As utilized herein, the terms “e.g.,” and “for example” set off lists of one or more non-limiting examples, instances, or illustrations. As utilized herein, circuitry is “operable” and/or “configured” to perform a function whenever the circuitry comprises the necessary hardware and code (if any is necessary) to perform the function, regardless of whether performance of the function is disabled, or not enabled, by some user-configurable setting.

Other embodiments may provide a computer readable device and/or a non-transitory computer readable medium, and/or a machine readable device and/or a non-transitory machine readable medium, having stored thereon, a machine code and/or a computer program having at least one code section executable by a machine and/or a computer, thereby causing the machine and/or computer to perform the steps as described herein for automatically detecting an ultrasound image view and focus to provide feedback on measurement suitability.

Accordingly, the present disclosure may be realized in hardware, software, or a combination of hardware and software. The present disclosure may be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited.

Various embodiments may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.

While the present disclosure has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from its scope. Therefore, it is intended that the present disclosure not be limited to the particular embodiment disclosed, but that the present disclosure will include all embodiments falling within the scope of the appended claims.