Source: https://patents.justia.com/patent/10470677
Timestamp: 2020-02-28 07:05:14
Document Index: 625112022

Matched Legal Cases: ['art 110', 'art 110', 'art 110', 'art 110', 'art.\n2', 'art.\n8']

US Patent for Artificially intelligent ejection fraction determination Patent (Patent # 10,470,677 issued November 12, 2019) - Justia Patents Search
Justia Patents Tomography (e.g., Cat Scanner)US Patent for Artificially intelligent ejection fraction determination Patent (Patent # 10,470,677)
Embodiments of the invention provide a method, system and computer program product for artificially intelligent ejection fraction determination. In a method for artificially intelligent ejection fraction determination, a neural network is loaded into memory of a computer, that has been trained with different sets of cardiac imaging data acquired during imaging of a ventricle for different hearts and a known ejection fraction for each of the sets. Then, a contemporaneous set of imaging data is acquired of a ventricle of a heart and the contemporaneous set of imaging data is provided to the neural network. Finally, an ejection fraction determination output by the neural network is displayed in a display of the computer without tracing a ventricle boundary of the heart.
The present invention relates to the systolic function of the heart and more particularly ejection fraction measurement.
The ejection fraction is a measurement of the systolic function of the heart that refers to the percentage of blood leaving the heart at each contraction. Specifically, during each pumping cycle of the heart, the heart both contracts and also relaxes. When the heart contracts, the heart ejects blood from its two pumping chambers, known as the left ventricle and the right ventricle. Conversely, when the heart relaxes, both ventricles refill with blood. Of note, no matter how forceful the contraction of the heart, the heart is not able to pump all of the blood out of each ventricle. Instead, some blood remains. Hence, the term “ejection fraction” refers to the percentage of blood that is able to be pumped out of a filled ventricle with each heartbeat.
Of the two ventricles, the left ventricle is the main pumping chamber of the heart that pumps oxygenated blood through the ascending aorta to the rest of the body, and the right ventricle is the chamber that pumps blood to the lungs for oxygenation. The ejection fraction of the left or right ventricle may be measured through the use of several different imaging techniques. The most common technique is the echocardiogram in which the ejection fraction is measured by sound-wave produced images of the heart and the blood pumping through the heart. Other alternatives to echocardiography include the use of magnetic resonance imaging (MRI), computerized tomography (CT) and nuclear medicine scanning, catheter-based imaging.
Current ejection fraction measurement methods tend to inaccurately assess disease conditions of the heart. This error can lead to the delayed treatment of patients, and the significant worsening of disease conditions during the delay. In this regard, echocardiography relies upon Simpson Biplane methodology to produce a measurement. In particular, in the Simpson Biplane methodology, the end systolic and end diastolic volumes of the ventricle are measured so as to compute a fractional difference. But, in doing so, the ventricle border needs to be manually traced by a human reader, which is subjective to the person. Then, the ventricle volume is assumed to be composed of a finite number, usually twenty, of elliptical cylinders, which while convenient, is not accurate. Moreover, this methodology relies on finding the exact end systolic and end diastolic image frames, often nontrivial step that can introduce errors if not done accurately. As a result, current methods in ejection fraction measurement perform measurements in a way that is neither optimized nor reproducible despite the advances in modern diagnostic technologies.
Embodiments of the present invention address deficiencies of the art in respect to ejection fraction measurement and provide a novel and non-obvious method, system and computer program product for artificially intelligent ejection fraction determination. In an embodiment of the invention, a method for artificially intelligent ejection fraction determination includes training a neural network with different sets of cardiac imaging data acquired of a ventricle for different hearts and a known ejection fraction for each of the sets. Then, the trained neural network is loaded into memory of a computer. A contemporaneous set of imaging data is acquired of a ventricle of a heart whether in the form of clip imagery or data produced during the imaging process of the ventricle of the heart ultimately which is transformed into spatial data of the heart over time that is able to be visualized in a display. Thereafter, the contemporaneous set of imaging data is provided to the neural network. Finally, an ejection fraction determination output by the neural network is displayed in a display of the computer without having traced the ventricle boundary of the heart.
In one aspect of the embodiment, the acquired contemporaneous set of imaging data is filtered to include only imaging data acquired utilizing a specified modality of imaging and representing a specific view of the ventricle. As well, in another aspect of the embodiment, only a portion of the contemporaneous set of imaging data is provided to the neural network and, upon receiving an indication from the neural network of an inconclusive output, an additional portion of the contemporaneous set of imaging data is provided to the neural network in order to receive a conclusive ejection fraction output from the neural network. In yet another aspect of the embodiment, a multiplicity of different neural networks each are trained with different sets of cardiographic imaging data acquired of the ventricle for different hearts and a known ejection fraction for each of the sets utilizing respectively different views and modalities. As such, only a particular one or more of the trained neural networks is loaded into the memory of the computer corresponding to an identified modality and at least one specified view of the contemporaneous set of imaging data. Finally, in even yet another aspect of the embodiment, the contemporaneous set of imaging data, in the form of video clip imagery, is pre-processed through re-sizing and cropping of each video clip of the imagery.
In another embodiment of the invention, a cardiographic data processing system is configured for artificially intelligent ejection fraction determination. The system includes a host computing platform that includes one or more computers each with memory and at least one processor. The system also includes an ejection fraction determination module. The module includes computer program instructions enabled upon execution in the memory of the host computing platform to load into the memory a neural network trained with different sets of cardiac imaging data acquired of a ventricle for different hearts and a known ejection fraction for each of the sets, to acquire a contemporaneous set of imaging data of a ventricle of a heart, to provide the contemporaneous set of imaging data to the neural network and to display in a display of the computer an ejection fraction determination output by the neural network without tracing the ventricle border of the heart.
FIG. 1 is pictorial illustration of a process for artificially intelligent ejection fraction determination;
FIG. 2 is a schematic illustration of a cardiographic data processing system configured for artificially intelligent ejection fraction determination; and,
FIG. 3 is a flow chart illustrating a process for artificially intelligent ejection fraction determination.
Embodiments of the invention provide for artificially intelligent ejection fraction determination. In accordance with an embodiment of the invention, different sets of cardiac imaging data in the form of different video clips are acquired of the ventricle, left, right or both, for different hearts and a known ejection fraction corresponding to each of the sets is associated with the sets. The sets are then supplied as training input to a neural network with respect to the associated ejection fractions so as to train the neural network. In this regard, the sets are supplied as training input to the neural network in connection with a particular modality utilized to acquire the video clips, such as ultrasound echocardiography, CT or MRI, along with an indication of a particular type of view presented in each of the video clips, for instance parasternal long and short axis, apical two, three, four and five chamber and sub-coastal views. The neural network as trained may then be stored in connection with the particular modality and view.
Thereafter, a contemporaneous set of cardiac imaging data of a ventricle of a heart is acquired and a modality and view are identified for the contemporaneous set of imagery. Optionally, video clips determined to be of sub-standard quality are removed from the set. Then, a neural network is selected that corresponds to the identified modality and view of the contemporaneous set of imagery and the contemporaneous set of imagery is provided to the selected neural network so as to receive an ejection fraction value as output from the selected neural network without tracing the ventricle border of the heart. In this way the ejection fraction of the ventricle may be determined without reliance upon the traditional manual tracing of the ventricle by a human reader, manual determination of end systole or diastole frames, or a false presumption of modeling the ventricle volume with elliptical cylinders.
In further illustration, FIG. 1 is pictorial illustration of a process for artificially intelligent ejection fraction determination. As shown in FIG. 1, a cardiographic data processing system 120 acquires imaging data 130 of a ventricle of a heart 110 during imaging of the ventricle of the heart 110. The imaging data 130 may be acquired according to one or more different modalities, such as imaging data ultrasonically acquired, CT imagery or MRI imagery. The imaging data also may include spatial data over time, either one-dimensional, two dimensional or three-dimensional, from many different views of the heart 110 including parasternal long and short axis, apical two, three, four and five chamber and sub-coastal views. The video clip imagery 130 is then pre-processed so as to filter the imaging data 130 to a selection of imaging data 150 corresponding to one or more selected views of the heart 110 acquired by way of one or more selected modalities, for example either a two chamber or four chamber view of appropriate quality including omitting partial views or off-angle views, acquired by way of B-mode echocardiography, Doppler mode echocardiography, M-mode echocardiography, CT or MRI.
Then, a subset 160A of the selection of imaging data 150 are provided as input to a neural network 170 that had been previously training in connection with a selection of different views and modalities of ventricles of hearts and corresponding known ejection fractions for those different views. To the extent that the neural network outputs an inconclusive result 190, a new subset 160B of the selection of imaging data 150 is selected and provided to the neural network 170. This process may continue until no imaging data in the selection of imaging data 150 remain, or until the neural network 170 outputs an ejection fraction 180. In that instance, the ejection fraction 180 is displayed in a display for review by the end user. Optionally, the output of the neural network 170 may instead be stored in connection with a picture archiving and communication system (PACS), an electronic medical record (EMR) system, or a echocardiography/radiology information management system
The process described in connection with FIG. 1 may be implemented in a cardiographic data processing system. In yet further illustration, FIG. 2 schematically shows a cardiographic data processing system configured for artificially intelligent ejection fraction determination. The system includes a host computing platform 210 that may include one or more computers, each with memory and at least one processor. A neural network 240 is loaded into memory of the host computing platform 210 that has been trained using training module 220 with input of a set of video clip imagery of different hearts each with a known ejection fraction. Optionally, the neural network 240 may include a selection of different neural networks, each trained with a different video clip imagery acquired according to different modalities and presenting different views of the heart.
Of note, an ejection fraction determination module 300 is provided. The ejection fraction determination module 300 includes computer program instructions that when executed in the memory of the host computing platform 210, is enabled to process a contemporaneously acquired selection of imaging data of a heart stored in image store 230. The program instructions are further enabled to pre-process the selection of imaging data by filtering the imaging data to only imaging data of a selected view sourced of a particular modality. The program instructions are yet further enabled to pre-process the imaging data, when in the form of video clip imagery, to crop the video clip imagery so as to remove extraneous material unrelated to the heart from the video clip imagery, and to rotate the video clip imagery to a correct angular orientation and to resize the video clip imagery.
In one aspect of the embodiment, the entirety of the filtered and pre-processed imaging data may be submitted to the neural network 240 for a determination of an ejection fraction. In this regard, on submission, each video clip in the contemporaneous set of video clip imagery is decomposed into a multiplicity of frames and the frames are submitted, in parallel, to the neural network 240. However, as an alternative, the program instructions may be enabled to select only a portion of the filtered and pre-processed imaging data for submission to the neural network 240 with imaging data for each video clip in the portion being decomposed into a multiplicity of frames and the frames being submitted, in parallel, to the neural network 240. In either circumstance, the program instructions are enabled to display an ejection fraction produced by the neural network 240 when provided by the neural network 240. However, in response to the condition when the neural network 240 produces an inconclusive result, the program instructions are enabled to select an additional portion of the filtered and pre-processed images for additional submission to the neural network 240 in a follow-on attempt at determining the ejection fraction of the ventricle of the heart.
In even yet further illustration of the operation of the ejection fraction determination module 300, FIG. 3 is a flow chart illustrating a process for artificially intelligent ejection fraction determination. Beginning in block 310, imaging data in the form of a set of video clip imagery of the ventricle of a heart are loaded into memory and in block 320, the video clip imagery in the set are filtered to include only one or more selected views of images acquired according to a selected modality. Then, in block 330 the filtered video clip imagery each are corrected through cropping, padding or rotation functions. Finally, a subset of the filtered video clip imagery is selected in block 340 as input in block 350 to the neural network. Optionally, a particular neural network trained in association with the selected views and selected modality is selected to receive the input in block 350. In decision block 360, it is determined if the neural network is able to output an ejection fraction. If so, the ejection fraction is displayed in block 390. Otherwise, in decision block 370 it is determined if additional images remain to be processed by the neural network. If so, an additional subset of the filtered images is selected in block 340 for as input in block 350 to the neural network. If no further images remain to be processed, an error condition results in block 380.
1. A method for artificially intelligent ejection fraction determination, the method comprising:
training a neural network with different sets of cardiac imaging data acquired of a ventricle for different hearts and a known ejection fraction for each of the sets;
loading the trained neural network into memory of a computer;
acquiring a contemporaneous set of imaging data of a ventricle of a heart;
providing the contemporaneous set of imaging data to the neural network; and,
displaying in a display of the computer an ejection fraction determination output by the neural network without tracing a ventricle boundary of the heart.
2. The method of claim 1, further comprising filtering the acquired contemporaneous set of imaging data to only imaging data pertaining to video clips acquired utilizing a specified modality of imaging and representing a specific view of the ventricle.
only a portion of the contemporaneous set of imaging data is provided to the neural network; and,
upon receiving an indication from the neural network of an inconclusive output, providing an additional portion of the contemporaneous set of imaging data to the neural network in order to receive a conclusive ejection fraction output from the neural network.
training a multiplicity of different neural networks with different sets of cardiac imaging data acquired of the ventricle for different hearts and a known ejection fraction for each of the sets utilizing respectively different views and modalities;
loading a particular one of the trained neural networks into the memory of the computer corresponding to an identified modality and at least one specified view of the contemporaneous set of imaging data.
5. The method of claim 1, further comprising pre-processing the contemporaneous set of imaging data in the form of video clip imagery through re-sizing and cropping of each video clip in the video clip imagery.
6. The method of claim 1, wherein providing the contemporaneous set of imaging data to the neural network comprises decomposing each video clip of the contemporaneous set of imaging data when in the form of video clip imagery into a multiplicity of frames of a movie clip and repeatedly submitting the frames, in parallel, to the neural network.
7. An cardiographic data processing system configured for artificially intelligent ejection fraction determination, the system comprising:
a host computing platform comprising one or more computers each with memory and at least one processor; and,
an ejection fraction determination module comprising computer program instructions enabled upon execution in the memory of the host computing platform to perform:
loading the trained neural network into the memory;
acquiring a contemporaneous set of video clip imagery of a ventricle of a heart;
displaying in a display of at least one of the computers an ejection fraction determination output by the neural network without tracing a ventricle boundary of the heart.
8. The system of claim 7, further comprising filtering the acquired contemporaneous set of imaging data to only imaging data pertaining to video clips acquired utilizing a specified modality of imaging and representing a specific view of the ventricle.
10. The system of claim 7, wherein the computer program instructions are further enabled upon execution in the memory of the host computing platform to perform:
11. The system of claim 7, wherein the computer program instructions are enabled upon execution in the memory of the host computing platform to further perform pre-processing the contemporaneous set of imaging data in the form of video clip imagery through re-sizing and cropping of each video clip in the video clip imagery.
12. The system of claim 7, wherein the computer program instructions providing the contemporaneous set of imaging data to the neural network comprises computer program instructions for decomposing each video clip of the contemporaneous set of imaging data when in the form of video clip imagery into a multiplicity of frames of a movie clip and repeatedly submitting the frames, in parallel, to the neural network.
13. A computer program product for artificially intelligent ejection fraction determination, the computer program product including a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to perform a method including:
training a neural network with different sets of cardiac imagery acquired of a ventricle for different hearts and a known ejection fraction for each of the sets;
14. The computer program product of claim 13, further comprising filtering the acquired contemporaneous set of imaging data to only imaging data pertaining to video clips acquired utilizing a specified modality of imaging and representing a specific view of the ventricle.
17. The computer program product of claim 13, wherein the method further comprises pre-processing the contemporaneous set of imaging data in the form of video clip imagery through re-sizing and cropping of each of the video clips in the video clip imagery.
18. The computer program product of claim 13, wherein providing the contemporaneous set of video clip imagery to the neural network comprises decomposing each video clip of the contemporaneous set of imaging data when in the form of video clip imagery into a multiplicity of frames of a movie clip and repeatedly submitting the frames, in parallel, to the neural network.
6368285 April 9, 2002 Osadchy
20140272842 September 18, 2014 Kwiatkowski
20170109881 April 20, 2017 Avendi
106503610 March 2017 CN
Lou et al. “A novel left ventricular volumes prediction method based on deep learning network in cardiac MRI.” 2016 Computing in Cardiology Conference. Sep. 11, 2016. 89-92.
Lou et al. “Cardiac left ventricular volumes prediction method . . . ” IEEE International Conference on Bioinformatics and Biomedicine. Dec. 15, 2016. 1604-1610.
Kabani et al.“Estimating Ejection Fraction and Left Ventricle Volume Using Deep Convolutional Networks.” MICCAI 2015.Jul. 1, 2016.
Ira Korshunova. “Diagnosing Heart Diseases with Deep Neural Networks.” Mar. 15, 2016. http://irakorshunova.github.io/2016/03/15/heart.html.
Florian Muellerklein. “Measuring cardiac ejection fraction with deep convolution neural networks.” Mar. 17, 2016. http://florianmuellerklein.github.io/DSB/.
Wang et al. “Diagnosis of Heart Disease via CNNs.” Mar. 24, 2016. http://cs231n.stanford.edu/reports/2016/pdfs/331_Report.pdf.
Patent Publication Number: 20190104949
Assignee: BAY LABS, INC. (San Francisco, CA)
Inventors: Charles Cadieu (San Francisco, CA), Michael G. Cannon (Haverford, PA), Ha Hong (Pleasant Hill, CA), Kilian Koepsell (San Francisco, CA), Johan Mathe (San Francisco, CA), Nicolas Poilvert (Seattle, WA)
Application Number: 15/730,377
International Classification: A61B 5/029 (20060101); G06T 7/00 (20170101); G06N 3/08 (20060101); G16H 50/20 (20180101);