Abstract:
A system for marking cardiac time intervals from heart valve signals includes a non-invasive sensor unit for capturing electrical signals and composite vibration objects, a memory containing computer instructions, and one or more processors coupled to the memory. The one or more processors causes the one or more processors to perform operations including separating a plurality of individual heart vibration events into heart valve signals from the composite vibration objects, and marking cardiac time interval from the heart valve signals by detecting individual heartbeats and processing cumulative energy within the individual heartbeat to set a threshold to set a marking point.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This Application claims the priority benefit of Provisional Application Nos. 62/274,766, 62/274,761, 62/274,763, 62/274,765, and 62/274,770 each of which were filed on Jan. 4, 2016, the entire disclosure of each are incorporated herein by reference. 
     
    
     FIELD 
       [0002]    The embodiments herein relate generally to cardiopulmonary health monitoring and more particularly to analysis software combined with transducers to capture multi-channel vibration signals along with an electrocardiogram signal for the measurement of heart functions. 
       BACKGROUND 
       [0003]    Heart disease is the leading cause of death accounting for more than one-third (33.6%) of all U.S. deaths. Overall cardiac health can be significantly improved by proper triage. Low invasive and non-invasive ultrasound techniques (e.g. echocardiogram) are standard procedures, but the requirement of expensive devices and skilled operators limit their applicability. The following are the various types of heart disease that can be diagnosed and treated using the separated signal, namely, Coronary artery disease, Heart murmurs and valve abnormalities, Heart failure, Heart rhythm abnormalities (arrhythmias), Vascular disease, congenital heart disease, Cardiac resynchronization and Risk factor modification. A physician can work with patients to perform a comprehensive evaluation and design a personalized plan of care aimed at keeping them healthy. 
         [0004]    The cardio pulmonary system which consists of the respiratory components, snoring components, and cardiac components, creates vibrations during each cardiac cycle. The vibrations are the result of the lung sounds, heart sounds, acceleration and deceleration of blood due to abrupt mechanical opening and closing of the heart valves during the cardiac cycle. 
       SUMMARY 
       [0005]    The exemplary embodiments herein provide a method and system of marking cardiac time intervals from the source separated heart valve signals from the composite cardiac vibration objects. In some embodiments, data is obtained using a tri-axial accelerometer or multiple tri-axial accelerometers placed on different points of torso. The present technology pertains in general to technology for assessment of cardiac contractility in a subject from the source separated signals from recorded precordial acceleration signals. The embodiments herein can use machine learning, Principal Component Analysis (PCA), Singular Value Decomposition (SVD), k nearest neighbors, Linear LDA, Quadratic LDA, Linear SVM, or rbf SVM or others. 
         [0006]    Examples of cardiac vibration objects are the first sound, the second sound, the third sound, the fourth sound, ejection sounds, opening sounds, murmurs, heart wall motions, coronary artery sounds, and valve sounds of the Mitral valve opening and closing, Aortic valve opening and closing, Pulmonary valve opening and closing, Tricuspid valve opening and closing. Examples of the pulmonary vibration objects are the respiratory lung sounds, breathing sounds, tracheobronchial sounds, vesicular sounds, Broncho vesicular sounds, snoring sounds. A portion of the energy produced by these vibrations lies in the infra-sound range, which falls in the inaudible and low sensitivity human hearing range. A portion of the energy produced by these vibrations falls in the audible hearing range. For example, the vibration objects from the Mitral, Tricuspid, Aortic, and Pulmonary valve openings fall in a lower range of vibrations such as 0 to 60 Hertz, whereas vibration objects from the Mitral, Tricuspid, Aortic, and Pulmonary valve closings fall in a higher range of vibrations such as 50 to 150 Hertz. Accelerometer transducers placed on the chest capture these vibrations from both these ranges. 
         [0007]    Source separation analysis extract individual vibration objects from the composite vibration signal captured on the surface (of the torso or elsewhere). The individual vibration signals are identified to be from the mitral valve, aortic valve, tricuspid valve, and the pulmonary valve during individual heart beats. Along with separating breathing sounds, and heart wall motion. The identified valve signals are marked to indicate their start and end of the event with respect to the start of the EKG to provide the cardiac time intervals as described in the embodiments herein. These events correspond to the opening and closing of each valve . Further note that the techniques and methods herein are not limited to acoustic, electrical or vibrational data as might be used in some stethoscopes, but can also be applied to other forms of monitoring such as echo imaging or sonograms, magnetic resonance imaging (MRI), computed tomography (CT) scanning, positron emission tomography (PET) scanning, and monitoring using various forms of catheterization. The techniques and methods herein are primarily applicable to monitoring of heart valve events, but can be alternatively applied to other types of involuntary biological signaling emanating from the brain, intrauterine, pre-natal contractions, or elsewhere within both humans and other species. 
     
    
     
       DESCRIPTION OF THE DRAWINGS 
         [0008]      FIG. 1A  illustrates a system for the extraction, identification, marking and display of the heart valve signals in accordance with one embodiment; 
           [0009]      FIGS. 1B and 1C  illustrate cardio pulmonary signal capture at the chest in accordance with various embodiments; 
           [0010]      FIG. 2  is a flowchart of a method practiced by the system in accordance with one embodiment; 
           [0011]      FIG. 3  illustrates multichannel signals captured from the sensor array on the chest shown in accordance with one embodiment; 
           [0012]      FIG. 4  illustrates a cardiac cycle in relation with Electrocardiogram, acoustic and accelerometer sensors of the system in accordance with one embodiment; 
           [0013]      FIG. 5  illustrates a heart anatomy and schematic representation of the cardiopulmonary sounds in relation to electrocardiogram; 
           [0014]      FIGS. 6A, 6B, and 6C  illustrate a method and a cardiac time interval measurement in accordance with one embodiment; 
           [0015]      FIGS. 7A and 7B  illustrate the marking of vibration objects or each valve into individual streams in accordance with one embodiment; and 
           [0016]      FIGS. 8A, 8B, and 8C  illustrate the comparison of M 1 , T 1 , A 2 , and P 2  timings and comparison of time calculations using different energy thresholds in accordance with one embodiment. 
       
    
    
     DETAILED DESCRIPTION 
       [0017]    The exemplary embodiments may be further understood with reference to the following description and the appended drawings, wherein like elements are referred to with the same reference numerals. The exemplary embodiments describe a system and method of marking the cardiac time intervals and display of the heart valve signals. Specifically, psychoacoustics are considered in identifying the separated cardiac vibration signals captured through the transducers. The system, the psychoacoustics, and a related method will be discussed in further detail below. 
         [0018]    The exemplary embodiments provide a novel approach for small, portable, robust, fast and configurable source separation based software with transducer hardware. The use of a vibration signal pattern and novel psychoacoustics help bypass conventional issues faced by linear time invariant systems. Clinical indices of myocardial contractility can be categorized as follows based on pressure measurements (such as dP/dtmax), volume and dimension (such as stroke volume and ejection fraction) and systolic time intervals (such as pre-ejection period, left ventricular ejection time and isovolumic contraction time). dP/dtmax is the gold standard of measurement of myocardial contractility. Some of the cardiac time intervals can include Left Ventricular Systolic Time (LVST), Left Ventricular Diastolic Time (LVDT), Pre-atrial Diastolic Filling Time (PADT), Accelerated Atrial Filling Time (AAFT), QS1 (Electromechanical activation time), QS2, Pre-Ejection Period (PEP), Right Ventricular Systolic Time (RVST), Left Atrial Systolic Time (LAST), Right Atrial Systolic Time (RAST), Right Ventricular Ejection Fraction (RVEF), Right Ventricular Diastolic Time (RVDT), Left Atrial Diastolic Time (LADT), Right Atrial Diastolic Time (RADT), Systolic Time Interval (PEP/LVST). 
         [0019]    The exemplary embodiments of the system and method proposed here are shown in  FIGS. 1A, 1B, and 1C . System  100  shown in  FIGS. 1A and 1B  is an embedded platform which can be any smart processing platform with digital signal processing capabilities, application processor, data storage, display, input modality like touch-screen or keypad, microphones, speaker, Bluetooth, and connection to the internet via WAN, Wi-Fi, Ethernet or USB. This embodies custom embedded hardware, smartphone, iPad-like and iPod-like devices. Area  101  in  FIGS. 1A and 1B  is the auditory scene at the chest locations. Array  102  in  FIGS. 1A and 1B  is the transducer array used to capture the heart signal(s). In some embodiments, the transducer array includes a pad that includes a vibration sensor such as a vibration sensor  102   b  and an electrode  102   a  for an ECG sensor. In some embodiments, the transducer array can include a single pad, two pads as shown in  FIG. 1B  or more than two pads as shown in  FIG. 1C . In the particular embodiment of  FIG. 1C , a transducer array  110  includes three pads ( 102 ) where each pad includes the vibration sensor  102   b  and the ECG electronic  102   a . Other embodiments can include three or more pads where each pad would have at least a vibration sensor and optionally an electrode for the ECG sensor. Hardware  103  in  FIGS. 1A-C  is the wearable microprocessor hardware with digital signal processing capabilities, application processor, Analog to digital frontend, data storage, input modality like buttons, and wireless connection via Bluetooth, Bluetooth low energy, near field communication transceiver, Wi-Fi, Ethernet or USB. 
         [0020]    Processor  112  shown in  FIG. 1C  comprises of the signal processing module on the wearable device that captures synchronized sensor data from the transducer array  102 . The processor  112  is configured to save the synchronized sensor data to memory and communicate it with the system  100  for data transfer. Module  105  in  FIG. 1A  is the module that calculates vital sign from the input sensor stream coming from hardware  103  for the Heart rate, breathing rate, EKG signal, skin temperature, and associated vitals. The hardware  103  can optionally encrypt the raw sensor data for transmission to the cloud computing module  106 . It can also communicate with a dashboard on module  105  or  106  for data exchange, login, alerts, notifications, display of processed data. Computing device  106  in  FIG. 1A  is the cloud module that processes the individual streams for eventual source separation. In some embodiments, the system  100  could include a connected display or other modality of display or presentation device. In some embodiments the system  100  allows a user to visually see the individual streams and information of the different cardiopulmonary signals. 
         [0021]    The transducer array  102  can include multiple sensor transducers that capture the composite signal that includes the electrocardiogram signals, heart sounds, lung sounds and snoring sounds for example. The module  103  can be in the form of wearable hardware that synchronously collects the signals across the transducers and is responsible for the analog to digital conversion, storage and transmission to a portable unit  104 . Note that the embodiments herein are not limited to processing the individual streams for source separation, identification and marking of the heart valve signals at the cloud computing module  106  only. Given sufficient processing power, the aforementioned processing can occur at the microprocessor hardware module  103 , at the module  105 , or at the cloud-computing module  106 , or such processing can be distributed among such modules  103 ,  105 , or  106 . 
         [0022]    The exemplary embodiments of the system and method proposed here for the source identification of the cardiopulmonary signals  200  are shown in  FIG. 2 . Block  201  indicates the separation of sources from the composite signals. Block  202  represents the phase estimation between the separated sources at each of the sensor position. Block  203  represents calculating the time stamps of individual sources at each heartbeat with respect to the synchronized EKG signal and the other sensor or sensors. Block  204  represents the source identification module responsible for tagging each of the separated source in individual heart beats to be one of the heart valve event, namely Mitral valve closing and opening, Tricuspid valve closing and opening, Aortic valve opening and closing, and the Pulmonic valve opening and closing. Block  205  represents the time marking module to estimate the time of occurrence of the above mentioned valve events with respect to the start of the EKG signal. 
         [0023]    The exemplary embodiments of the system and method proposed here for the source identification of the cardiopulmonary signals from the composite signal  300  are shown in  FIG. 3 . Area(s)  101  in  FIG. 1B  indicate the locations at which the composite heart signal can be captured. A vibration signal  302  as charted on the first line in  FIG. 3  represents a signal captured at the aortic auscultation location. A vibration signal  303  shows the vibration signal captured at the pulmonic auscultation location. A vibration signal  304  shows the vibration signal captured at the tricuspid auscultation location. A vibration signal  305  represents a vibration signal captured at the mitral auscultation location. The last or bottom line in  FIG. 3  represents an electrocardiogram signal  306  captured. In some embodiments, note that the number of sensors used (such as in the sensor array  102  of  FIG. 1 ), are less than the number of vibration sources. For example, 3 sensors can be used to ultimately extract signals for 4 (or more) vibration sources; or 2 sensors can be used to ultimately extract signals for 3 or 4 (or more) vibration sources; or 1 sensor can be used to ultimately extract signals for 2, or 3, or 4 (or more) vibration sources. 
         [0024]    The exemplary embodiments of the system and method proposed here draw inspirations from biology with respect to the cardiac cycle in-relation with electrocardiogram and accelerometer transducer captured cardiac signal. A timeline chart  400  in  FIG. 4  shows a cardiac cycle. Lines or signals  401   a,    401   b,  and  401   c  represent or indicate the pressure changes during a cardiac cycle for aortic pressure ( 401   a ), atrial pressure ( 401   b ) and ventricular pressure ( 401   c ) measured in measured in millimeters of mercury (mmHg). Line or signal  402  represents or indicates the volume changes during a cardiac cycle in milliliters (ml). Line or signal  403  represents or indicates the electrical changes during a cardiac cycle captured by an electrocardiogram. Line or signal  404  represents or indicates the acoustic changes during a cardiac cycle captured by an acoustic sensor such as a phonocardiogram or PCG. S 1  represents the first heart sound or the “lub” sound and the S 2  represents the second heart sound or “dub” sound. Line or signal  405  represents or indicates the vibration changes during a cardiac cycle captured by an accelerometer transducer at the location of the device. Pattern  406  in  FIG. 4  indicates the different valve opening and closing seen in line or signal  405  as captured by the accelerometer sensor or sensors. More specifically, a closer inspection of the pattern  406  reveals the closing of the mitral valve (M 1 ) and tricuspid valve (T 1 ) during the S 1  or first heart sound and the closing of the aortic valve (A 2 ) and pulmonary valve (P 2 ). 
         [0025]      FIG. 5  goes on to further show a representation  510  of the human heart relevant for the generation of the sounds and corresponding graph  500  representing the sounds belonging to coronary artery, murmurs, first sound, second sound, third sound, fourth sound, ejection sounds, opening sounds, respiratory sound, breathing, and snoring during individual heart beats, with respect to the electrocardiogram signal. 
         [0026]    The exemplary embodiments of the system and method proposed here provide a source marking algorithm for the vibrations from the cardiohemic system. In some embodiments, the system next uses PCA to determine which source is associated with which event (e.g., Mitral closing &amp; opening, Tricuspid closing &amp; opening, Aortic opening &amp; closing, Pulmonic opening and closing). The following describes the architecture for automatic source tagging and timing of valvular events. One way to identify which events are relevant to a source is by manually tagging the sources against the synchronized EKG signal and taking advantage of the timings relative to a QRS wave (identification of the S 1  and S 2  sounds using the EKG signal as the reference has been widely researched in studies). Another approach is an automatic tagging algorithm. The tagging is composed of a classifier preceded by a feature extraction algorithm. For the timing, the system exploits the computations of one of the feature extraction algorithms to obtain an energy contour from which the time location of a given event can be inferred. Because the embodiments here build upon having the ability to capture the signal at different locations simultaneously, to the proposed system exploits the relations among channels to extract additional information about the sources. Likewise, since some source separation algorithms where channels relations are associated with location, the system can leverage on the intrinsic relations among the channels to extract relevant information that helps the system distinguish among the events. In some embodiments, the system hypothesizes that phase information between channels is relevant for distinguishing among cardiac events since valves are located at different positions within the heart. Perhaps, one of the most distinctive features of the cardiac events is their relative order of occurrence, which repeats periodically with each heartbeat. Time information extracted from the set of sources can be utilized to localize the occurrence of each source signal within the heart cycle. Therefore, the features proposed here are conceived to provide three aspects: 1) Spectral information, 2) Relations among channels, and 3) Relations among events in the form of relative times of occurrence. 
         [0027]    The automated timing is obtained from the separated sources. The embodiments can employ the eigenfilter approach described above to extract energy envelopes that can be easily detected and further processed to extract a time point. In this case, the system uses the two leading right singular vectors of the tap-delay matrix. It has been observed that, for a single channel, the first two right singular vectors of the tap-delay matrix contain oscillatory components with π/2 phase delay. This behavior can be extended to the two-channel case by noticing that the first half of the two leading singular vectors contain an oscillatory component of similar frequency with the above mentioned π/2 phase difference for channel 1, and that the same result applies to the second half for channel 2. From the above observation, we can consider the first 2 leading right singular vectors as a quadrature pair of eigenfilters. In other words, these filters have the same magnitude in frequency with a π/2 phase difference. The sum of instantaneous energies for the quadrature pair provides an energy envelope that, for the source signals, can be processed in a simple way to obtain time stamps on the occurrence of the events associated with the source. Let u1 and u2 be the two leading right singular vectors of Δi. Let s 1 =Δ i  u 1  and s 2 =Δi u 2  be the score vectors. The energy envelope s can be calculated as (s) l =(s 1 ) l   2 +(s 2 ) l   2 . From the sparsity property of the heart sounds, it is possible to detect single heart beats from the energy contour s since the source signal is mostly zeroes followed by the oscillations related to the event at each heart beat. A simple marking procedure can be obtained by first detecting individual heartbeats and then processing the cumulative energy within a heartbeat to set a threshold that defines the marking point. Process  602  shown in the box  610  of  FIG. 6B  describes the procedure. A resulting time stamp (black vertical lines)  601  (in chart  600  of  FIG. 6A ) using the energy threshold can be marked. Notice that the endpoints of the Heart valve signal have been also detected as part of the procedure in determining the time stamps  601 . The chart  600  shows the resulting markings using a cumulative energy to provide a threshold. In this case 1% of cumulative energy was selected to provide the threshold value. Chart  603  shows the time intervals found for the Mitral closing ( 611 ), Tricuspid closing ( 612 ), Aortic opening ( 613 ), Aortic closing ( 614 ) and Pulmonic closing ( 615 ). 
         [0028]    The exemplary embodiments of the system and method proposed here provide a source marking algorithm that allows from the explanation earlier for the marking of the Mitral valve closing (MC), Mitral valve opening (MO), Aortic valve opening (AO), Aortic valve closing (AC), Tricuspid valve closing (TC), Tricuspid valve opening (TO), Pulmonary valve closing (PC) and Pulmonary valve opening (PO) signals. The extracted individual valve vibration objects are aligned into a signal for each of the four valves across multiple heart beats. The chart  700  in  FIG. 7A  shows the source separation of heart valve opening and closing signals. Line  701  indicates the length or duration of the vibration signal for the Mitral valve closing (M 1 ). Line  702  indicates the length or duration of the vibration signal for the Tricuspid valve closing (T 1 ). Line  703  indicates the length or duration of the vibration signal for the Aortic valve closing (A 2 ). Line  704  indicates length or duration of the vibration signal for the Pulmonic valve closing (P 2 ). Signal  705  indicates the composite vibration signal captured by a particular transducer. Signal  706  indicates the EKG signal captured by the system. Referring to chart  710  of  FIG. 7B , the Line  707  indicates the length or duration of the vibration of the Aortic valve opening (AO). Line  708  indicates the length or duration of the vibration of the Pulmonic valve opening (PO). Further note that the lines or signals  709  in  FIG. 7A or 711  in  FIG. 7B  are actually several separated superimposed signals representing the vibration signals from separate sources coming from the mitral valve, tricuspid valve, aortic valve, and pulmonary valve (using less than 4 vibration sensors to extract such separated signals in some embodiments. 
         [0029]    It was observed that peak of T 1  timing distribution is close to that of AO. The reason is that the length of M 1  and T 1  Source Separation vibrations is longer than the length of AO Source Separation vibrations. So when the mid-point of accumulative energy is calculated, M 1  and T 1  timings are already shifted forward and don&#39;t represent the start of the vibration. Such a timing shift exists for AO but it&#39;s not as big as M 1  and T 1 . To verify and compare, the following time information on some patients helps provide different approaches: Mean length of M 1 , T 1  vibration, Mean start point of M 1 , T 1  vibration, Mid-energy point is obtained from PCA algorithm. A shift back in timing of M 1 , T 1 , A 2 , P 2  by reducing the 50% of accumulative energy to 30%, 20%, and 10%. The results are demonstrated in  FIGS. 8A, 8B and 8C . 
         [0030]    In the exemplary embodiments, a novel way of calculating the timing of the source separated individual heart vibration events from the composite vibration objects captured via multiple transducers is used to work on a single package, easy-to-use and portable device. 
         [0031]    The exemplary embodiments develop a novel method of source timing, which in one embodiment using the Pulmonary and Aortic, and in addition possibly the Tricuspid and Mitral auscultation locations, lends the system capable of calculating the time intervals of individual valve events from the vibrations with respect to the electrocardiogram. 
         [0032]    The exemplary embodiments develop a novel method of time interval calculation, which in one embodiment using the Pulmonary and Aortic, and in addition possibly the Tricuspid and Mitral auscultation locations, lends the system capable of marking the time of occurrence of the individual valve events with respect to the electrocardiogram. The novel method lends the system capable of measuring the cardiac time intervals. 
         [0033]    The exemplary embodiments develop a novel method of providing time intervals of individual valve signals over time. The novel method allows for both short-term and long-term discrimination between signals. Short-term pertains to tracking individual stream when they are captured simultaneously as part of the composite signal. Long-term tracking pertains to tracking individual streams across multiple heart beats, tracking valve signals as they transition in and out during each cardiac cycle. 
         [0034]    The exemplary embodiment of system and method described is the development on an embedded hardware system, the main elements required to capture body sounds are the sensor unit that captures the body sounds, digitization, and digital processing of the body sounds for noise reduction, filtering and amplification. Of course, more complicated embodiments using the techniques described herein can use visual sensors, endoscopy cameras, ultrasound sensors, MRI, CT, PET, EEG and other scanning methods alone or in combination such that the monitoring techniques enable improvement in terms of source separation or identification, and/or marking of events such as heart valve openings, brain spikes, contractions, or even peristaltic movements or vibrations. Although the focus of the embodiments herein are for non-invasive applications, the techniques are not limited to such non-invasive monitoring. The techniques ultimately enable diagnosticians to better identify or associate or correlate detected vibrations or signaling with specific biological events (such as heart valve openings and closings, brain spikes, fetal signals, or pre-natal contractions.) 
         [0035]    It will be apparent to those skilled in the art that various modifications may be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the method and system described and their equivalents.