Phono- and Electrocardiogram Assisted Detection of Valvular Disease
Conditions: Aortic Valve Stenosis, Mitral Regurgitation, Heart Murmurs, Valvular Heart Disease
Sponsor: University of California, San Francisco

Study Title: Phono- and Electrocardiogram Assisted Detection of Valvular Disease 
 
NCT Number: Pending 
 
UCSF CHR Number: 17-21881 
 
Date: 3/1/18 
 
Study Protocol and Statistical Analysis Plan 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
Phono- and Electrocardiogram Assisted Detection of Valvular Disease 
(PEA-Valve Study) 
 
Investigator team: John Chorba, Carol Maguire, John Maidens, [Study coordinator to be 
named], [Jeff Olgin/Mark Pletcher] 
 
Abstract 
The diagnosis of valvular heart disease (VHD), or its absence, invariably requires 
cardiac imaging. A familiar and inexpensive tool to assist in the diagnosis or exclusion 
of significant VHD could both expedite access to life-saving therapies and reduce the 
need for costly testing.  The FDA-approved Eko Duo device consists of a digital 
stethoscope and a single-lead electrocardiogram (ECG), which wirelessly pairs with the 
Eko Mobile application to allow for simultaneous recording and visualization of phono- 
and electrocardiograms. These features uniquely situate this device to accumulate large 
sets of auscultatory data on patients both with and without VHD. 
In this study, we seek to develop an automated system to identify VHD by phono- 
and electrocardiogram.  Specifically, we will ask whether we can develop machine 
learning algorithms to learn the phonocardiograms of patients with clinically important 
aortic stenosis (AS) or mitral regurgitation (MR), and then task the algorithms to identify 
subjects with clinically important VHD, as identified by a gold standard, from naïve 
phonocardiograms. We anticipate that our study has the potential to revolutionize the 
diagnosis of VHD by providing a more accurate substitute to traditional auscultation. 
 
Specific aim(s) 
Aim 1: Can a machine learning algorithm derived from simultaneous phono- and 
electrocardiogram recordings reliably diagnose clinically important aortic stenosis? 
 
Aim 2: Can a machine learning algorithm derived from simultaneous phono- and 
electrocardiogram recordings reliably diagnose clinically important mitral regurgitation? 
 
Significance 
Valvular heart disease (VHD) is a common global health problem, with 
population-based studies showing a prevalence of 10% for aortic stenosis (AS) and 
20% for mitral regurgitation (MR).1,2  New surgical and interventional advances allow us 
to treat patients at an older age or whose risk of intervention would previously have 
been untenable.3  Given that the incidence of both MR and AS increases with 
increasing age, there is a growing need to identify these conditions so as to offer 
disease-altering therapies. 
In current clinical practice, the diagnosis of VHD relies heavily on 
echocardiography.4  This, in turn, requires both a referral from a provider with a clinical 
suspicion for VHD, typically from an abnormality on auscultation, as well as access to 
the echocardiogram itself.  MR and AS both result in reliably reproducible auscultatory 
findings: holosystolic and systolic crescendo-decrescendo murmurs, respectively.5  Yet 
despite this, auscultation as a diagnostic tool is notoriously poor: its accuracy to detect 
MR and AS ranges only from 5-40%.6,7  These factors all lead to concerns for 
underdiagnosis of these increasingly treatable conditions. 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
Here, we will address the needs for both greater access to and improved 
diagnostic accuracy of testing for VHD.  We will utilize a combination of 
phonocardiogram (PCG) and single-lead electrocardiogram (ECG) recordings, synced 
in real-time to a secure cloud-based server, using the Eko Duo electronic stethoscope.  
With these data, we will develop and validate a machine learning algorithm to diagnose 
clinically important AS or MR.  As the Eko Duo is essentially similar to a traditional 
stethoscope, an iconic tool widely accepted by patients and providers alike, its use to 
drive an automated detection algorithm is both feasible and attractive as a substitute for 
traditional auscultation.  Furthermore, by shifting the burden of test interpretation away 
from the clinician and onto the algorithm, we hypothesize that we will improve overall 
diagnostic accuracy. 
 
Methods 
Overview of design: Cross-sectional study of all subjects undergoing clinical 
echocardiograms at the UCSF adult echocardiography laboratory 
 
Study subjects 
Overview: We will enroll adult subjects undergoing clinical echocardiograms at the 
UCSF Parnassus campus.  These subjects will be grouped into derivation and 
validation cohorts sequentially, stratified by case status, so as to reach the expected 
sample size.  Such grouping will occur after subject enrollment and data collection. 
 
Target Population: Adults with either moderate-to-severe to severe AS or moderate-to-
severe to severe MR (cases) and adults with structurally normal hearts with minimal 
VHD (controls).  In a more-stringent, parallel analysis, a target population of controls will 
be defined as having any degree of AS or MR less than moderate-to-severe.  
 
Accessible Population: Adults meeting the entry criteria undergoing clinical 
echocardiograms at the UCSF echocardiography laboratory amenable to participation. 
 
Inclusion Criteria: 
o Adults > 18 years of age 
o Able to provide consent 
o Undergoing complete (i.e. not limited) echocardiogram studies 
Exclusion Criteria: 
o Refusal to participate 
 
Sampling Scheme: We will approach subjects presenting to the adult echocardiography 
laboratory at UCSF Parnassus consecutively.  Additionally, we will pre-screen subjects 
for a high likelihood of having AS or MR (based on indication for study and prior 
diagnoses in the APEX medical record) and selectively their enrollment during situations 
where the enrollment capacity of the study coordinator is saturated. 
 
Recruitment Strategy:  Introduction of study at time of registering for echocardiogram 
with a brochure or flyer, followed by in-person approaching of potential subjects while 
awaiting the clinical echocardiogram. 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
 
Retention Strategy: None.  We will retain a master file of the medical record numbers to 
identify contact information in the future if deemed necessary. 
 
Measurements 
Overview:  Our study will focus on two measurements: 1) the gold standard assessment 
of VHD by echocardiogram, as reported by the UCSF echocardiography laboratory.  2) 
10 second simultaneous PCG and single-lead ECG recordings by the Eko Duo device 
at each of the four standard cardiac auscultatory positions.  Our study takes advantage 
of the fact that all clinical echocardiogram reports include these valvular assessments. 
 
Gold Standard:  The echocardiogram is accepted as the gold standard for diagnosis of 
VHD severity by the cardiology community.4 To minimize the burden on the 
investigators, as well as reduce costs, we will take advantage of that all clinical 
echocardiogram reports include assessments of VHD, which will serve as our gold 
standard.  These reports follow American Society for Echocardiography (ASE) 
guidelines,8,9 which allow grading of VHD as follows: none, mild, moderate, or severe.  
The UCSF echocardiography laboratory includes additional categories of trace, mild-to-
moderate, moderate-to-severe, and critical, allowing for interpretations where individual 
parameters within the study conflict.  Our primary measurement will be the final 
conclusion of severity of VHD for MR or AS, as reported by a board-certified 
cardiologist.  We will define “clinically important” VHD as that graded moderate-to-
severe or worse, as this would encompass all levels of disease which could require 
direct intervention.  In addition, we will extract the entire echocardiography report, as 
well as the images of the echocardiogram files, so as to save the data for future use as 
new research questions arise.  
 
Device Measurements:  Recordings of the simultaneous PCG and single-lead ECG will 
be performed for each subject in a standardized manner.  Each subject will undergo 10 
second recordings using the Eko Duo device at the four standard auscultation positions.  
Observers will be trained on the systematic method of obtaining measurements.  Time 
and patient permitting, we will also obtain the same recordings using the Eko Core 
device, which uses the same software but does not include ECG recording.  As the 
device will allow visualization of the PCG during recording, the observer will get real-
time feedback on positioning of the device to maximize the quality of the recording at 
each position.  We will plan for periodic review of recordings to ensure adequate data 
quality.  We anticipate that this flexibility and real-time feedback will improve the 
generalizability of the use of the device to a real-world (i.e. non-study) clinical situation. 
 
Primary Outcome:  Our primary outcome will be the identification of either clinically 
important MR or AS (either present or absent) by the algorithm on the device 
measurements from subjects in the validation set.  For the primary outcome, we will 
define our controls as patients with VHD no greater than mild severity as per the gold 
standard.  As our algorithm will provide a score comparing similarity of the learned 
characteristics with test (validation set) data, we will plot our results as a receiver 
operating characteristic (ROC) curve at each score.  In addition, we will plan to calculate 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
the likelihood ratios (both positive and negative) of the test at sensitivity cutoffs of 0.9, 
0.95, and 0.99. 
 
Secondary Outcome: Our secondary outcome will be substantially the same as our 
primary outcome, but in this case, we will define our controls as all subjects with less 
than moderate-to-severe or greater disease of the valvular condition (MR or AS) in 
question.  This will necessitate the generation of a separate algorithm with the 
derivation set, which we will perform in parallel. 
 
Confounders and Bias: The derivation and validation of our algorithms will occur 
remotely, after the clinical echocardiogram has been performed, and therefore with no 
effect on the outcome of the gold standard.  Our test may be influenced by the presence 
of other conditions causing systolic murmurs (including VHD other than AS or MR, or 
congenital heart disease).  We will include these measures from the echocardiogram 
report to compare test performance in those with and without these other conditions. 
 
Statistical issues 
Null Hypothesis:  A machine learning algorithm cannot predict the presence of clinically 
important AS or MR. 
 
Sample Size Justification: 
o Sensitivity of Algorithm: 90% 
o Specificity of Algorithm: 90% 
o Target Likelihood Ratio (+) of Algorithm [LR(R)]: 9 (derived from Sn / [1 – Sp]) 
o Minimum Likelihood Ratio (+) of Algorithm [LR(R)]: 5 
o Confidence Level = 0.95 (alpha = 0.05)  
o Confidence Interval [LR(R)]: 5.120-15.820 
o Sample Size = 110 per group; 330 per cohort (control, AS case, MR case); 660 
overall (training and validation cohorts) 
o Summary: Assuming the sensitivity and specificity of the machine learning 
algorithm for detection of clinically important AS or MR are both 0.9, a total 
sample size of 660 is not expected to go below the threshold likelihood ratio of 
5.0 in the 95% confidence interval in either the derivation or validation cohorts. 
o Justification of Critical Assumptions:  We assume that our algorithm can produce 
a sensitivity and specificity of 90% in detecting aortic stenosis or mitral 
regurgitation compared to hearts with no valvular disease, based upon prior 
published reports using neural networks, using sample sizes of under 100 
cases.10  Furthermore, we estimate a minimum likelihood ratio of 5 would be 
necessary for the test to be clinically useful.  While this estimate works well for 
our validation set, the number needed for our derivation set is less clear; the 
estimates above are a conservative number.  We anticipate training the algorithm 
after enrollment of every 20 cases using a bootstrapping approach; this will 
provide interim test characteristics and help determine the true number needed 
for the derivation set.  As this is, in part, a pilot study, identification of the true 
sensitivity and specificity of the test is in itself a valuable result. 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
Analysis approach: We will generate ROC curves (plotting Sn vs. 1-Sp) for algorithm 
scores for the validation set.  Ultimately we will generate 4 curves: two each for MR and 
AS, using algorithms generated by our primary (defining controls as having no greater 
than mild VHD) and secondary (defining controls as not having moderate-to-severe or 
greater VHD) approaches.  Additionally, we anticipate performing exploratory, 
descriptive analyses of the algorithm itself, by identifying clinical correlates to the 
characteristics most heavily weighted in detecting AS or MR. 
 
Miscellaneous 
Ethical considerations: No major concerns. Data will be securely stored on HIPAA 
compliant platforms.  The study qualifies as minimal risk by UCSF CHR criteria. 
 
Pretest plans: Prior to study recruitment, study staff will collect data on themselves and 
providers to test the data collection system.  During initial subject recruitment, study 
staff will review the process after each day to discuss roadblocks or concerns. 
 
Data Management Plan: Data from our study will come from two sources.  Reports of 
echocardiograms, extracted from the electronic medical record (APEX), will be reviewed 
by study staff to generate our main database of disease characteristics.  A master file 
linking subject identifiers with identifiable information, as well as extracted and de-
identified echocardiogram reports and de-identified raw echocardiogram images will be 
stored on a secure research server used by the Division of Cardiology.  Recorded PCG 
and ECG data (the actual study measurements) will be synced in real-time to a secure, 
HIPAA-compliant, cloud-based server managed by Eko Devices.  At pre-specified times 
of algorithm training, the machine learning team (coordinated by Eko Devices) will be 
provided keys to the assignment of subject identifier to VHD category. 
 
Quality control measures: Periodic review of the recorded data will be performed by the 
study PI to ensure appropriate data quality. 
 
Timetable: 
Contract/Logistics  
 
Subject Enrollment 
 
 
Algorithm Development 
 
 
Analysis 
 
 
Publication 
Overall Timetable: 9 Months 
 
Budget: 
Item 
Estimated Cost 
Notes 
Research Coordinator 
$88000 
Costs not included: 
Eko Duo & Eko Core Device
Salary support for machine  
learning (headed by  
John Maidens) 
 
PI (10% FTE) 
$25000 
Statistician (10% FTE) 
$20000 
Division Research Overhead $14000 
Indirect Costs (58.5%) 
$79000 
Total 
$226000 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
References 
 
1.  
Jones EC, Devereux RB, Roman MJ, et al. Prevalence and correlates of mitral 
regurgitation in a population-based sample (the Strong Heart Study). Am J 
Cardiol. 2001;87(3):298-304. http://www.ncbi.nlm.nih.gov/pubmed/11165964. 
Accessed August 30, 2017. 
2.  
Eveborn GW, Schirmer H, Heggelund G, Lunde P, Rasmussen K. The evolving 
epidemiology of valvular aortic stenosis. the Tromsø study. Heart. 
2013;99(6):396-400. doi:10.1136/heartjnl-2012-302265. 
3.  
Faxon DP, Williams DO. Interventional Cardiology. Circulation. 
2016;133(25):2697-2711. doi:10.1161/CIRCULATIONAHA.116.023551. 
4.  
Nishimura RA, Otto CM, Bonow RO, et al. 2014 AHA/ACC Guideline for the 
Management of Patients With Valvular Heart Disease. J Am Coll Cardiol. 
2014;63(22):e57-e185. doi:10.1016/j.jacc.2014.02.536. 
5.  
Mann DL, Zipes DP, Libby P, Bonow RO, Braunwald E. Braunwald0s Heart 
Disease: A Textbook of Cardiovascular Medicine.; 2015. 
doi:10.1001/jama.294.3.376-a. 
6.  
Mangione S, Nieman LZ. Cardiac auscultatory skills of internal medicine and 
family practice trainees. A comparison of diagnostic proficiency. JAMA. 
1997;278(9):717-722. http://www.ncbi.nlm.nih.gov/pubmed/9286830. Accessed 
August 30, 2017. 
7.  
Mangione S. Cardiac auscultatory skills of physicians-in-training: a comparison of 
three English-speaking countries. Am J Med. 2001;110(3):210-216. 
http://www.ncbi.nlm.nih.gov/pubmed/11182108. Accessed August 30, 2017. 
8.  
Zoghbi WA, Adams D, Bonow RO, et al. ASE GUIDELINES AND STANDARDS 
Recommendations for Noninvasive Evaluation of Native Valvular Regurgitation A 
Report from the American Society of Echocardiography Developed in 
Collaboration with the Society for Cardiovascular Magnetic Resonance. 2017. 
doi:10.1016/j.echo.2017.01.007. 
9.  
Baumgartner H, Hung J, Bermejo J, et al. Recommendations on the 
Echocardiographic Assessment of Aortic Valve Stenosis: A Focused Update from 
the European Association of Cardiovascular Imaging and the American Society of 
Echocardiography Introduction 373 Aetiologies and Morphologic Assessment 373 
Basic Assessment of Severity 375 Recommendations for Standard Clinical 
Practice 375 Peak Jet Velocity 375 Mean Pressure Gradient 378 Aortic Valve 
Area 379. 2017. doi:10.1016/j.echo.2017.02.009. 
10.  Pretorius E, Cronje ML, Strydom O. Development of a pediatric cardiac computer 
aided auscultation decision support system. Conf Proc  . Annu Int Conf IEEE Eng 
Med Biol Soc IEEE Eng Med Biol Soc Annu Conf. 2010;2010:6078-6082. 
doi:10.1109/IEMBS.2010.5627633. 
 
 
 

John Chorba 
PEA-Valve Study Protocol 
2.2018 
 
Appendix: Data Dictionary for Echocardiography Report Abstraction 
 
Data Dictionary 
 
# 
Name 
Type 
Label 
Values 
Notes 
1 
Rec 
Binary 
Device Recordings 
0 – No 
1 – Yes 
 
2 
Ind 
Text 
Indications for Study 
Free Text 
May need to 
include some 
common 
sample 
indications (i.e. 
murmur) 
3 
AS 
Integer 
Aortic Stenosis 
0 – None 
1 – Trace 
2 – Mild  
3 – Mild/Moderate 
4 – Moderate 
5 – Moderate/Severe 
6 – Severe 
7 – Critical 
 
4 
MR 
Integer 
Mitral Regurgitation 
Same as 1 
 
5 
TR 
Integer 
Tricuspid Regurgitation 
Same as 1 
 
6 
AR 
Integer 
Aortic Regurgitation 
Same as 1 
 
7 
PR 
Integer 
Pulmonic Regurgitation 
Same as 1 
 
8 
TS 
Integer 
Tricuspid Stenosis 
Same as 1 
 
9 
MS 
Integer 
Mitral Stenosis 
Same as 1 
 
8 
PS 
Integer 
Pulmonic Stenosis 
Same as 1 
 
9 
VR 
Binary 
Valve Replacement 
0 – No 
1 – Yes 
 
9a 
Loc 
Integer 
Valve Replacement Location 
1 – Aortic 
2 – Pulmonic 
3 – Tricuspid 
4 – Mitral 
 
9b 
VType 
Integer 
Valve Replacement Type 
0 – Annular Ring 
1 – Bioprosthetic 
2 – Mechanical 
 
10 
PDA 
Binary 
Patent Ductus Arteriosus 
0 – No 
1 – Yes 
 
11 
VSD 
Binary 
Ventricular Septal Defect 
0 – No 
1 – Yes 
 
12 
ASD 
Binary 
Atrial Septal Defect 
0 – No 
1 – Yes 
 
13 
Cong 
Binary 
Congenital Heart Disease 
0 – No 
1 – Yes 
 
14 
Other 
Text 
Other – Free Text 
Free Text 
Free text 
required given 
complexity