Do Script Concordance Tests Correlate With Family Medicine Standardized Tests and Failing Rotation Grades?
Conditions: Knowledge, Attitudes, Practice
Sponsor: Pamela Hughes

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
1 
PROTOCOL FOR NON-EXEMPT RESEARCH INVOLVING HUMAN SUBJECTS 
Title: 
Do Script Concordance Tests correlate with Family Medicine standardized tests and failing rotation grades? 
IRB #: 
FWH20190010H 
Principal 
Investigator (PI) 
Rank / Civ 
Rating 
Branch 
AD/DoD Civ/ 
Ctr/Civilian 
Dept/Base 
Phone # 
E-mail 
Pamela Hughes 
Maj 
USAF 
AD 
FMR/99MDG 
702-653-3298 
pamela.r.hughes4.mil@mail.mil 
 
The research relevance of this protocol focuses on: 
☐ 
Diagnosis 
☐ 
Treatment 
☐ 
Medical Utilization/Managed Care 
☐ 
Prevention 
☐ 
Medical Readiness 
☒ 
Other: Education evaluation improvement 
 
Does the research fall under the purview of any other departments or committees? 
No 
1. LOCATION AND SPONSOR 
Collaborating Facilities: None 
AF Sites Seeking Regional IRB:  
Jill Clark, 99MDG, (702) 653-3298, jill.m.clark15.ctr@mail.mil 
Study Sponsors: None 
2. RESEARCH PLAN 
Purpose of Study:  
To determine if a Script Concordance Test (SCT) correlates with Family Medicine In Training Exam (ITE) scores, American Board 
of Family Medicine (ABFM) certification exam scores, Accreditation Council for Graduate Medical Education (ACGME) medical 
knowledge and patient care milestones, and failing family medicine inpatient clinical rotation grades. 
Hypotheses, Research Questions, or Objectives:  
• Objective 1: Investigate whether SCT scores correlate with Family Medicine ITE scores.    
• Null Hypothesis 1: SCT scores do not correlate with Family Medicine ITE exam scores.  
• Alternative Hypothesis 1: SCT scores do correlate with Family Medicine ITE exam scores. 
• Objective 2: Investigate whether SCTs correlate with ABFM certification exam scores.  
• Null Hypothesis 2: SCT scores do not correlate with  ABFM certification exam scores 
• Alternative Hypothesis 2: SCT scores do correlate with ABFM board scores. 
• Objective 3: Investigate whether SCT scores correlate with failing family medicine inpatient clinical rotation grades  
• Null Hypothesis 3: SCT scores do not correlate with failing family medicine inpatient rotation grades, 
• Alternative Hypothesis 3: SCT scores do correlate with failing family medicine inpatient rotation grades. 
• Objective 4: Investigate whether SCT scores correlate with ACGME medical knowledge and patient care milestones.   
• Null Hypothesis 4: SCT scores do not correlate with ACGME medical knowledge or patient care milestones.  
• Alternative Hypothesis 4: SCT scores do correlate with ACGME medical knowledge and patient care milestones. 
• Objective 5: Investigate whether ACGME medical knowledge and patient care milestones correlate to failing rotation 
grades.  
• Null Hypothesis 5:  ACGME medical knowledge and patient care milestones do not correlate to failing family medicine 
inpatient rotation grades.  
• Alternative Hypothesis 5: ACGME medical knowledge and patient care milestones do correlate with failing family medicine 
inpatient rotation grades. 
Significance:  
Delayed or poor identification and remediation of clinical reasoning difficulties can lead to clinician underperformance and can 
ultimately compromise patient care.  To date, no research has been done to see if SCTs correlate with current grading 
standards (ITE, ABFM certification exam score, ACGME milestones, or failing clinical rotation grades) in Family Medicine 
Residents. If we can identify that an SCT correlates with current standardized testing of Family medicine residents then it could 
be possible to identify struggling learners prior to poor scores on the ITE, ACGME milestones, or clinical rotations.   If we can 
identify a learner that does poorly on an SCT given early in the academic year the learner can begin a remediation plan to 
improve their deficits before receiving a failing grade, poor ITE or ACGME milestone scores.   
Military Relevance:  

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
2 
Delayed or poor identification and remediation of clinical reasoning difficulties can lead to clinician underperformance and can 
ultimately compromise patient care both at home and in austere environments.  Clinical reasoning and decision making in a 
context of uncertainty reflects real life situations where the provider/s may have limited information and limited time to make 
an assessment and treatment decision. In this proposed study, the SCT may help to identify those residents who are struggling 
with uncertainty in decision making prior to a failing rotation or exam score. Remediation plans could then be made for those 
residents who score poorly on the SCT before they perform poorly on a clinical rotation or ITE exam. Ensuring that resident 
doctors are adept with clinical reasoning and patient assessment better prepares them for patient care as doctors of Family 
Medicine in standard and austere environments. 
Background and Review of Literature:  
The script concordance test (SCT) is an instrument for evaluating reasoning in a context of uncertainty – clinical data 
interpretation1. 
 
The SCT: 
• Confronts candidates with a problematic situation, probes their reasoning and give them a score by comparing their 
answers to that of a reference panel. 
• Format: the case is described in a vignette, it is followed by one or more items (tasks requested from the candidate and 
subject to a measurement). 
 
In their everyday practice, health professionals encounter a variety of complex problems. In practice, the available data are 
often incomplete, sometimes ambiguous or inaccurate. This type of problem requires an ability to reason in a context of 
uncertainty; for some, this ability represents the essence of professional competence. 
 
Theory and principles: SCT is based on hypothetico-deductive theory of clinical reasoning and script theory. The theory of 
scripts comes from cognitive psychology. It implies that to make sense and act effectively in a situation, clinicians activate 
scripts (knowledge networks organized for the resolution of specific tasks). 
 
The scripts appear when students begin to face real clinical tasks. They are then developed and refined throughout their 
working life. They are used to understand information and to actively treat patients. 
 
In a clinical situation, clinicians quickly generate hypotheses that they then try to confirm or refute, looking for positive or 
negative clinical signs that reinforce or eliminate them. This is how they progress towards solving clinical problems. Similar 
patterns occur when reasoning about options for investigation or treatment. The clinical reasoning is made of multiple clinical 
judgments that can be measured and compared with those of a panel of experienced clinicians. 
 
The name of the test, script concordance, reflects this study of the concordance between the judgments of the candidate and 
those of the panel members2. The probe tests if knowledge is organized according to clinical tasks and the quality of 
candidates' scripts. 
 
The principles of the SCT are described below3: 
• The candidate is placed in an authentic clinical context, described in a vignette. 
• Several options for diagnosis, investigation or treatment are relevant in the situation. 
• The items represent questions experienced by experienced clinicians in a clinical context. 
• The task for the student is to determine the impact this new data has on the status of the option. 
• The candidate's answer is collected on a Likert scale. The different points of the scale correspond to positive values (the 
option is reinforced by the new data), neutral (the data does not modify the status of the option) or negative (the option is 
invalidated by the data). 
 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
3 
(3) 
It is unknown whether SCTs correlate with ITE scores, ABFM certification exam scores, ACGME milestones or to failing clinical 
rotation grades, and, if there is a correlation, whether struggling learners can be identified earlier than they are with current 
standards.  
Bibliography:  
1. https://www.cpass.umontreal.ca/recherche/groupe-de-recherche-cpass/axes-de-recherches/concordance/sct/learn-
more-about-the-sct/ 
2. Gagnon R, Charlin B, Coletti M, Sauv´e E, van der Vleuten C. Assessment in the context of uncertainty: how many members 
are needed on the panel of reference of a script concordance test? Med Educ. 2005;39:284–291. 
3. Fournier JP, Demeester A, Charlin B. Script Concordance Tests: Guidelines for Construction. BMC Medical Informatics and 
Decision Making. 2008;8:18. doi:10.1186/1472-6947-8-18. 
4. O’Neill TR, Zijia Li Z, Peabody MR, Lybarger M, Royal K, Puffer JC. The Predictive Validity of the ABFM’s In-Training 
Examination. Fam Med 2015;47(5):349-56.  
5. Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, 
and biomedical sciences. Behavior Research Methods. 2007; 39 (2): 175-191. 
6. R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, 
Vienna, Austria. URL http://www.R-project.org/. 
3. RESEARCH DESIGN AND METHODS 
Research Design and Methods: 
This study is a longitudinal observation study. A panel of experts (numbering 15-20) consisting of Family Medicine attending’s 
who have been out of residency greater than 3 years will complete the SCT. They will be provided an Informational Consent. 
Their willingness to be a panel member is their implied consent to be part of the research study. Gagnon has shown that, for 
high stake examinations, 15 panel members are required in order to obtain acceptable reliability estimates (Cronbach's alpha 
coefficient)2. The values of these estimates rise with larger panels, but with more than 20 members, improvement is only 
marginal. For lower stake examinations, for instance formative assessment within a clinical rotation, smaller panels can be 
used. However, panels with less than 10 persons are associated with more error in reliability estimates. 
 
Composition is another important issue. The basic idea behind SCT is to compare residents' performance with a group of 
persons who are representatives of the profession (or the specialty) to which they wish to belong. Therefore, panels are made 
up of physicians with good overall clinical experience in the field rather than experts from narrow parts of the field. Panel 
composition also depends on the assessment goal. 
 
The SCT will be made up of 25-30 vignettes, each with 3-4 question items; topics will cover inpatient, obstetrics, outpatient, 
sports medicine, pediatrics, etc. (see list of example topics below). 
• Chest pain:  diagnosis and/or investigation and/or treatment 
• Shortness of breath: diagnosis and/or investigation 
• Pediatrics rash 
• Adult rash 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
4 
• Knee pain 
• Back pain 
• Diabetes 
• Abdominal pain:   diagnosis and/or investigation and/or treatment  
• Pregnancy related complications 
• Dizziness or syncope: diagnosis and/or investigation and/or treatment 
• Paresthesia (ddx: cts, dm peripheral neuro, stroke) 
• Headache: diagnosis and/or investigation and/or treatment 
• Fever (peds) 
• Anxiety/depression: diagnosis and/or investigation  
• Dementia: diagnosis and/or investigation and/or treatment 
• Abnormal Uterine Bleeding: diagnosis and/or investigation (endometrial ca, pregnancy, female triad) 
 
The SCT will be given to PGY1, 2, and 3 residents each year during the Residents three year residency, in June or July.   
 
Each SCT will be scored using an aggregate method3. For each answer the credit is the number of expert members that chose 
that answer divided by the modal value for the question.  Example: 18 experts used.  15 chose -2 (15/15 =1), two chose -1 
(2/15 = 0.13), and one chose 0 (1/15 = 0.06). So a resident that chooses -1 as the answer would be given a score of 0.13 for 
that question.  Each question score will be added to give a total score for the test.  That number will then be divided by the 
number of questions and multiplied by 100 to get a percentage score.  
 
For comparison with the SCT, data will be collected on each resident’s annual ITE score family medicine inpatient rotation 
grades, and ACGME medical knowledge and patient care milestones. For PGY3s the ABFM certification exam score will be 
collected.  ITE and ABFM certification exam scores, family medicine inpatient rotation grades, and ACGME milestone grades 
will be obtained from the residency coordinator and research staff will blind the results prior to results being given to the 
statistician. 
 
The family medicine inpatient rotation will be the only clinical rotation used for the rotation grade component. For ACGME 
milestones, we will collect data on two milestones, Medical Knowledge and Patient Care.  An average of the Medical 
knowledge will be used for MK portion (MK-1, MK-2) and an average of the Patient Care section (PC-1, PC-2, PC-3, PC-4, PC-5) 
for the data points.  Thus, dependent variables will 1) a combined MK score, 2) a combined PC score, and 3) a FM inpatient 
rotation score.  
 
The post graduate year (PGY) will be a co-variant. When comparing to categorical value (pass/fail) we will convert the SCT to a 
pass/fail based on 1 SD below the respective PGY mean.  For the rest of the numerical scales we will do correlations.  Power 
will be set at 0.8. 
 
The results of the SCT will not be used to inform any evaluations or academic milestone determinations. The SCT will be 
blinded and by research staff prior to scoring.   
 
The SCTs will be given once a year during the Residents three year residency.  The research coordinators will place the unique 
subject ID (code) on the SCT and hand out to the enrolled PGY 1, PGY 2, and PGY 3 residents to complete during a didactic 
session in either June or July for that academic year. The participants can either return the completed SCT to the research 
coordinator during the didactic session or at a later date. Once the research coordinator has collected all the SCTs, they will 
then provide the completed and coded SCTs to the panel to score. The research coordinators will be responsible for obtaining 
all the ITE and ABFM scores, family medicine rotation grades, and ACGME milestones from the Residency Coordinator.  Each 
subject’s identifiable demographic information, i.e., name, phone # and e-mail address, will be placed in a Master Key and 
coded.  Access to the Master Key will be restricted to a designated Research Coordinator on the study.  All other investigators 
in this study will not have access to any identifiable information regarding the participating Residents. 
a. Interventions and Observations:  
 SCT scores compared to ITE and ABFM scores, family medicine rotation grades, and ACGME milestones.  
b. Setting: 
Residents would be given a paper copy of the SCT in a didactic session by the research coordinators. 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
5 
 
f. 
Source of Research Material:  
Will you be using private information in this study? 
☒Yes 
If Yes,  
☐protected health information (PHI) held by a covered entity 
☒other types of private information  
Describe: name, email address, phone number 
☒research information (non-PHI) that 
is not publically available (i.e., student 
records) 
Describe: SCT scores, ITE and ABFM scores, failing family medicine 
rotation grades, and ACGME milestones which will be obtained from 
the Residency Coordinator. 
Use of identifiers with private information 
 
Identifiers to be Used? 
 
Column A-
Looked at by 
research team 
Column B-
Recorded on 
enrollment log, 
subject list, or 
key list 
Column C-Recorded 
on data collection 
tool (survey, 
spreadsheet, etc.) 
Column D-
Recorded on 
specimen 
containers 
Column E-
Shared w/ 
others not on 
research team 
Column F-
Stored after 
study ended 
Names 
☒ 
☒ 
☐ 
☐ 
☐ 
☐ 
Study codes linked to 
individuals’ identities using 
a key only accessible by the 
researcher 
 
☒ 
☒ 
☐ 
☐ 
☐ 
Phone/Fax Numbers 
☒ 
☒ 
☐ 
☐ 
☐ 
☐ 
E-mails 
☒ 
☒ 
☐ 
☐ 
☐ 
☐ 
Coding Plan? 
 
Describe the method that will be 
used to create and assign unique 
study codes to data. 
The unique study code will be assigned in sequential order beginning with 001. The code 
will be placed in a Master Key of identifiable PHI/PII for each subject. 
Describe the method that will be 
used to create and assign unique 
study codes to specimens. 
☒N/A, not collecting specimens 
What is the format of the key? 
☒Paper 
☒Electronic 
Who will have access to the key? 
Designated Research Coordinator 
Where will the key be stored 
and how will it be protected? 
 
Location(s): We will maintain a Master Key of identifiable PHI/PII that will be kept in an 
electronic database, which will be encrypted, password protected and the access will be 
restricted to a Designated Research Coordinator.  The Master Key will be electronically 
stored separately from the coded de-identified research data.  The Master Key will not be 
stored on any non-government or personal computers or laptops.   At the conclusion of the 
study, the data will be de-identified prior to review and analysis. 
 
Confidentiality measures: The coded research data will be kept in a locked cabinet in a 
locked office and only the research department has the key. The coded research data will 
be retained until the conclusion of the research study. Once a Final Report has been 
approved by the IRB, all the paper records will be de-identified and any key linking the 
subject to their records will be destroyed, based on AFI 33-332, “The Air Force Privacy and 
Civil Liberties Program” and the National Institute of Standards and Technology Special 
Publication (NIST SP 800-88) for the approved methods to destroy PII. The anonymized 
c. 
Date(s):  
November 2018-November 2023 
d. Subjects: 
Family medicine residents at 99MDG 
e. Inclusion/Exclusion Criteria:  
Inclusion Criteria 
Family Medicine resident at 99MDG 
Exclusion Criteria 
None 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
6 
research data will not be utilized for further research activity beyond the protocol 
stipulations without additional IRB approval. 
Complete the table.  
 
Source of Research Material per Participant (Procedures) 
# Routine Care 
# Research Driven 
# Total Procedures 
 
Script concordance tests 
0 
3 
3 
g. Instrumentation:  
The attached Script Concordance Test vignettes are an example of the vignettes that will be used to collect data. 
4. HUMAN SUBJECT PROTECTION 
Recruitment and Consent Processes:  
Male and Female Active Duty military Family Medicine Residents will be recruited through announcements at a didactic 
session. All potentially eligible residents, will be offered an opportunity to participate by the research coordinators. Since the 
subjects will be subordinates in a training program, the study coordinators (contract personnel) will recruit all subjects in this 
study to prevent any misconception of coercion or undue influence. 
 
Consent Processes: 
Informed Consent and HIPAA authorization will be sought in advance of any screening and study-related procedures from each 
prospective study subject and appropriately documented in accordance with 32 CFR 219.117. Potential candidates will be 
notified about the study either through announcements at a didactic session and will be given the opportunity to consent by 
one of the referred study coordinators. The study coordinator will provide a written copy of the Informed Consent Document 
(ICD). The subject may decline to consent without prejudice. At the subjects’ discretion, they may take the ICD home to discuss 
further with family members or another physician, prior to making a decision. If they decide they are interested in participating 
in the study, they can contact the research department. If the subject consents, a copy of the signed ICD and HIPAA 
Authorization Document will be given to the subject. No vulnerable populations are included in this research study. Subjects 
who cannot provide Informed Consent will not be allowed to participate. No Legally Authorized Representatives (LAR) will be 
utilized. Each subject will be asked to place their de-identified research data into the “Mike O'Callaghan Military Medical 
Center General Research Data Repository” (FWH20180064H) for future research. If the subject does not give their 
authorization, then the de-identified research data will be destroyed no later than 3 years following the closure of the study. 
 
Recruiting Service Members 
Will you be recruiting service members in a group setting?  
☒Yes 
☐No 
 
Participation Compensation:  
Subjects will not be paid for participation in this study. 
 
Assent Process: N/A 
 
Benefits:  
There may be no direct benefit to participants. If the correlation between SCT scores is proven, future residents may benefit 
from learning about potential clinical reasoning deficiencies earlier in their residency. 
 
Risks:  
There may be a risk of accidental breach of confidentiality. 
 
Costs: N/A 
 
Safeguards for Protecting Information:  
All identifiable PHI/PII data will be kept in an electronic database stored separately from the coded, de-identified research 
data, which will be encrypted, double password protected and the access will be restricted to the Designated Research 
Coordinator. The research data will be coded and we are not collecting any Protected Health Information. The anonymized 
research data will not be utilized for further research activity beyond the protocol stipulations without additional IRB approval. 
Data and Specimen Storage Plan 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
7 
How will coded or identifiable data/specimens be stored? 
 
☒ 
Paper data, including completed consent forms 
The research consents and HIPAA Authorization Documents will 
be stored in a locked cabinet in a locked room with restricted 
access. 
☒ 
Electronic data 
All coded, de-identified research data will be electronically 
stored separately from the Master Key of identifiable patient 
demographics and PHI/PII at the Nellis Family Medicine site. 
☐ 
Long-term storage (following completion of the 
study and inactivation of IRB approval) 
The research data will be coded and any links to identifiable 
data will be destroyed (an approved shredding bin) as soon as 
possible or no later than at the closure of the study, with the 
exception of those study subjects that consent to place their de-
identified research data into the “Mike O'Callaghan Military 
Medical Center General Research Data Repository” 
(FWH20180064H) for future research. The anonymized research 
data will not be utilized for further research activity beyond the 
protocol stipulations without additional IRB approval.  All de-
identified research data will be maintained for 3 years following 
study closure. 
 
Safeguards for Protecting Subjects Relative to Reasonably Expected Risks:  
There are minimal risks to the subjects and include those related to an inadvertent breach of confidentiality. Research 
Coordinators will recruit and code data so that the PI cannot identify the subjects.  A Designated Research Coordinator will 
have the only direct access to the Master Key of identifiable PHI/PII. 
 
Categories of subjects 
☒Family Medicine Residency Students 
 
Clinical Care: N/A 
 
Injury Compensation: N/A 
 
Data Safety Monitoring 
☒N/A – none of the situations listed above apply 
5. ALTERNATIVES 
Alternatives:  
The alternative is not to participate. 
6. DATA ANALYSIS 
Data Analysis:  
Outcome Measures:  
There are two outcome measures.  The first being the interval SCT score produced as described in the Research Design and 
Methods section. SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 
subjects enrolled in the study each year of their residency in June or July.  The second outcome measure is a binomial SCT 
classification derived from each interval SCT score. A “fail” classification is defined as an SCT score 1 SD below the respective 
PGY mean.  All other SCT scores will be classified as “pass”.   
 
These outcome measures will serve as independent variables to explore statistical associations with several dependent 
variables. These are the previously described ITE score, family medicine inpatient rotation grade, ACGME medical knowledge 
and patient care milestone averages, and ABFM certification exam score (applicable to PGY3 only).  As the ITE score has been 
shown to have its greatest improvement from residency year 1 to 24, year of residency will serve as a covariate. 
 
Sample Size Estimation/Power Analysis:  
Subjects will be a random sample of PGY1, 2, and 3 family medicine residents in a primary care setting at 99 MDG. The subjects 
are presumed to be randomly selected from the population of similar residents at other Air Force medical treatment facilities. 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
8 
There being no prior information on which to base an effect size for the outcome measures, sample size will be based on an 
assumption of a moderate effect size to achieve an a priori power of 0.80.   
 
Though the study is primarily exploratory, examining the nature of statistical associations between independent SCT scores 
and the several dependent variables, an a priori power analysis may be based a statistical hypothesis derived from a mixed 
effects, randomized complete block design with repeated measures employing the interval SCT score as an independent 
variable, regression year as a covariate, and ITE score as the dependent variable. Subject is a random effect as explained 
above.  Residency year covariate is also a random effect as this is a random characteristic of subjects.  Time of repeated 
measure is a fixed effect as these times cannot be generalized to other times. 
 
A priori power for a repeated measures analysis of covariance (rANCOVA), mixed effects design, was assessed using G*Power 
Version 3.1.9.2 5. Results indicate a sample size of 104 subjects will have a power of 0.80 at α = 0.05 to detect a moderate 
effect size of 0.25 over the repeated measures. 
 
Statistical Analysis:  
Sample and residency year group means, standard deviations, medians, quantiles and interquartile ranges (IQR) will be 
calculated for descriptive interval variables. Frequency distributions of descriptive nominal variables will be calculated. 
Differences in residency year group means and medians for interval variables will be tested using ANOVA and Kruskal-Wallis 
One-way Analysis of Variance by ranks respectively.  For nominal variables, proportions will be calculated for each group and 
statistical association tested with Chi Squared or Fisher's Exact Test. Odds ratio or Cramer’s V will be used to indicate strength 
of association for significant tests. 
 
Exploration of the linear bivariate relationships between the independent interval SCT score and the several dependent 
interval variables will be done using Pearson correlation, Kendall rank correlation, and/or Spearman correlation.  Exploration of 
the statistical associations between the binomial SCT pass/fail classification and the several dependent interval variables will 
be done by testing differences of dependent interval means and medians between SCT pass/fail groups using univariate t-test 
and Mann-Whitney U test respectively.  
 
Null hypotheses for longitudinal interval data will be tested by a mixed effects rANCOVA.  In the event the rANCOVA null 
hypothesis is rejected, contrasts will be used to investigate effects and differences within time intervals.  In the event multiple 
comparison tests are used to investigate effects, the Holm method will be used to correct the significance level of α to p = .05. 
 
Mr. Danny Sharon, Senior Research Biostatistician Subject Matter Expert for Clinical Research Management under contracts 
OMNI 0004 3-82 and OMNI 0005 3-126, is the statistical consultant supporting this study. Statistical analysis will be performed 
with R Version 3.3.2.7 6. 
 
Number of Subjects:  
 
# Planned to Enroll 
# Enrolled 
# Planned to Complete Study 
TOTAL 
Number of Subjects at 99 MDG 
130 
N/A 
104 
130 
7. STUDY DURATION 
Duration of Study:  
Approximate duration of the study: 5 years 
8. LOCAL AND EXTERNAL SUPPORT SERVICES 
Local and External Support Services: None 
Describe the plan for training personnel 
☒Not applicable – no training of non-study personnel required 
9. INTRAMURAL (GME) AND EXTRAMURAL FUNDING SUPPORT 
Intramural (GME) and Extramural Funding Support: None 
10. DRUGS, BIOLOGICS, DIETARY SUPPLEMENTS, AND MEDICAL DEVICES 
Does the study plan dictate the use of any of the following? 
A drug 
☒No 
A biologic 
☒No 

59th Medical Wing Institutional Review Board 
Version: 4 Apr 2018 
 
 
 
 
 
 
 
 
 
 
 
 
9 
A compound intended to affect structure or any function of the body 
☒No 
A dietary supplement or substance generally recognized as safe that will be used to diagnose, cure, mitigate, treat, or 
prevent disease 
☒No 
A medical device 
☒No 
 
Is this research an “applicable clinical trial” which must be registered on ClinicalTrials.gov? 
☒No 
 
Use of a placebo in place of standard therapy: 
Is a placebo being used in place of standard therapy? 
☒No 
☐Yes 
11. MEDICAL RESEARCH AREA 
☒Other: 
Family Medicine Residency 
12. ATTACHMENTS 
1. Form A, Signature Sheet 
2. Form A-2, Study Personnel Listing 
3. Form D, Informed Consent Document 
4. H15 Template, HIPAA Authorization Document 
5. Script concordance questions