37

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

APPENDIX A. SEARCH STRATEGIES

Table A-1. Search strategy for RCTs and observational studies (PubMed, April 2012)

Step Category Terms Result

1 Terms for “group 

appointment”

(visit[ti] OR visits[ti] OR appointment[ti] OR appointments[ti] OR clinic[ti] 

OR clinics[ti] OR “Appointments and Schedules”[Mesh]) AND (group[ti] 

OR shared[ti] OR cluster[ti])

744

2 Terms for “shared 

visits”

(“shared medical appointment”[tiab] OR “shared medical 

appointments”[tiab] OR “group care”[tiab] OR “group medical 

appointment”[tiab] OR “group medical appointments”[tiab] OR “cluster 

visit”[tiab] OR “cluster visits”[tiab] OR “group visit”[tiab] OR “group 

visits”[tiab] OR “shared medical visit”[tiab] OR “shared medical visits”[tiab] 

OR “group medical clinic”[tiab] OR “group medical clinics”[tiab])

313

3 Combined 

intervention terms

#1 OR #2 961

4 Terms for RCT 

study design

(randomized controlled trial[pt] OR controlled clinical trial[pt] OR 

randomized[tiab] OR randomised[tiab] OR randomization[tiab] 

OR randomisation[tiab] OR placebo[tiab] OR drug therapy[sh] OR 

randomly[tiab] OR trial[tiab] OR groups[tiab] OR Clinical trial[pt] OR 

“clinical trial”[tw] OR “clinical trials”[tw] OR “evaluation studies”[Publication 

Type] OR “evaluation studies as topic”[MeSH Terms] OR “evaluation 

study”[tw] OR evaluation studies[tw] OR “intervention studies”[MeSH 

Terms] OR “intervention study”[tw] OR “intervention studies”[tw] 

OR “cohort studies”[MeSH Terms] OR cohort[tw] OR “longitudinal 

studies”[MeSH Terms] OR “longitudinal”[tw] OR longitudinally[tw] 

OR “prospective”[tw] OR prospectively[tw] OR “follow up”[tw] OR 

“comparative study”[Publication Type] OR “comparative study”[tw] OR 

systematic[subset] OR “meta-analysis”[Publication Type] OR “metaanalysis as topic”[MeSH Terms] OR “meta-analysis”[tw] OR “metaanalyses”[tw]) NOT (Editorial[ptyp] OR Letter[ptyp] OR Case Reports[ptyp] 

OR Comment[ptyp]) NOT (animals[mh] NOT humans[mh])

4216173

5 Intervention AND 

RCT

#3 AND #4 466

6 Terms for 

observational 

studies

“pre-post”[tiab] OR “post-test”[tiab] OR “post test”[tiab] OR pretest[tiab] 

OR pre-test[tiab] OR “pre test”[tiab] OR quasi-experiment*[tiab] OR 

quasiexperiment*[tiab] OR quasirandom*[tiab] OR quasi-random*[tiab] 

OR quasi-control*[tiab] OR quasicontrol*[tiab] OR (“time series”[tiab] AND 

interrupt[tiab]) OR (“time points”[tiab] AND (multiple[tiab] OR one[tiab] 

OR two[tiab] OR three[tiab] OR four[tiab] OR five[tiab] OR six[tiab] OR 

seven[tiab] OR eight[tiab] OR nine[tiab] OR ten[tiab] OR month*[tiab] OR 

day*[tiab] OR week*[tiab] OR hour*[tiab]) OR (before[tiab] AND after[tiab]) 

OR (*before[tiab] AND during[tiab])

44225

7 Intervention and 

Observational 

#3 AND #6 11

8 Applies limits to 

combined RCT 

and observational 

studies

#5 OR #7 with limits: English, Publication Date from 1996 to 2011 323
38

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

APPENDIX B. EXCLUDED STUDIES

All articles listed below were reviewed in their full-text version and excluded for the reason 

indicated. An alphabetical reference list follows the table.

Table B-1. Excluded studies with reasons

Reference

Not full 

publication,

peerreviewed, or 

primary data

Not study 

population 

of interest

Not eligible

study 

design

Comparator 

not of 

interest

Intervention 

does not 

meet 

protocol 

definition

Not an 

outcome 

of interest 

reported at 

≥3 months

AHRQ, 2003 (943) X

AHRQ, 2007 (844) X

Anonymous, 1996 (946) X

Anonymous, 2001 (913) X

Anonymous, 2001 (944) X

Anonymous, 2003 (351) X

Antonucci, 2008 (835) X

Barud, 2006 (730) X

Block, 2010 (747) X

Bray, 2005 (299) X

Bronson, 2004 (1331) X

Brooks, 2007 (265) X

Campbell, 2007 (518) X

Clancy, 2003 (347) X

Clancy, 2007 (259) X

Conrad, 2008 (775) X

Desouza, 2010 (157) X

Anonymous, 2001 (373) X

Dontje, 2011 (607) X

Falck-Ytter, 2009 (1286) X

Geller, 2011 (142) X

Harris, 2010 (178) X

Jaber, 2006 (780) X

Jeanfreau, 2008 (732) X

Anonymous, 2002 (955) X

Katz, 1975 (596) X

Kirsh, 2006 (1312) X

Krywkowski-Mohn, 2009 

(1508)

X

Lin, 2008 (214) X

Loney-Hutchinson, 2009 

(703)

X

Mackay, 2011 (649) X

Masley, 2001 (386) X

Mayo Clinic Proceedings, 

2008 (507)

X

McCulloch, 1998 (410) X

McHugh, 1998 (420) X

Miller, 329 (329) X

Murray, 2005 (313) X

Ostroff, 2010 (1278) X

Palaniappan, 2011 (135) X

Peterson, 2007 (929) X

Porta, 2004 (326) X

Reiber, 2004 (328) X
39

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reference

Not full 

publication,

peerreviewed, or 

primary data

Not study 

population 

of interest

Not eligible

study 

design

Comparator 

not of 

interest

Intervention 

does not 

meet 

protocol 

definition

Not an 

outcome 

of interest 

reported at 

≥3 months

Rivard, 2009 (498) X

Rossi, 2011 (1269) X

Salinas, 2006 (1308) X

Salinas-Martinez, 2009 

(1282) X

Sanchez, 2011 (608) X

Scott, 1998 (426) X

Shahady, 2008 (795) X

Shahady, 2010 (465) X

Stoner, 2001 (375) X

Taveira, 2008 (1686) X

Thompson, 2000 (875) X

Thompson, 2001 (389) X

Trento, 2006 (291) X

Trento, 2008 (1291) X

Trento, 2008 (238) X

Trento, 2009 (1283) X

Trento, 2009 (1284) X

Trento, 2009 (904) X

Vachon, 2007 (237) X

Vinci, 2006 (1311) X

Watkinson, 2004 (976) X

Watts, 2009 (1582) X

Weber, 2004 (1333) X

Westheimer, 2009 (666) X

Wheelock, 2009 (211) X

Worth, 1990 (557) X

Yehle, 2007 (689) X

Yehle, 2009 (213) X

Yu, 2010 (165) X

LIST OF EXCLUDED STUDIES

Agency for Healthcare Research and Quality. 

Group visits to primary care doctors by 

disadvantaged diabetes patients result in better 

diabetes care than individual visits. AHRQ 

Research Activities. 2003(278):14-14.

Agency for Healthcare Research and Quality. 

Studies examine medication adherence and 

group medical visits among persons with high 

blood pressure. AHRQ Research Activities. 

2007(326):16-17.

Anonymous. Colorado HMO trades traditional 

doctor visits for group clinics. Disease State 

Management. 1996;2(2):13-17.

Anonymous. DIGMAs (drop-in group medical 

appointments): satisfaction Rx for doctors and 

patients. Hosp Peer Rev. 2001;26(6):81-2.

Anonymous. Researchers find new value in group 

visit concept among chronically ill adults. Disease 

Management Advisor. 2001;7(8):121-125.

Anonymous. Take steps to ensure group visits 

are successful: start with chronic patient groups. 

Patient Education Management. 2001;8(8):92.

Anonymous. Group appointments for seniors 

continue to prove their worth. Senior Care 

Management. 2002;5(4):57-59.
40

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Anonymous. Shared medical appointments save 

money for capitated groups. Capitation Manag 

Rep. 2003;10(2):20-4, 17.

Antonucci J. A new approach to group visits: 

helping high-need patients make behavioral 

change. Fam Pract Manag. 2008;15(4):A6-8.

Barud S, Marcy T, Armor B, et al. Development 

and implementation of group medical visits at a 

family medicine center. Am J Health Syst Pharm. 

2006;63(15):1448-1452.

Block JP. Outcomes research in review. Group 

visits for the treatment of hypertension among 

diabetics: success without a pill? Journal of 

Clinical Outcomes Management. 2010;17(8):343.

Bray P, Roupe M, Young S, et al. Feasibility and 

effectiveness of system redesign for diabetes 

care management in rural areas: the eastern 

North Carolina experience. Diabetes Educ. 

2005;31(5):712-8.

Bronson DL, Maxwell RA. Shared medical 

appointments. Increasing patient access without 

increasing physician hours. Cleve Clin J Med. 

2004;71(5):369-+.

Brooks AD, Rihani RS, Derus CL. Pharmacist 

membership in a medical group’s diabetes health 

management program. Am J Health Syst Pharm. 

2007;64(6):617-21.

Campbell BB, Gosselin D. High patient 

satisfaction amongst males participating in men’s 

educational group appointments (MEGA) for 

routine physical exams. Journal of Men’s Health 

and Gender. 2007;4(3):266-270.

Clancy DE, Cope DW, Magruder KM, et al. 

Evaluating group visits in an uninsured or 

inadequately insured patient population with 

uncontrolled type 2 diabetes. Diabetes Educ. 

2003;29(2):292-302.

Clancy DE, Yeager DE, Huang P, et al. Further 

evaluating the acceptability of group visits in 

an uninsured or inadequately insured patient 

population with uncontrolled type 2 diabetes. 

Diabetes Educ. 2007;33(2):309-14.

Conrad D, Fishman P, Grembowski D, et al. Access 

intervention in an integrated, prepaid group practice: 

effects on primary care physician productivity. 

Health Serv Res. 2008;43(5 Part 2):1888-1905.

Desouza CV, Rentschler L, Haynatzki G. The 

effect of group clinics in the control of diabetes. 

Prim Care Diabetes. 2010;4(4):251-4.

Dontje K, Forrest K. Implementing Group Visits: 

Are They Effective to Improve Diabetes SelfManagement Outcomes? Journal for Nurse 

Practitioners. 2011;7(7):571-577.

Falck-Ytter CD, Ellert R, Denise K, et al. Effect 

of a Diabetes Group Clinic Using Shared Medical 

Visits on Outcomes for Diabetic Patients Cared 

for in an Urban Resident Clinic. J Gen Intern 

Med. 2009;24:252-252.

Geller JS, Orkaby A, Cleghorn GD. Impact of a 

group medical visit program on Latino health-related 

quality of life. Explore (NY). 2011;7(2):94-9.

Harris MD. Shared medical appointments after 

cardiac surgery-the process of implementing a 

novel pilot paradigm to enhance comprehensive 

postdischarge care. J Cardiovasc Nurs. 

2010;25(2):124-9.

Jaber J, Braksmajer A, Trilling J. Group Visits 

for Chronic Illness Care: Models, Benefits and 

Challenges. Fam Pract Manag. 2006;13(1):37-40.

Jeanfreau S. Group visit intervention to improve 

diabetes care -- a program utilizing group visits led 

by a nurse practitioner to improve outcomes for the 

medically underserved in diabetes care. Southern 

Online Journal of Nursing Research. 2008;8(2):2p.

Katz G, Hollander FL. From clinic to group 

practice. Hospitals. 1975;49(5):67-71.

Kirsh S, Watts S, Aron D, et al. Integrating 

residency training and diabetes shared medical 

appointments. J Gen Intern Med. 2006;21:74-75.

Krzywkowski-Mohn SM. Diabetic control and 

patient perception of the scheduled in group 

medical appointment at the Cincinnati veterans 

administration medical center. ProQuest 

Information & Learning; 2009.

Lin A, Cavendish J, Boren D, et al. A pilot 

study: reports of benefits from a 6-month, 

multidisciplinary, shared medical appointment 

approach for heart failure patients. Mil Med. 

2008;173(12):1210-3.
41

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Loney-Hutchinson LM, Provilus AD, Jean-Louis 

G, et al. Group visits in the management of 

diabetes and hypertension: effect on glycemic and 

blood pressure control. Current Diabetes Reports. 

2009;9(3):238-242.

Mackay FD. Well Woman’s Group Medical 

Appointment: For screening and preventive care. 

Can Fam Physician. 2011;57(4):e125-7.

Masley S, Phillips S, Copeland JR. Group office 

visits change dietary habits of patients with 

coronary artery disease-the dietary intervention 

and evaluation trial (D.I.E.T.). J Fam Pract. 

2001;50(3):235-9.

Mayo Clinic Proceedings. Chronic care model 

and shared care in diabetes: Randomized trial 

of an electronic decision support system (Mayo 

Clinic Proceedings (2008) 83, 7, (747-757)). 

Mayo Clin Proc. 2008;83(10):1189.

McCulloch DK, Price MJ, Hindmarsh M, et 

al. A population-based approach to diabetes 

management in a primary care setting: early 

results and lessons learned. Eff Clin Pract. 

1998;1(1):12-22.

McHugh F, Lindsay G. A study of nurse-led 

shared care for coronary patients. Nurs Stand. 

1998;12(45):33.

Miller D, Zantop V, Hammer H, et al. Group 

medical visits for low-income women with 

chronic disease: a feasibility study. J Womens 

Health (Larchmt). 2004;13(2):217-25.

Murray JL, Everson K. Group medical visits 

and lifestyle modifications. J Fam Pract. 

2005;54(1):64.

Ostroff ABM, Sundel S, Bradley S, et al. 

Group Visits: A Model to Improve Patient SelfManagement in Geriatric Ambulatory Practice. J 

Am Geriatr Soc. 2010;58:228-228.

Palaniappan LP, Muzaffar AL, Wang EJ, et al. 

Shared medical appointments: promoting weight 

loss in a clinical setting. J Am Board Fam Med. 

2011;24(3):326-8.

Peterson C, Chicano K, Capone-Swearer D, et al. 

Shared medical appointments. Nurse Practitioner 

World News. 2007;12(6):16-16.

Porta M, Trento M. ROMEO: rethink organization 

to improve education and outcomes. Diabet Med. 

2004;21(6):644-5.

Reiber GE, Au D, McDonell M, et al. Diabetes 

quality improvement in Department of Veterans 

Affairs Ambulatory Care Clinics: a grouprandomized clinical trial. Diabetes Care. 2004;27 

Suppl 2:B61-8.

Rivard L, Walker C, Arias A, et al. Group medical 

visits achieve improvements of clinical outcomes 

in diverse ethnic, underinsured populations: A 

project dulce(trademark) disease management 

intervention. Diabetes. 2009;58.

Rossi AM, Tucker AL, Hedelt AC. Cardiovascular 

Disease Prevention Tailored for Women: Shared 

Medical Appointments. J Cardiovasc Nurs. 

2011;26(4):276-277.

Salinas AM, Irizar JF, Garza MG, et al. The 

cluster visit along with all-inclusive management: 

A good plan for improving diabetes primary care 

effectiveness? Diabetes. 2006;55:A552-A552.

Salinas-Martinez AM, Garza-Sagastegui MG, 

Cobos-Cruz R, et al. Effects of incorporating 

group visits on the metabolic control of 

type 2 diabetic patients. Rev Med Chil. 

2009;137(10):1323-1332.

Sanchez I. Implementation of a Diabetes Selfmanagement Education Program in Primary Care 

for Adults Using Shared Medical Appointments. 

Diabetes Educ. 2011;37(3):381-391.

Scott J, Gade G, McKenzie M, et al. Cooperative 

health care clinics: a group approach to 

individual care. Geriatrics. 1998;53(5):68-70, 

76-8, 81; quiz 82.

Shahady EJ. Diabetes management: an approach 

that improves outcomes and reduces costs. 

Consultant (00107069). 2008;48(4):331.

Shahady EJ. Group visits for diabetes: An 

innovative way to overcome barriers and achieve 

quality care. Cons_ltant. 2010;50(11).

Stoner KL, Lasar NJ, Butcher MK, et al. 

Improving glycemic control: can techniques used 

in a managed care setting be successfully adapted 

to a rural fee-for-service practice? Am J Med 

Qual. 2001;16(3):93-8.
42

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Taveira TH, Pirraglia PA, Cohen LB, Wu WC. 

Efficacy of a pharmacist-led cardiovascular risk 

reduction clinic for diabetic patients with and 

without mental health conditions. Prev Cardiol

2008 Fall;11(4):195-200.

Thompson E. Physicians. The power of group 

visits: improved quality of care, increased 

productivity entice physicians to see up to 15 

patients at a time. Mod Healthc. 2000;30(23):54.

Thompson M, Gee S, Larson P, et al. Health and 

loyalty promotion visits for new enrollees: results 

of a randomized controlled trial. Patient Educ 

Couns. 2001;42(1):53-65.

Trento M. ROMEO (Rethink Organization to 

iMprove Education and Outcomes). A 4-year 

multicentre randomised controlled trial of group 

care for the management of type 2 diabetes. 

Diabetologia. 2008;51:S69-S69.

Trento M, Passera P, Bajardi M, et al. 

Lifestyle intervention by group care prevents 

deterioration of Type II diabetes: a 4-year 

randomized controlled clinical trial. Diab tologia. 

2002;45(9):1231-9.

Trento M, Passera P, Miselli V, et al. Evaluation 

of the locus of control in patients with type 2 

diabetes after long-term management by group 

care. Diabetes Metab. 2006;32(1):77-81.

Trento M, Porta M. Romeo (Rethink Organization 

to Improve Education and Outcomes). A 4-Year 

Multicentre Randomised Controlled Trial of Group 

Care for the Management of Type 2 Diabetes. 

Atherosclerosis Supplements. 2009;10(2).

Trento M, Sicuro J, Semperbene L, et al. Costefficacy of group care in the management of 

type 2 diabetes. Economic evaluation of the 

ROMEO (Rethink Organization to iMprove 

Education and Outcomes) data set. Diabetologia. 

2009;52:S391-S391.

Trento M, Tomelini M, Basile M, et al. The locus 

of control in patients with Type 1 and Type 2 

diabetes managed by individual and group care. 

Diabet Med. 2008;25(1):86-90.

Vachon GC, Ezike N, Brown-Walker M, et al. 

Improving access to diabetes care in an inner-city, 

community-based outpatient health center with 

a monthly open-access, multistation group visit 

program. J Natl Med Assoc. 2007;99(12):1327-36.

Vinci LM, Clark J, Buchholz N, et al. Improving 

diabetes care with nurse practitioner-led group 

visits and simple case management. J Gen Intern 

Med. 2006;21:72-72.

Watkinson M. Group visits improved concordance 

with American Diabetes Association practice 

guidelines in type 2 diabetes. Evidence Based 

Nursing. 2004;7(2):57-57.

Watts SA, Gee J, O’Day ME, et al. Nurse 

practitioner-led multidisciplinary teams to 

improve chronic illness care: The unique strengths 

of nurse practitioners applied to shared medical 

appointments/group visits. J Am Acad Nurse 

Pract. 2009;21(3):167-172.

Weber V, Bulger J, Sim J. Implementing a group 

visit model for adult patients with diabetes: A 

pilot. J Gen Intern Med. 2004;19:105-106.

Westheimer JM, Capello J, McCarthy C, et al. 

Employing a group medical intervention for 

hypertensive male veterans: an exploratory 

analysis. Journal for Specialists in Group Work. 

2009;34(2):151-174.

Wheelock C, Savageau JA, Silk H, et al. 

Improving the health of diabetic patients through 

resident-initiated group visits. Fam Med. 

2009;41(2):116-9.

Worth RC, Nicholson A, Bradley P. Shared care 

for diabetes in Chester: Preliminary experience 

with a ‘clinic-wide’ scheme. Practical Diabetes. 

1990;7(6):266-268.

Yehle KS. A comparison of standard office visits 

and shared medical appointments in adults with 

heart failure. Touro University International; 2007.

Yehle KS, Sands LP, Rhynders PA, et al. The 

effect of shared medical visits on knowledge and 

self-care in patients with heart failure: a pilot 

study. Heart Lung. 2009;38(1):25-33.

Yu GC, Beresford R. Implementation of a chronic 

illness model for diabetes care in a family 

medicine residency program. J Gen Intern Med. 

2010;25 Suppl 4:S615-9.
43

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

APPENDIX C. DATA ABSTRACTION ELEMENTS 

Study Characteristics:

•	 Study Sites and Setting

•	 Study Design

•	 Comparator Type 

•	 Enrollment Approach

•	 Study Enrollment/Study Completion (N’s)

•	 Patient Eligibility Criteria for Study

Population Characteristics:

•	 Demographic

•	 Baseline Biophysical Characteristics

Intervention Components:

•	 Time period of intervention

•	 Type of model session and care team

•	 Number and duration of visits planned

•	 Number of health professionals present

•	 Was the prescribing clinician present?

•	 Size of patient group

•	 Were family members/friends invited to participate?

•	 Were medication changes made during the SMA visit?

•	 Did any clinician spend time with group members individually?

•	 Was the contact with the patients over the telephone outside of the SMA?

•	 Health professionals who conducted the educational session

•	 Theoretical orientation of the intervention

•	 Did group member have input on education topics?

•	 Topics covered during the session

•	 Strategy used with SMA group

•	 Were printed materials provided, and were they tailored?

Outcome Components:

•	 Target conditions

o Biophysical markers postintervention values

	HbA1c

	Blood Pressures

	Lipids

•	 Patient and staff experience

•	 Adherence (medication, visit, and self-management)

•	 Symptom severity

•	 Quality of life

•	 Functional status

•	 Resource utilization

•	 Direct cost and total cost

•	 Adverse effects
44

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

APPENDIX D. CRITERIA USED IN QUALITY ASSESSMENT 

General Instructions:

For each risk of bias item, rate as “Yes,” “No,” or “Unclear.” After considering each of the 

quality items, give the study an overall quality rating of good, fair, or poor.

Detailed Quality Items:

If an item is rated as “No,” describe why in the comments column.

Randomization and allocation concealment:

a. *Randomization adequate? Was the allocation sequence adequately generated? 

Yes No Not

 reported/Unclear

b. *Allocation concealment? Was allocation adequately concealed?

Yes No Not

 reported/Unclear

Outcomes:

a. *Outcome assessors blinded (hard outcomes)? Were Outcome assessors blind to 

treatment assignment for “hard outcomes” such as mortality?

Yes No Not

 reported/Unclear

b. *Outcome assessors blinded (soft outcomes)? Were Outcome assessors blind to treatment assignment for “soft outcomes” such as symptoms?

Yes No Not reported/Unclear

c. Lack of measurement bias? Were the measures used reliable and valid? If so, choose 

“Yes,” indicating no important measurement bias. 

Yes No Not

 reported/Unclear

Data analysis:

a. *All outcomes reported? Are reports of the study free of suggestion of selective outcome reporting (systematic differences between planned and reported findings)?

Yes No Not reported/Unclear
45

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

b. *Incomplete outcome data adequately addressed?

 Yes (no systematic differences between groups in withdrawals from study and 

no high overall loss to follow-up; all eligible, randomized patients are included in 

analysis (ITT)

 No

 Not reported/Unclear

c. Adequate power for main effects?

Yes No Not

 reported/Unclear

Results:

a. Other selection bias? Were systematic differences observed in baseline characteristics 

and and prognostic factors across the groups compared?

Yes No Not

 reported/Unclear

b. *Comparable groups maintained? (Includes crossovers, adherence, and contamination). Consider issues of crossover (e.g., from one intervention to another), adherence 

(major differences in adherence to the interventions being compared), contamination 

(e.g., some members of control group get intervention), or other systematic differences in care that was provided.

Yes No Not

 reported/Unclear

Conflict of interest:

a. Was there the absence of potential important conflict of interest? The focus here is 

financial conflict of interest. If no financial conflict of interest (e.g., if funded by 

government or foundation and authors do not have financial relationships with drug/

device manufacturer), then answer “Yes.”

Yes No Not

 reported/Unclear

* Items contained in Cochrane Risk of Bias Tool
46

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Overall study rating:

Choose an item.

Please assign each study an overall quality rating of “Good,” “Fair,” or “Poor” based on the 

following definitions:

A “Good” study has the least bias, and results are considered valid. A good study has a clear 

description of the population, setting, interventions, and comparison groups; uses a valid 

approach to allocate patients to alternative treatments; has a low dropout rate; and uses 

appropriate means to prevent bias, measure outcomes, and analyze and report results.

A “Fair” study is susceptible to some bias but probably not enough to invalidate the results. 

The study may be missing information, making it difficult to assess limitations and potential 

problems. As the fair-quality category is broad, studies with this rating vary in their strengths and 

weaknesses. The results of some fair-quality studies are possibly valid, while others are probably 

valid.

A “Poor” rating indicates significant bias that may invalidate the results. These studies have 

serious errors in design, analysis, or reporting; have large amounts of missing information; or 

have discrepancies in reporting. The results of a poor-quality study are at least as likely to reflect 

flaws in the study design as to indicate true differences between the compared interventions.

Comments:

Form status:

 Fully complete – ready for export 

 Not ready for export – should be discussed further/changes reconciled with the abstractor 
47

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

APPENDIX E. PEER REVIEW COMMENTS 

Reviewer Comment Response

Question 1: Are the objectives, scope, and methods for this review clearly described?

1 Yes. The authors present the objectives and scope in a very succinct 

fashion. The methods are described in great detail. The key questions 

for review are very relevant in my opinion. Key question 1(KY 1) was 

well defined and the authors found 18 studies to evaluate. However 

KY2 and 3 are quite broad and not as clearly defined as KY 1. (Page 

1, line 43, Page 2, Line 2). As there were not enough studies to 

address these questions, I can only speculate that if the questions were 

more focused, that the authors would have had better luck. They are 

to be commended for a thorough and detailed lit review, anyway to 

answer the call. 

Thank you. The key 

questions were developed 

in collaboration with our 

stakeholders. 

2 Yes. The terms “objectives” and “scope” were not used exactly; 

however the intent of this section was clearly described. The methods 

were superbly articulated.

Thank you.

3 Yes. Very clear. Acknowledged

4 Yes. No comment. Acknowledged

5 Yes. No comment. Acknowledged

6 Yes. But comments under question 4. Acknowledged

7 Yes. No comment. Acknowledged

Question 2: Is there any indication of bias in our synthesis of the evidence?

1 No. No comment. Acknowledged

2 No. Risk of bias was evaluated when rating the body of evidence. 

Threats to internal validity of the systematic review conclusions 

were accounted for in potential selection bias, performance bias, and 

attribution and detection bias. Bias was accounted for by using criteria 

in the quality assessment tool in Appendix D for the review of the 

literature.

Thank you.

3 No. No comment. Acknowledged

4 No. No comment. Acknowledged

5 No. No comment. Thank you.

6 No. But comments under question 4. Acknowledged

7 No. No comment. Acknowledged

Question 3: Are there any published or unpublished studies that we may have overlooked?

1 No. Not that I am aware of. Acknowledged

2 No. Not that I am aware of. Acknowledged

3 No. No comment. Acknowledged

4 No. I am not a SME on this topic so I may not be aware of some 

overlooked studies.

Acknowledged

5 Yes. I have unpublished retrospective pre-test/post-test study data 

awaiting consideration for publication from Diabetes Care Journal. 

N=1170 with ~ 1% A1c level drop.

Thank you for informing 

us about your data. We 

were unable to obtain a 

copy of this manuscript 

prior to finalizing our 

report.
48

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

6 No. I am not aware of any, but I would be surprised if I knew of them 

from routine practice while the authors used a rigorous process. 

They do not include the many group interventions directed at weight 

control. I am not sure why these escaped their search criteria. I think 

they are not viewed as medical appointments by indexers, even 

though most would think the programs defined by the AHEAD study 

or even many MOVE! programs are medical encounters

Our focus on specific 

chronic conditions—

asthma, coronary artery 

disease, congestive 

heart failure, chronic 

obstructive pulmonary 

disease, diabetes 

mellitus, hyperlipidemia, 

hypertension—was 

developed in 

collaboration with 

stakeholders. Obesity 

was considered but 

not included since 

medication management 

is not as prominent a 

component compared to 

the included conditions.

7 Yes. Published - Cohen, L. B., Taveira, T. H., Khatana, S. A., Dooley, 

A. G., Pirraglia, P. A., & Wu, W. C. (2011). Pharmacist-led shared 

medical appointments for multiple cardiovascular risk reduction in 

patients with type 2 diabetes. [Randomized Controlled Trial Research 

Support, Non-U.S. Gov’t]. Diabetes Educ, 37(6), 801-812. doi: 

10.1177/0145721711423980

Unpublished - Pharmacist-led Group Medical Visits to Help With 

Diabetes Management (MEDIC-1 year), NCT00554671

Thank you for making 

us aware of this study. 

It was published after 

our literature search but 

is now included in the 

review. 

Thank you for making 

us aware of this study. 

It has been added to the 

appendix of ongoing 

clinical trials.

Question 4: Please write additional suggestions or comments below. If applicable, please indicate the page and 

line numbers from the draft report

1 Few suggestions include:

1. (Page 3, Line 13) of executive summary: Mentioning the duration 

of the studies for the reader to get a snapshot would beneficial, as 

meaningful change in chronic disease takes time. See below.

2. (Page3, Line 37) of the executive summary: SMAs were associated 

with lower A1c than usual care (mean difference=0.58; 95% CI, -1.05 

to -0.11) Again mentioning the time frame would be useful, especially 

as this was one of the main outcomes of their work. It is described on 

(page 22, line 18), assessed at 6 months to 4 years.

3. Page 24: Figure 4. Forest plots for meta-analysis on cholesterol. 

The headings for mean and SD says (mm Hg). Should say (mg/dl). I 

believe this is an error.

4. Page 24: Figure 5. Headings for mean SBP, should say (mmHg) 

1. We added the range 

of followup to this 

section.

2. Thank you. This 

addition was made.

3. Thank you. This 

correction was 

made.

4. Thank you. This 

correction was 

made.
49

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

1 cont. I would like to commend the authors very highly for undertaking this 

extensive review. There is certainly great need to recognize, investigate 

and assess newer models of care for chronic illness in the 21st century. 

They present the current state of chronic illness care clearly, (page 8, line 

22-26) which forms a nice background to the topic The methodology 

used is described in great detail and clear. Their conclusion after an 

extensive rigorous analysis of the literature highlights how complicated 

and inter-linked management of chronic disease really is. Their 

conclusion is not overstated. 

In the past decade, several breakthrough collaboratives introducing 

quality improvement methodology, rather than RCT’s have been 

implemented in the US, mainly focusing on system improvements to 

address some of the key questions cited by the authors and addressing 

the six aims outlined by the Institute of medicine. This review did not 

analyze them. Only 18 studies qualified for analysis by the methods 

for SMA’s being a newer model of care. 

As several components are inter-linked that leads to improvement 

in this model, it was hard to show significance with the rigor used in 

RCT’s, the gold standard, as seen in this review. So in terms of future 

research, I am not convinced that large scale RCT’s to address for e.g. 

diabetes outcomes will be feasible and answer the question described 

in (Table ES-2: evidence gaps and future research). One thought is to 

look at different models of care scientifically, to identify best practices 

and health systems with improved outcomes along with the economic 

cost for chronic illness management.

In conclusion, this paper has many strengths. (As cited by the 

authors). It is well described, clear and thoughtful with an exhaustive 

review and analysis of the literature. Going forward, it is an important 

topic for discussion in primary care and I greatly appreciate the 

opportunity to review this work. 

Thank you.

We followed the Cochrane 

Effective Practice and 

Organization of Care guidance, and included comparative patient or cluster 

RCTs, nonrandomized 

cluster controlled trials, 

controlled before-and-after 

studies, and interrupted 

time series designs. Any 

published breakthrough 

collaborative studies meeting these design specifications would have been 

included.

As stated above, we 

included comparative 

nonrandomized designs. 

We have modified the 

future research table 

to include these study 

design options.

Thank you.

2 On page 8 of the document (numbered as page 4), I noted in this 

paragraph, there needs to be a change in this word (see below in 

red). Put page/paragraph here

All three studies showed fewer hospital admissions in ......

In addition, on page 13 under the table where there are words 

describing what is meant by “provider”, would you consider changing 

advance nurse practitioner to this Advanced Practice Registered 

Nurse (APRN) as this is the more correct term to describe the nurse 

provider. Thank you.

We have made this 

change.

The typo “shower” was 

changed to “showed.”

We made the suggested 

change.

3 No comment from reviewer 5. Acknowledged
50

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

4 The scope of this data synthesis severely limited by limited number of 

high quality studies. It would seem that sufficient studies of sufficient 

design were not available to address KQ2 and KQ3. KQ1 of interest but 

only addresses initial questions of SMA effectiveness, and then only in 

diabetes. Some Tables not self-explanatory, such as Table ES-1 which 

refers to consistency, directness and precision but the definitions and 

measurement not clearly described. 

More analysis of issues raised in page 9 paragraph 2 would be helpful. 

Most of the SMA intervention studies suggest deployment of significant 

employee resources, sometimes on a limited number of patients in the 

group visit. These additional resources may have been largely responsible 

for the small improvement in intermediate outcomes seen in diabetes 

SMA. Left unresolved is whether it is worthwhile for facilities to invest 

these resources without clear return on investment. And for which 

patients? 

We reviewed all Tables 

and edited or footnoted so 

that tables will “stand on 

their own.”

We agree that this is 

a critical issue and 

attempted to determine 

factors associated with 

effect. Given the limits on 

intervention reporting and 

relatively small number of 

studies, our analysis did 

not identify the critical 

factors. For resource use, 

we were limited to the 

small amount of data 

reported.

5 p.9 2nd paragraph additional reasons for improved outcomes of SMA to 

consider-interprofessional synergism and motivational interviewing that 

goes on in the discussion section of SMA’s.

p.33 Table 6 Implementation issues-not clear that MD was most prevalent 

prescribing clinician if pharmacist & nurse practitioners listed with MD 

or without an MD were mentioned 11 studies and MD prescribers only as 

8 times -may want to say have a prescribing clinician present.

Thank you. These are 

good points that have 

been added to the 

introduction.

We have attempted to 

clarify by describing 

the clinical leads and 

team composition in 

more detail (see KQ 3 

results) and modifying the 

implementation table to 

reflect that a prescribing 

clinical is needed, rather 

than an MD.
51

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

6 RE: EXECUTIVE SUMMARY

I note that the executive summary is readable in length and well 

formatted. The writing is clear and crisp, which will be helpful to the 

eventual consumer. 

However, I am disappointed in several aspects of the presentation and 

data selected for presentations:

The Executive Summary emphasizes quantitative over qualitative 

commentary. There is very little discussion of the specific aspects of 

the interventions that are studied; if the reader is not familiar with 

what goes on in an SMA, then they won’t know after reading this, 

either. Similarly, one has very little information about what the usual 

care control treatments are, or how one qualifies to get into the study. 

There is no exploration of the mechanism of action.

Although I recognize that the summary must be brief, it is hard to 

imagine that all the tables pointing out that it is not possible to draw 

definitive conclusions based on a couple of studies of non-diabetic 

high users, or draw conclusions about economic effects when most 

studies did not report them. 

This is especially disappointing since the readers are not naïve – 

they are likely to be aware of the literature suggesting a benefit 

from SMA based on prominent studies, several in VA, that have had 

this finding. What the reader needs is guidance regarding what one 

should or should not do if one is attempting to be evidence based. I 

think that requires description of commonalities in the study designs, 

even if you can’t comment on whether one aspect or another was 

demonstrably better.

Since the introduction suggests a couple mechanisms of action, 

perhaps one could say whether the results say anything about these. 

In the case of improved access, for example, I guess the answer is 

that this is NOT the mechanism, since the control people would have 

also benefited from the provider having more clinic slots. Similarly, 

if the mechanism were expertise, I would like to see commentary on 

the expertise of the providers who participated in the SMA – my sense 

from the articles I have reviewed is that they are not content area 

experts. 

Thank you.

Thank you. We added 

a description of the 

common intervention 

components to the 

executive summary and 

the KQ 3 results. We have 

also added a section on 

the comparison condition 

in the initial description of 

the studies.

Results are briefly 

summarized in text. The 

table also summarizes 

results but adds the 

strength of evidence 

(SOE). Unfortunately, the 

SOE was insufficient for 

many outcomes

Unfortunately, the 

literature does not 

yet establish the 

characteristics of SMA 

associated with benefit. 

However, in the KQ 3 

results, we describe the 

common features and 

echo these findings in the 

discussion.

Thank you. This is 

a good point, but 

few studies reported 

intermediate outcomes 

(e.g., self-management 

behaviors) or provider 

training. We report the 

available results and have 

added to the discussion 

the point that few studies 

report on the potential 

mechanisms of action.
52

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

6 cont. RE: METHODS

I think that these are well described. But I wish that there was more 

of a sense that clinical experts were doing the synthesis and they 

were thinking about how they made sense. The AHRQ methodology 

does not seem to exclude this. Moreover, I acknowledge it seems like 

it is not very rigorous and might be open to bias on the part of the 

reviewers. So I think that this is why you have people like me review 

the opinion and offer a counterpoint. And I am confident that the 

Durham VA and Duke has a lot of people who could offer opinions 

and give the consumer confidence that the recommendations are in the 

end based on opinion, but very well informed and vetted opinion. 

Thus, I would like to see an attempt to present the rationale for why they 

think they won’t change their mind (i.e., strength of evidence “High”) 

or that they are not yet convinced. I would not want this to be devoid 

of quantitative thinking, rather, I would like an exposition of why they 

think what they think – this can be that it just does not make sense 

or that it is very consistent with lots of less strong studies or that the 

quantitative analysis is particularly convincing or is subject to error due 

to some methodological consideration, despite a nominally significant 

p value or important effect size (e.g., the condemnation of IMG carotid 

endarterectomy complication rates based on a trivial number of cases – 

Ann Intern Med 1990;113:747-753). 

RE: RESULTS:

I am surprised that there is no description of the eligibility criterion. 

Obviously, the DM trials required the patients to have DM, but there is 

no information about whether this was to be poorly controlled or of a 

certain duration, or if the patients had to have a stable medical regimen 

or be taking or not taking insulin or ?? 

Outcomes are well reported in tabular form. Although there are no 

statistically effects by study or patient characteristics, I would have 

liked some exploration of individual examples with greater or smaller 

effect size, and some assessment of why one had more effect than 

another.

The research team 

included physicians (one 

who is expert in shared 

medical appointments) 

and psychologists. Our 

goal was to summarize 

the evidence so that 

policy makers could 

incorporate the best 

available evidence into 

decision making.

The approach to 

assigning SOE is 

summarized in the 

methods section. The 

summary table in the 

discussion presents our 

judgments about each 

domain (study design, 

risk of bias, consistency, 

directness, precision) 

that forms the foundation 

for these judgments.

Thank you. We have 

added descriptions of the 

eligibility criteria to the 

results (see KQ 1).

Effects were consistent 

for blood pressure 

outcomes but varied 

for glucose control 

and HRQOL. We 

explored three factors 

hypothesized a priori 

to be related to effect 

size: baseline severity, 

intervention robustness, 

and study quality. 
53

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

6 cont. It is presumably because the study authors did not include the 

information, but one wonders if there are any process measures like 

medication changes or behavior changes that could explain the change 

in control. For example, with all the BP improvement, did this reflect a 

drop in weight, a change in prescribed medication, a change in medication 

adherence, a change in physical activity or what ? Or do we just not 

know? It seems most likely that this reflects a medication effect, since 

the effect on A1c, LDL and BP are all sensitive to this, and there is no 

evidence of a change in weight – I assume this was examined and will be 

reported in the revised version.

RE: DISCUSSION: 

I love that there is an explicit paragraph labeled “Should SMAs be 

Implemented?” But I am disappointed that it does not appear that they 

answer this question. Since the VA in particular paid for this synthesis, 

one would think there would at least be an answer for VA. It might be 

something along the lines of “The evidence available suggests that the 

VA should be implementing an SMA in all hospitals. This should be made 

available for all patients who have inadequate control. However, one 

should require referral by the primary care provider.” 

One would obviously provide caveats and nuance the presentation, but it 

is unfortunate that when the explicit goal is to help policy makers change 

policy, the guidance is vague. 

RE: APPENDIX C – DATA ELEMENTS ABSTRACTED

Hard to imagine that weight was not abstracted from at least some of the 

studies, given that all the disease specific studies were about diabetes. 

“Health professionals who conducted the educational session” is not 

reported with much specificity – that is, we see only MD as the descriptor. 

Is there evidence of specialists versus generalists? The same question 

applies to the pharmacists. Or are they just people who are interested in 

the area or researcher team members. 

RE: APPENDIX G – Study characteristics of included studies

The column labeled “Target condition HBA1c % (for total population)” 

appears to be mislabeled. The number in parentheses seems to be the standard 

deviation, not the value of A1c for the total population. Or the percent of the 

total population included – I am not sure what they mean by this. 

Usual care could have been described in more detail. Particularly in 

situations where the DM control was poor at baseline, one would imagine 

that the correct comparator would be some other, non-group approach to 

improved control – probably as simple as having the prescribing provider 

see the person for better control. 

Since the introduction suggests that access to an expert provider might 

be the mechanism of action, I wonder if there is information about the 

training of the MD or pharmacist.

The authors of the 

individual studies did 

not provide this data 

consistently. A few 

studies report effects 

on self-management 

behaviors and these results 

are reported in KQ 1 

“treatment experience and 

adherence outcomes.” We 

also abstracted information 

on medication changes and 

report these results in the 

same section

Our goal is to synthesize 

the evidence to inform 

policymaking. Although 

we try to describe some 

of the considerations (in 

addition to evidence of 

intervention effect) that 

might influence policy, 

it is not our role to 

prescribe policy. 

Weight was not specified 

as an outcome of interest 

by our study team or 

stakeholders. 

Specialty, training and 

experience of the MD 

professionals were almost 

never described.

Thank you. We relabeled 

the column to improve 

clarity. 

We have added a 

paragraph describing the 

Comparison Conditions in 

more detail in the “Study 

Characteristics” section of 

the results.

Unfortunately, information 

about the specialty training 

and experience of the 

MD clinician or clinical 

pharmacist was give rarely.
54

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

7 Other elements to assess in explaining heterogeneity of study results in 

diabetes is the clinician expertise in diabetes and group management, e.g. 

do the clinician(s) have training in diabetes or conduction of group visits 

prior to starting the trial – such as being a certified diabetes educator, do 

the clinician(s) manage diabetes in other settings – such as individual 

diabetes clinics, how many disciplines were in the team, etc.

Although this is a 

good idea, and could 

explain heterogeneity 

in intervention effects, 

clinician expertise and 

certification as a diabetes 

educator were not 

described consistently 

enough for analysis.

Optional Dissemination and Implementation Questions

Question 5: Are there any VA clinical performance measures, programs, quality improvement measures, 

patient care services, or conferences that will be directly affected by this report? If so, please provide detail.

1 I do not work in the VA setting, so am unable to comment on this. Acknowledged

2 The Office of Nursing Service (ONS) conducts courses on EBP; use 

of this document may enhance learning for the participants who would 

attend this conference; given that the EBP course is for the properly 

targeted audience. 

The other group that could benefit is the Clinical Practice Portfolio of the 

ONS as this ES is timely to impact recommendations for intervention for 

both PACT and Specialty Care services. In addition, a presentation on the 

findings would be beneficial for the Advanced Practice Nursing Advisory 

Group as this group of providers would be influenced by these findings.

Acknowledged 

Acknowledged

3 PACT compass, Office of Patient Care Service, Office of Nursing Service, 

Office of Academic Affiliations: Centers of Excellence in Primary Care 

Education

Acknowledged

4 PACT implementation included an emphasis on SMA and group visits as 

a means to enhance access and implement a chronic care model. 

The ability to improve access is not addressed by this data synthesis. A 

SMA of reasonable size might improve access by reducing reliance on 

routine clinic visits. This hypothesis was not evaluated. A concern from 

the perspective of PACT is that the SMA might adversely affect patient 

continuity with their primary care provider. The information regarding 

whether the intervention preserved patient continuity with the provider 

or not is not provided. An evaluation of the benefits of SMA versus the 

trade-offs might ultimately be very helpful.

Acknowledged 

Most studies describe the 

intervention as additional 

care, rather than a 

substitute for primary 

care. We have added a 

statement to this effect in 

the discussion. No studies 

reported access outcomes 

directly or information on 

continuity with their PCP. 

Few reported effects on 

patient experience.

5 PACT initiative in Primary Care is promoting SMA’s. Acknowledged

6 The authors are clear that there is not enough evidence that we can say 

that use or non-use of SMA is a quality issue. I think there are a number 

of conferences where the results will be of interest – certainly HSR and 

QUERI and hopefully the leadership meetings. 

The results of this 

review and a parallel 

“realist” review are being 

presented at the July 2012 

HSR&D conference.

7 PACT compass that includes SMA as one of its components, in addition 

to the traditional diabetes performance measures in Primary care

Acknowledged 
55

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Reviewer Comment Response

Question 6: Please provide any recommendations on how this report can be revised to more directly address or 

assist implementation needs.

1 No comments by reviewer 2

2 Report is very well done. Are companion presentations done in 

conjunction with the report (PowerPoint, to be used for presenting to 

interested groups)?

Thank you. There will be 

a presentation at the July 

2012 HSR&D/QUERI 

meeting.

3 Executive summary very direct and to the point. Recommendations for 

future research offer usable directions

Thank you.

4 More information about the intervention would have been helpful. Key 

elements that may result in successful implementation are hinted at in 

Table 6, but are not fully discussed. This table is also not self-explanatory: 

for example what is meant by “Team continuity”? 

We revised Table 6 and 

footnoted other tables 

to more improve clarity. 

Detailed information 

on the intervention is 

presented in Appendix H 

and summarized in Table 

3.

5 A table of the roles of each providers (along with quality) in the studies 

would help as sites try to implement SMA’s

Although team 

composition was 

described, roles of 

individual providers were 

not described consistently 

enough to develop the 

requested table.

6 See response to #4 Addressed above.

7 I believe this report is an unbiased assessment and synthesis of current 

literature and clearly points out the strengths and weaknesses of the 

available data

Thank you. 

Question 7: Please provide us with contact details of any additional individuals/stakeholders who should be 

made aware of this report.

1 No. No comment Acknowledged 

2 Once completed: Christine Engstrom, Anna Alt-White Acknowledged 

3 PACT e-mail group, Centers of Excellence in Primary Care Education Acknowledged 

4 Richard Stark; Joanne Shear Acknowledged 

5 No comment Acknowledged 

6 None in particular Acknowledged 

7 No comment Acknowledged 
56

for Chronic Medical Conditions Evidence-based Synthesis Program

OING CLINICAL TRIALS

als

Organization Intervention Comparator Sponsor and ClinicalTrials.gov ID Funding Start/Stop Status

overnment Shared medical 

appointments to promote establishing collaborative teams (ReSPECT)

Traditional diabetes education and teleconsultation

Department of Veterans AffairsNCT00854594

Sep 2010–Sep 2012 Recruiting

niversity

filiated clinic Group Visit Standard individual medical appointment Oregon Health and Science University

NCT01497301

Feb 2012–Feb 2013 Not yet open for 

participation

niversity

filiated clinic Heart Failure Group Clinic Appointments Standard heart failure education Carol Smith, RN, PhD, FAAN (NHLBI)

NCT00439842

Mar 2007–Sep 2012 Ongoing, but not 

recruiting 

overnment Pharmacist-led group 

medical visits for patients with type 2 diabetes mellitus

Usual care Department of 

Veterans AffairsNCT00554671

May 2008–June 2012 Ongoing, but 

not recruiting participants

NHLBI=National Heart, Lung, and Blood Institute
57

for Chronic Medical Conditions Evidence-based Synthesis Program

 STUDY CHARACTERISTICS

eristics

ion

ng

zation

 N

Age in Years (SD)

Sex (%)

Race/ethnicity (%)

Target Condition

Mean Baseline HBA1c % (SD)

(for Diabetes Studies)

SMA Planned Visits

Study Duration Comparator Quality

73.5 (NR)Male (34%)NR

Chronically ill older adultsNot applicable

12>12 months

Usual care Poor

ted clinic

59.4 (14.3)Male (44%)NR

Diabetes; hypertensionHBA1c: 8.2 (2.4)

412 months

Usual care Fair

ted clinic

54.0 (NR)Male (22%)Black (77.5%) 

Type 2 diabetes HBA1c: 10.4 (NR)

66 months

Usual care Good

ted clinic

56.0 (NR)Male (28%)Black (83.3%)

Type 2 diabetes HBA1c: 9.1 (2.0)

1212 months

Usual care Fair

m

69.8 (10.7)Male (100%)NR

Type 2 diabetesHbA1c: 7.8 (1.0)

4 once weekly + 5 monthly booster session6 months

Usual care Fair

ed health 

56.8 (NR)Male (36%)NR

Type 2 diabetes HBA1c: 9.4 (NR)

7 visits28 months

Usual care Poor

m

62.0 (9.7)Male (96%)Black (59.0%)

Diabetes; hypertensionHBA1c: 9.2 (1.4)

7 visits12 months

Usual care Good

ted clinic

NR (NR)Male (NR)Hispanic (100%) 

Type 2 diabetes HBA1c: NR (NR)

36 visits offered17 months

Usual care Poor
58

for Chronic Medical Conditions Evidence-based Synthesis Program

ion

ng

zation

 N

Age in Years (SD)

Sex (%)

Race/ethnicity (%)

Target Condition

Mean Baseline HBA1c % (SD)

(for Diabetes Studies)

SMA Planned Visits

Study Duration Comparator Quality

m

61.0 (9.9)Male (NR)NR

Type 2 diabetes HBA1c: 10.1 (NR)

NA (drop-in)4 months )

Usual care Fair

78.2 (7.2)Male (35%)NR

High usage of clinic servicesNot applicable

12 visits12 months

Usual care Fair

m

63.6 (7.9)Male (NR)Black (31.0%)

Type 2 diabetes HBA1c: 8.8 (1.3)

4 visits3 months intervention; 12 months followup

Enhanced usual care (2 required diabetes group education sessions)

Good

56.0 (9.1)Male (57%)White (74.6%)

Types 1 and 2 diabetes HBA1c: 9.7 (1.7)

6 visits6 months

Usual care Good

74.1 (7.5)Male (41%)NR

Older; high usage of clinic servicesNot applicable

24 visits24 months

Usual care Fair

m

64.4 (10.3)Male (95%)White (91.0%)

Type 2 diabetes HBA1c: 8.0 (1.3)

4 visits1 month (outcomes reported at 4 months)

Usual care Fair

m

60.8 (9.6)Male (98%White (99%)

Types 1 and 2 diabetesHBA1c: 8.4 (1.8)

9 visits6 months

Usual care Good

ted clinic

61.5 (NR)Male (54%)NR

Type 2 diabetes HBA1c: 7.4 (1.4)

7-8 visits24 months

Usual care plus individual education sessions

Fair

ted clinic

Median 27-31 (NR)Male (60%)NR

Type 1 diabetesHBA1c: 8.7 (1.2)

15 visits36 months

Usual care plus individual education sessions

Fair
59

for Chronic Medical Conditions Evidence-based Synthesis Program

ion

ng

zation

 N

Age in Years (SD)

Sex (%)

Race/ethnicity (%)

Target Condition

Mean Baseline HBA1c % (SD)

(for Diabetes Studies)

SMA Planned Visits

Study Duration Comparator Quality

ted clinic

69.3 (8.4)Male (51%)NR

Type 2 diabetes HBA1c: 7.8 (1.6)

14 visits48 months

Usual care; followup scheduled every 3 months

Good

60.7(NR)Male (53%)White (72.8%)

Types 1 and 2 diabetesHBA1c: 7.5 (NR)

8 visits24 months

Usual care Fair
60

for Chronic Medical Conditions Evidence-based Synthesis Program

 INTERVENTION COMPONENTS

 team and process components

Clinical Team Group Group Visit Processes

disciplines

Team 

continuity

Team size

Closed?

Group size

Family or peers 

allowed?

Individual 

breakouts?

Medication 

changes?

Visit 

duration

(minutes)

Telephone 

contacts?

psychologists (as 

s)

Specific group 

but rotated

≥2

Yes

8a

Yes Yes Yes 120 Yes

or others (type 

Yes

2

Yes

3-12

NR Yes Yes 120 NR

ctitioner, and 

ers

Yes

2-3

Yes

19-20

NR Yes Yes 120 No

Yes

2

Yes

6-7

No Yes Yes 120 No

N, physical 

dietitian

Yes

4

Yes

4-6

Yes NR Yes 60 No

ialist, MD, RN, 

rker

Yes

7

Yes

10-16

Yes NR Yes 210 NR

or, MD, 

 RN

Yes

3

Yes

7-8

No Yes Yes 90-120 Yes

st, RN, and 

NR

7

No

NR

NR NR NRb NR NR

ctitioner, 

sychologist, and/

Yes

5

No

≤8

NR Yes Yes Varied

>60 

NR

 practitioner Yes

3

Yes

25

NR Yes Yes 90 NR/unclear

Yes

≥2

Yes

5-7

No Yes Yes 120 No
61

for Chronic Medical Conditions Evidence-based Synthesis Program

Clinical Team Group Group Visit Processes

disciplines

Team 

continuity

Team size

Closed?

Group size

Family or peers 

allowed?

Individual 

breakouts?

Medication 

changes?

Visit 

duration

(minutes)

Telephone 

contacts?

etician, 

nd RN

Yes

4

Yes

10-18

No Yes Yes 120 Yes

st, RN, physical 

occupational 

Yes

≥2

No

7.7a

Yes Yes Yes 120 No

rmacist, physical 

RN

Yes

Unclear

Yes

4-8

Yes NR Yes 120 No

d RN Yes

Unclear

No

4-6

Yes No Yes 120 No

or and MD Partialc

2-3

Yes

9-10

Yes Yes Yes 50-80 No

opedagogist Partialc

2

Yes

6-7

Yes Yes Yes 40-80 No

or and MD Partialc

2

Yes

9-10

Yes Yes Yes 60 No

st, and RN Yes

3

Yes

6-10

NR Yes NR 60 No

e rather than a range is reported in the article.

ot clearly report whether medication changes were made as part of the group process; however, it is implied in that an MD and a pharmacist were usually 

 lowered their HbA1c and started taking more aspirin than the control group.

 on a pool of health providers for group intervention, which may provide patients with the possibility to see the same provider more than once—hence, team 
62

for Chronic Medical Conditions Evidence-based Synthesis Program

 educational and behavioral components

(s) of Educational Session Behavioral Approach Patients Input on Topics? Topics Behavioral Strategies Print Material?

macist, RN, or other 

mber 

NR No Medication management, nutrition, 

physical activity

NR Yes, generic

er team member NR NR Disease-specific education, 

medication management, nutrition 

Goal-setting, personalized plan 

NR

r guest lecturers NR Yes Disease-specific education, 

medication management, nutrition, physical activity 

NR NR

NR Yes Disease-specific education, 

medication management, nutrition, physical activity 

NR NR

st NR No Disease specific education, 

medication management, nutrition, physical activity

Goal-setting, homework assignments, personalized care plan, self-monitoring

Yes

NR No Disease-specific education, 

medication management, nutrition, physical activity 

Goal-setting, problem-solving skills

NR

ucator or RN NR Yes Disease-specific education, 

medication management, nutrition, physical activity 

Personalized care plan

Yes, generic

ker NR NR NR NR NR

ychologist NR NR Disease-specific education 

nutrition, smoking cessation 

Personalized plan Yes, generic

 NR Yes Medication management, nutrition, 

physical activity 

NR Yes, generic

ician NR No Disease-specific education, 

medication management 

Goal-setting, personalized plan, problem-solving skills, self- monitoring

Yes, tailored 

health behavior  pharmacist, 

or RN

NR Yes Disease-specific education, 

physical activity 

Personalized plan NR

MD, pharmacist, 

herapist, or RN

NR Yes Disease-specific education, 

medication management, nutrition, physical activity 

NR Yes tailored 

pharmacist, physical 

or RN 

Stages of change No Disease-specific education, 

medication management, nutrition, physical activity, smoking cessation 

NR Yes, tailored
63

for Chronic Medical Conditions Evidence-based Synthesis Program

(s) of Educational Session Behavioral Approach Patients Input on Topics? Topics Behavioral Strategies Print Material?

t, pharmacist, or RN Stages of change No Disease-specific education, 

medication management, nutrition, physical activity, smoking cessation 

NR Yes, tailored

lth educator Patient-centered 

adult learning

No Disease-specific education, 

medication management, nutrition, physical activity, smoking cessation 

Homework assignment, problem-solving skills, self-monitoring 

Yes, tailored

lth educator Patient-centered 

adult learning

Yes Disease-specific education, 

medication management, nutrition, physical activity, self-care

Homework assignment, problem-solving skills, self-monitoring 

Yes, tailored 

lth educator Patient-centered 

adult learning

No Disease-specific education, 

medication management, nutrition, physical activity, smoking cessation

Homework assignment, problem-solving skills, self-monitoring 

Yes, tailored 

er health professional NR No Disease-specific education Self-monitoring Yes, generic
64

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

APPENDIX I. GLOSSARY

Abstract screening 

The stage in a systematic review during which titles and abstracts of articles identified in the 

literature search are screened for inclusion or exclusion based on established criteria. Articles 

that pass the abstract screening stage are promoted to the full-text review stage.

Allocation concealment

The method by which randomization assignment is concealed from participants and investigators 

before and during the enrollment process. Common processes are central allocation (telephone 

or web-based, pharmacy or off-site statistician controlled randomization sequence generation 

and sequentially numbered, opaque, sealed envelopes. Allocation concealment concentrates on 

preventing selection and confounding biases, safeguards the assignment sequence before and 

until allocation, and can always be successfully implemented

Area under the curve (AUC)

The area under the receiver operating characteristic (ROC) curve. The summary receiver operator 

characteristic (SROC) curve and the AUC have been proposed as a way to assess diagnostic data 

in the context of a meta-analysis. The accuracy of a diagnostic test depends on how well the test 

separates the group being tested into those with and without the condition in question. 

Case-control study

A retrospective, analytical, observational study often based on secondary data in which the proportion 

of cases with a potential risk factor are compared to the proportion of controls (individuals without the 

disease or condition) with the same risk factor. The common association measure for a case-control 

study is the odds ratio. These studies are commonly used for initial, inexpensive evaluation of risk 

factors and are particularly useful for rare conditions or for risk factors with long induction periods. 

Unfortunately, due to the potential for many forms of bias in this study type, case control studies 

provide relatively weak empirical evidence even when properly executed. 

Case report 

A description of a single case, typically describing the manifestations, clinical course, and 

prognosis of that case. Due to the wide range of natural biologic variability in these aspects, a 

single case report provides little empirical evidence to the clinician. A case report does describe 

how others diagnosed and treated the condition and what the clinical outcome was.

Case series 

A descriptive, observational study of a series of cases, typically describing the manifestations, 

clinical course, and prognosis of a condition. A case series provides weak empirical evidence 

because of the lack of comparability unless the findings are dramatically different from 

expectations. Case series are best used as a source of hypotheses for investigation by stronger study 

designs, leading some to suggest that the case series should be regarded as clinicians talking to 

researchers. Unfortunately, the case series is the most common study type in the clinical literature. 
65

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

ClinicalTrials.gov

A registry and results database of federally and privately supported clinical trials conducted in 

the United States and around the world. ClinicalTrials.gov provides information about a trial’s 

purpose, location, participant characteristics, among other details. 

Cochrane Database of Systematic Reviews

A bibliographic database of peer-reviewed systematic reviews and protocols prepared by the 

Cochrane Review Groups in The Cochrane Collaboration. 

Cochran’s Q test

A nonparametric statistic to test for differences in intervention effects between studies. Because 

the test statistic is often underpowered, the threshold for statistically significant differences in 

intervention effects is often set at p<0.10. 

Cohort study

A prospective, analytical, observational study based on data, usually primary, from a followup 

period of a group in which some have had, have, or will have the exposure of interest, to determine 

the association between that exposure and an outcome. Cohort studies are susceptible to bias by 

differential loss to followup, the lack of control over risk assignment, and the potential for zero time 

bias when the cohort is assembled. Because of their prospective nature, cohort studies are stronger 

than case-control studies when well executed, but they also are more expensive. Because of their 

observational nature, cohort studies do not provide empirical evidence that is as strong as that 

provided by properly executed randomized controlled clinical trials. 

Companion article

A publication from a trial that is not the article containing the main results of that trial. It may be 

a methods paper, a report of subgroup analyses, a report of combined analyses, or other auxiliary 

topic that adds information to the interpretation of the main publication.

Confidence interval (CI)

The range in which a particular result (such as a laboratory test) is likely to occur for everyone 

in the population of interest a specified percentage of the time known as the confidence level or 

confidence coefficient. It is an interval calculated from a study’s observations used to estimate 

the reliability of the estimate of a parameter. The most common confidence level is 95%. For 

example, a confidence interval with a 95% confidence level is intended to give the assurance that, 

if the statistical model is correct, then taken over all the data that might have been obtained, the 

true value of the parameter will be found within the given interval 95% of the time. 

Consistency

Extent to which effect size and direction vary within and across studies; inconsistency may be 

due to heterogeneity across PICOTS.
66

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Cumulative Index to Nursing and Allied Health Literature (CINAHL)

A collection of medical databases of nursing and allied health literature.

Data abstraction

The stage of a systematic review that involves a pair of trained researchers extracting reported 

findings specific to the research questions from the full-text articles that met the established 

inclusion criteria. These data form the basis of the evidence synthesis. 

Directness

Degree to which outcomes that are important to users of the comparative effectiveness review 

(patients, clinicians, or policymakers) are encompassed by trial data.

Embase

A database containing bibliographic records with citations, abstracts, and indexing derived from 

biomedical and pharmacological articles in peer-reviewed journals.

Exclusion criteria

The criteria, or standards, set out before a study or review. Exclusion criteria are used to 

determine whether a person should participate in a research study or whether an individual 

study should be excluded in a systematic review. Exclusion criteria may include age, previous 

treatments, and other medical conditions. 

External validity

The extent to which clinical research studies apply to broader populations. A research study has 

external validity if its results can be generalized to the larger population.

Forest plot

A visual display of information from individual studies in a meta-analysis. A forest plot shows 

the amount of variation between the results of the studies as well as an estimate of the overall 

result of all the studies together. A horizontal line represents the 95% confidence interval (CI) of 

the “effect” observed in the studies. 

Full-text review

The stage of a systematic review in which a pair of trained researches evaluates the full-text of 

study articles for potential inclusion in the review.

GRADE

Grading of Recommendations Assessment, Development and Evaluation (GRADE), a systematic 

approach to evaluating the overall body of research evidence and rating the quality of medical 

evidence and the strength of clinical recommendations. 
67

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Health-related quality of life (HRQOL)

Aspects of overall quality of life that can be clearly shown to affect health—either physical or 

mental health.

I2

A statistic that describes the percentage (range from 0-100%) of total variation across studies 

due to heterogeneity between study characteristics rather than due to chance. Heterogeneity 

is categorized as low, moderate or high based on I2 values of 25, 50 or 75%, respectively. It is 

considered an indication of consistency or inconsistency across studies in a meta-analysis.

Inclusion criteria

The criteria, or standards, set out before the systematic review. Inclusion criteria are used to 

determine whether an individual study can be included in a systematic review. Inclusion criteria 

may include population, study design, gender, age, type of disease being treated, previous 

treatments, and other medical conditions. 

Intent-to-treat analysis

A method of analyzing results of a randomized controlled trial that includes in the analysis 

all cases that should have received a treatment regimen but for some reason did not. All cases 

allocated to each arm of the trial are analyzed together as representing that treatment arm, 

regardless of whether they received or completed the prescribed regimen. 

Interquartile range (IQR)

A measure of the spread of or dispersion within a data set. The IQR is the width of an interval 

that contains the middle 50 percent of the sample, so it is smaller than the range and its value is 

less affected by outliers.

Meta-analysis

A way of combining data from many different research studies. A meta-analysis is a statistical 

process that combines the findings from individual studies.

Meta-regression analyses

An extension of meta-analysis to subgroups that allows the effect of continuous, as well 

as categorical, characteristics to be investigated if sufficient studies examining the same 

characteristics may be compared. In principle, it allows the effect of multiple factors to be 

investigated simultaneously. In meta-regression, the outcome variable is the effect estimate 

(e.g., a mean difference, etc.). The explanatory variables are characteristics of studies that might 

influence the size of the intervention effect.

Mixed effects

Statistical models that include both fixed (nonrandom) and random effects.
68

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

National Committee for Quality Assurance (NCQA) 

A nonprofit organization dedicated to improving health care quality. 

National Quality Forum (NQF)

A nonprofit organization that promotes change through development and implementation of a 

national strategy for health care quality measurement and reporting.

Negative predictive value (NPV)

The likelihood that people with a negative test result would not have a condition. The higher the 

value of the negative predictive value (for example, 99 percent would usually be considered a 

high value), the more useful the test is for predicting that people do not have the condition.

Nonrandomized study

Any quantitative study estimating the effectiveness of an intervention (harm or benefit) that 

does not use randomization to allocate units to comparison groups (including studies where 

“allocation” occurs in the course of usual treatment decisions or peoples’ choices; i.e., studies 

usually called “observational”). There are many possible types of nonrandomized intervention 

studies, including cohort studies, case-control studies, controlled before-and-after studies, 

interrupted-time-series studies, and controlled trials that do not use appropriate randomization 

strategies (sometimes called quasi-randomized studies).

Observational study

A study in which the investigators do not seek to intervene but simply observe the course of 

events. Changes or differences in one characteristic (e.g., whether or not people received the 

intervention of interest) are studied in relation to changes or differences in other characteristics 

(e.g., whether or not they died), without action by the investigator. Observational studies provide 

weaker empirical evidence than do experimental studies because of the potential for large 

confounding biases to be present when there is an unknown association between a factor and an 

outcome. 

Odds ratio

A ratio of the odds of having the outcome of interest in a group with a particular exposure, 

symptom, or characteristic of interest, to the odds of outcome in a group that does not have the 

exposure/symptom/characteristic. An odds ratio of 1 indicates that the outcome is equally likely 

to occur in both groups. An odds ratio of 4 indicates that the outcome is 4 times more likely to 

be present in the group that has the symptom or characteristic of interest, compared with the 

group that does not have this symptom. When outcomes are infrequent, the odds ratio is a good 

approximation of the risk ratio.

Outlier

An observation in a data set that is far removed in value from the others in the data set. It is an 

unusually large or an unusually small value compared to the others. 
69

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Patient-centered adult learning

An approach used in the professional–patient interaction. Common elements are empathic 

communication, acknowledgement, realistic expectations, goal negotiation, guided problemsolving, individualized strategies, and ongoing support.

PICOTS

Population, intervention, comparator, outcome, timing, setting. 

Positive predictive value (PPV)

Indicates the likelihood that a person with a positive test result would actually have the condition 

for which the test is used. The higher the value of the positive predictive value (for example, 

90 percent would be considered a high value), the more useful the test is for predicting that the 

person has the condition. 

Precision

The degree of certainty for estimate of effect with respect to a specific outcome.

Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA)

An evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

Probability

The likelihood (or chance) that an event will occur. In a clinical research study, it is the number 

of times a condition or event occurs in a study group divided by the number of people being 

studied.

Process of care or performance measure

Quality measures used to gauge how well an entity provides care to its patients. Measures are 

based on scientific evidence and usually reflected in guidelines, standards of care or practice 

parameters.

Prospective observational study

A clinical research study in which people who presently have a certain condition or receive a 

particular treatment are followed over time and compared with another group of people who are 

not affected by the condition.

PsycINFO

An abstracting and indexing database of peer-reviewed literature in the behavioral sciences and 

mental health.

Publication bias

The tendency of researchers to publish experimental findings that have a positive result, 

while not publishing the findings when the results are negative or inconclusive. The effect of 
70

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

publication bias is that published studies may be misleading. When information that differs 

from that of the published study is not known, people are able to draw conclusions using only 

information from the published studies.

PubMed

A database of citations for biomedical literature from MEDLINE, life science journals, and 

online books in the fields of medicine, nursing, dentistry, veterinary medicine, the health care 

system, and preclinical sciences.

QUADAS

Quality Assessment of the Diagnostic Accuracy Studies, a tool that uses a standard methodology 

to judge the quality of individual studies in a systematic review.

Quasi-experimental study 

 A quasi-experimental study manipulates a variable between two or more groups, but participants 

are not randomly assigned to groups. Quasi-experimental study designs, such as nonrandomized 

pre-post studies, are frequently used when it is not logistically feasible or ethical to conduct a 

randomized controlled trial. 

Quasi-random allocation

Methods of allocating people to a trial that are not random but were intended to produce similar 

groups when used to allocate participants. Quasi-random methods include allocation by the 

person’s date of birth, by the day of the week or month of the year, by a person’s medical record 

number, or just allocating every alternate person. In practice, these methods of allocation are 

relatively easy to manipulate, introducing selection bias.

Randomized controlled trial

A prospective, analytical, experimental study using primary data generated in the clinical 

environment. Individuals similar at the beginning of the trial are randomly allocated to two or 

more treatment groups and the outcomes the groups are compared after sufficient followup time. 

Properly executed, the RCT is the strongest evidence of the clinical efficacy of preventive and 

therapeutic procedures in the clinical setting. 

Relative risk (RR)

A comparison of the risk of a particular event for different groups of people. Relative risk is 

usually used to estimate exposure to something that could affect health. In a clinical research 

study, the experimental group is exposed to a particular drug or treatment. The control group is 

not. The number of events in each group is compared to determine relative risk.

Reporting bias

A bias caused by only a subset of all the relevant data being available. The publication 

of research can depend on the nature and direction of the study results. Studies in which 

an intervention is not found to be effective are sometimes not published. Because of this, 
71

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

systematic reviews that fail to include unpublished studies may overestimate the true effect of 

an intervention. In addition, a published report might present a biased set of results (e.g., only 

outcomes or subgroups where a statistically significant difference was found).

Risk

A way of expressing the chance that something will happen. It is a measure of the association 

between exposure to something and what happens (the outcome). Risk is the same as probability, 

but it usually is used to describe the probability of an adverse event. It is the rate of events (such 

as breast cancer) in the total population of people who could have the event (such as women of a 

certain age). 

Robustness score

A score developed to indicate the number of intervention components hypothesized to be 

associated with greater treatment effects.

Sensitivity

The ability of a test to identify correctly people with a condition. A test with high sensitivity will 

nearly always be positive for people who have the condition (the test has a low rate of falsenegative results). Sensitivity is also known as the true-positive rate. 

Shared medical appointment (SMA)

A group visit where multiple patients are seen together for followup or routine care.

Spearman’s correlation

A rank correlation coefficient that is usually calculated on occasions when it is not convenient, 

economical, or even possible to give actual values to variables but only to assign a rank order to 

instances of each variable. It may also be a better indicator that a relationship exists between two 

variables when the relationship is nonlinear.

Specificity

The ability of a test to identify correctly people without a condition. A test with high specificity 

will rarely be wrong about who does not have the condition (the test has a low rate of falsepositive results). Specificity is also known as the true-negative rate.

Stages-of-change model

A common health behavioral model consisting of these components: precontemplation, 

contemplation, preparation, action, and maintenance.

Standard error

The standard deviation of the sampling distribution of a statistic. Measurements taken from a 

sample of the population will vary from sample to sample. The standard error is a measure of 

the variation in the sample statistic over all possible samples of the same size. The standard error 

decreases as the sample size increases.
72

Shared Medical Appointments for Chronic Medical Conditions Evidence-based Synthesis Program

Standardized mean difference (SMD)

The difference between two estimated means divided by an estimate of the standard deviation. 

It is used to combine results from studies using different ways of measuring the same concept, 

e.g. mental health. By expressing the effects as a standardized value, the results can be combined 

since they have no units. 

Statistical significance

A mathematical technique to measure whether the results of a study are likely to be true. 

Statistical significance is calculated as the probability that an effect observed in a research study 

is occurring because of chance. Statistical significance is usually expressed as a P-value. The 

smaller the P-value, the less likely it is that the results are due to chance (and more likely that 

the results are true). Researchers generally believe the results are probably true if the statistical 

significance is a P-value less than 0.05 (p<.05).

Strength of evidence (SOE)

A measure of how confident reviewers are about decisions that may be made based on a body 

of evidence. SOE is evaluated using one of four grades: (1) High confidence that the evidence 

reflects the true effect; further research is very unlikely to change reviewer confidence in the 

estimate of effect; (2) moderate confidence that the evidence reflects the true effect; further 

research may change the confidence in the estimate of effect and may change the estimate; (3) 

low confidence that the evidence reflects the true effect; further research is likely to change the 

confidence in the estimate of effect and is likely to change the estimate; and (4) insufficient; the 

evidence either is unavailable or does not permit a conclusion.

Summary receiver operating characteristic (SROC)

A data analysis approach that combines independent studies of diagnostic tests. The SROC curve 

and the area under the curve (AUC) have been proposed as a way to assess diagnostic data in the 

context of a meta-analysis.

Systematic review

A summary of the clinical literature. A systematic review is a critical assessment and evaluation 

of all research studies that address a particular clinical issue. The researchers use an organized 

method of locating, assembling, and evaluating a body of literature on a particular topic using a 

set of specific criteria. A systematic review typically includes a description of the findings of the 

collection of research studies. The systematic review may also include a quantitative pooling of 

data, called a meta-analysis.

Time-series study

A quasi-experimental research design in which periodic measurements are made on a defined 

group of individuals both before and after implementation of an intervention. Time series studies 

are often conducted for the purpose of determining the intervention or treatment effect.