Presenting Summary Information From Cochrane Systematic Reviews to Physicians: Protocol for Research of Infographics Presentation vs. Plain Language Summary
Phase: NA
Conditions: Health Knowledge, Attitudes, Practice
Sponsor: University Hospital of Split

Protocol: 
 
Presenting summary information from Cochrane systematic reviews: randomized 
controlled trial of infographics presentation vs. standard word summaries 
 
 
Authors (provisional by line): Ivan Buljan, Mario Malički, Liz Wager, Livia Puljak, 
Darko Hren, Frances Kellie, Helen West, Žarko Alfirević, Ana Marušić 
 
 
Trial registration 
 
NCT03002610 
 
Protocol version 
 
This version of protocol was written in November 2016. 
 
 
 
Unique protocol ID:  
 
IP-2014-12-7672 
 
 
 
   
Funding 
 
This research is funded by the Croatian Science Foundation in “Professionalism and 
Health”scientific project (Grant No. IP-2014-09-7672).Croatian Science Foundation had no 
role in the design of this study and will not have any role during its execution. 
 
 
Roles and responsibilities 
 
The idea for the study was provided by AM and ŽA. Research group from the 
University of Split (IB, AM, MM, LW, LP) drafted the study design. Research group from 
the University of Liverpool (ŽA, FK, HW) provided the infographics material and 
contributed to question development. All authors contributed to the refinement of the study 
protocol and approved the final manuscript. 
 
 
Introduction 
 
Background 
 
The presentation of research evidence to the lay audience has become an important 
aspect of the Cochrane. Results of studies on evidence presentation are not always easy to 
interpret (Mazur & Hickam, 1991; Mazur & Merz, 1994; Carrasco-Labra et al, 2015) and the 
efforts continue to develop more understandable ways of scientific data presentation to the 
lay public, and ultimately better knowledge translation. 
 
Key messages of individual Cochrane Systematic reviews are presented in two ways. One 
is a scientific summary (SciSumm), aimed at researchers and health practitioners and the 

other is a plain language summary (PLS), aimed at the lay public (Santesso et al, 2014). 
Recently, Cochrane started developing infographics, where short textual information about 
research is supported by visual representations of the main findings and is also aimed at the 
lay public (www.visuallycochrane.net.) 
 
A recent systematic review (Gagliardi et al, 2015) showed that there was no clear 
evidence for the best type of intervention for healthcare knowledge information translation, 
so further research about different interventions is needed. 
 
 
Objectives 
 
The objective of our study is to evaluate the efficacy of infographics in presenting 
information, in terms of understanding and remembering research results, compared to 
standard PLS formats and scientific summary format. 
 
The null hypothesis is that the proportion of the correct answers in all three groups is 
not going to differ significantly. The equation for that hypothesis is: 
 
 
H0: Infographic = PLS = SciSumm 
 
 
The alternative hypothesis is that participants in the infographic group (Infographics) 
will have significantly more correctly answered questions about the topic than those in PLS 
group (PLS), meaning that infographics enables a better understanding of scientific findings 
than PLS. Also, the scientific summary group will have the lowest knowledge score of the three 
groups, meaning than scientific summary is not the suitable way of data presentation about a 
Cochrane review for lay population. The equation for that hypothesis is: 
 
 
Ha: Infographic > PLS >SciSumm 
 
 
Trial design 
 
We will conduct a randomized control trial (RCT) with three different formats of the 
same systematic review summary (infographics, PLS and scientific summary). The content of 
these three formats is based on the same systematic review, but the ways of data presentation will 
differ: visual presentation in plain language, plain language, and scientific language. The trial 
will be performed at the beginning of the 2016/2017 academic year, with physicians as 
participants. The trial will be voluntary and anonymous. The survey will consist of 4 parts: 1) 
demographic data, 2) one format of the summary (randomly assigned), 3) comprehension test 

of the information given in the summary, 4) accessibility of information and overall 
satisfaction with the given summary assessed by survey and 5) health numeracy test. All 
materials will be in Croatian. The materials used in this research will be assessed by experts 
in order to confirm face validity of the survey. 
 
 
Methods: Participants, interventions, and outcomes 
 
 
Setting and recruitment 
 
The trial will be performed online using SurveyMonkey software (A/B 
testing method). 
 
Eligibility criteria 
 
Eligible participants will be medical doctors in any specialty or field of work. Authors 
will make a list of publicly available e-mail addresses of doctors–clinical researchers from 
the School of Medicine and University Hospital Center in Split to make an email list of 
potential participants 
 
 
Interventions 
 
Infographics format of a Cochrane systematic review summary represents the 
experimental intervention in the trial. The control groups will receive either PLS (which 
represent text format with simple explanation of the survey topic main findings and is intended 
for lay audience) or the scientific summary (the text written for the academic population and 
practitioners) groups. We will use formats already available for the systematic review on external 
cephalic version for breech presentation before term (Hutton et al, 2015.). 
 
All three types of Cochrane summaries will be translated into Croatian by the trial 
organizers, and back translated to English by and independent language expert to ensure 
the quality of the Croatian translation. The survey instrument will be in Croatian. 
 
 
Outcomes 
 
The results for each participant will be the sum of correct answers at the knowledge test 
as the primary outcome, while secondary outcomes will be the number of correct answers on 
health numeracy and the scores on Likert-type scales for of accessibility and satisfaction. 
 
 
a) Demographic characteristics 

Participants will first provide information about their gender, age, knowledge about or prior 
experiences with Cochrane reviews and Cochrane Library, obtained specialization or PhD. 
 
 
b) Primary outcome: Understanding the information from the review summary 
 
The primary outcome of the study is the score on a knowledge test with ten questions about 
 
information contained in all three forms of presentation. The questions will focus on 
 
understanding the benefits and risks of the intervention and the quality of evidence described 
 
in the systematic review. 
 
Each correctly answered question will be awarded one point, with a maximum of 10 
points. Prior research on PLSs (Vandvik et al., 2012) used multiple choice questions with a 
single correct answer to assess understanding of PLS. However, multiple choice questions are 
closed questions and the correct answer may provide a visual reminder to the participant and 
thus do not measure real understanding of the material (Choi & Pak, 2005). In order to reduce 
this measurement bias, we will use open questions to assess knowledge of the participants. 
 
 
c) Secondary outcomes 
 
C.1. Reading experience: This section of the survey will include 5 questions about 
the experience of participants about the text they read, measured on a 10-point Likert type 
scale, where 1 means do not agree at all and 10 means fully agree. The total score is the sum 
of scores on all five answers (maximum 50). 
 
 
Survey C1 
 
Please tell us how much you agree with the following statements: 
 
 
1. I am satisfied with the overall look of how the information was presented. 
 
 
I do not agree at 
I fully agree 
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
2. I would understand the text better if there was more information about ECV procedure 
when turning babies.* 

I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
3.  I wish this information was presented with more pictures or graphs.* 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
4. 
This information was hard to follow.* 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
5. I think this way of information presentation is easily memorized. 
 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
*Reverse scored item. 
 
C.2. Accessibility of relevant information: This section of the survey will have 5 questions 
concerning how easy it was for the participant to find relevant information, measured by a10-
point Likert type scale where the answer 1 means I do not agree at all and 10 means I fully 
agree. The total score is the sum of scores on all the answers (maximum 50). The items for 
measuring accessibility of information will be: 
 
 
Survey C2 
 
 
1. It was easy to recall the information about EVC procedure needed to answer the test 
questions. 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 

2. I think this information is understandable to someone who does not have an 
experience with the topic. 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
3.  This topic is not possible to grasp with this small amount of information.*
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
4. I think I need more information in order to decide about the benefits of ECV 
procedure.* 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
5. I think I could explain to someone the benefits of ECV procedure using 
the information from this text only. 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
*Reverse scored item 
 
 
C3. Health numeracy: This section will use 6-item General health numeracy test 
(Osborne et al., 2013) in order to determine how much our participants understand the basic 
health instructions regarding numeracy dimension. For each correct answer, the participants 
receive one point and the total score is the sum of all correct answers. 
 
 
Survey C3 
 
1. Call your physician if you have a temperature of 38°C or higher. The 
thermometer shows the following temperature: 

37.9°C 
Will you call your physician? 
 
ANSWER: 
YES 
NO 
 
2. If 4 out of 20 persons are at risk of getting a cold, what would the risk of getting a cold 
be? 
 
ANSWER:_______________.% 
 
3. Let us assume that a 60-year-old woman has a maximum heartrate of 160 heartbeats 
per minute and that she was told to exercise until she reaches 80% of her maximum 
heartrate. How many heartbeats per minute isequal to 80% of her maximum heartrate? 
 
ANSWER:_______________.heartbeats per minute 
 
 
 
 
4. You have eaten half a bowl of carrots. How many grams of carbohydrates have 
you eaten? 
 
 
Food information 
Serving: 1 portion (85 g) 
 
Portions per bowl: 2.5 
 
 
Amount per portion 
 
Calories 45 
Calories from fat 0 
Daily amount % 
Fats 0g 
0% 
Saturated fats 0g 
0% 
Cholesterol 0g 
0% 
Sodium 55g 
2% 
Total carbohydrates 10g 
3% 
Dietary fiber 3g 
12% 
Sugars 5g 
Protein 1g 
 
ANSWER:_______________grams 
 
5. Your physician has informed you that your cholesterol is high. He explains to you that 
you have a 10% chance (risk) of having a heart attack in the next 5 years. If you start 
taking anti-cholesterol medication, you can reduce your risk by 30%. 
 
What is your risk of heart attack in the next five years if you take the medication? 
Answer:_______________.% 

6. Mammography is used to discover breast cancer in women. False positive tests are 
those tests that erroneously show a positive result. 85% positive mammographs are 
actually false positives. If 1000 women get mammography results, and 200 of 
them are informed that the results are abnormal, how many women are likely to 
actually have breast cancer? 
 
Answer:_______________.women 

Participant timeline 
 
The participants from the School of Medicine will be tested at the first course at the 
beginning of the first academic year. The participants from the School of Humanities will be 
tested during their course on psychology in the same academic year. The testing is 
completed in a single day. 
 
 
Sample size 
 
The sample size was calculated using the Sample Size Calculator (Medcalc, 
Ostend, Belgium). 
 
We calculated the sample size based on the main outcome of the study: score on the 
knowledge/understanding test. We used the alpha of 0.05 and 80% power to detect the 
difference of average result in groups of people who correctly answered the questions. We 
used the data from the studies by Santesso et al. (2015) and Schwartz et al.(2009) to estimate 
that the difference of 40% in the knowledge score would be the size of effect relevant for our 
study. In those studies, 80% of the participants who received a summary of information in a 
table with event rates answered questions correctly compared with 20-40% of those who did 
not receive this information. We thus estimated that, at a minimum, 15 people in each group 
need to complete the survey so it can be compared between with the other group. 
 
 
Methods: Assignment of interventions 
 
 
Sequence generation 
 
The curriculum for the Medical Humanities course is organized in three seminar 
groups, to which students are normally assigned in alphabetical order. For the purposes of this 
research, we will use a computer program (www.randomisation.com, permuted block 
method) to randomly assign students to one of the three groups and this list will be posted on 
the course web-site, as is usual practice for courses at the School. 
 
 
Concealment of allocation 
 
The participants will not be aware of intervention allocation before and during the 
trial as they will be invited to participate in research about the readability of new Cochrane 
formats. 
 
 
Blinding 

Participants will not be blinded to type of format, but the authors who analyse the 
answers will be blinded to the type of formats. 
 
Methods: Data collection, management, and analysis 
 
Data will be collected electronically, using the SurveyMonkey platform. The data 
will be fully anonymous and will be kept on a secure server of the School of Medicine. 
 
 
Descriptive statistics 
 
Descriptive statistics will include participants’ baseline characteristics and 
outcomes, means and standard deviations for continuous variables and proportions for 
dichotomous variables. 
 
Claiming of the non-inferiority 
 
For the primary outcome, non-inferiority of the experiment format to the control formats 
will be claimed if the lower limit of the CI (for the difference in average results between 
intervention group and control groups) is higher than the non-inferiority margin of 20%.The 
tests of superiority will be applied to the secondary outcomes. We want to perform the non-
inferiority testing in order to prove that infographic is no worse than PLS and scientific 
summaries for information translation in terms of understanding of information. Also, the 
superiority tests will show that the infographic is better in terms of user satisfaction and 
accessibility of information compared to standard PLS and scientific summaries. 
 
 
Inferential statistics 
 
The primary outcome of the trial understands of research results presented in 
various summary formats by lay persons. Each question has a single possible answer and 
the participant’s result is the sum of the correct answers. Also, it is possible to determine 
how each one of the demographic characteristics predicts correct answer on the knowledge 
test using logistic regression. 
 
 
For overall reading experience and accessibility of information between group testing we 
plan to test the distributions of the variables, and then to choose the non-parametric test 
(Kruskall-Wallis test) if the distribution of results is not normal or to choose the parametric 
test (ANOVA) if the distribution of results is normal. 
 
 
Table 1 Overview of the outcome measures 

Outcome 
Scale 
Measure 
Methods of analysis 
Understanding 
Dichotomous 
Sum of correct 
Kruskall- 
answers 
Wallis/ANOVA 
Accessibility of 
Continuous 
Likert–type scale 
Kruskall- 
information 
from 1 to 10 
Wallis/ANOVA 
Reading experience 
Continuous 
Likert–type scale 
Kruskall- 
from 1 to 10 
Wallis/ANOVA 
Health numeracy 
Dichotomous 
Sum of correct 
Kruskall- 
answers 
Wallis/ANOVA; 
Spearman correlation 
coefficient with 
understanding 
outcome 
 
 
Data Management 
 
Since the trial will be performed during the first lecture of a mandatory graduate 
course, we do not expect significant dropout. If a participant decides to leave during the trial, 
his or her results will not be included in the analysis. The participants will be not able to 
learn about their results after the trial as the identity of the participants will be unknown to 
the researchers. We will try to reduce the possibility of missing data in a way that the testing 
interface will not allow the respondent to move to the next page until all questions on the 
current page are answered. As the primary outcome is knowledge examination, the order of 
the questions will be the same for all the participants so that we could avoid interference of 
other contents, which could have an adverse impact to the results. 
 
 
Ethics and dissemination 
 
The approval of the Ethical Committee of University of Split School of Medicine has 
been obtained under the ethics approval for the grant “Professionalism and Health” funded 
by the Croatian Science Foundation (Grant No. IP-2014-09-7672). 
 
 
References: 
 
Bell DS, Harless CE, Higa JK, Bjork EL, Bjork RA, Bazarghan M et al. Knowledge retention 
after an online tutorial: A randomized educational experiment among resident physicians. J 
Gen Intern Med.2008;23:1164–71. 

Carrasco-Labra A, Brignardello-Peetersen R, Santesso N, Neumann I, Mustafa RA, 
Mbuagbaw L, et al. Comparison between the standard and a new alternative format of 
the Summary-of-Findings tables in Cochrane review users: study protocol for a 
randomized controlled trial. Trials.2015;16:164. 
 
Choi BC, Pak AW.A catalog of biases in questionnaires. Prev Chronic Dis. 2005;2:A13. 
Gagliardi AR, Légaré F, Brouwers MC, Webster F, Badley E, Straus S. Patient-mediated 
knowledge translation (PKT) interventions for clinical encounters: a systematic review. 
 
Implementation Science.2015;11:26. 
 
Osborn CY, Wallston KA, Shpigel A, Cavanaugh K, Kripalani S, Rothman RL. 
Development and Validation of the General Health Numeracy Test (GHNT). Pat educ 
counsel.2013;91:350-6. 
 
Hutton EK, Hofmeyr GJ, Dowswell T. External cephalic version for breech 
presentation before term. Cochrane Database Syst Rev.2015;7:CD000084. 
 
Santesso N, Rader T, Stromme Nilsen E, Glenton C, Rosenbaum S, Ciapponi A, et al. A 
summary to communicate evidence from systematic reviews to the public improved 
understanding and accessibility of information: a randomized controlled trial. J Clin 
Epidemiol.2014;68:182–90. 
 
Schwartz LM, Woloshin S, Welch HG. Using a drug facts box to communicate drug 
benefits and harms: two randomized trials. Ann Intern Med.2009;150:516–27. 
 
Vandvik PO, Santesso N, Akl EA, You J, Mulla S, Spencer FA, et al. Formatting 
modifications in GRADE evidence profiles improved guideline panelists comprehension 
and accessibility to information: a randomized trial. J Clin Epidemiol.2012;65:748–55.