Presenting Summary Information From Cochrane Systematic Reviews: Randomized Controlled Trial of Infographics Presentation vs. Standard Word Summaries
Phase: NA
Conditions: Understanding Cochrane Systematic Reviews
Sponsor: University Hospital of Split

Protocol: 
 
Presenting summary information from Cochrane systematic reviews: randomized 
controlled trial of infographics presentation vs. standard word summaries 
 
 
Authors (provisional by line): Ivan Buljan, Mario Malički, Liz Wager, Livia Puljak, 
Darko Hren, Frances Kellie, Helen West, Žarko Alfirević, Ana Marušić 
 
 
 
 
Protocol version 
 
This version of protocol was written in September 2016. 
 
Unique Protocol ID: ID: IP-2014-09-7672 
 
 
NCT02918656 
 
 
Funding 
 
This research is funded by the Croatian Science Foundation in “Professionalism and 
Health”scientific project (Grant No. IP-2014-09-7672).Croatian Science Foundation had no 
role in the design of this study and will not have any role during its execution. 
 
 
Roles and responsibilities 
 
The idea for the study was provided by AM and ŽA. Research group from the 
University of Split (IB, AM, MM, LW, LP) drafted the study design. Research group from 
the University of Liverpool (ŽA, FK, HW) provided the infographics material and 
contributed to question development. All authors contributed to the refinement of the study 
protocol and approved the final manuscript. 
 
 
Introduction 
 
Background 
 
The presentation of research evidence to the lay audience has become an important 
aspect of the Cochrane. Results of studies on evidence presentation are not always easy to 
interpret (Mazur & Hickam, 1991; Mazur & Merz, 1994; Carrasco-Labra et al, 2015) and the 
efforts continue to develop more understandable ways of scientific data presentation to the 
lay public, and ultimately better knowledge translation. 
 
Key messages of individual Cochrane Systematic reviews are presented in two ways. 
One is a scientific summary (SciSumm), aimed at researchers and health practitioners and 

the other is a plain language summary (PLS), aimed at the lay public (Santesso et al, 2014). 
Recently, Cochrane started developing infographics, where short textual information about 

research is supported by visual representations of the main findings and is also aimed at the 
lay public (www.visuallycochrane.net.) 
 
A recent systematic review (Gagliardi et al, 2015) showed that there was no clear 
evidence for the best type of intervention for healthcare knowledge information translation, 
so further research about different interventions is needed. 
 
 
Objectives 
 
The objective of our study is to evaluate the efficacy of infographics in presenting 
information, in terms of understanding and remembering research results, compared to 
standard PLS formats and scientific summary format. 
 
The null hypothesis is that the proportion of the correct answers in all three groups is 
not going to differ significantly. The equation for that hypothesis is: 
 
 
H0: Infographic = PLS = SciSumm 
 
 
The alternative hypothesis is that participants in the infographic group (Infographics) 
will have significantly more correctly answered questions about the topic than those in PLS 
group (PLS), meaning that infographics enables a better understanding of scientific findings 
than PLS. Also, the scientific summary group will have the lowest knowledge score of the three 
groups, meaning than scientific summary is not the suitable way of data presentation about a 
Cochrane review for lay population. The equation for that hypothesis is: 
 
 
Ha: Infographic > PLS >SciSumm 
 
 
Trial design 
 
We will conduct a randomized control trial (RCT) with three different formats of the 
same systematic review summary (infographics, PLS and scientific summary). The content 
of these three formats is based on the same systematic review, but the ways of data 
presentation will differ: visual presentation in plain language, plain language, and scientific 
language. The trial will be performed at the beginning of the 2016/2017 academic year, with 
first year university students as participants. The trial will be voluntary and anonymous. The 
survey will consist of 4 parts: 1) demographic data, 2) one format of the summary (randomly 
assigned), 3) comprehension test of the information given in the summary, 4) accessibility of 
information and overall satisfaction with the given summary assessed by survey and 5) health 

numeracy test. All materials will be in Croatian. The materials used in this research will 
be assessed by experts in order to confirm face validity of the survey. 
 
 
Methods: Participants, interventions, and outcomes 
 
 
Setting and recruitment 
 
The setting of the trial will be at the two schools of the University of Split: the 
School of Medicine and the Faculty of Humanities and Social Sciences. The students 
attending the first year Medical Humanities course at the School of Medicine and the 
students attending a course in psychology at the Faculty of Humanities and Social Sciences 
will be invited to participate in the trial. 
 
 
Eligibility criteria 
 
Eligible participants will be first-year university students without experience with 
Cochrane review summaries, because prior familiarization with Cochrane summaries could 
make information uptake easier for those participants. Participants who give the positive 
answer to the survey question about the familiarity with the Cochrane will be excluded 
from the study analysis. 
 
 
Interventions 
 
Infographics format of a Cochrane systematic review summary represents the 
experimental intervention in the trial. The control groups will receive either PLS (which 
represent text format with simple explanation of the survey topic main findings and is intended 
for lay audience) or the scientific summary (the text written for the academic population and 
practitioners) groups. We will use formats already available for the systematic review on external 
cephalic version for breech presentation before term (Hutton et al, 2015.). 
 
All three types of Cochrane summaries will be translated into Croatian by the trial 
organizers, and back translated to English by and independent language expert to ensure 
the quality of the Croatian translation. The survey instrument will be in Croatian. 
 
The students will come to the course group according to the group assignment published 
on the course web-site. Before the course break on the first day, the tutor will invite the students 
to participate in research regarding reading of medical research information. After the students 
have read the text the students will have to close the computer monitors and will continue with 
the course teaching after the break. The course activity will consist of a 

video-material related to the topic of the course (unrelated to the topic of the infographics) 
and lasting for about 30 minutes. We will use a delayed testing method because it has been 
shown that, although immediate knowledge post tests can show significant improvement in 
knowledge retention, such knowledge is mostly rapidly lost with time (Bell et al, 2008.). As 
our aim is to examine what amount of information is kept after reading the summaries so it 
can be used at a later time, we will use a delayed testing format. The students will be asked 
to turn off the monitors of the computers in the teaching lab. 
 
After the end of the course day, the participants will be asked to complete a survey 
about the text they have read at the beginning of the course day. All study groups will have 
the same time allocated to this task, and the time each individual participant takes to 
complete the survey will be noted. The participants will not be able to use the Internet to get 
more information about the topic or to look at each other computers. Two tutors will be 
overseeing the survey and ensuring that cell phones are turned off and put on the desk; they 
will not otherwise interact with the participants. 
 
 
Outcomes 
 
The results for each participant will be the sum of correct answers at the knowledge test 
as the primary outcome, while secondary outcomes will be the number of correct answers on 
health numeracy and the scores on Likert-type scales for of accessibility and satisfaction. 
 
 
a) Demographic characteristics 
 
Participants will first provide information about their gender, age, knowledge about or prior 
experiences with Cochrane reviews and Cochrane Library, as well as habits in searching 
for health information. 
 
 
b) Primary outcome: Understanding the information from the review summary 
 
The primary outcome of the study is the score on a knowledge test with ten questions about 
 
information contained in all three forms of presentation. The questions will focus on 
 
understanding the benefits and risks of the intervention and the quality of evidence described 
 
in the systematic review. 
 
Each correctly answered question will be awarded one point, with a maximum of 10 
points. Prior research on PLSs (Vandvik et al., 2012) used multiple choice questions with a single 
correct answer to assess understanding of PLS. However, multiple choice questions are closed 
questions and the correct answer may provide a visual reminder to the participant and 

thus do not measure real understanding of the material (Choi & Pak, 2005). In order to reduce 
this measurement bias, we will use open questions to assess knowledge of the participants. 
 
 
c) Secondary outcomes 
 
C.1. Reading experience: This section of the survey will include 5 questions about 
the experience of participants about the text they read, measured on a 10-point Likert type 
scale, where 1 means do not agree at all and 10 means fully agree. The total score is the sum 
of scores on all five answers (maximum 50). 
 
 
Survey C1 
 
Please tell us how much you agree with the following statements: 
 
 
1. I am satisfied with the overall look of how the information was presented. 
 
 
I do not agree at 
I fully agree 
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
2. I would understand the text better if there was more information about ECV procedure 
when turning babies.* 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
3.  I wish this information was presented with more pictures or graphs.* 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
4. 
This information was hard to follow.* 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
5. I think this way of information presentation is easily memorized. 

 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
*Reverse scored item. 
 
C.2. Accessibility of relevant information: This section of the survey will have 5 questions 
concerning how easy it was for the participant to find relevant information, measured by a10-
point Likert type scale where the answer 1 means I do not agree at all and 10 means I fully 
agree. The total score is the sum of scores on all the answers (maximum 50). The items for 
measuring accessibility of information will be: 
 
 
Survey C2 
 
 
1. It was easy to recall the information about EVC procedure needed to answer the test 
questions. 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
2. I think this information is understandable to someone who does not have an 
experience with the topic. 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
3.  This topic is not possible to grasp with this small amount of information.*
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
4. I think I need more information in order to decide about the benefits of ECV 
procedure.* 

I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
 
5. I think I could explain to someone the benefits of ECV procedure using 
the information from this text only. 
 
I do not agree at 
I fully agree
all 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
 
*Reverse scored item 
 
 
C3. Health numeracy: This section will use 6-item General health numeracy test (Osborne 
 
et al., 2013) in order to determine how much our participants understand the basic health 
 
instructions regarding numeracy dimension. For each correct answer, the participants receive 
 
one point and the total score is the sum of all correct answers. 
 
 
Survey C3 
 
1. Call your physician if you have a temperature of 38°C or higher. The 
thermometer shows the following temperature: 
 
37.9°C 
Will you call your physician? 
 
ANSWER: 
YES 
NO 
 
2. If 4 out of 20 persons are at risk of getting a cold, what would the risk of getting a cold 
be? 
 
ANSWER:_______________.% 
 
3. Let us assume that a 60-year-old woman has a maximum heartrate of 160 heartbeats 
per minute and that she was told to exercise until she reaches 80% of her maximum 
heartrate. How many heartbeats per minute isequal to 80% of her maximum heartrate? 
 
ANSWER:_______________.heartbeats per minute 
 
 
 
 
4. You have eaten half a bowl of carrots. How many grams of carbohydrates have 
you eaten? 

 
Food information 
Serving: 1 portion (85 g) 
 
Portions per bowl: 2.5 
 
 
Amount per portion 
 
Calories 45 
Calories from fat 0 
Daily amount % 
Fats 0g 
0% 
Saturated fats 0g 
0% 
Cholesterol 0g 
0% 
Sodium 55g 
2% 
Total carbohydrates 10g 
3% 
Dietary fiber 3g 
12% 
Sugars 5g 
Protein 1g 
 
ANSWER:_______________grams 
 
5. Your physician has informed you that your cholesterol is high. He explains to you that 
you have a 10% chance (risk) of having a heart attack in the next 5 years. If you start 
taking anti-cholesterol medication, you can reduce your risk by 30%. 
 
What is your risk of heart attack in the next five years if you take the medication? 
Answer:_______________.% 
 
6. Mammography is used to discover breast cancer in women. False positive tests are 
those tests that erroneously show a positive result. 85% positive mammographs are 
actually false positives. If 1000 women get mammography results, and 200 of 
them are informed that the results are abnormal, how many women are likely to 
actually have breast cancer? 
 
Answer:_______________.women 

Participant timeline 
 
The participants from the School of Medicine will be tested at the first course at the 
beginning of the first academic year. The participants from the School of Humanities will be 
tested during their course on psychology in the same academic year. The testing is 
completed in a single day. 
 
 
Sample size 
 
The sample size was calculated using the Sample Size Calculator (Medcalc, 
Ostend, Belgium). 
 
We calculated the sample size based on the main outcome of the study: score on the 
knowledge/understanding test. We used the alpha of 0.05 and 80% power to detect the 
difference of average result in groups of people who correctly answered the questions. We 
used the data from the studies by Santesso et al. (2015) and Schwartz et al.(2009) to estimate 
that the difference of 40% in the knowledge score would be the size of effect relevant for our 
study. In those studies, 80% of the participants who received a summary of information in a 
table with event rates answered questions correctly compared with 20-40% of those who did 
not receive this information. We thus estimated that, at a minimum, 15 people in each group 
need to complete the survey so it can be compared between with the other group. As the 
number of students in the Medical Humanities course is about 90, we will include all of them, 
which should ensure that each group has at least 25 participants, taking into account that 
some students will refuse to participate. 
 
 
Methods: Assignment of interventions 
 
 
Sequence generation 
 
The curriculum for the Medical Humanities course is organized in three seminar 
 
groups, to which students are normally assigned in alphabetical order. For the purposes of this 
 
research, we will use a computer program (www.randomisation.com, permuted block method) 
 
to randomly assign students to one of the three groups and this list will be posted on the 
 
course web-site, as is usual practice for courses at the School. The familiarity of the students 
 
with Cochrane summaries will be controlled by the following questions in the survey: 1. 
 
“Have you heard of Cochrane systematic reviews before?”, 2. “Have you ever used Cochrane 
 
library?” and 3. “What is your primary source of health information?” The participants 
 
positively answering the questions 1 and 2 will be excluded from the trial analysis. 

 
Concealment of allocation 
 
The participants will not be aware of intervention allocation before and during the 
trial. Each participant will have his or her own computer, with computer screens all showing 
the same instructions for the research and informed consent, which they will read in a given 
time. The student will be seated in such a way that they will not be able to see each other 
screens and the tutors present during the intervention will ensure that there is no exchange 
of information among the students. 
 
 
Blinding 
 
The participants will be blinded to the group they will be allocated to because they 
will not see each other’s computer screens.The lecturer and two tutors will be able to look 
at the monitors, but they will not be familiar with the study and its design.None of the 
researchers will not be involved as the instructor of the course. 
 
 
Methods: Data collection, management, and analysis 
 
Data will be collected electronically, using the SurveyMonkey platform. The data 
will be fully anonymous and will be kept on a secure server of the School of Medicine. 
 
 
Descriptive statistics 
 
Descriptive statistics will include participants’ baseline characteristics and 
outcomes, means and standard deviations for continuous variables and proportions for 
dichotomous variables. 
 
Claiming of the non-inferiority 
 
For the primary outcome, non-inferiority of the experiment format to the control formats 
will be claimed if the lower limit of the CI (for the difference in average results between 
intervention group and control groups) is higher than the non-inferiority margin of 20%.The 
tests of superiority will be applied to the secondary outcomes. We want to perform the non-
inferiority testing in order to prove that infographic is no worse than PLS and scientific 
summaries for information translation in terms of understanding of information. Also, the 
superiority tests will show that the infographic is better in terms of user satisfaction and 
accessibility of information compared to standard PLS and scientific summaries. 
 
 
Inferential statistics 

The primary outcome of the trial understands of research results presented in 
various summary formats by lay persons. Each question has a single possible answer and 
the participant’s result is the sum of the correct answers. Also, it is possible to determine 
how each one of the demographic characteristics predicts correct answer on the knowledge 
test using logistic regression. 
 
 
For overall reading experience and accessibility of information between group testing we 
plan to test the distributions of the variables, and then to choose the non-parametric test 
(Kruskall-Wallis test) if the distribution of results is not normal or to choose the parametric 
test (ANOVA) if the distribution of results is normal. 
 
 
Table 1 Overview of the outcome measures 
 
Outcome 
Scale 
Measure 
Methods of analysis 
Understanding 
Dichotomous 
Sum of correct 
Kruskall- 
answers 
Wallis/ANOVA 
Accessibility of 
Continuous 
Likert–type scale 
Kruskall- 
information 
from 1 to 10 
Wallis/ANOVA 
Reading experience 
Continuous 
Likert–type scale 
Kruskall- 
from 1 to 10 
Wallis/ANOVA 
Health numeracy 
Dichotomous 
Sum of correct 
Kruskall- 
answers 
Wallis/ANOVA; 
Spearman correlation 
coefficient with 
understanding 
outcome 
 
 
Data Management 
 
Since the trial will be performed during the first lecture of a mandatory graduate 
course, we do not expect significant dropout. If a participant decides to leave during the trial, 
his or her results will not be included in the analysis. The participants will be not able to 
learn about their results after the trial as the identity of the participants will be unknown to 
the researchers. We will try to reduce the possibility of missing data in a way that the testing 
interface will not allow the respondent to move to the next page until all questions on the 
current page are answered. As the primary outcome is knowledge examination, the order of 

the questions will be the same for all the participants so that we could avoid interference 
of other contents, which could have an adverse impact to the results. 
 
 
Ethics and dissemination 
 
The approval of the Ethical Committee of University of Split School of Medicine has 
been obtained under the ethics approval for the grant “Professionalism and Health” funded 
by the Croatian Science Foundation (Grant No. IP-2014-09-7672). The approval of the 
Faculty of Humanities and Social Sciences will be obtained before the trial. 
 
 
References: 
 
Bell DS, Harless CE, Higa JK, Bjork EL, Bjork RA, Bazarghan M et al. Knowledge retention 
after an online tutorial: A randomized educational experiment among resident physicians. J 
Gen Intern Med.2008;23:1164–71. 
 
Carrasco-Labra A, Brignardello-Peetersen R, Santesso N, Neumann I, Mustafa RA, 
Mbuagbaw L, et al. Comparison between the standard and a new alternative format of 
the Summary-of-Findings tables in Cochrane review users: study protocol for a 
randomized controlled trial. Trials.2015;16:164. 
 
Choi BC, Pak AW.A catalog of biases in questionnaires. Prev Chronic Dis. 2005;2:A13. 
Gagliardi AR, Légaré F, Brouwers MC, Webster F, Badley E, Straus S. Patient-mediated 
knowledge translation (PKT) interventions for clinical encounters: a systematic review. 
 
Implementation Science.2015;11:26. 
 
Osborn CY, Wallston KA, Shpigel A, Cavanaugh K, Kripalani S, Rothman RL. 
Development and Validation of the General Health Numeracy Test (GHNT). Pat educ 
counsel.2013;91:350-6. 
 
Hutton EK, Hofmeyr GJ, Dowswell T. External cephalic version for breech 
presentation before term. Cochrane Database Syst Rev.2015;7:CD000084. 
 
Santesso N, Rader T, Stromme Nilsen E, Glenton C, Rosenbaum S, Ciapponi A, et al. A 
summary to communicate evidence from systematic reviews to the public improved 
understanding and accessibility of information: a randomized controlled trial. J Clin 
Epidemiol.2014;68:182–90. 
 
Schwartz LM, Woloshin S, Welch HG. Using a drug facts box to communicate drug 
benefits and harms: two randomized trials. Ann Intern Med.2009;150:516–27. 

Vandvik PO, Santesso N, Akl EA, You J, Mulla S, Spencer FA, et al. Formatting 
modifications in GRADE evidence profiles improved guideline panelists comprehension 
and accessibility to information: a randomized trial. J Clin Epidemiol.2012;65:748–55.