Readers' Attention of Shorter Versus Longer Abstracts of Systematic Reviews: A Randomized Controlled Trial
Phase: NA
Conditions: Researchers Attention
Sponsor: University of Oldenburg

Readers' attention of shorter versus longer abstracts of systematic reviews: A randomized 
controlled trial 
Jasmin Helbach1, Kathrin Wandscher1, Dawid Pieper2,3, Falk Hoffmann1  
1Department of Health Services Research, School of Medicine and Health Sciences, Carl von Ossietzky 
University Oldenburg, Oldenburg, Germany 
2Faculty of Health Sciences Brandenburg, Brandenburg Medical School Theodor Fontane, Institute for 
Health Services and Health System Research, Rüdersdorf, Germany 
3Center for Health Services Research, Brandenburg Medical School Theodor Fontane, Rüdersdorf, 
Germany 
 
 
Correspondence to: 
Helbach J., Department of Health Services Research, School of Medicine and Health Sciences, Carl 
von Ossietzky University Oldenburg, Oldenburg, Germany 
e-mail: jasmin.helbach@uni-oldenburg.de 
 
Protocol version:  
Final version Date: 24.07.2024 
 
 
 
 

Background:  
The number of systematic reviews (SRs) indexed in PubMed continues to grow1, highlighting the 
importance of well-written abstracts. Researchers often do not have the time or interest to read all 
of the available research and instead base their decision to read the full article on the information 
presented in the title and abstract2–4. Abstracts should therefore reflect all the essential content of 
the main manuscript in an informative, accurate, attractive, and concise form that attracts 
researchers' attention2. 
Several articles have been published on how to write good abstracts, emphasizing that the abstract is 
one of the most important parts of the manuscript2,5,6. Abstracts serve to 'sell' the work to editors, 
reviewers, and potential readers and help to improve the findability of the article2. Due to the 
importance of the abstract, the preferred reporting items for systematic reviews and meta-analysis 
for abstracts (PRISMA-A) was developed in 20137 and updated in 20208. It provides guidance on 
which items should be reported in an abstract alongside with examples.  
In our previous studies on the reporting of abstracts according to the PRISMA-A guidelines, we found 
that there are differences in structure, length, and reporting quality between Cochrane Reviews (CRs) 
and non-Cochrane Reviews (non-CRs)9,10. In particular, CRs are better reported and always 
structured. While non-Cochrane reviews consistently remain at a median length of 250 words, CR 
abstract's length has increased steadily from 353 words in 2000 to 838 words in 202210. It is 
questionable whether such long abstracts meet the goal of being informative, accurate, attractive, 
and concise. However, there are no guidelines regarding the optimal length of well-written SR 
abstracts, nor are there any studies indicating whether longer abstracts are more likely to attract 
readers' attention.    
Objective:  
Primary:  
The primary aim of the study will be to investigate readers' attention of shorter versus longer 
abstracts, based on trial participation after reading the abstract.  
Secondary:  
The secondary objective will be to assess the perception of an abstract based on the four indicators 
of a well-written abstract, outlined by Bahadoran et al2. These indicators state that abstracts should 
be: a) informative, b) accurate, c) attractive, and d) concise. 
Study design:  
This will be a two-arm, single-blinded, superiority, parallel-group randomized controlled trial (RCT) 
with 1:1 allocation of intervention and control groups.  
Reporting: 
This protocol was reported in line with the SPIRIT checklist11.  
Study setting: 
The study will involve researchers from all over the world who recently published a SR, regardless of 
their research field.  
Participants:  
Eligibility criteria 
The corresponding authors of SRs that were indexed in PubMed between January 1, 2024, and March 
26, 2024, and for which an English abstract is available will be included. These include authors of all 

types of SR as defined by Munn et al.12, with the majority likely representing SRs on effectiveness13. 
Authors of other types of evidence syntheses (e.g. scoping reviews, methodological papers), 
editorials, protocols, corrections, retractions, erratums, or summaries of SRs will be excluded. For 
identifying the SRs from which the corresponding authors will be contacted we will use the following 
eligibility criteria.  
 
Table 1 Eligibility criteria 
Inclusion 
Exclusion 
Systematic reviews of all types (Munn et al.12) 
No English abstract 
Other types of evidence syntheses (e.g. scoping 
reviews, methodological papers)  
Protocols 
Editorials 
Others: e.g. Corrections, retractions, erratums, 
or summaries of SRs 
No email address available  
Identification of participants:   
To identify eligible authors, MEDLINE (via PubMed) was searched with the following search strategy 
(date of search: 23.05.2024). To ensure the extracted contact data was up-to-date, we included SRs 
from January 2024 onwards until the minimum sample size of 6200 was reached. This was the case 
on March 26, 2024. 
 
"systematic review"[Filter] AND ("2024/01/01"[Date - Entry]: "2024/03/26"[Date - Entry]) NOT 
("1000"[Date - Publication]: "2023/12/31"[Date - Publication] OR "2024/03/27"[Date - Publication]: 
"3000"[Date - Publication]) 
 
All references will be screened by two reviewers independently. Articles that identify themselves as a 
study type other than SR, corrections or summaries in the title or abstract (e.g. explicitly labeled as a 
protocol or scoping review) will be excluded from this study. For all included SRs, the name, country, 
and email address of the corresponding author will be extracted manually from PubMed, the 
journal's page or the full text by the same independent reviewers. The extraction process will 
proceed according to the following procedure: 
• 
If only one email address is provided in PubMed, this author is defined as the corresponding 
author.  
• 
If multiple email addresses are provided, the first and then the last author will be given 
priority. If no e-mail address is provided for either of them, the first author to appear with 
an email address will be listed as the corresponding author.   
• 
If there´s no email address listed in PubMed, we will first search the journal page and then 
the full text to determine the identity of the corresponding author.  
• 
If more than one email address is provided for a single author, the first email address and 
affiliation will be utilized. 
• 
If no email address can be identified in PubMed the journal page or the full text, the article 
will be excluded. 
All data is extracted from a single source, with PubMed being the primary source of information. If 
no information can be found in PubMed the journal page or the full text being consulted. The source 
of the information will be extracted. Disagreements between reviewers will be resolved by discussion 
or, if necessary, by a third reviewer.   
 

Before randomization, duplicate e-mail addresses will be excluded using SAS for Windows version 9.4 
(SAS Institute Inc, Cary, North Carolina) to prevent the cover letter from being sent to the same email 
address on more than one occasion. 
Intervention:   
In this study, we will attempt to emulate the reading process observed in PubMed. When searching 
PubMed, researchers are initially presented with an overview of the references identified. After 
clicking on the title, the corresponding abstract appears. For full-text access, researchers must 
actively follow a separate link to the journal page. 
In this RCT, participants are randomly assigned to two groups (those receiving a long abstract vs. 
those receiving a short abstract). Both groups receive the same cover letter by e-mail explaining the 
purpose of the study, the use of the data, measures to ensure anonymity of participation and a link 
for participation. When participants click on the link, they automatically confirm their participation in 
the study.  
The link will direct participants to SoSci Survey, an online survey tool. Those in the group receiving 
the long abstract will see an abstract with a length of 771 words (PMID: 37955353). Those in the 
group receiving the short abstract will see an abstract with a length of 277 words (PMID: 37942649). 
Both abstracts are structured and pertain to the same review by Soderberg et al. entitled 
"Percutaneous nephrolithotomy vs. retrograde intrarenal surgery for renal stones". This review was 
published once in the Cochrane Database of Systematic Reviews and once in the journal BJU 
International.   
The design of the abstract presented is similar to that of PubMed, but no title or authors will be 
indicated. After reading the abstract, participants must actively click the "Continue" button to 
proceed.  
Outcomes: 
Primary outcome:  
The primary outcome of this RCT will be the trial participation, which is defined as the proportion of 
participants who read the abstract and then clicked on the "Continue" button. This will be quantified 
by dividing the number of participants who read the abstract and proceeded to complete the survey 
by the total number of individuals who clicked on the link in the email. Information will be 
electronically captured in SoSci survey. 
Secondary outcomes:  
The secondary outcomes will be the four criteria outlined by Bahadoran et al.2 as indicators of a well-
written abstract: 
a) Informativeness 
b) Accuracy 
c) Attractiveness 
d) Conciseness.  
The extent to which participants perceive these criteria to be fulfilled will be quantified on a 4-point 
Likert scale, e.g. very little informative, little informative, informative, very informative. For data 
analysis, the data will be dichotomized. For instance, the statements "very little informative" and 
"little informative" will be summarized in the category "non-informative," while the statements 
"informative" and "very informative" will be summarized in the category "informative." 
Additionally, the time needed to read the abstract will be quantified. The time interval is defined as 
the period between the participant clicking on the link in the email and clicking on the „Continue" 
button to proceed to the survey. This time will be automatically recorded by SoSci Survey.  

Additional outcomes:   
In addition, data on some formal and structural characteristics of an Abstract (structure, length, 
funding, and registration) as well as participant characteristics (age, gender, country, and experience 
in writing SRs) will be collected. 
Sample size:   
Assuming an 80% trial participation in the control group, a two-sided significance level of 5% (α 
=0.05), and a power of 90% (β=10%), 558 participants (279 per group) are needed to detect a relative 
difference in trial participation of 15% (80% vs. 68%). The sample size calculation was carried out 
using the PROC POWER function in SAS. Given an expected response of 9.6%14, the sample size was 
estimated to be 5813. Additionally, expecting an approximately 5% exclusion due to study types 
other than SR13 or duplicate email addresses, the sample size was increased to 6103. To achieve this, 
a total of at least 6200 SRs will be selected from PubMed and assessed for eligibility.  
Recruitment and incentives   
The corresponding authors of the identified SRs will be contacted by email with a link to the survey 
and invited to participate. To achieve the highest possible response, recruitment will follow the 
recommendations for increasing the response of electronic questionnaires from Edwards et al.15.  
Accordingly, we will:  
• 
use a short and personalized cover letter that includes a picture 
• 
use a simple but meaningful header, not including the term “survey” 
• 
use a brief e-questionnaire that is designed in an attractive way 
• 
give participants enough time, but also set and communicate a firm deadline. 
• 
send the email by a female investigator. 
The initial email invitation to participate in the study will be sent in August 2024. After two weeks, a 
first reminder will be sent out. A second reminder will be sent after four weeks. Data collection will 
conclude eight weeks after the initial e-mail has been sent. 
No financial incentives are planned. 
Participant timeline 

 
Figure 1. Flowchart: Preparation for enrollment and recruitment  
Allocation: 
Sequence generation and implementation: 
In a 1:1 ratio, the corresponding authors will be randomly assigned to either the group receiving the 
long abstract or the group receiving the short abstract using the PROC PLAN procedure in SAS.  
Allocation concealment:  
Recruitment 
Pre-enrollment preparation 
Randomization 
Exclude: 
No English abstract (n=XXXX) 
Other study type (n=XXX) 
Protocols & editorials (XXX) 
Others (e.g. Corrections) (n=XXX) 
No email address available (XXX) 
Duplicate Authors (n=XXX) 
Identification of participants  
Sample of corresponding authors of SRs indexed 
in PubMed between January 1, 2024, and March 
26, 2024  
Search date: 23.05.2024 
1st reminder: after two weeks 
2st reminder: after four weeks 
Group receiving the short 
abstract: (n=XXX) 
Receipt the non-CR abstract with a 
length of 277 words 
(PMID: 37942649) 
Invitation to participate 
An invitation to participate was sent 
to every available email address of 
the corresponding authors. 
(Date: 01.08.2024) 
 
 
Response: (n=XXX) 
The number of individuals who 
clicked on the invitation link within 
eight weeks. 
 
Group receiving the long 
abstract: (n=XXX) 
Receipt the CR abstract with a length 
of 771 words; 
(PMID: 37955353) 
Invitation to participate  
An invitation to participate was sent 
to every available email address of 
the corresponding authors. 
(Date: 01.08.2024) 
Response: (n=XXX) 
The number of individuals who 
clicked on the invitation link within 
eight weeks. 
Survey closed: after eight weeks 
Undelivered email: 
(n=XXX) 
Undelivered email: 
(n=XXX) 

A third party (FH) not directly involved in the data extraction will generate the randomization plan 
(PROC PLAN procedure) in SAS.  
Blinding: 
The participants will be blinded to the intervention. In the cover letter, participants will be informed 
that the study aims to examine the characteristics of an abstract. However, they will not be informed 
that the primary focus of this study is the influence of the abstract´s length. Furthermore, the 
participants will be not aware of the number of different abstracts that were randomly allocated.  
Data collection and management:  
All data will be collected and managed using the online survey tool SoSci Survey.   
Statistical methods:  
Data will be analysed descriptively, using frequencies and proportions for categorical and means with 
standard deviations (SDs) as well as medians with interquartile ranges (IQRs) for continuous 
variables. All p-values less than 0.05 will be considered statistically significant. All statistical analyses 
will be conducted with SAS. 
Table 1. Variable, Measures, and Methods of Analysis 
Variable/Outcome 
Outcome measure 
Methods of analysis 
Primary:  
Trial participation 
Proportion (percentage) of participants 
who continued to participate after 
reading the abstract 
Chi-square test 
Secondary: 
Informativeness 
Likert scale 1 to 4, dichotomized as 
informative and non-informative 
Chi-square test 
Secondary: 
Accuracy 
Likert scale 1 to 4, dichotomized as 
accurate and non-accurate 
Chi-square test 
Secondary: 
Attractiveness 
Likert scale 1 to 4, dichotomized as 
attractive and non-attractive 
Chi-square test 
Secondary: 
Conciseness 
Likert scale 1 to 4, dichotomized as 
concise and non-concise 
Chi-square test 
Secondary: 
Time needed to 
read the abstract 
Time spent on the first page that presents 
the abstract  
Mann-Whitney U-Test 
Data monitoring:  
Not applicable. 
Harms: 
We do not anticipate any harm to the participants.  
Ethical approval:  
Participation in this study is entirely voluntary, and the data will be collected anonymously. The study 
has been presented to the Medical Ethics Committee of the [Carl von Ossietzky University of 
Oldenburg] for review and has received a waiver (2024-113). 
Consent: 

The cover letter will inform participants of the objectives of the study, as well as ethical, legal, and 
anonymity information. Participants will be informed that their click on the link will be their consent 
for participation in the study. 
Confidentiality: 
No personal or medical data will be collected.  
Access to data: 
We do not plan to publish our raw data.  
Ancillary and post-trial care: 
Not applicable. 
Dissemination policy: 
We plan to communicate study results in a peer-reviewed journal. 
Auditing:  
Not applicable. 
Author contribution to the protocol  
JH and FH conceived the study idea. JH, KW, DP, and FH further developed the study design. JH and 
KW developed the contact letter for the study participants and the survey via SoSci-Survey. The 
authors' contact information was extracted manually by JH, KH, SS, SS, PL, LB, ES, and LL. All authors 
contributed to the refinement of the study protocol and approved the final version of the protocol. 
Roles and responsibilities in the conduct of the study  
Principal investigator: JH  
Conduct of the trial: JH, KW, DP, FH 
Data Analysis: JH, KW 
Writing the manuscript: JH 
Approval of the manuscript: JH, KW, DP, FH 
Conflict of interest:  
The authors declare that there are no conflicts of interest. 
Funding:  
This research did not receive any specific grant from funding agencies in the public, commercial, or 
not-for-profit sectors. 
Acknowledgements:  
We would like to thank Sara Stubenrauch, Sonja Sydekum, Paula Lienesch, Leah Barghorn, Elisa-
Marie Speckmann and Laura-Lee Linngrön for their support in data collection and for their efforts in 
compiling the contact list for participant recruitment. 

References 
 
1.  
Hoffmann F, Allers K, Rombey T, et al. Nearly 80 systematic reviews were published each day: 
Observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin 
Epidemiol 2021; 138: 1–11. 
2.  
Bahadoran Z, Mirmiran P, Kashfi K, et al. The Principles of Biomedical Scientific Writing: 
Abstract and Keywords. Int J Endocrinol Metab 2020; 18: e100159. 
3.  
Squires BP. Structured abstracts of original research and review articles. C  Can Med Assoc J = 
J l’Association medicale  Can 1990; 143: 619–622. 
4.  
Shiely F, Gallagher K, Millar SR. How, and why, science and health researchers read scientific 
(IMRAD) papers. PLoS One 2024; 19: e0297034. 
5.  
Tullu MS. Writing the title and abstract for a research paper: Being concise, precise, and  
meticulous is the key. Saudi J Anaesth 2019; 13: S12–S17. 
6.  
Cals JWL, Kotz D. Effective writing and publishing scientific papers, part II: title and abstract. J 
Clin Epidemiol 2013; 66: 585. 
7.  
Beller EM, Glasziou PP, Altman DG, et al. PRISMA for Abstracts: Reporting Systematic Reviews 
in Journal and Conference Abstracts. PLoS Med 2013; 10: e1001419. 
8.  
Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline 
for reporting systematic reviews. PLoS Med 2021; 18: 1–15. 
9.  
Helbach J, Hoffmann F, Pieper D, et al. Reporting according to the preferred reporting items 
for systematic reviews and  meta-analyses for abstracts (PRISMA-A) depends on abstract 
length. J Clin Epidemiol 2022; 154: 167–177. 
10.  
Helbach J, Wandscher K, Pieper D, et al. Cochrane review abstracts are getting longer, but this 
has no large impact on the  reporting quality. J Evid Based Med 2023; 16: 294–302. 
11.  
Chan A-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining standard protocol 
items for clinical trials. Ann Intern Med 2013; 158: 200–207. 
12.  
Munn Z, Stern C, Aromataris E, et al. What kind of systematic review should i conduct? A 
proposed typology and guidance for systematic reviewers in the medical and health sciences. 
BMC Med Res Methodol 2018; 18: 1–9. 
13.  
Hoffmann F, Allers K, Rombey T, et al. Nearly 80 systematic reviews were published each day: 
Observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin 
Epidemiol 2021; 138: 1–11. 
14.  
Barcot O, Ivanda M, Buljan I, et al. Enhanced access to recommendations from the Cochrane 
Handbook for improving  authors’ judgments about risk of bias: A randomized controlled trial. 
Res Synth Methods 2021; 12: 618–629. 
15.  
Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic 
questionnaires. Cochrane database Syst Rev 2023; 11: MR000008.