Evaluating Online Messages About Colon Cancer Screening
Conditions: Colonic Cancer
Sponsor: University of Utah

Official Title:   
Messages about Cancer Screening 
NCT number:  
NCT06134089 
Date of document:  28 Feb 2023 
 

RESEARCH STRATEGY 
A. SIGNIFICANCE  
Colorectal cancer (CRC) is the third most diagnosed and third most deadly cancer among men and 
women in the United States (US).1 Although CRC rates have decreased over the past several years, 
progress in decreasing mortality has slowed down and morbidity/mortality rates of CRC in the US trail behind 
only lung, prostate (for men), and breast (for women) cancers.1 In 2020, the American Cancer Society 
estimates that over 145,000 people will be diagnosed with CRC and over 50,000 deaths will be attributed to 
CRC.1 Incidence and mortality related to CRC was most prominent among older populations (e.g., 50 years of 
age or older) for the past few decades, but new evidence suggests that CRC incidence has been increasing 
consistently for younger cohorts even as incidence and mortality have declined in older populations.2-4 
Racial disparities exist among those affected by CRC in terms of stage of diagnosis and mortality 
rates. CRC incidence and death rates are higher among Black Americans than non-Hispanic White 
Americans.1 While some CRC-related disparities have decreased (e.g., incidence and stage of presentation),5 
disparities continue to present in the context of CRC screening (CRCS).6 CRCS has been identified as a major 
contributing factor to recent decreases in CRC incidence and mortality.7 Thus, working to reduce CRCS 
disparities and increase overall screening rates continues to be an important public health goal. 
CRC screening (CRCS) serves as a tool for prevention and early detection, but often receives less 
public attention than other cancer screening tests. There is sufficient evidence regarding the benefits of 
CRCS that the United States Preventive Services Task Force (USPSTF) recommends CRCS for individuals 
between the ages of 50 and 75.8 The American Cancer Society recommends similar screening guidelines, with 
a caveat that starting noninvasive screening at 45 would have benefits.9 Recommendations are for people to 
complete a stool test (e.g., fecal immunochemical test or fecal occult blood test) annually, a flexible 
sigmoidoscopy every five years, or a colonoscopy every 10 years. The public health goal (e.g., Healthy People 
2020) for CRCS was to have screening rates reach 70.5% by this year, though estimates fall below that goal.10 
Currently, screening rates for Black Americans are lower than those of non-Hispanic White Americans.10 
Barriers to screening for Black Americans are varied, but include lack of accurate information about 
screening options, as well as limited access to necessary healthcare resources. Complex individual- and 
system-level factors contribute to disparities in CRCS rates for Black Americans. Individual factors include 
limited knowledge about benefits of CRCS, low perceived personal risk, feelings of fear and anxiety about 
potential diagnoses, cost of CRCS, lack of interest in CRCS, concerns about invasiveness and intrusiveness, 
as well as low perceived risk of the disease.11,12 System-level factors include inequalities in access to cancer 
screening information, limited continuity of care and lack of a physician recommendation about screening.12 
Studies also suggest that medical mistrust contributes to lower rates of CRCS among Black Americans.11 
While the factors influencing low screening adherence are complex, misinformation about risk and screening 
procedures are the most prominent modifiable individual-level factors influencing low screening rates. 
Studies suggest that supportive and information-rich social networks, both online and offline, could 
improve CRCS among Black Americans. Studies have found that social support, family influence, social 
interaction, and social network density all contribute to increased CRCS rates among Black Americans.13-15 
Having larger social networks provides more access to information and conversation, offline or online, about a 
variety of topics like cancer screening.16 People might intentionally be exposed to new information via targeted 
attempts at obtaining information (i.e., information seeking) or through their normal routines of reading the 
news, browsing online content, and talking to people in their social networks (i.e., information scanning). More 
social ties and social network members increases the likelihood of exposure to information via scanning, which 
may in turn result in a person seeking additional information. These are important to consider in the context of 
CRCS, as information scanning and seeking have been associated with cancer screening behaviors.17-19  
Older Americans use online, digital media information sources more frequently each year for seeking, 
scanning, and sharing health information about cancer screening. Black Americans were more likely to 
use social network sites to seek out colorectal cancer information compared to White Americans.20 
Additionally, Black Americans who seek information are more likely than White Americans to use that health 
information to improve their health.21 In addition, Black Americans appear to cast a wider net, reporting using 
more sources when scanning and seeking CRCS related information.22 This growing body of evidence 
indicates the importance of online sources for health information seeking and scanning about CRC and CRCS, 
but little is known about the impact of the messages that individuals are encountering on these platforms. Prior 
research has focused on the topics of such messages and their overall credibility, but not the specific 
behaviors, attitudes, or expectations they encourage. One study of Twitter messages found, across all 
analyzed content on the platform (>75,000 tweets), interest and engagement with information about CRC from 
a variety of users, with most (~85%) Twitter information about CRC scored as credible.23 Looking at Instagram 

posts about CRC, researchers discovered that content most often focused on patient support or treatment 
stories, though screening featured in content (15% of posts) as well.24 Most research on social media in cancer 
contexts is focused on describing content without examining effects of that content and has infrequently 
focused on CRC or CRCS,25 though some rigorous studies in the area examine interactivity (e.g., likes, 
comments, shares) with social media posts.26 Research on the content and volume of messages White and 
Black Americans encounter from online health information sources is still unclear—particularly regarding any 
disparities that exist about what specific information is sought, scanned, or shared by Black Americans. 
There is a critical need to understand which messages resonate among populations at-risk for specific 
diseases (e.g., CRC) and who may have concerns about engaging in early detection behaviors (e.g., 
CRCS) and may face disparities in exposure to (mis)information from online sources. Public health 
communicators have growing concerns about health misinformation on social media.27 Evidence that 
misinformation spreads more widely and easily than accurate information online28 suggests remedies may be 
needed to compete with such misinformation. For instance, researchers have used a text mining approach to 
identify misinformation about cancer screening on websites to assist health communicators to counter such 
messages publicly.29 Other researchers found that gynecologic cancer misinformation appeared in about 30% 
of messages on the Chinese platform Weibo, with more misinformation presented about treatment than 
prevention.30 That percentage mirrors work in the US on cancer information posted by parents on Facebook, 
where among posts including medical information, only 67% were accurate, with other posts being either 
inaccurate (19%) or conveying unproven treatments (14%).31 Research considering YouTube videos and 
respective page content about prostate cancer found that 77% of studied video included misinformation or 
biased content, and that there was a negative correlation between the scientific quality of a video and 
engagement (e.g., views and likes).32 Taken together, these descriptive studies suggest there is considerable 
misinformation present via social media, though none of the studies examined CRC or CRCS. Additionally, 
these descriptive studies provide insights into what information and misinformation exists in the public 
communication environment, but lack follow-up work to assess how people assess or intend to utilize this 
information in the future. So, while novel methods are being used to monitor the communication environment, 
the utility of these methods is falling short of making direct contributions to future communication efforts.  
Combining computational methods of monitoring the public information environment with 
crowdsourcing methods to evaluate public communication preferences is an important next step at 
improving health communication in dynamic online information environments. Crowdsourcing refers to 
“an online, distributed problem-solving and production model that leverages the collective intelligence of online 
communities to serve specific organizational goals” (p. xix).33 Crowdsourced data function as an important 
source of knowledge about improving public health34 and have been suggested as an outlet to improve 
decision making related to cancer self-examinations.35 Communication researchers have adopted an approach 
to using crowdsourcing in message selection, which is referred to as a wiki survey.36 Wiki surveys offer 
researchers the opportunity to evaluate and generate messages on topics of interest by presenting a pair of 
messages and asking people to choose which message they believe is a stronger argument (or some other 
criterion of interest). Participants are allowed to choose “neither,” or add their own message/argument. Recent 
communication science research, for example, requested participants to evaluate and generate arguments 
related to recreational marijuana legalization.37 This crowdsourcing approach offers an innovation in measuring 
perceived argument strength and other perceptions of messages quickly and efficiently, while also offering 
researchers and practitioners an opportunity to incorporate dynamic, user-generated content during a research 
study. The combination of computational methods to monitor content and crowdsourcing methods to provide 
an initial assessment of content offers a novel approach to study the public communication environment about 
CRC and CRCS, while testing an approach that could be scaled to different users and cancer sites over time.  
Crowdsourcing formative evaluation research offers an outlet to identify strong message content, but 
there is limited evidence on its predictive validity in identifying messages that increase screening and 
sharing intentions. Extant research supports the use of crowds to assess content quality and generate new 
content, but there is still a need to establish predictive validity of the approach to determine that crowd selected 
messages are persuasive.37 Establishing predictive validity of this approach requires consideration of how 
selected messages can affect key outcomes (e.g., CRCS intentions) and provide content that can be used for 
recommended communication strategies (e.g., targeting). Targeted delivery approaches have been successful 
promoting CRCS behaviors38 and targeting by race has been effective in promoting cancer screening 
behaviors.39 Message targeting by race involves tradeoffs for financially constrained campaigns, however, in 
that the strategy often requires more resources for message production and placement,40 and few studies 
directly compare if race-targeted content outperforms messages found in pre-testing to appeal to a broad 
cross-section of the population. The most scientifically rigorous way to establish predictive validity of 
crowdsourced formative evaluation research and test content targeting by racial group preference is to conduct 
a randomized controlled trial (RCT). Selecting messages from the crowd rankings allows for categorization of 

high-quality messages both overall and by racial group. This will allow us to design an RCT that provides direct 
evidence of predictive validity of the crowdsourcing approach for CRCS, and to determine if selecting targeted 
messages via crowdsourcing improve outcomes for primary (targeted) and secondary audience segments. 
Summary of study significance. Disparities in CRC diagnosis and outcomes, as well as suboptimal 
screening levels, may be due in part to limited knowledge and misinformation. Determining how the public 
health information environment contributes to reducing information disparities, and in turn screening disparities, 
offers an important opportunity to address this public health priority. Developing and validating a novel 
approach to monitoring and evaluating the public information environment using methodological innovations 
increasingly available to communication scientists. The proposed project directly responds to the program 
announcement focus of utilizing and applying new cancer communication surveillance approaches to 
examine public health communication about CRC prevention and control. Additionally, the project 
engages with a traditionally underserved population, Black Americans, who are at elevated risk of CRC. 
B. INNOVATION 
The present proposal’s innovation is developing an overarching framework that functions to monitor and use 
the online information and message environment to facilitate efficient, timely, evidence-based communication 
recommendations for cancer control communication efforts. Having a comprehensive framework to 
monitor, evaluate, generate, and disseminate accurate information that also demonstrates the ability to 
motivate action is a necessary next step to strategic cancer communication efforts. The proposal 
accomplishes this by using a three-step process that (1) captures and analyzes digital and social media 
information and misinformation about CRCS, (2) uses a crowdsourcing approach to evaluate and generate 
information and messages found through monitoring efforts that are most appealing and effective with specific 
population segments at greater risk for CRC, and (3) tests if the messages identified as most appealing do 
indeed improve screening and message sharing intentions among target audiences. As such, this project will 
provide evidence on the efficacy of a conceptual framework to monitor and implement messages from the 
public information environment to improve cancer communication across the cancer continuum. 
Comprehensive monitoring of online information and misinformation about CRCS. This project will offer 
a comprehensive assessment of public information about CRCS across multiple social network and related 
platforms (e.g., Twitter, YouTube, WebMD patient portal). As part of the monitoring, we will test and refine 
computational processes to identify demographic characteristics of messages shared via social media (e.g., 
age) to offer tools for future researchers to continue to improve public cancer information surveillance. 
Testing a new method for evaluating and generating messages about CRCS. Determining what 
messages and information to disseminate via social media is always a challenge for public health 
communicators. There have been few methodological innovations to engage in formative research for strategic 
health communication efforts, particularly methods that do not require intensive, in-person research activities 
like focus groups and copy testing. Given interest in exploring distance and remote outlets for conducting 
research in unprecedented pandemic conditions, our test of the wiki survey crowdsourcing approach will offer 
important evidence to determine the applicability of the method across various cancer communication topics. 
Assessing the predictive validity of the novel methodological approach. While computational methods 
have been used in some capacity in certain health contexts (e.g., tobacco-related messages)41 where 
messaging is typically recommending one specific behavior (e.g., quit), but infrequently with topics where 
recommendations are less straightforward (e.g., multiple options for screening and varied recommendations 
based on risk and demographic factors).42 Using the computational methods to feed into crowdsourced 
message testing has not been tested previously either, which is another part of our novel methodological 
approach. Finally, our randomized message testing experiment brings the entire approach together by testing 
the validity of wiki surveys to identify messages that are persuasive among key target audiences. 
C.1 APPROACH: Theoretical Basis, Design Overview, and Preliminary Studies  
Public health communication research and the need for methodological innovation offers a guide to 
testing our conceptual framework. Communication scientists have focused on integrating innovative 
methodological approaches as large amounts of public data become more easily accessible and as the ability 
to connect with audiences of interest has become easier. Our proposal applies approaches advocated by 
those interested in computational communication science43 and crowdsourcing for strategic communication 
improvement37 to improve cancer communication practice. While past work has implemented computational 
approaches in monitoring cancer prevention related social media messages,44-46 prior work compartmentalizes 
this monitoring process. Often the conclusion of that work is that health communicators will somehow improve 
their practice and ability to target a future audience of interest because of knowing more about the public 
information environment. While this certainly may be true and a significant benefit of such approaches, 
stopping the process there fails to take advantage of the wealth of information collected via these rigorous and 

innovative computational approaches that allow for more in-depth monitoring of the public information 
environment than ever before.47 Indeed, the traditional trajectory of strategic health communication efforts is to 
begin with formative research, which often includes taking inventory of what competing or contradictory 
messages exist in the public information environment. This is particularly important for messages about CRCS 
due to past research discovering that being exposed to two or three messages promoting CRCS can increase 
participation rates meaningfully.48 Messages targeted to particular audiences that share motivations and needs 
related to information are likely to be particularly effective for CRCS.38 The computational, natural language 
processing approach to monitoring social media messages is a type of formative research. The next step in 
strategic health communication efforts is formative evaluation, a process that typically involves focus groups or 
copy testing approaches.49 No existing health campaign or strategic health message research has attempted 
to use a crowdsourcing approach to distribute formative evaluation tasks among a large group of relevant 
audience members. Unlike focus group approaches that might include the perspectives of 45 to 50 people, 
comparing 5 to 6 different message possibilities, the crowdsourcing approach to formative evaluation considers 
hundreds of people’s evaluations of dozens of message possibilities. Determining the validity and scalability of 
these novel methods is essential to innovate formative research and evaluation approaches in the future. 
Design overview. For Study 2, we will contract a custom, crowdsourced wiki survey using a national sample 
of adults 45 to 74 years old—including 1000 individuals who self-identify as White or Caucasian and 1000 
individuals who self-identify as Black or African American—to choose preferred messages curated from the 
social media monitoring occurring during the first year of Study 1. Data collection will occur over two and a half 
months, with 100 people from each targeted group participating every week. This approach allows us to 
integrate messages generated by participants, as well as new messages discovered during the continuing 
monitoring of social media messages. We will include 40 to 60 messages, to be displayed randomly in pairs, 
for the first week of data collection (see Figure 1). The original message set will be supplemented with crowd-
generated messages on a weekly basis. Based on past wiki survey research,37 we expect participants to 
generate approximately 800 message suggestions of which approximately 5% (n = 40) will be novel (i.e., not 
duplicates) to the message set. We expect each participant will respond to at least 15 randomized pairs of 
messages, totaling at least 30,000 message choices crowdsourced in the study.  
C.2 APPROACH: Planning and Preparation, IRB  
We will develop study protocols that protect 
human subjects in accordance with a single-site 
IRB (at Iowa State University), in collaboration 
with IRBs at collaborating institutions, as required 
by federal regulations and university operating 
policies. Research related to Aim 1 does not 
require IRB approval because it does not involve 
human participants. Studies for all three aims will 
be registered with OSF.io, the registration website 
of The Open Science Framework, and we will 
also register the RCT (Aim 3) on clinicaltrials.gov. 
The PI will work with the survey company 
(AmeriSpeak/NORC) to program and implement 
the studies to be carried out for Aims/Studies 2 and 3 to ensure that they protect the health, safety and rights of 
human subjects. 
C.3 APPROACH: Aim 2, Crowdsourcing Message Evaluation and Generation Using Wiki Surveys 
The purpose of Aim 2 is to offer a crowdsourced alternative to traditional formative evaluation research that 
requires considerable economic and temporal resources, as well as face-to-face contact, to conduct. Wiki 
surveys, if demonstrating validity in identifying effective messages, also serve to promote rapid deployment of 
messages and information to counter widespread misinformation or contradictory public information about 
CRCS. 
Wiki Surveys. Compared to traditional formative evaluation research options, wiki surveys allow us to collect 
as much data as each participant is willing to provide (i.e., participants can choose to complete as many 
comparisons as they like). Wiki surveys also afford us the opportunity to integrate participants’ new ideas into 
the survey that may not have been captured via social media monitoring or that newly emerge over time. Wiki  
surveys, while not used frequently in communication research, have been deployed successfully since 2010. 
An online resource (https://allourideas.org) offers researchers an adaptable, open source platform to 
implement on online platforms. Another example of the wiki survey application include using the approach to 
obtain residents’ suggestions for a citywide sustainability plan.36 

A wiki survey asks participants to choose their preferred message from a pair of randomly selected messages 
(e.g., between message A and message B, randomly selected from a seeded pool of messages). In addition to 
choosing between messages, participants can also contribute new messages, which are subsequently 
presented to future participants. Using this approach, data collected from multiple pairwise comparisons from a 
large number of participants can be analyzed to determine the relative rankings of each message across the 
entire message set. For the current study, we will present participants with message choices and ask which of 
the messages offers a better reason to adhere to CRCS recommendations. 
Message Selection. The grant team will seed the initial list of choices with at least 40 to 60 messages 
selected from Study 1. Messages will be selected from the library collected during Study 1 from social media 
platforms and popular health websites. All messages presented to participants will be reviewed by the medical 
consultant on the grant (Dr. Rich Hoffman) to ensure participants are not rating messages that include any 
misleading or incomplete information. Messages will be selected by the grant team based on their review of the 
library of message and suggestions from the medical consultant who has research and practice experience 
with cancer screening and health disparities. Once the messages are selected, we will work with our 
contracted survey provider—AmeriSpeak/NORC—to input the messages into the customized wiki survey tool. 
Participants. AmeriSpeak/NORC will provide a sample that combines members of their panel and a third-party 
non-probability panel for a portion of the sample of Black Americans. This is necessary given the goal of 
having a large sample of a traditional under researched (in communication science) and underserved 
population (Black Americans between the ages of 45 and 74). For Study 2, the total recruited sample (N = 
2,000) will be evenly split between White American adults aged 45 to 74 (n = 1,000) and Black American 
adults aged 45 to 74 (n = 1,000). The reason for sampling participants from age 45 is that many screening 
recommendation options for higher risk populations—including Black Americans and individuals with other 
comorbidities—begin at age 45. Individuals who participate in Study 2 will not be eligible for Study 3. We 
estimate Study 2 will take participants approximately 15 to 20 minutes to complete. 
Procedure and Message Integration. Once participants complete informed consent procedures, 
administered electronically by the survey contractor, the survey will begin. We will first present demographics 
questions followed by the wiki survey task, embedded within the larger survey platform. We will provide 
participants with instructions about how to complete the wiki survey component, as well as basic information 
about what the research study references when it mentions colon cancer screening. Based on procedures and 
our agreement with the contractor, participants will be told they can respond to as many message pairs that 
they like. The contractor estimates that participants will likely complete 20 to 25 choice pairs based on typical 
user engagement. After the first week of data collection, the PI (King) and one co-I (Niederdeppe) will oversee 
graduate students who will sift through crowd-generated messages to remove messages that duplicate 
information already in the message set, as well as messages that might be offensive or unrelated to the study. 
We will integrate valid, novel messages generated via crowdsourcing into the wiki survey on a weekly basis. 
The wiki survey will be programmed to prioritize presentation of the newly-added messages to even out how 
many times messages are evaluated—though presented messages will continue to be randomly presented in 
pairs to participants. This will occur every week for 10 weeks until data collection is complete for Study 2. With 
each pair of messages displayed, participants will be prompted with the following question: “Which of the two 
statements below is a better reason for you to have a colon cancer screening test?” 
Measured Variables. The survey contractor will monitor and ensure data quality for participants within the 
AmeriSpeak/NORC panel and from the third-party provider. We will collect demographic information on 
participants including age, sex/gender identity, educational obtainment, ethnicity, income, employment status, 
housing type, access to health insurance, and CRCS status. Outside of demographic variables, the key 
outcomes for Study 2 will be the choices from the presented message pairs, as well as crowd-generated 
messages. We will use this information to rank the quality of messages but also provide information on how 
many messages researchers might expect to be generated by the crowd using this methodological approach, 
including considering how sex (as a biological variable) affects rankings. 
C.4 APPROACH: Specific Research Questions and Hypotheses 
Study/Aim 2 – Wiki Survey 
Wiki surveys offer a method for using the perceptions and perspectives of large groups of targeted audiences 
in evaluating and generating messages about a topic of interest. Still, less is known about how many unique 
messages researchers and practitioners are likely to generate using this approach. Related, there is scant 
evidence regarding the quality of messages generated by this approach to soliciting message ideas. Based on 
these unknowns about the utility of a crowdsourced approach to message generation via wiki surveys, we 
propose the following two research questions. 

RQ: How many unique messages about CRCS are generated by wiki surveys in the context of CRCS? 
C.6 APPROACH: Data Analysis Plan 
Analyses for Study/Aim 2. The wiki survey approach of Study 2 requires analysis and aggregate ranking of 
people’s message selections from message choice pairs. To analyze these data, we will fit a Bradley-Terry 
model. Engaging this approach allows us to generate a relative ranking of statements. More specifically, 
Bradley-Terry models use a generalized linear modeling approach to provide a log odds estimate regarding if 
one message will outperform another in a paired comparison with a one statement set as a reference item. 
Past work using wiki surveys has used the median item as the reference category,37 which is the approach we 
will use for analyzing data from Study 2. The coefficients produced by the procedure can be used to rank 
messages where better performing messages have a higher value than lower performing messages. We will 
work with the statistical expert on the project to ensure the Bradley-Terry models fit for the Study 2 data, 
including fitting separate models to consider individual differences62 for factors such as education and access 
to insurance, and to account for non-independence of observations that arise from the fact that some 
respondents will choose to rate more argument pairs than other respondents. 
Power analysis. Study/Aim 2 will fit a series of Bradley-Terry models to the choice data provided by the wiki 
survey and this analytical approach produces ranking that can be produced with varied sample sizes. Given 
the potential to test up to 100 messages by adding crowd-generated messages, the study design will display 
each message at least 300 times and all possible choice combinations (n = 9,900 assuming 100 message 
options) will be evaluated at least once, which is recommended in paired comparison data.64