Improving the Behavioural Impact of Air Quality Alerts in London
Phase: NA
Conditions: Pollution; Exposure, Health Behavior
Sponsor: King's College London

 
 
 
 
 
STUDY PROTOCOL 
Title: Improving the behavioural impact of air quality 
alerts 
NCT03552198 
 
July 2017 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
2 
 
STUDY PROTOCOL 
Title: Improving the behavioural impact of air quality alerts 
 
1 
BACKGROUND 
In 2014, with a news release, the WHO revealed that in 2012 around 3.7 million people had died prematurely 
in the world as a result of exposure to ambient air pollution. These deaths were attributed to specific 
diseases such as heart disease, stroke, chronic obstructive pulmonary disease (COPD), lung cancer and acute 
respiratory infections in children (1). In this context, it has been recommended to inform the general 
population, and in particular individuals who are more susceptible to experience symptoms (e.g. due to lung 
or heart problems), with the aim of raising awareness about air pollution and its health impact, and to 
provide advice on how to reduce exposure (2).However, the evidence shows that adherence to air quality 
advice to modify behaviours during pollution episodes is often suboptimal (D’Antoni et al., unpublished 
systematic review(3),(4)), and that the traditional strategy of simply informing people about high pollution 
episodes is not effective (5).  
The aim of this project is to improve the behavioural impact of air quality alerts. In particular, I have 
conducted a systematic literature review of the predictors of adherence to air quality alerts. This review has 
informed the development of specific communication strategies, which after being piloted, will be used with 
the users of an existing air alert smartphone App developed by the KCL ERG group. Implications of this study 
include the potential to reduce the health burden of air pollution, through the development of more 
effective communication strategies provided via existent air quality alert systems.  
 
1.1. Systematic literature review: Psychosocial predictors of adherence and non-adherence to health 
advice provided through air quality information services 
The aim of this systematic review was to understand to what extent the general public adopt protective 
behaviours in response to hearing or reading air quality warning systems and the health advice 
accompanying them, and to identify the demographic and psychosocial factors associated with adherence 
and non-adherence to the received health advice. 
1.1.1. 
Methodology 
The present review is conducted and reported in accordance with the Preferred Reporting Items for 
Systematic reviews and Meta-Analysis (PRISMA) guidelines (6), using systematic methods to identify and 
select studies, and assess their risk of bias.  
1.1.2. 
 Search Strategy 
We searched electronic databases such as Web of Science Core Collection, OVID (Global Health 1973 to 2016 
week 18), PsycINFO (1806 to May 2016), Social Policy and Practice (2016), Embase (1947 to May 2016), Ovid 
MEDLINE(R) (1946 to present), Science Direct, Scopus, Pubmed, CINAHL in August 2016. In addition, 
OpenGrey.eu, EThos, and google were used to identify relevant unpublished studies and reports (e.g. 
governmental reports). No date limit or study type limit were applied, however papers published only as 
abstracts were excluded. We only searched for literature written in English. The search strategy combined 
terms related to air quality alerts (i.e. ‘air quality alert’, ‘air quality index’, ‘air quality advisories’, ‘smog 
alert’), and adherent and protective behaviours (i.e.  ‘adherence’, ‘compliance’, ‘health behaviour’, ‘risk 
reduction’, ‘public response’). The search was conducted to make sure that both the general public and 
vulnerable population (e.g. asthmatics) were included. Additional articles were included through manually 
screening reference lists of relevant articles and reports. 
 
 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
3 
 
1.1.3. 
 Inclusion criteria 
The inclusion criteria were based on the Participants, Interventions, Outcomes, and Setting -PI(C)OS - 
approach in the PRISMA guidelines. Studies were included if they met the following criteria: 
i. 
Participants: people who read or heard of air quality reports, alerts, indices or other sources of health 
information related to air quality (e.g. users of air quality warning systems, people familiar with air quality 
forecasts). Participants could be drawn from the general public, patient groups or specific occupational groups. 
ii. 
Interventions: exposure to information about air quality and/or health advice associated with air quality levels, 
including information related to actual and/or hypothetical levels of air pollution. 
iii. 
Outcomes/Predictors: Actual and/or intended adherence/ behaviour change in response to air quality 
information,  
AND/OR 
Predictors of, and/or self-reported reasons for adherence or non-adherence to health advice associated with 
air quality information. 
 
iv. 
Study reporting: All study designs, aside from those published only as editorials or abstracts, were included.  
Studies were excluded if they met the following criteria: 
v. 
Were based on the assumption that the respondents were aware of air quality alerts Involved a population 
only assumed to be aware of air quality information services during alert days (i.e. every time an alert was 
issued in a specific area), but did not collect evidence of this; 
vi. 
Analysed behaviour change in response to air quality as driven by people’s own perception of air quality, 
without the involvement of any official information; 
vii. 
Analysed only information seeking behaviour and/or frequency of access of air quality information services, 
without investigating behavioural changes in response to this activity; 
viii. 
Investigated pro-environmental behaviours only. 
 
 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
4 
 
 
1.1.4. 
 Studies characteristics 
Design 
The majority of the studies were  conducted in the USA (7–16), whereas five were conducted in the UK (17–
21), four in Canada (22–25), one in Hong Kong (26), and one in Denmark (27). Data collection covers the 
period from 1982 to 2016. The vast majority of the included studies were cross-sectional surveys (7,10–
18,20,21,24–27). Although there was only one qualitative study (23), other studies collected some qualitative 
data through asking people to report the reasons for adherence and/or non-adherence (12,17,24,25). The 
studies included samples of general public, service users of air alert systems, asthmatic patients and people 
with other respiratory and/or heart condition, elderly, people who spend most of their time working in busy 
streets, communities involved in wildfire events, parents of healthy and parents of asthmatic children, and 
health care professionals. The majority of the studies focused on air quality forecasts and alerts with 
associated health advice, one study used a web-based asthma action plan smartphone application, and one 
study investigated responses to emergency risk communications during a local wildfire episode. Air quality 
information was provided through different public media, with almost half of the studies assessing actual 
adherence with messages received also through personal channels such as telephone messages, emails 
notifications, and smart phone applications.  
 
Outcomes (adherence and predictors) 
Adherence to health advice associated to air quality information was investigated via non-validated self-
report questionnaires or interviews in all but one study (19), which used objective emergency department 
attendances. The majority of the studies investigated actual adherence, with only three studies investigating 
intended adherence (8,9,20), and one study assessing both actual and intended behaviour (27).  
 
1.1.5. Results: adherence rates 
Actual behaviour: reducing/rescheduling outdoor activities 
In the studies investigating actual adherence to the health advice to avoid, reduce or reschedule outdoor 
activities during poor air quality, overall adherence rates ranged from 9.7% (12) to 70.7% (27), with median 
adherence rates of 36% (11). 
Actual behaviour: additional protective behaviours 
In the studies investigating a wider range of actual protective behaviours, going beyond the decision to simply 
reduce or reschedule outdoor activities, overall adherence (including also behaviours performed less than 
monthly during moderate or high pollution episodes) ranged from 17.7% (26) to 98.1% (13), with median 
adherence rates of 42% (18). The most common responses reported by all study participants included: avoiding 
busy or polluted road (with adherence rates ranging from about 10% to 52.5% (17,18,21), spending more time 
indoors (ranging from about 30% to 58.7% (13,18,21), adjusting or rescheduling travels or other outdoor 
activities (41.4%), changing means of travel (38.6%)(21), and avoiding strenuous exercise or other outdoor 
activities (ranging from 17.4% to 88.4% (13,17,18,22,26). Other behaviours included taking a reliever 
medication (ranging between 30.5% and 50% (17,18), taking a preventative medication (30.5%-38.5% (17)), 
getting advice from GP (about 1% (18,26)), and wearing a mask (6.4% to 8.1% (13,26)). 
 
Intended behaviour 
These studies showed an overall intention to adhere ranging from 36.4% (20) to about 81.8% (27) (the latter 
refers to a group of respondents with severe lung disease).  
 
1.1.6. 
Psychosocial predictors of adherence and non-adherence 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
5 
 
Table 1 categorises within the COM-B framework the main factors affecting adherence to health advice 
provided with air quality information services, as found in our systematic review. Socio-demographic factors 
are not reported here as not modifiable and therefore not relevant to this project. Full details on the 
predictors reported in D’Antoni et al. systematic review (submitted for publication). 
 
 
Table I. Factors influencing adherence to health advice provided in association with air quality information services within the COM-B 
models. 
CAPABILITY 
MOTIVATION 
OPPORTUNITY 
Psychological 
 
Knowing where to check AQHI (Air Quality 
Health Index) numbers (24) 
 
Understanding the air quality indices/ health 
messages (17,23,24) 
 
Confusion between different indices (23) 
 
Awareness of media alerts (28) 
 
Information seeking behaviour (11) 
 
 
Reflective 
Health messages able to reduce both concern about, 
and perceived susceptibility to, air pollution (9) 
Experiencing symptoms ascribed to air pollution 
(beliefs about the illness & threat) (21,27) 
Beliefs that smog can have negative health effects 
(beliefs about the health threat) (7,21) 
Beliefs that something can be done to reduce smog 
(outcome expectancies) (7) 
Perceived benefits of AQI (Air Quality Index) adoption 
(beliefs about the recommendation) (17,23,24) 
Perception of lack of necessity of AQI adoption 
(beliefs about the recommendation) (24,25) 
Lack of message relevance (23,25) 
Self-efficacy/locus of control (24,25) 
Physical 
 
Wearable device option/ smartphone 
applications (19,21,23) 
 
Exposure to visible air pollution (13,15) 
Automatic 
Depression (13) 
Reliance on sensory cues (12,17,23,24) 
 
Social 
 
Professional health care network 
promotion/GP advice (13,23,28) 
 
Neighbourhood scale focus (23) 
 
Local media reporting (23,24) 
 
Use of different sources of information (21) 
 
Necessity to continue with everyday life/ Lack 
of time (17,23,24) 
 
 
2 
THEORETICAL FRAMEWORK 
The Capability, Opportunity and Motivation (COM-B) 
framework, developed from existing theories of 
behaviour change (29), proposes that to better 
understand the determinants of behaviour we should 
consider the interactions existing between capability 
(C), opportunity (O) and motivation (M), where 
individual, group and environmental determinants are 
equally 
considered 
in 
controlling 
behaviours. 
‘Capability’ is defined as the individual’s psychological 
(e.g. knowledge, understanding) and physical capacity 
to engage in the targeted activity. ‘Opportunity’ refers 
to all the external factors that make the behaviour 
possible or prompt it, and ‘motivation’ to the mental 
processes that energise and direct behaviour, 
including also habitual processes and emotional 
responses. The COM-B framework and the BCW (Behavioural Change Wheel) can be used as a guide to both 
identify specific determinants of target behaviours as well as identify behaviour change techniques (BCTs) 
aimed at modifying these determinants and in turn the relevant target behaviours (29–31). Given our focus on 
improving written health communications, we decided to address ‘reflective motivation’ as key determinant 
of adherence behaviour (see Table 1).  Once identified this determinant, the next step involved an accurate 
Figure 1 Behavioural Change Wheel (Michie et al. 2011) 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
6 
 
analysis of different intervention functions aiming at addressing the targeted determinant – as reported by 
Michie and colleagues (29)-. We established to adopt two main functions: education, and persuasion, where 
education refers to increasing knowledge and understanding, persuasion refers to the use of communication 
to induce positive or negative feelings or stimulate action. Table 2 shows the messages we are going to use. 
These alternative health messages were developed based on the psychosocial factors identified in the 
systematic review earlier discussed.  
In addition, these alternative messages were developed to provide more specific health advice. This is based 
on evidence showing  the importance of designing health risk communications that provide specific, and clear 
information (32), so that sufficient information is provided to enable decisions about health protection. 
Specificity refers to the extent to which a message provides a detailed description of the recommended 
behaviour (387).   
A meta-analysis of 18 studies (388) found that messages providing health recommendations with a more 
specific description seem to be significantly more persuasive than generic recommendations (r=.10, k=18, 
N=11,105). For instance, Leventhal and colleagues (389) found that health messages recommending tetanus 
vaccination that provided a more explicit description of the steps to take towards the recommended action 
were more persuasive than less detailed information. In addition, Frantz found that adding procedurally 
explicit precaution for safe use of products (e.g. ‘Open windows to vent vapours to outdoors’ or ‘Wear rubber 
gloves and protective glasses) increased adherence to recommendations on how to use these products, 
compared to more generic instructions (e.g. ‘Use in a well-ventilated area’ or  ‘Avoid contact with eyes and 
skin’) (391). As to why message specificity seems to be more effective, there are no conclusive explanations, 
however some potential mechanisms have been suggested. It is plausible that more specific descriptions of 
recommended behaviours make it easier for the targeted audience to imagine themselves performing that 
action, which in turn enhances persuasiveness (388). Moreover, as a person imagines themselves performing 
a specific action, their perceived ability to engage with that specific behaviour (i.e. self-efficacy) might be 
enhanced, and in turn this would increase actual adherence (388).  
 
 
4 
RESEARCH QUESTION/AIM(S) 
The purpose of the present study is to test whether theory and evidence-based alternative communication 
formats, targeting message specificity and previously identified psychosocial predictors of adherence, could 
improve the behavioural impact of existing alert systems, compared to the official messages sent in association 
with the UK AQIs. 
 
Research questions 
Primary questions: 
1) What is the main effect of using behaviourally enhanced messages, compared to the currently used DAQI 
messages, to present the health advice associated with air quality notifications for a hypothetical high air 
pollution scenario on adherence intentions? 
Prediction: 
A) The behaviourally enhanced messages will lead to stronger intentions to adhere to recommendations. 
2) What is the main effect of using a behaviourally enhanced message, compared to the currently used DAQI 
messages, to present the health advice associated with air quality notifications on actual behaviour changes 
at four weeks? 
Prediction: 
A) The enhanced messages will lead to greater behaviour changes. 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
7 
 
3) In case of a real alert being issued during the study period, how do the different messages affect actual 
behaviour change? 
Prediction: 
A) The enhanced messages will lead to greater behaviour change. 
Secondary questions: 
4) If there is a format effect, which variables mediate the relationship between the information format and 
behaviour change? 
5) Which factors are associated with greater behaviour change across all groups? 
 
5. METHODS 
5.1 DESIGN 
This is a randomised control trial using a 2-way factorial design, with target population (2 levels: general 
population vs. individuals with a pre-existing health condition) and message format (2 levels: usual message 
format vs. behaviourally enhanced messages) as between-factors. Participants will be randomised to either 
the usual message group or the enhanced message.  

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
8 
 
 
 
5.2 
STUDY SETTING 
Participants will be able to complete the relevant questionnaires at home (Via the app on their own 
smart device) 
 
5.3 
SAMPLE  
Inclusion criteria  
Sent only when 
an actual air 
quality alert is 
issued 
Sent for 3 
weeks 
independently 
from air quality 
levels
Eligible participants accept to take part in the study 
They are asked to indicate whether they have a pre-existing 
health condition in order to receive targeted health advice 
about air pollution 
General public 
At risk individuals 
Usual health 
advice 
Usual health 
advice 
Alternative 
health advice 
Alternative 
health advice 
Additional 
messages 
targeting 
beliefs 
Additional 
messages 
targeting 
beliefs
High air pollution hypothetical scenario (the same for all 
conditions) with targeted advice for general public and at risk 
Initial questionnaire measuring intentions to change behaviours 
in relation to the hypothetical scenario 
Final questionnaire measuring intentions (using the same hypothetical 
scenario and health advice as above), actual changes in the past 4 weeks, and 
beliefs about air pollution and relative health advice 
Questionnaire measuring actual behaviour change in response to 
receiving a real air quality alert 
4 WEEKS
General public- 
Control 
General public- 
Intervention 
At risk- 
Intervention 
At risk -Control 
Usual health 
advice 
Usual health 
advice 
Alternative 
health advice 
Alternative 
health advice 
Randomisation
Figure 2 Study flow diagram 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
9 
 
Inclusion: 
 
Users of air alert systems managed by the ERG 
 
Age: > 18 years 
 
Working or living in Greater London. 
 
 
Exclusion criteria  
 
younger than 18 years 
 
not working or living in Greater London 
 
no longer users of the air quality alert smartphone application. 
 
 
Sampling 
Sample size was calculated using the G*Power statistical power analysis (32), to give 80% power to detect 
a statistically significant difference in the main outcome measures at α=0.05, if a small to medium effect 
size of f=0.02 or higher is observed (9), adjusting for one covariate. Given the possibility of dropout, we 
inflated the sample size by 20%, and aimed to achieve a minimum sample size of n=240.  
 
5.4 
STUDY PROCEDURE 
5.4.1 CityAir smartphone application 
This is an application designed and developed by the City of London Corporation and King's College 
London. The CityAir app version 2.0.4 (available at: 
https://www.cityoflondon.gov.uk/business/environmental-health/environmental-protection/air-
quality/Pages/New-CityAir-App.aspx) is currently available for iOS and Android and is compatible with 
iPhone, iPad and iPod touch and Android devices. This application allows users to sign up for air pollution 
alerts and find less polluted routes when levels of air pollution are high in London. In addition, it provides 
users with targeted advice for generally healthy and at risk individuals on how to reduce exposure to air 
pollution. Such advice is based on the official UK AQIs. The application also provides simple advice for 
users registered as cyclists, pedestrian, jogger, drivers, and businesses on how to reduce emissions and 
exposure (e.g. advice for drivers: ‘Turn off your engine when parked’; for joggers: ‘Consider delaying your 
jog till pollution levels are lower’; for businesses: ‘Check out the CityAir guides to help lower pollution 
levels in the City’).  
For the purpose of the current study, KCL will launch an alternative version, which will allow the 
researchers to send different types of messages for the control and intervention groups during the study 
period. To increase message relevance, targeted advice will be sent to people with lung and/or heart or 
other existing condition, and the generally healthy public (see Table 3). It is worth noting that the current 
UK AQI does not provide targeted advice for people with heart conditions apart from reducing exertion, 
and the main advice for people with lung conditions is to consider using their reliever inhaler more often. 
Usually CityAir sends alerts for moderate, high and very high air pollution to users in the at-risk group, 
whilst sending only alerts for high and very high air pollution to users in the other groups. However, for 
the purpose of the current study, the alternative version of the application also will send moderate alerts 
and associated advice to the general population group. This will allow to actually test the UK AQI advice in 
case of a real moderate air pollution episode during the study period. To keep the study design simple, we 
will not differentiate between cyclists, joggers, and pedestrians. However, general messages also will 
include advice on how to adjust these types of physical activity to reduce exposure. Personal device IDs 
(i.e. an anonymous and unique identifier attributed to an individual mobile device) will be used to link 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
10 
 
questionnaires completed at different time points by the same study participant. At the end of the study, 
a new version of CityAir will be issued to allow the application to go back to its usual functions.  
 
 
5.4.2 
 Recruitment and study procedure 
All CityAir application users will be sent a notification about the current study containing a link to the 
participant information sheet. If eligible and happy to take part in the study, then they will be redirected 
to an online survey, using a well-known online platform (https://www.surveymonkey.co.uk/).  
Once potential participants had agreed to take part, they will be asked to indicate whether they had a 
pre-existing health condition. Based on their answers, they will be divided into two groups (general public 
and at risk respondents). Respondents in both groups will be randomised via an algorithm run by CityAir 
to either a control or intervention condition.   
Participants will be asked to read the scenario of a hypothetical high pollution episode and to indicate 
their intentions to follow the health advice given (i.e. baseline adherence intentions measure). After 
completion of the first questionnaire, all participants will be able to received real-time CityAir 
notifications about real air pollution episodes. Whilst the control group will receive air quality notification 
and associated health advice in the usual UK AQI format, the intervention group will receive health advice 
in an alternative format (see Materials).  
 
5.5 MATERIALS: 
Table 2 shows the health advice accompanying air quality alerts in both the usual and alternative format 
(i.e. targeting message specificity). Table 3 shows some of the additional messages that will be sent to the 
intervention group and targeting specific psychosocial factors identified in our previous systematic 
review. Readability scores were calculated for all messages (using a readability tool available at: 
https://www.webpagefx.com/tools/read-able/check.php) and reported in Tables 2-3. To reduce the 
likelihood of confounding effects, the alternative messages have a similar word count and identical 
readability scores of 8 (i.e. they should be easily understood by 13 to 14 years old). 
 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
11 
 
Table 2 The health advice associated with the current UK AQI compared with the alternative/more detailed health messages developed for this study. 
  Level of pollution 
Usual messages of UK AQI* 
Alternative messages (targeting message specificity) 
General public  
At risk 
General public  
At risk  
Moderate 4-6 
Enjoy your activities as 
usual a 
Adults and children with lung problems, and 
adults with heart problems, who experience 
symptoms, should consider reducing strenuous 
physical activity, particularly outdoors. b 
Enjoy your activities as usual. 
However, if you wanted you, you 
can reduce your exposure to 
pollution by reducing levels and 
length of physical activity outdoors.  
 
You can also consider changing: 
• Travel route or 
• Exercise location (e.g. use our app 
to find less polluted roads or parks) 
or 
• Time (e.g. mornings or less 
polluted times). g 
Adults and children with lung 
problems, adults with heart 
problems, and older people, should 
reduce levels and length of physical 
activity outdoors.  
 
Where possible, change: 
• Travel route or 
• Exercise location (e.g. use our app 
to find less polluted roads or parks) 
or 
• Time (e.g. mornings or less 
polluted times). h 
High 7-9 
Anyone experiencing 
discomfort such as sore 
eyes, cough or sore 
throat should consider 
reducing activity, 
particularly outdoors. c 
Adults and children with lung problems, and adults 
with heart problems, should reduce strenuous 
physical exertion, particularly outdoors, and 
particularly if they experience symptoms. People 
with asthma may find they need to use their 
reliever inhaler more often. Older people should 
also reduce physical exertion. d 
Anyone who experiences sore eyes, 
cough or sore throat should consider 
limiting their exposure to air 
pollutants.  
 
To do this, you don’t need to stay 
indoors, instead you can reduce 
levels and length of physical activity 
outdoors.  
 
You can also consider changing: 
• Travel route or 
• Exercise location (e.g. use our app 
to find less polluted roads or parks) 
or 
Adults and children with lung 
problems, adults with heart 
problems, and older people, should 
reduce levels and length of physical 
activity outdoors.  
 
People with asthma may find they 
need to use their reliever inhaler 
more often.  
 
Where possible, change:  
• Travel route or 
• Exercise location (e.g. use our app 
to find less polluted roads or parks) 
or 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
12 
 
• Time (e.g. mornings or less polluted 
times). i 
• Time (e.g. mornings or less 
polluted times). i 
Very high 10 
Reduce physical exertion, 
particularly outdoors, 
especially if you 
experience symptoms 
such as cough or sore 
throat. e 
Adults and children with lung problems, adults 
with heart problems, and older people, should 
avoid strenuous physical activity. People with 
asthma may find they need to use their reliever 
inhaler more often. f 
Reduce levels and length of physical 
exertion, in particular outdoors, even 
more so if you experience symptoms 
such as cough or sore throat.  
 
Where possible, change: 
• Travel route or 
• Exercise location (e.g. use our app 
to find less polluted roads or parks) 
or 
• Time (e.g. mornings or less polluted 
times). l 
Adults and children with lung 
problems, adults with heart 
problems, and older people, should 
avoid intense physical activity.  
 
People with asthma may find they 
need to use their reliever inhaler 
more often.  
 
Where possible, change: 
• Travel route or 
• Exercise location (e.g. use our app 
to find less polluted roads or parks) 
or 
• Time (e.g. mornings or less 
polluted times). i 
Note: For each message word count and readability score are reported (i.e. higher scores indicate more complex sentences). a. Word count: 5, readability score: 12 (It should be easily understood by 17 to 18 year olds); b. 
Word count: 22, readability score: 20; c. Word count: 17, readability score: 18 (understood by 23 to 24 year olds); d. Word count: 45, readability score: 14 (understood by 19 to 20 year olds); e. Word count: 16, readability 
score: 16 (understood by 21 to 22 year olds); f. Word count: 32, readability score: 12; g. Word count: 53, readability score: 7 (understood by 12 to 13 years old); h. Word count: 49, readability score: 8 (understood by 13 to 14 
years old); i. Word count: 62, readability score: 8; l. Word count: 50, readability score: 8. These messages were reviewed by both the ERG and PPI groups. 
 
 
 
 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
13 
 
Table 3 A few examples of additional messages developed and aiming to target specific psychosocial factors. 
Targeted Variable 
General population 
At risk - other 
At risk: lung-specific 
At risk: heart-specific 
At-risk: combo 
 
Response efficacy 
  
  
 
Taking a side street route cuts a 
person's exposure to air pollution 
by half 
 
Taking a side street route cuts a 
person's exposure to air 
pollution by half 
 
Taking a side street route cuts a 
person's exposure to air pollution 
by half 
 
Taking a side street route cuts a 
person's exposure to air pollution 
by half 
 
Taking a side street route cuts a 
person's exposure to air pollution by 
half 
  
  
If you have a preventer inhaler, 
using it every day will reduce the 
inflammation that causes 
asthma.  Your reliever inhaler will 
help to ease symptoms straight 
away. 
  
If you have a preventer inhaler, using 
it every day will reduce the 
inflammation that causes 
asthma.  Your reliever inhaler will 
help to ease symptoms straight away. 
 
Response   costs (i.e. 
negative consequences 
associated with the 
recommended behaviour) 
  
  
The CityAir App can help you find 
side street routes, which do not 
necessarily add to your total 
travel time, and may give you a 
more pleasant journey.  
The CityAir App can help you find 
side street routes, which do not 
necessarily add to your total 
travel time, and may give you a 
more pleasant journey.  
The CityAir App can help you find 
side street routes, which do not 
necessarily add to your total travel 
time, and may give you a more 
pleasant journey.  
The CityAir App can help you find 
side street routes, which do not 
necessarily add to your total 
travel time, and may give you a 
more pleasant journey.  
The CityAir App can help you find side 
street routes, which do not 
necessarily add to your total travel 
time, and may give you a more 
pleasant journey.  
Before you go out walking, 
jogging or cycling check our App 
to find less polluted areas where 
exercising. Start by exploring 
green areas and parks close to 
your house or office. 
Before you go out walking, 
jogging or cycling check our App 
to find less polluted areas where 
exercising. Start by exploring 
green areas and parks close to 
your house or office. 
Before you go out walking, jogging 
or cycling check our App to find 
less polluted areas where 
exercising. Start by exploring green 
areas and parks close to your 
house or office. 
Before you go out walking, 
jogging or cycling check our App 
to find less polluted areas where 
exercising. Start by exploring 
green areas and parks close to 
your house or office. 
Before you go out walking, jogging or 
cycling check our App to find less 
polluted areas where exercising. Start 
by exploring green areas and parks 
close to your house or office. 
 
 
People with asthma may worry 
about using their preventer 
inhalers every day. However, you 
should know that these 
medications are safe to use daily. 
They should also reduce your need 
for the reliever inhaler  
 
People with asthma may worry about 
using their preventer inhalers every 
day. However, you should know that 
these medications are safe to use 
daily. They should also reduce your 
need for the reliever inhaler  

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts v.3 
14 
 
The table reports the specific variable targeted by the messages, and who is the target of the messages. Although the majority of massages targeting the at-risk group are identical, some are slightly adjusted depending on 
whether people had a pre-existing lung and/or heart disease, or if they report to be at risk for other conditions including old age. This is to make sure that participants are not receiving messages irrelevant to them, as this 
may lead to confusion, loss of interest, and mistrust (e.g. (33)). The average readability scores were 9.3 (i.e. easily understood by 14 to 15 year olds) and 9.8 (i.e. easily understood by 15 to 16 year olds), for the general 
population and at-risk groups (all messages considered) respectively. These messages were reviewed by both the ERG and PPI groups.

 
 
 
5.6 MEASURES 
 
Table 4. Summary of measures.  
Timing 
Measure 
How it will be measured 
First Online 
Questionnaire 
– After reading hypothetical 
high air pollution scenario 
Baseline intentions a 
to adopt protective 
behaviours in 
response to 
hypothetical, high air 
pollution alert 
Assuming the situation described was happening right now, how much would you agree 
with the following statements? 
- I intend to follow the recommendations received with the air alert to reduce exposure 
to air pollution 
- I will avoid going outdoors 
- I intend to reduce length or level of my physical activity outdoors 
- I intend to change my travel route  
- I intend to change my exercise location 
- I intend to change the time when I travel  
- I intend to change the time when I exercise outdoors  
Baseline medication 
adherence 
intentions a 
If you have lung problems:  
- I intend to use my preventer inhaler daily  
- I intend to carry my reliever inhaler with me 
Other baseline 
intention a  
(not advised 
behaviours) 
- I will wear a mask as a protection from air pollution 
- I intend to reduce length or level of my physical exercise indoors 
Previous protective 
behaviour b 
- In the past 4 weeks, how often have you taken action to reduce exposure to air 
pollution, I response to hearing or reading an air quality forecast? 
Planning 
- In the last 4 weeks, to reduce your exposure to air pollution, have you considered 
making permanent changes to your daily travel route or exercise place and time?  c 
- In the last 4 weeks, how often have you checked air quality information before doing 
physical activity outdoors? b 
Physical activity d 
- To stay healthy, the NHS recommends at least 150 minutes of moderate physical 
activity (e.g. cycling, fast walking, swimming) every week  (e.g. 30 minutes 5 days a 
week). Based on this, how physically active were you in the last week? 
Symptoms c 
- Have you ever experienced symptoms caused by air pollution? 
Air Alert period – 
Online questionnaires 
Alert recall e 
- Recently we sent you an air quality alert about poor air quality. What was the level of 
air pollution reported? 
Actual Behaviour 
change c 
In response to receiving the air quality alert: 
- I reduced length or level of my physical activity outdoors 
- I changed my travel route  
- I changed my exercise location 
- I changed the time when I travelled  
- I changed the time when I exercised  outdoors  
- If you answered no, please report the reasons [text box]. 
Medication 
adherence c 
If you have lung problems: 
- I used my preventer inhaler daily  
- I carried my reliever inhaler with me when going outdoors. 
- If you answered no, please report the reasons [text box]. 
Final Online Questionnaire –  
at 4 weeks 
Behaviour change at 
4 weeks b 
- In the past 4 weeks, how often have you taken action to reduce exposure to air 
pollution, I response to hearing or reading an air quality forecast? 
Physical activity d 
- To stay healthy, the NHS recommends at least 150 minutes of moderate physical 
activity (e.g. cycling, fast walking, swimming) every week  (e.g. 30 minutes 5 days a 
week). Based on this, how physically active were you in the last week? e 
Action planning 
- In the last 4 weeks, to reduce your exposure to air pollution, have you considered 
making permanent changes to your daily travel route or exercise place and time?  c 
- In the last 4 weeks, how often have you checked air quality information before doing 
physical activity outdoors? b 
Worry a 
 
Adapted from Witte et al. (34) 
- The information received through the CityAir app made me worry about the 
possibility of suffering health effects from exposure to air pollution 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts 
v.3 
 
   
 
16 
 
- The information received through the CityAir app made me nervous and tense about 
the possibility of suffering health effects from exposure to air pollution 
Severity a 
Adapted from Witte et al. (34) 
- Air pollution is a severe threat to my health 
Conditional 
susceptibility f 
 
- How likely do you think you are to suffer from health effects due to air pollution if you 
do not take any action to reduce exposure?  
- How likely do you think people of your same age and sex are to suffer from health 
effects due to air pollution if they do not take any action to reduce exposure? 
Response efficacy a 
Adapted from Witte et al. (34) 
- Following the health advice received through the CityAir app is effective in protecting 
me from exposure to air pollution  
Medication response 
efficacy a 
 
If you have lung problems: 
- Taking my preventer medications daily is effective in helping me to control my 
asthma 
- Carrying my reliever medication with me is an effective way to protect my health 
Self-efficacy a 
Items adapted from Witte et al (34) and Rhode et al (35) 
- I am confident I would be able to follow the health advice received through the 
CityAir app, if I wanted to  
Medication self-
efficacy a 
 
Items adapted from Witte et al (34) and Rhode et al (35) 
- If you have lung problems: 
I am confident I would be able to take my preventer inhaler every day , if I wanted to 
- I am confident I would be able to remember to carry my reliever inhaler with me 
during days of poor air quality, if I wanted to 
Response costs a 
- I do not have enough time to follow the health advice received through the CityAir 
app 
- Taking side roads makes my journey too long 
Medication response 
costs a  
If you have lung problems: 
- Using the preventer inhaler daily is not safe 
- Using the preventer inhaler daily makes me dependent on it 
Unwanted 
consequences of 
intervention a 
- In the last 4 weeks I stopped exercising altogether due to receiving alerts about poor 
air quality 
- In the last 4 weeks, I made an emergency /unplanned visit to A&E or visited my GP 
due to symptoms caused by air pollution 
Sensory cue 
prevalence a 
- I trust my own perception of air quality more than any official air quality alert 
Credibility g  
- Meyer’s credibility index (36,37) (α=.67) 
Clarity h 
- Overall, how clear were the messages received? 
Perceived informed  
choice a 
- I received enough information to make an informed choice on how to reduce my 
exposure to air pollution 
Intentions a to adopt 
protective behaviour 
in response to 
hypothetical, high air 
pollution alert 
[After reading again the high air pollution hypothetical scenario and tailored advice] 
- I intend to follow the recommendations received with the air alert to reduce exposure 
to air pollution 
- I will avoid going outdoors 
- I intend to reduce length or level of my physical activity outdoors 
- I intend to change my travel route  
- I intend to change my exercise location 
- I intend to change the time when I travel  
- I intend to change the time when I exercise outdoors  
Medication 
adherence 
intentions a 
If you have lung problems: 
- I intend to use my preventer inhaler daily 
- I intend to carry my reliever inhaler with me 
Other intentions (not 
advised behaviours) 
a 
- I will wear a mask as a protection from air pollution 
- I intend to reduce length or level of my physical exercise indoors 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts 
v.3 
 
   
 
17 
 
 
Notes: a Measured on 9-point scales, where 1=strongly disagree to 9=strongly agree; b From 1 Not at all to 9 all of the time; c Possible answers: yes, no, 
not sure; d From 1 Not at all, to 7 more than 150 minutes; e  Possible answers: Low, Moderate, High, Very High, Not sure, I did not receive any alert; f From 
1=not at all likely to 9=extremely likely; g on a 5-point scale; h 1= not clear at all to 9= extremely clear. Demographic data will be collected for all the 
participants in the first questionnaire. 
 
 
6. STATISTICAL ANALYSES 
6.1 Quantitative data 
ANCOVAs will be performed for all the intentions measures in relation to the high air pollution hypothetical 
scenario, adjusting for baseline measures collected about one month earlier (target population and 
intervention/control group entered as fixed factors). Both main and interaction effects will be tested. ANCOVAs 
will also be performed for the actual behaviour change at 4 weeks (with the intervention/control group 
entered as fixed factor). These latter analyses will also be ran in the subgroups of healthy and at-risk 
participants. Chi square tests will be performed to analyse differences in proportion of self-reported actual 
behaviour change between groups, in relation to a real moderate air pollution episode. A two-way MANOVA 
(1000 bootstraps) will test whether there are differences between groups (both main effects and interactions) 
in perceived susceptibility, worry, severity, perceived response efficacy, perception of time, self-efficacy, trust, 
clarity, credibility and perceived informed choice measures. Multiple linear regressions will be used to assess 
the associations between main predictor variables and self-reported behaviour change at 4 weeks. Mediations 
effects of predictor variables on significant behavioural changes will be tested using PROCESS macro (38). 
 
6.2 Qualitative data 
Qualitative data will be subject to content analysis, with the aim of summarising and systematising the data 
(39). Respondents’ answers will be read repeatedly to familiarise the researcher with the data. Deductive 
analysis will be used to identify barriers and facilitators to behaviour change in response to receiving air quality 
alerts. The identified factors will be categorised within the components of the COM-B model (29) to facilitate a 
comparison with previous literature. Triangulation with a member of the research team will be used to validate 
the themes developed. 
 
7   
Consent 
As this research uses online surveys, no written consent will be taken. However, all potential participants will be 
provided with full details in the participant information sheet (PIS), which clearly states that participation is 
entirely voluntary, and that completing any surveys online will indicate their consent to participate.  
The PIS will be made available to potential participants both when first contacted via the app. They will have all 
the time they want to consider participation, and to get in touch with the researchers for questions and 
clarifications.  
Participants will be also informed that while they can withdraw from participation at any time, once they have 
submitted their answers, they will not be able to withdraw their data. 
8 
ETHICAL AND REGULATORY CONSIDERATIONS 
8.1 
Assessment and management of risk 
As part of the study participants will be presented with one of two versions of health advice associated with 
real-time air quality notifications. All registered users who accept to take part in the study will receive real-time 
air quality notifications as usual. The control group will receive associated health advice in the usual format, 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts 
v.3 
 
   
 
18 
 
whilst the intervention group will receive advice in an alternative format. Which version of the advice they are 
given will be determined by random. Because we do not want to bias participants, we will not tell them exactly 
which parts have been changed. In line with the same need to avoid biasing participants, we will withhold one 
of the details of the hypothesis under test (i.e., that we want to specifically test whether the change in wording 
will have an impact on their intention and actual responses to the information received).  
We have no reasons to believe that this may lead to any discomfort or known risks. However, to obviate any 
potential risk arising from this, we will provide a debriefing at the end of the study (via the online survey or via 
the app) and provide participants with all the researchers' contact details for doubts or concerns. The 
debriefing will explain the study hypotheses, methods and potential implications, and thank them for their 
contribution. 
8.2  
Research Ethics Committee (REC) review & reports 
 
Ethics approval granted by the BDM Research Ethics Panel at King’s College London on the 2nd March 
2017 [ref:  LRS-16/17-4286]. 
 
All correspondence with the REC will be retained. 
 
8.4  
Patient & Public Involvement 
All research materials and questionnaires will be reviewed by experts from the ERG, and tested through Patient 
and Public Involvement (PPI) workshops. In particular, members of a PPI panel will review the Participant 
Information Sheet, the study procedure and participants' proposed involvement, and all the study material that the 
participants will read.  
 
8.7 
Data protection and patient confidentiality  
In accordance with the UK Data Protection Act 1998, to the best of our ability, participants' answers in this 
study will remain confidential. The questionnaires are anonymous – we will not ask for participants' name. We 
will be using the data from participants in aggregate. The responses of individual participants cannot be 
identified, and any reports or publications from this research will provide only broad descriptions of the 
sample, for example, noting the age, gender, ethnicity, occupation and health status. 
As with any online related activity the risk of a breach is minimal but always possible, however we will minimize 
any risks by storing electronic data in a secure password-locked computer files. No data will be accessed by 
anyone other than the research team. Anonymity of the material will be protected by using anonymous 
participant ID numbers. Any electronic data containing sensitive confidential data (including email addresses) 
will be stored in password-protected files in King's College computers and deleted after data collection is 
complete. 
 
9 
DISSEMINIATION POLICY 
The research data will be used for publication in DD’s PhD thesis, in scientific journals, and conferences. 
This will be clearly stated in the Participant Information Sheet. 
 
10.2  
Appendix 1 – Schedule of Procedures  
 
Procedures 
 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts 
v.3 
 
   
 
19 
 
Screening 
Baseline 
Weeks 1 to 3 
Week 4 
Informed consent 
x 
 
 
 
Demographics 
 
x 
 
 
Health status 
 
x 
 
 
Baseline intention 
 
x 
 
 
Actual adherence 
 
 
x 
x 
End of study intentions 
 
 
 
x 
 
 
 
REFERENCES 
 
1.  
World Health Organization (WHO). Burden of disease from air pollution [Internet]. 2014 [cited 
2017 Mar 18]. Available from: 
http://www.who.int/phe/health_topics/outdoorair/databases/FINAL_HAP_AAP_BoD_24March
2014.pdf?ua=1. 
2.  
NICE Public Health and Social Care Guidelines Team. Air pollution: outdoor air quality and 
health: Draft for consultation [Internet]. 2016 [cited 2017 Mar 18]. Available from: 
https://www.nice.org.uk/guidance/indevelopment/gid-phg92 
3.  
D’Antoni D, Smith L, Auyeung V, Weinman J. Psychosocial and demographic predictors of 
adherence and non-adherence to health advice accompanying air quality warning systems: A 
systematic review. (Unpublished). 2017;  
4.  
Oltra C, Sala R. Communicating The Risks Of Urban Air Pollution To The Public. A Study Of 
Urban Air Pollution Information Services. Rev Int Contam Ambient. 2015;31(4):361–75.  
5.  
Johnson BB. Experience with urban air pollution in Paterson, New Jersey and implications for 
air pollution communication. Risk Anal. 2011/09/03. 2012;32(1):39–53.  
6.  
Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred Reporting Items for Systematic 
Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med [Internet]. Public Library of 
Science; 2009;6(7):e1000097. Available from: 
http://dx.doi.org/10.1371%2Fjournal.pmed.1000097 
7.  
Evans GW, Colome SD, Shearer DF. Psychological reactions to air pollution. Env Res. 
1988/02/01. 1988;45(1):1–15.  
8.  
Johnson BB. Gender and race in beliefs about outdoor air pollution. Risk Anal. 2002;22(4):725–
38.  
9.  
Johnson BB. Communicating air quality information: Experimental evaluation of alternative 
formats. Risk Anal. 2003;23(1):91–103.  
10.  
McDermott M, Srivastava R, Croskell S. Awareness of and compliance with air pollution 
advisories: A comparison of parents of asthmatics with other parents. J Asthma. 
2006;43(3):235–9.  
11.  
Reams MA, Lam NS, Cale TM, Hinton CM. Applying a community resilience framework to 
examine household emergency planning and exposure-reducing behavior among residents of 
Louisiana’s industrial corridor. J Emerg Manag [Internet]. 2013;11(2):107–20. Available from: 
http://shibboleth.ovid.com/secure/?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medl&AN=
24180091 
12.  
Semenza JC, Wilson DJ, Parra J, Bontempo BD, Hart M, Sailor DJ, et al. Public perception and 
behavior change in relationship to hot weather and air pollution. Environ Res [Internet]. 
2008;107(3):401–11. Available from: 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts 
v.3 
 
   
 
20 
 
http://www.sciencedirect.com/science/article/pii/S0013935108000704 
13.  
Sugerman DE, Keir JM, Dee DL, Lipman H, Waterman SH, Ginsberg M. Emergency Health 
Risk Communication During the 2007 San Diego Wildfires: Comprehension, Compliance, and 
Recall. J Health Commun. 2012;17(6):698–712.  
14.  
Wen XJ, Balluz L, Mokdad A. Association Between Media Alerts of Air Quality Index and 
Change of Outdoor Activity Among Adult Asthma in Six States, BRFSS, 2005. J Community 
Health. 2009;34(1):40–6.  
15.  
Mansfield  C. C. C, Research Triangle Park NRTII. Task 4: Analysis of Survey Data on Ozone 
Alert Days. Final Report [Internet]. Research Triangle Park, NC: RTI International.; 2003. 
Available from: https://www.rti.org/publication/task-4-analysis-survey-data-ozone-alert-days 
16.  
KHIP [Kentucky Health Issues Poll]. Air quality concerns and responses to alerts [Internet]. 
2012. Available from: https://www.interactforhealth.org/kentucky-health-issues-poll-archive 
17.  
Hartill S. (Personal communication). Environmental Health Service. Air Alert User 
Questionnaire.  
18.  
Kilbane-Dawe IJ, Potter S, Johnson L, Carruthers DJ, Honore C, Stidworthy A. First results 
from Croydon airtext. In: European Space Agency, (Special Publication) ESA SP [Internet]. SP-
636. 2007. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-
36448995468&partnerID=40&md5=204f7af315c914d21fc3dfc493cb8109 
19.  
Lyons RA, Rodgers SE, Thomas S, Bailey R, Brunt H, Thayer D, et al. Effects of an air 
pollution personal alert system on health service usage in a high-risk general population: A 
quasi-experimental study using linked data. J Epidemiol Community Health [Internet]. 2016; 
Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-
84971536979&partnerID=40&md5=86baaf32c316dcefb7171de9cf4b9246 
20.  
Smallbone K. Report: Individuals interpretation of air quality information: customer insight and 
awareness study [Internet]. Department for Environment Food & Rural Affairs; 2010. Available 
from: https://uk-air.defra.gov.uk/library/reports?report_id=719 
21.  
Laube M. Changing mobility through health information on the mobile internet. Vol. MSc, 
Oxford Internet Institute. University of Oxford; 2012.  
22.  
Licskai C, Sands TW, Ferrone M. Development and pilot testing of a mobile health solution for 
asthma self-management: asthma action plan smartphone application pilot study. Can Respir J. 
2013/08/13. 2013;20(4):301–6.  
23.  
Radisic S, Newbold KB. Factors influencing health care and service providers’ and their 
respective “at risk” populations’ adoption of the Air Quality Health Index (AQHI): a qualitative 
study. BMC Health Serv Res. 2016;16:11.  
24.  
Radisic S, Newbold KB, Eyles J, Williams A. Factors influencing health behaviours in response 
to the air quality health index: a cross-sectional study in Hamilton, Canada. Environ Heal Rev 
[Internet]. Canadian Institute of Public Health Inspectors; 2016;59(1):17–29. Available from: 
http://dx.doi.org/10.5864/d2016-002 
25.  
Stieb DM, Paola J, Neuman K. Do smog advisories work? Results of an evaluation of the 
Canadian Smog Advisory Program. Can J Public Heal Can Sante Publique. 1996/05/01. 
1996;87(3):166–9.  
26.  
Mak KL, Linh TVP, Hung WT. Public appraisal of Air Pollution Index - A case study in Hong 
Kong. Proceedings of the First International Conference on Sustainable Urbanization (Icsu 
2010). 2010.  
27.  
Skov T, Cordtz T, Jensen LK, Saugman P, Schmidt K, Theilade P. Modifications of health 
behaviour in response to air pollution notifications in Copenhagen. Soc Sci Med. 1991/01/01. 
1991;33(5):621–6.  
28.  
Wen X-JJ, Balluz L, Mokdad A. Association Between Media Alerts of Air Quality Index and 
Change of Outdoor Activity Among Adult Asthma in Six States, BRFSS, 2005. J Community 
Health [Internet]. 2009;34(1):40–6. Available from: http://dx.doi.org/10.1007/s10900-008-9126-
4 

SHORT TITLE/ACRONYM: Improving the health impact of air quality alerts 
v.3 
 
   
 
21 
 
29.  
Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for 
characterising and designing behaviour change interventions. Implement Sci [Internet]. 
2011;6(1):42. Available from: http://dx.doi.org/10.1186/1748-5908-6-42 
30.  
Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior 
change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an 
international consensus for the reporting of behavior change interventions. Ann Behav Med. 
2013/03/21. 2013;46(1):81–95.  
31.  
Jackson C, Barber N, Weinman J, Eliasson L. Applying COM-B to medication adherence. Eur 
Heal Psychol [Internet]. 2014;16(1):7–17. Available from: 
http://openhealthpsychology.net/ehp/issues/2014/v16iss1February2014/16_1_EHP_Februay201
4 Jackson et al.pdf 
32.  
Faul F, Erdfelder E, Buchner A, Lang A-G. Statistical power analyses using G*Power 3.1: Tests 
for correlation and regression analyses. Behav Res Methods [Internet]. 2009;41(4):1149–60. 
Available from: http://www.springerlink.com/index/10.3758/BRM.41.4.1149 
33.  
Sillence E, Briggs P, Fishwick L, Harris P. Trust and mistrust of online health sites. Proc 2004 
Conf Hum factors Comput Syst  - CHI ’04 [Internet]. 2004;(January):663–70. Available from: 
http://portal.acm.org/citation.cfm?doid=985692.985776 
34.  
Witte K, Meyer G, Martell D. Effective health risk messages. Newbury Park, CA: Sage; 2001.  
35.  
Rhodes RE, Blanchard CM, Matheson DH. A multicomponent model of the theory of planned 
behaviour. Br J Health Psychol [Internet]. Blackwell Publishing Ltd; 2006;11(1):119–37. 
Available from: http://dx.doi.org/10.1348/135910705X52633 
36.  
Meyer P. Defining and Measuring Credibility of Newspapers: Developing an Index. Journal 
Mass Commun Q [Internet]. 1988;65(3):567–74. Available from: 
http://jmq.sagepub.com/content/65/3/567.short 
37.  
McComas KA, Trumbo CW. Source credibility in environmental health-risk controversies: 
application of Meyer’s credibility index. Risk Anal. 2001/09/27. 2001;21(3):467–80.  
38.  
Hayes AF. Introduction to Mediation, Moderation, and Conditional Process Analysis: A 
Regression-Based Approach. New York, NY: The Guilford Press; 2013.  
39.  
Wilkinson S. Focus groups. In: Smith JA, editor. Qualitative Psychology A practical guide to 
research methods. Second. London: Sage; 2011. p. 186–206.