# Evaluating the Public's Reaction to Simulated Alerts on Mobile Phones in France: Scale Effects, and Challenges

Johnny Douvinet, Camille Cavaliere, Esteban Bopp, Karine Emsellem, Béatrice Gisclard, Karine Weiss

► To cite this version:

Johnny Douvinet, Camille Cavaliere, Esteban Bopp, Karine Emsellem, Béatrice Gisclard, et al. Evaluating the Public's Reaction to Simulated Alerts on Mobile Phones in France: Scale Effects, and Challenges. 19th International Conference on Information Systems for Crisis Response and Management, Ecole Nationale des Ingénieurs de Tarbes, May 2022, Tarbes, France. halshs-03677680

HAL Id: halshs-03677680

https://shs.hal.science/halshs-03677680

Submitted on 24 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Evaluating the Public's Reaction to Simulated Alerts on Mobile Phones in France: Scale Effects, and Challenges

**Johnny DOUVINET**

Avignon University,
UMR ESPACE 7300 CNRS,
Institut Universitaire de France
Johnny.douvinet@univ-avignon.fr

**Camille CAVALIERE**

Grenoble-Alpes University,
UMR LIG-LAG 5127 CNRS
[camille.cavaliere@univ-grenoble-alpes.fr](mailto:camille.cavaliere@univ-grenoble-alpes.fr)

**Esteban BOPP**

Franche-Comté University
UMR 6049 ThéMA CNRS
esteban.bopp@univ-fcomte.fr

**Karine EMSELLEM**

Côte d'Azur University,
UMR ESPACE 7300 CNRS
[Karine.emsellem@unice.fr](mailto:Karine.emsellem@unice.fr)

**Béatrice GISCLARD**

Nîmes University,
EA PROJEKT 7447
[Beatrice.gisclard@unimes.fr](mailto:Beatrice.gisclard@unimes.fr)

**Karine WEISS**

Nîmes University
EA CHROME 7352
[Karine.weiss@unimes.fr](mailto:Karine.weiss@unimes.fr)

**ABSTRACT**

In case of sudden and extreme life-threatening situation, the most efficient system fulfils the requirement of mobile phone emergency alerting information services. But how much information and what type of content must be used in a textual message, and how adapt this alert to a large audience and diversified contexts? This paper proposes to address this challenge, 1) by presenting results obtained in France during simulated “alerting days”, with sampling revealing the perception and understanding of SMS and CB by a non-specialized audience (161 respondents), and 2) by comparing findings obtained in different contexts (students and teaching staff in a university, crisis managers and stakeholders in an industrial context, citizens working near high-risky areas). Even if reactions to an alert are individual, similar patterns emerge, suggesting that standardized and non-random needs exist.

**Keywords**

Alerting message, Design, Perception, Feedbacks, France.

**INTRODUCTION**

Compared to sirens, that make people only aware of one alert, PWS (Public Warning System) that use Electronic Communications Services (ECS-PWS) offer several advantages, such as increased reachability, as a large part of the population always has a mobile with them (Hauri et al., 2022). Perhaps the two most practical systems, that fulfil the requirements of mobile phone emergency alert information service are the Location-Based SMS (LB-SMS) and Cell Broadcast (CB). With LB-SMS, message is sent from point-to-point to a predefined set of phones. This channel is an individual addressable channel, has a “store and forward” mechanism, and during times of an unavailable network coverage or temporary failure, SMS is stored in the Short Message Service Centre network that is delivered when recipients becomes available (Aloudat, 2010). With Cell Broadcast, the point-to-area notification is sent inside an area defined by cell towers. CB alerts require little bandwidth, whereas fast warning through multi-hazard warning apps or LB-SMS cannot always be guaranteed in congested.
Article 110 EECC (European Electronic Communications Code) mandates that by June 2022, every EU member state must implement a public warning system that can reach mobile phone users affected by major emergencies and disasters (EU 2018). In France, the French Government agreed the 7th September 2020 to deploy a new Public Warning System, that will combine CB and LB-SMS to previous siren network (so-called SAIP). This ECS-PWS (so-called FR-Alert) also comes in response to recommendations asked by a few deputies (Bellot in 2016, Bouchet in 2018), following the alerts not disseminated massively to the population, especially during the terrorist acts in Paris in 2015 or Nice in 2016. Their legislative proposals had remained without follow-up, until the event of September 26, 2019: the fire that occurred near the Lubrizol SEVESO High Threshold site, in Rouen, brought back to the forefront the need to inform the population in a different way (Bonfanti-Dossat and Bonnefoy, 2021), which also explains the announcement of the arrival of FR-Alert in Rouen, one year after this event that marked the inhabitants (Ferré and Daudé, 2020). Finally, it is also an unprecedented budgetary choice: the most expensive scenario (50 € million) has been accepted.

The two technologies will allow the French authorities to send a message with various information (the emergency situation, the location at danger, the safety guidelines), which will condition the people's capacities for action and reaction (Morss et al., 2018; Sutton & Vos, 2018; Cain et al., 2021). Their ability to interrupt the public activities will also be essential for their effectiveness (Potter, 2021). Nonetheless, designing a textual message remains hard and complex to define due to the current interactions between factors (i.e., the level of trust in the broadcaster, the language used, the feelings raised, or demographic parameters such as age, ethnicity or duration of residence) and the way in which people perceived the message and then responded to it is still difficult to address (Sellnow et al., 2012). Even if the FR-Alert platform sounds good, many challenges are still numerous: 1) an alerting message is still correlated with negative feelings such as fear, stress, the need to be assisted, helplessness or anxiety (Reed et al., 2010; Cvetković et al., 2019; Cain et al., 2021); 2) some people are not culturallyaccustomed to alerts (Ruin et al., 2007), which are considered as a constraint and a disturbance to daily life (Weiss et al., 2011; Douvinet, 2020); 3) the expectations of people have hardly been studied and are thus seldom considered by decision-makers, (Quinn, 2008; Bopp et al., 2021); 4) studies carried out on LB-SMS and CB alerts, as in the Netherlands (Jagtman, 2010), the United States (Gonzales et al., 2016; Parker et al., 2015), Australia (Aloudat et al., 2014), Sri Lanka (Samarajiva & Waidyanatha, 2009) or Maldives (Udu-Gama, 2009) have only focused on technical performance evaluation methods (Jagtman, 2010; National Research Council, 2013), without considering local effects, and the results cannot be generalized (McBride et al., 2020; Bopp, 2021); 5) even the best technology must rely on a clear policy and be integrated in a spatial and temporal daily continuum (Douvinet, 2018).

The majority of previous research has focused on experimental learning about participants' reactions to messages. Research gaps identified include proposals to develop more sophisticated experimentation of alert messaging in an integrated alert and warning ecosystem (Smith et al., 2022). Then, as a few researchers, we also decide to create protocol to answer to a big challenge: *what type of content is attempted by people in an alert message, and how adapt a single alert to a large audience located in various contexts?* In addition, we aim at providing insights into the deployment of the future "FR-Alert" platform, so it would become a real lever for cultural integration of alerts among the general public, and not only a technical tool designed to meet legal obligations without being accepted and appropriated by individuals. This paper is thus structured as follows. Section 2 presents the brief summary of the literature review on the structure of textual alerting messages. Section 3 explains the method and data used, and the choices made to set up the three field experimentations. Section 4 summarizes the main results obtained.

## BACKGROUND AND SCIENTIFIC INSIGTHS

Many studies have well-shown that the content and form of textual alerting messages influence the perception of the alert (Schultz et al., 2011; Liu et al., 2017; Kuligowski & Dootson, 2018; Cvetkovića et al., 2019; McBride et al., 2020). Users also benefit CB or LB-SMS as they can be addressed more selectively in real time, and be enabled to protect themselves by taking timely and effective countermeasures (Dallo and Marti 2021). But the needs differ according to the people. For example, a few researchers (McGee & Gow, 2012; Grant & Smith, 2019) have shown that people did not pay much attention to alerts received on their phone during the day or in working sites. Others have confirmed that the people still prefer their usual information channels, even in the event of an alert (Steelman et al., 2015). In the situation of extreme events, a large audience tend to favour familiar, local and interactive tools (Kuligowski & Dootson, 2018), more than scientific language. Whatever samplings, six variables (so-called Vn) are therefore highlighted (Figure 1), and advantages of each of them are summarized below.

### V1. A signal, to announce the alert message

A signal (using sound, light or vibration) is required to capture and focus people's attention in the event of an alert (Creton-Cazenave, 2010). Its detection is the first step in acknowledging and perceiving the nature of threat or the
danger (Bishop & Sonnenschein, 2012; McGee & Gow, 2012; Grant & Smith, 2019). It must create an interruption in people's daily activities, while being part of a certain continuity. This signal must help recipients identify the danger or threat (to civil security), the representation being constructed through learning, memorization (Edworthy et al. 2011) or according to own individual references (Weiss et al., 2011). In some cases, a specific sound can be associated with a type of risk (Edworthy et al, 2011), such as a fire alarm or a foghorn in case of dam failure.

## V2. The credibility and trust in the broadcaster

The broadcaster must be identified as quickly as possible, to be credible and well recognized (IBZ, 2017). Some researchers have confirmed that this variable influences the dynamics of the desired response (Trainor et al., 2015; Perreault et al., 2014; Lieu et al., 2016). This credibility stems from the possibility for the recipient to clearly trust the source of the message (McGee & Gow, 2012; Grant & Smith, 2019), in order to dispel any mistrust due to the many spam messages and fake news people receive daily (Lee & You, 2021). Providing the name of a defined entity is also recommended (IBZ, 2017; Kuligowski & Dootson, 2018; Grant & Smith, 2019). This entity varies according to the hazard or threat, as it is associated with organizations perceived as legitimate experts by the recipients (Australian Institute for Disaster Resilience, 2018).

## V3. The description of the event

The alert content must explain the nature of the risk, its physical description, its temporality and expected impact (Kuligowski & Dootson, 2018). The understandability of the message then depends on its ability to convey a sense of danger by using the appropriate language (Perrault et al., 2014). Regarding this, the *Australian Institute for Disaster Resilience* (2018) suggests replacing the standard wording by a clearer vocabulary (*dangerous* instead of *severe*, etc.), and banning technical and scientific language, in favour of common words that can be understood by a child aged 11-12. As an example, "*the lack of acute toxicity risk*" (i.e. a technical expression used during the Lubrizol fire occurred on September 24, 2019 in Rouen, France) is suitable.

## V4. The spatial information

The spatial information contained in the messages refers to the name and scope of the affected locations, areas to be avoided, or sectors to be evacuated. It is a key element that should be properly addressed (Cain et al., 2021). Recipients often have trouble locating the danger (Kuligowski, & Doermann 2018), especially when they are not familiar with the area or places mentioned. Likewise, people's own geographic representations of a familiar area can lead them to misunderstand the information provided (McGee & Gow, 2012). One map can improve risk perception and decision-making (Kuligowski & Doermann, 2018), but it cannot replace text (Cain et al., 2021). However, the spatial information recipients claim they would like to receive is particularly heterogeneous (Grant & Smith, 2019): exact location of the affected area, approximate distance between the recipient and the affected area, safe places to go, routes to follow, etc. This information should be provided using postcodes, names of districts and streets, or names of landmarks visible and familiar to recipients (*Australian Institute for Disaster Resilience*, 2018). A link or a web page can complete the alert message but it can be vulnerable to hacking.

## V5. The countermeasures and hierarchy of safety guidelines

Likewise, safety guidelines must be short, clear and provide key information. The amount and type of information provided influences the recipients' decision-making processes (Rehman et al., 2018). So, any inaccuracy must be avoided, as it could compromise the understanding of this parameter (McGee & Gow, 2012), and a hierarchy of instructions is required. But some studies show different outcomes. For some, the “ideal” message must be long (Potter, 2018) and it must provide detailed information so recipients do not have to take any additional steps to seek further information in order to confirm its authenticity (Chandler, 2010; Woody & Ellison 2014; *Australian Institute for Disaster Resilience*, 2018). For others, the content for the danger must be short to be better understood, notably in case of immediate events (Kuligowski & Dootson, 2018).

## V6. Additional information

If a "wall of words" is not appreciated (Grant & Smith, 2019), and if capitalization, bolding, and color should be used to highlight the crucial elements that should focus the recipients' attention (Kuligowski & Doermann, 2018), complements (via a URL link to a dedicated site for example) are expected to reinforce assessment of the situation. It is then necessary to ensure that this information is put online, and to anticipate an overload of connections. And if people receive too many warnings that are irrelevant to them, they may grow tired of the ECS-PWS and stop reacting to it. This calls for tailoring messages only to those at-risk, and not for warning (Reuter et al. 2017).
<figure><img src="image_1.png" /><figcaption>**Figure 1.** Conceptual design for a useful alerting message, and place of the six variables identified in the literature review (a fictive example here for a collision between 2 trains, inducing a possible chemical toxic cloud...)</figcaption></figure>

## METHOD AND DATA

Assessing the public's reactions during real alerts is difficult because it requires advance knowledge of the danger zone and real-time interviews with people. But evaluating the public's reception to simulated alerts, sent during experiences, offers opportunities (Cain et al., 2021; Smith et al., 2022). By the way, we created simulated “alerting days”, during those we asked to volunteer participants to answer in paper format questionaries about the content of SMS (in terms of clarity, length, words and location), we changed according to the nature of events, the contexts and the format (short or longer). This approach was carried out in various contexts in France in 2021 (despite the health context of COVID-19). All these experimentations render the observations of reception of the public placed in a fictive situation without using classical scripts of observation (Salès-Wuillemin, 2013). Unlike the observation that involves a minimal intervention by researchers, or surveys that are primarily interrogative, these “alerting days” consist of disconnecting participants from their daily activity, stimulated in their workplace during the day, and 2-3 observers also looked at the reactions of the respondents when alert messages were received.

## Research questions and hypotheses

Given the great number of questions asked for designing a notification, it was impossible to test everything with a large audience and in all experimented contexts. Thus, choices were made, to answer to 5 hypotheses (Table 1), and some of them are directly related to previous identified variables. The evaluation protocol was then deployed in each context, with similar questions, to allow for a comparative analysis.
<table><caption>Table 1. Hypotheses we want to test by measuring the public's reaction to alert messages (CB or LB-SMS)</caption><thead><tr><td>Number of hypothesis</td><td>Tested variables (literature review)</td><td>Comments and positive formulations</td></tr></thead><tbody><tr><td>H1</td><td>The description of one event (V3)</td><td>The nature of the event influences the form, content and understanding of the message</td></tr><tr><td>H2</td><td>Age of respondents</td><td>The age of the respondents plays a role on the expectations and content of the notifications;</td></tr><tr><td>H3</td><td>Previous alerting experience</td><td>The previous experience of an alert influences the expectations of future message</td></tr><tr><td>H4</td><td>Context and spatial information (V4)</td><td>Each site conditions the hazards, and thus the understanding of the alerting message</td></tr><tr><td>H5</td><td>CB notification Level 1 (V1)</td><td>The reception of the CB message Level 1 (with sound) does not induce the same reactions as an SMS</td></tr></tbody></table>

## Experimental conditions and study sites

The study sites are related to the positive responses of the stakeholders (local associations, company directors and prefectures). 1) At Avignon University, the respondents (33 students and 18 teachers) were put in a situation on the Hannah Arendt campus, located in the inner-city centre, knowing that scenarios are based on risks existing in the campus: flooding, fire, accident linked to the transport of dangerous materials or intrusion (Figure 2c). 2) At Port-Jerome, the respondents (12 elected officials and 22 crisis managers) were located near the EXXONMOBIL petrochemical platform (110 ha), where multiple risks exist: fire, technological accident or intrusion (Figure 2a); 3) At Le Havre, the respondents (11 truck drivers, 14 employees, 12 municipal agents, 6 parents, 9 firefighters) were put in place in 7 sites (SDIS-76, TGS accounting agency, in front of the Mayville school parking lot, in the SEVESO SEDIBEX site, in the Vendée Matériaux do-it-yourself store, in the ICPE SEPP and in the Le Cormoran roadside restaurant; Figure 2b). All of these sites have the common point of being close to the spatial scope of the Technological Risk Prevention Plan (PPRT), approved in 2015 by the State services.

<figure><img src="image_2.png" /><figcaption>Figure 2. Location of the different experimented sites in the three contexts: Le Havre, Port-Jérôme and Avignon</figcaption></figure>
Conditions of the experimentations were adapted to each site: 1) At Avignon University, the day was unmarked, and the 51 participants were distributed in 5 different rooms in order to respect the COVID rules (50% occupancy rate of the rooms; a minimum distance of 2m between the participants). Chasubles were also distributed (Figure 3), in order to be able to follow the movements of the individuals (filmed and photographed with their agreement). 2) In Port Jérôme, the elected officials and the administrative agents received 6 SMS in the context of a framework exercise organized by the Prefecture of Seine-Maritime (with the scenario of a fire generating a slight explosion and with a release of toxic gas that could cause respiratory problems). 3) In Le Havre, the experimentations were organized in a staggered way in time, on each of the 7 sites, and the protocols lasted 1 hour on average. Site visits were always organized before the experiments, to validate the participants' grouping rooms and to check the correct functioning of the alert tools. All participants were invited on a voluntary basis (with an e-mail sent by the directors of the establishments or by SMS). A declaration form respecting the *General Data Protection Regulation* (RGPD) was also drafted in order to keep anonymized data for research purposes for a period of 5 years. On the day of the experimentations, an automatic caller was used to simulate SMS arriving on mobiles of each participant (to gain credibility), and the CB notification was presented on a test cell phone. The participants were then given a guide booklet to fill in individually. For their part, the researchers had as main missions to explain the scientific approach, to ensure the smooth running of tests (Figure 3), and to ensure the moral integrity of the participants.

<figure><img src="image_3.png" /><figcaption>Figure 3. Alert experimentations in Avignon, and explanation of the test to the respondents (Douvinet@2021)</figcaption></figure>

## Number of respondents and nature of simulated scenario-based notifications

Different messages (Table 2) were sent to the respondents. SMS#1 asked to shelter themselves due to a progress fire in the building the respondents are located. No proactive action was therefore needed. SMS#2 warns people of the presence of a malicious person, without indicating its location; then, the respondents should consider how to protect themselves (escape or lockdown). Different gathering points were arranged to test the ability of people to orientate themselves in space. Each SMS were sent twice in two formats (short or longer, see one example in the Figure 4) and in French. The CB notification (SMS#3) was an intrusive notification indicating an earthquake: this simulated notification equals to a real message sent in Japan in 2021, but we aim here at measuring the sound effect on the public's response, even if such hazard cannot appear in the experimented contexts.

<table><caption>Table 2. Type of SMS and number of respondents in the different experimental contexts</caption><thead><tr><th>Nature of event<br>(scenario-based<br>messages)</th><th>Format</th><th>Language</th><th>AU</th><th>PJ</th><th>LH</th><th>Respondents<br>(N)</th><th>Par-<br>meters<br>(P)</th><th>Data<br>(N * P)</th></tr></thead><tbody><tr><td rowspan="2">SMS#1 (fire)</td><td>Short</td><td>French</td><td>51</td><td>34</td><td>63</td><td>148</td><td>5</td><td>740</td></tr><tr><td>Long</td><td>French</td><td>51</td><td>34</td><td>63</td><td>148</td><td>5</td><td>740</td></tr><tr><td rowspan="2">SMS#2 (a<br>malicious person)</td><td>Short</td><td>French</td><td>51</td><td>/</td><td>76</td><td>127</td><td>5</td><td>635</td></tr><tr><td>Long</td><td>French</td><td>/</td><td>/</td><td>76</td><td>76</td><td>5</td><td>380</td></tr><tr><td>CB Earthquake</td><td>Short</td><td>English</td><td>51</td><td>34</td><td>63</td><td>148</td><td>5</td><td>740</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>**Figure 4.** Example of SMS sent during the simulated fire alert in Le Havre (13 October 2021). Note: This message has been translated here in English, but it was sent in French, the native speaker language of participants.</figcaption></figure>

## Evaluation protocols

All the respondents were asked to answer various questions using paper questionaries (see Appendix A to see the basis for the data collection). First, questions on age, past experience of an alert (with a precision on the nature of the event and the related feeling), and the level of stress (noted from 0 to 10, using the Likert grid), to render the collection of some sociodemographic parameters easier. Then, after the reception of SMS, the respondents make a judgment on the length (*too short*, *too long*, *correct*), appropriation (no rereading, rereading necessary, or more) and level of knowledge of the places (I can identify the places right away, after thinking about it; it is not at all clear). They could also indicate whether there were items that needed to be reworded or deleted from the proposed notifications. For the CB, questions about the content, the sound (pleasant or unpleasant) and the feeling (*free response*) were asked. At the end of the various requests, the stress level was again assessed (to confirm that the research experiments had not created stress in accordance with ethical rules), and questions on the organization of the tests or the reasons for satisfaction were asked. A collective debriefing track was systematically organized to listen to the participants in a free way.

## Age and previous alerting experiences of the respondents

The volunteer participants have different ages. 55 people (34%) are between 19 and 29 years old (a majority being students enrolled in 2nd and 3rd year Bachelor's degree), 38 (23%) between 40 and 49 years old (almost half having been put in situation at the restaurant and at the SDIS in Le Havre, or being managers at Port Jérôme), 28 (17%) between 50 and 59 years old. Only 13 people are over 60 years old (Table 3). Out of the 64 persons declaring to have been alerted in the past, 20 confuse alerts with meteorological warnings (in particular those issued by the Prefectures, Météo-France and the insurers). If we remove these 20 responses, 24 people (out of 44) have already received an alert for an industrial accident (15 in the Le Havre area), 8 for a gas leak, 5 for a malicious act, 4 for a suspicious package (including 3 in Avignon), and 4 for an accident involving a vehicle transporting hazardous materials (TMD) (Table 3). Very few participants mentioned a high level of stress at the beginning and at the end of the experiments on the Likert grid (105 with a level of 0, 43 with a level of 1). Only one person indicated having been stressed (with a score ranging from 3 to 5), and she have been coached afterwards. As for representativeness, it is difficult to measure, as the participants registered on a voluntary basis. Therefore, the analysis of the results should allow us to validate the hypotheses and to identify trends that should be confirmed later.

<table><caption>**Table 3.** Age of the 161 respondents in the different experimental contexts</caption><thead><tr><th>Ages</th><th>Avignon (AU)</th><th>Le Havre (LH)</th><th>Port-Jérôme (PJ)</th><th>Total</th></tr></thead><tbody><tr><td>19-29</td><td>33</td><td>16</td><td>1</td><td><strong>50</strong></td></tr><tr><td>29-39</td><td>2</td><td>15</td><td>7</td><td><strong>24</strong></td></tr><tr><td>40-49</td><td>5</td><td>20</td><td>15</td><td><strong>40</strong></td></tr><tr><td>50-59</td><td>10</td><td>19</td><td>4</td><td><strong>33</strong></td></tr><tr><td>60 and more</td><td>1</td><td>6</td><td>7</td><td><strong>14</strong></td></tr><tr><td><strong>Total</strong></td><td><strong>51</strong></td><td><strong>76</strong></td><td><strong>34</strong></td><td><strong>161</strong></td></tr><tr><td><strong>Previous alerting experience</strong></td><td><strong>15</strong></td><td><strong>18</strong></td><td><strong>11</strong></td><td><strong>44</strong></td></tr></tbody></table>
## Statistical analyses

The data collected via the paper questionnaires were compiled in a spreadsheet, to allow filtering of the responses. Three approaches were carried out. First, for each type of message (fire, malicious act or earthquake), a qualitative overview is proposed, to observe differences according to the nature of alert and the experimentation sites. Second, inferential analyses (with the Chi-square tests) were tested on all collected parameters, to determine the relations between all collected data, by crossing the selected parameters in pairs. If null hypothesis was rejected (with a p-value lower than p<0.005 due to the weak number of data), the importance of the relationship was quantified using the Tschuprov test (easier to understand in comparison to the Cramer formula). Third, Multiple Correspondence Analyses (MCA) were applied to analyse the pattern of relations between the nature of events, socio-demographic characteristics and the evaluations on the length, clarity and spatial information, by sub-dividing the data collected for fire SMS, malicious act SMS and CB notification (earthquake). Profiles were then generated using Ascending Hierarchical Classifications (AHC), and the MCA/AHC couples were applied independently of sites, to measure the geographical influence on the public's reaction. These data processing techniques, performed with RStudio, offer synthetic view, and the dataset will be freely available online.

## RESULTS

### SMS: different reactions according to the nature of simulated events

Studying the answers of the respondents who receive the simulated fire and malicious act SMS is first interesting: trends appear on various sites, and differences according to the nature of the event are underlined.

Short and long SMS#1 (for the simulated fire alert) have been sent in the three contexts. At Avignon University (Figure 5a), the long SMS is considered as more adapted in terms of length (45/51) than the short SMS (37/51). Information contained in the shorter SMS is nevertheless easily retained, since the message is read only once by 36 respondents, against 18 for the long message (27 participants had to read the SMS again). If the long message contains more spatial information (43 people find it complete, against 32 for the short SMS), it takes much time of reflection and confusions (16 participants - all students - would have liked to know the place of gathering with more precision, against 3 participants only for the short SMS). In Le Havre, the short SMS is also considered as correct (51 against 37 for the long SMS). However, the spatial information is unclear (Figure 6a). 13 participants need time to think about it, and 13 people even did not understand it. In Port Jerome, only one reading was required (33/34 for the long SMS; 31/34 for the short SMS). The long SMS is considered the longest (25 against 20 for the short one), and the spatial information, more numerous, was understood immediately by 25 participants (against 16 for the short SMS). Finally, this analysis, qualitative, highlights the good reception of length and content, but differences exist in terms of clarity and understanding of spatial information.

Short or long SMS#2 (indicating a malicious act) have been only sent in two contexts. Trends are less obvious. At Avignon (Figure 5b), the long SMS is judged correct by a high number of respondents (44), as for the short SMS (41/51). The long message is more reread (22 vs. 15), but it is still judged as containing enough information (32, vs. 25 for the short SMS). However, respondents reported taking more time to locate the spatial information in both SMS (25 and 22). In addition, the term "suspicious person" in the long SMS was a source of stress and anxiety for 16 people (as testified by the observers and indicated in free discussions in the questionaries). In Le Havre (Figure 6b), the long SMS is correct for a high number of respondents (54), more than the short SMS (47/76). 38 people understand the locations indicated right away (versus 26 for the short SMS), and 26 participants do not have enough information in the short SMS. In addition, the term "run or hide" (indicated in the long SMS) received paradoxical reactions: 11 people laughed, while 14 people stressed.

### CB: different reactions according to the experimentation contexts

The alert traduced by a CB notification was sent in two contexts, and evaluated by 88 respondents (Figure 7). The analyses bring other results. 27 people (including 11 in Avignon) found the sound really surprising; 43 found the sound pleasant but 42 also found it unpleasant (including 22 in Le Havre). The observers also noticed that some respondents winced, moved their heads away from the handset, or covered their ears when hearing the notification. The length is not appreciated in the same way either: 31/51 found it "too short" in Avignon; 22/34 correct in Le Havre. 41/51 found it clear in Avignon, but 22/34 not clear at all in Le Havre. This is a result that supports the specific reactions to CB: at the University of Hull (Smith et al., 2022), the responses from 80 students demonstrate that the distinctive, loud, penetrating sound was distracting and unpleasant for 43% of individuals, and several participants covered their ears when hearing the sound of the notification. Finally, the SMS or CB support different reactions, that the practitioners and the authorities have to consider before sending a real message.
<figure><img src="image_5.png" /><figcaption>Figure 5. Results obtained in Avignon during the experimental alerting day.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 6. Results obtained in Le Havre during the experimental alerting day.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 7. Results obtained in Avignon and Le Havre after the CB reception.</figcaption></figure>

## Trends statistically confirmed on SMS collected data

Chi-square analyses (on all collected data) confirm that even if parameters are weakly related to each other (Table 4), the nature of the event is strongly related to the format (35.5%), and the clarity of messages (28.3%). Although other parameters present contributions ranged from 11 to 19% (e.g., the links between length and nature of events, or between the previous experience and spatial information), these relationships are statistically insignificant, as the null hypothesis is accepted if we consider a p-value > 0.005. Therefore, the levers to promote the understanding
of notifications and their acceptability seem to be founded on the format of message (short/longer) and the choice of language used to locate the alert zone, but do not depend on the age or previous experience. Thus, it is surprising that these parameters are not more emphasized since many studies had suggested their importance (Smith et al., 2022; McGee & Gow, 2012; Grant & Smith, 2019; Kuligowski & Dootson, 2018).

<table><caption>Table 4. Relation between parameters with Chi-square analyses</caption><thead><tr><th>Combined parameters</th><th></th><th>Chi Square</th><th>Theoretical Chi square</th><th>DDL</th><th>p-value</th><th>HO reject</th><th>Tschuprov</th><th>Link</th></tr></thead><tbody><tr><td rowspan="5">Nature of event (Scenario-based messages)</td><td>Age</td><td>11.52</td><td>16.92</td><td>9</td><td>0.24162</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Experience</td><td>1.58</td><td>7.81</td><td>3</td><td>0.66284</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Length</td><td>25.87</td><td>12.59</td><td>6</td><td>0.00023</td><td>YES</td><td>0.1383</td><td>Weak</td></tr><tr><td>Clarity</td><td>108.10</td><td>12.59</td><td>6</td><td>0.00000</td><td>YES</td><td><strong>0.2827</strong></td><td><strong>Medium</strong></td></tr><tr><td>Spatial Info.</td><td>8.97</td><td>12.59</td><td>6</td><td>0.17537</td><td>No</td><td>/</td><td>/</td></tr><tr><td rowspan="5">Clarity of SMS</td><td>Format</td><td>120.16</td><td>7.81</td><td>3</td><td>0.00000</td><td>YES</td><td><strong>0.3545</strong></td><td><strong>Medium</strong></td></tr><tr><td>Age</td><td>20.00</td><td>12.59</td><td>6</td><td>0.00276</td><td>YES</td><td>0.1123</td><td>Weak</td></tr><tr><td>Experience</td><td>9.99</td><td>5.99</td><td>2</td><td>0.00678</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Length</td><td>27.87</td><td>9.49</td><td>4</td><td>0.00001</td><td>YES</td><td>0.1588</td><td>Weak</td></tr><tr><td>Spatial Info.</td><td>37.50</td><td>9.49</td><td>4</td><td>0.00000</td><td>YES</td><td>0.1843</td><td>Weak</td></tr><tr><td rowspan="4">Previous Experience</td><td>Format</td><td>3.31</td><td>5.99</td><td>2</td><td>0.19077</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Age</td><td>8.73</td><td>7.81</td><td>3</td><td>0.03315</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Length</td><td>1.36</td><td>5.99</td><td>2</td><td>0.00678</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Location</td><td>15.16</td><td>5.99</td><td>2</td><td>0.00005</td><td>YES</td><td>0.1393</td><td>Weak</td></tr><tr><td rowspan="2">Spatial information</td><td>Format</td><td>0.08</td><td>3.84</td><td>1</td><td>0.78329</td><td>No</td><td>/</td><td>/</td></tr><tr><td>Age</td><td>11.79</td><td>12.59</td><td>6</td><td>0.06672</td><td>No</td><td>/</td><td>/</td></tr><tr><td></td><td>Length</td><td>27.41</td><td>9.49</td><td>4</td><td>0.00016</td><td>YES</td><td>0.1575</td><td>Weak</td></tr></tbody></table>

Even the best message will never speak to the majority!

Five Multiple Correspondence Analyses and Ascending Hierarchical Classifications (MCA/AHM) were tested, on data collected for short SMS#1, long SMS#1, short SMS#2, long SMS#2 and CB. The objective was to identify classes accounting for socio-demographic characteristics (age, experience), and evaluations on the length, clarity and spatial information, without including the experimentations contexts (used only as descriptive parameters) in the statistical process, and to see if the respondents present similar reactions to the 5 stimuli.

Results of the five MCA are summarized in Table 5. The 4 Factorial axes (F) retain various percent of the initial inertia, ranged from 64.1 to 75.8%. Interestingly, even if parameters do not contribute to the same axes, several parameters are systematically related. The SMS that takes time for understanding the spatial information and need 2 re-readings are often opposed (in F1 for short SMS#1 or F1 for short SMS#2) to the respondents who appreciate the content of SMS. The respondents who consider SMS too short had often experienced one alert in the past (F2 for short SMS#1). Those who consider spatial information unclear also do not appreciate the length of the message (too short for the short SMS#2 and too long for the short SMS#1).

<table><caption>Table 5. Results obtained with the Multiple Correspondence Analyses (MCA)</caption><thead><tr><th>Nature of event<br>(scenario-based<br>messages)</th><th>Format</th><th>F1<br>(%)</th><th>F2<br>(%)</th><th>F3<br>(%)</th><th>F4<br>(%)</th><th>Total %<br>(F1>F4)</th></tr></thead><tbody><tr><td rowspan="2">SMS#1 (fire)</td><td>Short</td><td>20.9</td><td>19.5</td><td>18.7</td><td>16.7</td><td>75.8</td></tr><tr><td>Long</td><td>19.7</td><td>15.6</td><td>13.8</td><td>12.6</td><td>61.7</td></tr><tr><td rowspan="2">SMS#2 (a<br>malicious person)</td><td>Short</td><td>19.8</td><td>16.9</td><td>16.5</td><td>13.3</td><td>66.5</td></tr><tr><td>Long</td><td>26.2</td><td>18.6</td><td>18.1</td><td>12.6</td><td>75.5</td></tr><tr><td>CB Earthquake</td><td>Short</td><td>21.2</td><td>18.4</td><td>14.0</td><td>11.3</td><td>64.9</td></tr></tbody></table>
The five related AHC allow for identifying 3 or 4 classes for the different stimuli. All the classes are not recurrent when we analyze the main characteristics of each of them. However, four groups (G) appear identical (Figure 7), especially for the class recalled A ("spatial information clear, 1 read and a correct message"), identified for the short SMS#1 and for the long SMS#2, the class B ("too long"), the class C ("spatial information unclear and no previous alerting experience") and the class D (only "correct message").

<table><caption>Figure 7. Results obtained with the MCA/AHC couple, for the four SMS messages</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="4">MCA Classes - Short SMS#1</th><th colspan="4">MCA Classes - Long SMS#1</th><th colspan="4">MCA Classes - Short SMS#2</th><th colspan="4">MCA Classes - Long SMS#2</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>1</th><th>2</th><th>3</th><th>4</th><th>1</th><th>2</th><th>3</th><th>1</th><th>2</th><th>3</th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><td rowspan="3">Spatial information</td><td>rigth away</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>a time to reflect</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>not at all</td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Length of the message</td><td>too short</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td>x</td><td></td></tr><tr><td>correct</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td></tr><tr><td>too long</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td></tr><tr><td rowspan="3">Clarity of the message</td><td>read once</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td>x</td></tr><tr><td>re-read once</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>more re-reads</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">Previous alert experience</td><td>yes</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>no</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td colspan="2">Interesting group</td><td>A</td><td>C</td><td></td><td>B</td><td>D</td><td></td><td>C</td><td></td><td>D</td><td>B</td><td></td><td></td><td>A</td><td></td><td></td><td></td><td></td></tr></tbody></table>

Thus, we pay attention on these identified groups, allocated to each respondent. G1 gathers 30 persons at Avignon, but only 4 persons in Rouen and 10 at Le Havre. It means that the "spatial information clear, 1 read and a correct message" is common, independently of the simulated based-scenario (fire or malicious act), but it strongly depends on the public profiles. At the opposite, G3 gathers only 6 respondents that evaluate SMS with "spatial information unclear and no previous alerting experience" at Le Havre. This group is not important, but it suggests that these persons will never appreciate the spatial information, whatever the nature of the event in progress. Finally, the G2 and the G4 do not group any respondent. "Too long" SMS are therefore one-time evaluations.

## CONCLUSIONS AND DISCUSSIONS

### Responses to initial hypotheses

Even if the samplings in each experimental site, and the number of respondents, are not necessarily representative of all contexts, our results first allow to answer to initial hypotheses (Table 5). The nature of event (V3), as well as the spatial information included in the alert (V1), are main variables (V3), in the design of the message but also for the respondents, whereas the age (H2) or the previous experience (H3) do not play a major influence. CB (V1) also induces differences in reactions, but trends are not clearly underlined.

<table><caption>Table 5. Hypotheses we wanted to test by measuring the public's reaction to alert messages, and results</caption><thead><tr><th>Number of hypothesis</th><th>Tested variables (literature review)</th><th>Comments and positive formulations</th><th>Results</th></tr></thead><tbody><tr><td>H1</td><td>The description of one event (V3)</td><td>The nature of the event influences the form, content and understanding of the message</td><td>Sure!</td></tr><tr><td>H2</td><td>Age of respondents</td><td>The age of the respondents plays a role on the expectations and content of the notifications;</td><td>No</td></tr><tr><td>H3</td><td>Previous alerting experience</td><td>The previous experience of an alert influences the expectations of future message</td><td>Not really</td></tr><tr><td>H4</td><td>Context and spatial information (V4)</td><td>Each site conditions the hazards, and thus the understanding of the alerting message</td><td>Sure!</td></tr><tr><td>H5</td><td>CB notification Level 1 (V1)</td><td>The reception of the CB message Level 1 (with sound) does not induce the same reactions as an SMS</td><td>Probably</td></tr></tbody></table>

By the way, such results allow to formulate a kind of "framing note" for designing optimal alerting messages. One example is given in the situation of a malicious act (Figure 8). The six variables need to account for the needs of population, so this framework should be implemented in the current and future ECS-PWS. Surely this grid needs to be adapted to the audience or by the authorities in charge of alert in each country, but it reveals invariant and non-invariant elements. And this framing note strongly improves the former studies (Sorensen, 2000).
<figure><img src="image_8.png" /><figcaption>Figure 8. A framing note applied for designing an optimal SMS accounting for the needs of population</figcaption></figure>

## Scale effects

Taken together, these results confirmed that there are normed reactions to alerting, and therefore not random. Some results are consistent with the scientific literature: individuals wish to receive a clear, precise, long, directive and coercive message, even for those who have already been informed in other ways (by sirens or loudspeakers). The nature of event, its location, the safety guidelines and/or the gravity of the situation are elements expected by the population, which underlines all the framing elements presented at the beginning of this article. In their study, Smith et al. (2022) tested for example flood CB notification with 188 students, and the most common response (34%, n = 66) indicated that being given direct instructions about what to do was important.

Second, we observed that in the situation of simulated fire or intrusion, the respondents take time to interpret the situation and take a decision, using the information available in the message, but also in their social and physical environment, which remains however uncertain and incomplete, to which are added the knowledge, beliefs and values. Social interactions are likely to reinforce perceptual biases, through effects that are now well known in the field of risk, such as comparative optimism (i.e. *thinking one's self is less risky than others*), or social proof (i.e. *if others act this way, there is a good reason*). Thus, cognitive and social factors interfere with the understanding of alerting messages, and the observers note that several responders looked around before answering.

But new elements have also appeared. **1.** The fire SMS is mostly recognized than a malicious act. This is probably due to the current exercises dealing with fire evacuating in numerous public sites, and to the stress more generated in case of terrorism in France (especially since the dramatic attacks in Paris in 2015). **2.** The authority perceived as legitimate is not always the territorially responsible actor, which leads to question the regulatory framework: if the officials and crisis managers trust in the prefecture, the students and staff of Avignon University would like the alert officer to be adapted to the nature of the event (the firefighters in case of fire; the prefecture in case of flood; the President of the University in case of malicious act...). **3.** The respondents wished to receive more than one SMS (for example, a SMS broadcasted every 15 minutes), to understand the evolution of the current situation and to be kept informed. Therefore, it is not necessary to limit the alert to a single message. **4.** Age and experience do not play a key role on alert reception, so these results differ from other studies (Smith et al., 2022). **5.** Most of the respondents prefer a directive first message (especially in case of immediate protection), and they would like to have more information in the following messages, or at the end of the alert; **6.** The knowledge, the use and the frequentation of the sites are discriminating factors in the strategies to go to safety or reach a place. **7.** The use of politeness (thank you, please...) is not desired in one alerting message.

## Supports and perspectives for FR-Alert in France

Several advices can be formulated for the broadcasters. **1.** Participants need to quickly authenticate the broadcaster of the alerting message, which should vary according to the nature of the hazard or threat. This entity must be the most legitimate entity in the “eyes of the population”, and not necessarily the administrative authority with territorial jurisdiction. **2.** Indicating the alert zones in a message (the place of threat or the place to be reached) is difficult because the people have to be identified them. The toponyms or the official names are not the best terms,
or even when the places are unknown by the individuals. In a message, the location of the place should be in relation to something recognizable and easily identifiable (nearby), taking into account the context (known place, specify the name in addition). 3. The first SMS is important since it creates the first relation between the authorities and the population, so the authorities need to send the alert as soon as possible, even if no additional information is known. Keep these messages short and remind them that people will be informed soon. 4. There is no magic formula for writing a SMS that will speak to everyone, but it would be enlightening to set up a typology of cases.

And precisely, results of this study have been presented to the French government in 2021, and to several prefects who will be the users of the future FR-Alert platform since March 2022. Regarding the choice of words and type of language recommended in our experimentations, discussions emphasize the wording in the command-and-control centers (such as the firefighters center) and changes are in progress: the instructions will be prioritized and a message will be proposed to the authorities to name the event and choose the appropriate words. The ORSEC messages will be also simplified (this list cannot be presented here since the last version is in the process of being validated). Other guidelines have been proposed to perform the trainings with the population. CB notifications should be sent to the population in May 2022, to see what kind of messages they could receive in case of real alert with FR-Alert; LB-SMS will be tested to check the number of antennas required to alert people located in the "point to point" zone (estimated with a threshold estimated at half a million people by some ECS-PWS providers).

Other issues should also be questioned because some people (impaired or disabled) will never be able to read the text messages: 1) how to reflect on cultural dimensions (language, practice, etc.) of the alert and the related biases (cognitive, perceptive)?, 2) how to focus on collective reactions following the reception of the messages (why did individuals react in such a way? what proportion of individuals was forced to reread the SMS? 3) how to evaluate the needs of a large audience in their diversity (handicapped people, children, etc.); 4) how to create automatic messages taking into account for context effects? Whatever these tracks, the alerting experience, the experiments and the risk awareness are essential elements to consider if we want to hope for an efficient warning.

## ACKNOWLEDGMENTS

The co-authors would like to express their sincere thanks to all authors, program and local committee members, and volunteers for their hard work and contributions to the 2022 ISCRAM conference. The co-authors also thank the ANR (French Agency for Research) and the SGDSN (Secrétariat Général de Défense et Sécurité Nationale) for founding the ANR project so-called Cap-4-Multi-Can'Alert (2020-2021). The co-authors also want to address their sincere thanks to partners who authorize the alerting days, the respondents and the observers, as well as the moderators, who give us answers and interesting comments. Finally, the co-authors also thank the other colleagues who work on the data analyses (M. Coulon, A. César, T. Jezirski, L. Roussel and G. Martin), and the anonymous reviewers, whose comments greatly improved the manuscript.

## REFERENCES

Aloudat, A. (2010). *Location-Based Mobile Phone Service Utilisation for Emergency Management in Australia*. Doctorate thesis, University of Wollongong, Wollongong, Australia, 245 p.

Aloudat, A., Michael, K., Chen, X., & Al-Debei, M.M. (2014). Social acceptance of location-based mobile government services for emergency management. *Telematics and Informatics*, 31, 153–171.

Australian Institute for Disaster Resilience (2018). *Guideline 1: Alert Message Construction: Choosing your Words*, Australian Disaster Resilience Handbook Collection.

Bean H., Sutton J., Lui B., Madden S., Wood, M., Mileti D. (2015). The study of mobile public alert messages: a research review and agenda, *The Review of Communication*, 15 (1), 60-80.

Bishop M.J. & Sonnenschein D. (2012). Designing with sounds to enhance learning: Four recommendations from the film industry, *The Journal of Applied Instructional Design*, 2 (1), 5-16.

Bonfanti-Dossat C., Bonnefoy N. (2021). *Risques industriels : prévenir et prévoir pour ne plus subir – Synthèse des auditions*. Report commissioned by the Fire Service of the Lubrizol plant,

Bopp E., Gisclard B., Douvinet J., Weiss K., Martin G. (2021). How to improve alert systems: the technical, human, environmental and structural aspects. *Australian Journal of Emergency Management* 36 (1), 125-135.

Burkett E., Given D. (2015). *Sounding the alert: Designing an effective voice for earthquake early alert*. Southern California Earthquake Center Annual Meeting, Palm Springs, California (Poster).
Cain L., Herovic E., Wombacher K. (2021). "You are here": Assessing the inclusion of maps in a campus emergency alert system, *J Contingencies and Crises Management*, 29 (3), 332-340.

Chandler R. C. (2010). *Emergency Notification*. Santa Barbara, CA: Praeger.

Créton-Cazanave, L. (2010). *Penser l'alerte par les distances. Entre planification et émancipation, l'exemple du processus d'alerte aux crues rapides sur le bassin versant du Vidourle*. Doctorate thesis, Joseph-Fourier University - Grenoble I, 350 p.

Cvetkovića V.C., Öcalb A., Ivanovc A. (2019). Young adults' fear of disasters: A case study of residents from Turkey, Serbia and Macedonia, *International Journal of Disaster Risk Reduction*, 35, 101085

Dallo I., and Marti M. (2021). Why Should I Use a Multi-Hazard App? Assessing the Public's Information Needs and App Feature Preferences in a Participatory Process, *International Journal of Disaster Risk Reduction*, 57.

Douvinet, J. (2018). Alerter la population face aux crues rapides en France : compréhension et évaluation d'un processus en mutation. Habilitation à diriger des recherches, Avignon University, 250 p.

Douvinet, J., Serra-Llobet, A., Bopp, E., Kondolf, G. M. (2021). Are sirens effective tools to alert the population in France?, Nat. Hazards Earth Syst. Sci., 21, 2899-2920;

Edworthy J. (2011). Designing effective alarm sounds, *Biomedical Instrumentation & Technology*, 45 (4), 290-294.

Gonzales, D., Miyashiro, L., Osburg, J., Shelton, S., & Woods, D. (2016). *Geo-Targeting Performance of Wireless Emergency Alerts in Imminent Threats Scenarios. Volume 1: Tornado Alerts*, Homeland Security, 124p.

Grant S., Smith K. (2019). *Emergency alerting in England: Alert simulation exercise*. Internal report, Department for the Environment, Food & Rural Affairs and the Environment Agency, United Kingdom.

Hauri A., Kolher K., and Scharte B. (2022). A comparative Assessment of Mobile Device-Based Multi-Hazard Warnings: Saving Lives through Public Alerts in Europe. *Risk and Resilience Report*, Zurich, 46 p.

IBZ (2017). *Principe de fonctionnement de Be-Alert*, Service Public Fédéral Intérieur, 42p.

Jagtman, H.M. (2010). Cell broadcast trials in The Netherlands: Using mobile phone technology for citizens' alarming. *Reliability Engineering & System Safety*, 95(1), 18-28.

Kuligowski E., Doermann J. (2018). *A review of public response to short message alerts under imminent threat*, National Institute of Standards and Technology.

Kuligowski E., Dootson P. (2018). *Emergency notification: Alerts and alerts*, in S. L. Manzello (ed) Encyclopedia of Wildfires and Wildland-Urban Interfaces (WUI) Fires, Springer.

Lee M., You M. (2021). Effects of COVID-19 emergency alert text messages on practising preventive behaviors: cross-sectional web-based survey in South Korea, *Journal of Medical Internet Research*, 23 (2).

Liu, B. F., Wood, M. M., Egnoto, M., Bean, H., Sutton, J., Mileti, D., & Madden, S. (2017). Is a picture worth a thousand words? The effects of maps and alert messages on how publics respond to disaster information. *Public Relations Review*, 43(3), 493-506.

McBride S.K. (2017). *The Canterbury tales: an insider's lessons and reflections from the Canterbury Earthquake Sequence to inform better public communication models*. Thesis presented in fulfilment of the requirements for the degree of Doctor of Philosophy in English and Media Studies at Massey University, Wellington, New Zealand (Doctoral dissertation), Massey University, 2017.

McBride S.K., Bostrom A., Sutton J., De Groot R.M., Baltay A.S., Terbush B., Bodin P., Dixon M., Holland E., Arba R., Laustsen P., Liu S., Vinci M. (2020). Developing post-alert messaging for ShakeAlert, the earthquake early alert system for the West Coast of the United States of America, *Internation Journal of Disaster Risk Reduction*, 50, 101713.

McGee T., Gow G. (2012). Potential responses by on-campus university students to a university emergency alert, *Journal of Risk Research*, 15 (6), 693-710.
Morss R., Cuite C., Demuth J., Hallman W., Shwom R. (2018). Is storm surge scary? The influence of hazard, impact and fear-based messages and individual differences on responses to hurricane risks in the USA, *International Journal of Disaster Risk Reduction*, 30, 44-58,

National Research Council (2013). *Geo-Targeted Disaster Alerts and Alerts: A Workshop*, Washington DC, 78p.

Parker, A.M., Jackson, B.A., Martinez, A., Sanchez, R., Shelton, S., & Osburg, J. (2015). *Exploring the Effect of the Diffusion of Geo-Targeted Emergency Alerts*, Homeland Security, 121p.

Perreault M., Wilkins L., Houston J. B. (2014). Does scary matters? Testing the effectiveness of new national weather service tornado alert messages, *Communication Studies*, 65 (5), 484-499.

Potter S. (2018). Recommendations for New Zealand Agencies in Writing Effective Short Messages. Lower Hutt, New Zealand: GNS Science, 2018.

Potter S. (2021). Why some people don't respond to alerts: writing effective short alert messages, *Australian Journal of Emergency Management*, 36 (1), 29-30.

Quinn S.C. (2008). Crisis and emergency risk communication in a pandemic: a model for building capacity and resilience of minority communities, *Health Promotion Practices*, 9 (4), 188-251.

Rehman A., Lyche T., Kwame A. O., Nadendla V.S.S. (2020). Effects of text message alerts on miner's evacuation decisions, *Safety Science*, 130 (10), https://doi.org/10.1016/j.ssci.2020.104875

Reuter C., Kaufhold M., Leopoled I, Knipp H. (2017). Katwarn, NINA or FEMA? Multi-Method Study on Distribution, Use, and Public Views on Crisis Apps, *Proceedings of the 25th European Conference on Information Systems (ECIS)*, 5-10 June 2017, 2187-2201.

Samarajiva, R., & Waidyanatha, N. (2009). Two complementary mobile technologies for disaster alert. *Info*, 11, 58-65.

Schultz, D.M., Gruntfest, E.C., Hayden, M.H., Benight, C.C., Drobot, S., Barnes, L.R., (2010). Decision making by Austin, Texas, residents in hypothetical tornado scenarios. *Weather Climate & Society*, 2, 249–254.

Sellnow T. & D. (2010). The Instructional Dynamic of Risk and Crisis Communication: Distinguishing Instructional Messages from Dialogue, *The Review of Communication*, 10, 112–26.

Sorensen, J.H. (2000). Hazard Alert Systems: Review of 20 Years of Progress. *Natural Hazards Review*, 1(2), 119-125.

Steelman T.A, McCaffrey S.M., Velez A.L., Briefel J.A. (2015). What information do people use, trust, and find useful during a disaster? Evidence from five large wildfires, *Natural Hazards*, 76, 615–634

Sutton J., Kuligowski E. (2019). Alerts and Alerts on Short Messaging Channels: Guidance from an Expert Panel Process, *Natural Hazards Review*, 20 (2), 1-10.

Sutton J., Vos S.C., Wood M.M., Turner M. (2008). Designing effective tsunami messages: examining the role of short messages and fear in alert response, *Weather, Climate, and Society*, 10 (1), 75–87.

Trainor J.E., Nagele D., Philips B., Scott B. (2015). Tornadoes, social science, and the false alarm effect, *Weather, Climate, and Society*, 7 (4), 333–352.

Wood, M.M., Bean, H., Liu, B.F., & Boyd, M. (2015). Comprehensive testing of imminent threat public messages for mobile devices: updated findings, *Homeland Security*, 119 p.

Woody, C., and Ellison, R. (2014). *Maximizing trust in the wireless emergency alerts (WEA) service*. Software Engineering Institute, 65 p.

APPENDIXES

Appendix A: Questions addressed for the data collection (adapted for this abstract – see in the following page)
## 1. Personal information

1.1. What is your age range?

* < 19 years

* 20-29 years

* 30-39 years

* 40-49 years

* 50-59 years

* > 60

1.2. Have you ever received an "alert" for an event that affected you?

* Yes

* No

1.3. What was the nature of the event?

* Fire

* Gas leak

* Malicious act

* Flood

* Threatening individual

* Suspicious package

Other (please specify in this case):

1.4. How did you feel about the occurrence of this previous event?

1.4a. What was the means of alert used?

* Siren

* SMS

* Other: ............................

1.4b. In your opinion, was this method appropriate?

* Yes

* No

* Why? ............................

1.5. By the way, are you ever stressed (1 : no, not at all 10 = very stressed)

• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10

## 2. SMS reception (repeated according to the number and nature of simulated based-scenario)

After receiving the SMS on your mobile phone, please answer to the following questions

2.1. The length of the message was

* Too short

* Correct

* Too long

* Read once

* Re-read once

* More re-reads

* Right away

* After a reflection time

* Not at all

* No answer

* Change? ............................

* Delete? ............................

2.2. The clarity: To understand the message, you need to...

2.3. The spatial information: You locate the event...

2.4. Is there any word(s) you want to...

## 3. CB notification (presented using a computer and a fictive phone)

After hearing this alert notification, please answer to the following questions

2.1. The length of the message was

* Too short

* Correct

* Too long

* Understood

* Not understood

* No answer

* Pleasant

* Unpleasant

* No answer

* Right away

* After a reflection time

* Not at all

* No answer

2.2. The clarity: Do you understand alerting information?

2.3. The voice: The vocal message was...

2.4. The spatial information: You locate the event...

2.5. What do you think about such message?