# Validity, reliability, and responsiveness of daily monitoring visual analog scales in MASK-air®

Bernardo Sousa-Pinto, Patrik E. Eklund, Oliver Pfaar, Ludger Klimek, Thorsten Zuberbier, Wienczyslawa Czarlewski, Annabelle Bédard, Carsten Bindslev-Jensen, Anna Bedbrook, Sinthia Bosnic-Anticevich, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Bernardo Sousa-Pinto, Patrik E. Eklund, Oliver Pfaar, Ludger Klimek, Thorsten Zuberbier, et al. Validity, reliability, and responsiveness of daily monitoring visual analog scales in MASK-air®. Clinical and Translational Allergy, 2021, 11 (7), 10.1002/clt2.12062. hal-03401816

HAL Id: hal-03401816

https://hal.science/hal-03401816

Submitted on 30 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
DOI: 10.1002/clt2.12062

REVIEW

<figure><img src="image_3.png" /></figure>

Validity, reliability, and responsiveness of daily monitoring
visual analog scales in MASK-air®

Bernardo Sousa-Pinto<fnref n="1" />,<fnref n="2" />,<fnref n="3" /> ID | Patrik Eklund<fnref n="4" /> | Oliver Pfaar<fnref n="5" /> ID | Ludger Klimek<fnref n="6" />,<fnref n="7" /> |
Torsten Zuberbier<fnref n="8" /> | Wienczyslawa Czarlewski<fnref n="9" />,<fnref n="10" />,<fnref n="11" /> | Annabelle Bédard<fnref n="12" />,¹³ |
Carsten Bindslev-Jensen¹⁴ ID | Anna Bedbrook<fnref n="9" />,<fnref n="11" /> | Sinthia Bosnic-Anticevich¹⁵,¹⁶ |
Luisa Brussino¹⁷ | Victoria Cardona¹⁸ ID | Alvaro A. Cruz¹⁹,²⁰ | Govert de Vries²¹ |
Philippe Devillier²² | Wytske J. Fokkens²³ | José Miguel Fuentes-Pérez²⁴ |
Bilun Gemicioğlu²⁵ | Tari Haahtela²⁶ | Yunen Rocío Huerta-Villalobos²⁴ |
Juan Carlos Ivancevich²⁷ | Inger Kull²⁸ | Piotr Kuna²⁹ | Violeta Kvedariene³⁰,³¹ |
Désirée E. Larenas Linnemann³² | Daniel Laune³³ | Michael Makris³⁴ |
Erik Melén³⁵,³⁶ | Mário Morais-Almeida³⁷ | Ralph Mösges³⁸,³⁹ ID |
Joaquim Mullol⁴⁰ | Robyn E. O'Hehir⁴¹,⁴² | Nikolaos G. Papadopoulos⁴³,⁴⁴ |
Ana Margarida Pereira<fnref n="1" />,<fnref n="2" />,<fnref n="3" /> | Emmanuel P. Prokopakis⁴⁵ | Fotis Psarros⁴⁶ |
Frederico S. Regateiro⁴⁷,⁴⁸ | Sietze Reitsma²³ | Boleslaw Samolinski⁴⁹ |
Nicola Scichilone⁵⁰ | Jane da Silva⁵¹ | Cristiana Stellato⁵² | Ana Todo-Bom⁴⁷,⁴⁸ |
Peter Valentin Tomazic⁵³ | Sanna Toppila Salmi²⁶ | Antonio Valero⁵⁴ |
Arunas Valiulis⁵⁵,⁵⁶ | Erkka Valovirta⁵⁷,⁵⁸ | Michiel van Eerd²¹ |
Maria Teresa Ventura⁵⁹ | Arzu Yorgancioglu⁶⁰ | Xavier Basagaña<fnref n="12" />,⁶¹,⁶² |
Josep M. Antó<fnref n="12" />,⁶¹,⁶²,⁶³ | Jean Bousquet<fnref n="8" />,<fnref n="11" />,⁶⁴ ID | João Almeida Fonseca<fnref n="1" />,<fnref n="2" />,<fnref n="3" />

<footnote n="1">¹MEDCIDS - Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal</footnote>

<footnote n="2">²CINTESIS - Center for Health Technology and Services Research, University of Porto, Porto, Portugal</footnote>

<footnote n="3">³RISE - Health Research Network, University of Porto, Porto, Portugal</footnote>

<footnote n="4">⁴Department of Computing Science, Umeå University, Umeå, Sweden</footnote>

<footnote n="5">⁵Department of Otorhinolaryngology, Head and Neck Surgery, Section of Rhinology and Allergy, University Hospital Marburg, Philipps Universität Marburg, Marburg, Germany</footnote>

<footnote n="6">⁶Department of Otolaryngology, Head and Neck Surgery, Universitätsmedizin Mainz, Mainz, Germany</footnote>

<footnote n="7">⁷Center for Rhinology and Allergology, Wiesbaden, Germany</footnote>

<footnote n="8">⁸Department of Dermatology and Allergy, Comprehensive Allergy Center, Charité Universitätsmedizin Berlin, Humboldt-Universität zu Berlin, Berlin, Germany</footnote>

<footnote n="9">⁹MASK-air, Montpellier, France</footnote>

<footnote n="10">¹⁰Medical Consulting Czarlewski, Levallois, France</footnote>

<footnote n="11">¹¹MACVIA-France, Montpellier, France</footnote>

<footnote n="12">¹²ISGlobAL, Universitat Pompeu Fabra (UPF), CIBER Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain</footnote>
¹³Equipe d'Epidémiologie Respiratoire Intégrative, Université Paris-Saclay, UVSQ, Université Paris-Sud, INSERM, CESP, Villejuif, France

¹⁴Department of Dermatology and Allergy Centre, Odense Research Center for Anaphylaxis (ORCA), Odense University Hospital, Odense, Denmark

¹⁵Quality Use of Respiratory Medicine Group, Woolcock Institute of Medical Research, The University of Sydney, Sydney, New South Wales, Australia

¹⁶Sydney Local Health District, Glebe, New South Wales, Australia

¹⁷Department of Medical Sciences, Allergy and Clinical Immunology Unit, University of Torino & Mauriziano Hospital, Torino, Italy

¹⁸Allergy Section, Department of Internal Medicine, Hospital Vall d'Hebron & ARADYAL research network, Barcelona, Spain

¹⁹Fundação ProAR -, Federal University of Bahia, Salvador, Brazil

²⁰WHO GARD Planning Group, Salvador, Brazil

²¹Peercode BV, Geldermalsen, The Netherlands

²²Laboratoire de Pharmacologie, UPRES EA220, Pôle des Maladies des Voies Respiratoires, Hôpital Foch, Université Paris-Saclay, Suresnes, France

²³Department of Otorhinolaryngology, Amsterdam University Medical Centres, Amsterdam, the Netherlands

²⁴Hospital General Regional, IMSS General Regional Hospital 1 Dr. Carlos Mac Gregor Sanchez Navarro, Mexico City, Mexico

²⁵Department of Pulmonary Diseases, Istanbul University-Cerrahpaşa, Cerrahpasa Faculty of Medicine, Istanbul, Turkey

²⁶Skin and Allergy Hospital, Helsinki University Hospital, Helsinki, Finland

²⁷Servicio de Alergia e Immunologia, Clinica Santa Isabel, Buenos Aires, Argentina

²⁸Department of Clinical Science and Education, Södersjukhuset, Karolinska Institutet, Stockholm, Sweden

²⁹Division of Internal Medicine, Asthma and Allergy, Barlicki University Hospital, Medical University of Lodz, Poland

³⁰Department of Pathology, Faculty of Medicine, Institute of Biomedical Sciences, Vilnius University, Vilnius, Lithuania

³¹Clinic of Chest Diseases and Allergology, Faculty of Medicine, Institute of Clinical Medicine, Vilnius University, Vilnius, Lithuania

³²Center of Excellence in Asthma and Allergy, Hospital Médica Sur, México City, Mexico

³³KYomed INNOV, Montpellier, France

³⁴Allergy Unit "D Kalogeromitros", 2nd Dpt of Dermatology and Venereology, National & Kapodistrian University of Athens, "Attikon" University Hospital, Athens, Greece

³⁵Sachs' Children and Youth Hospital, Södersjukhuset, Stockholm, Sweden

³⁶Institute of Environmental Medicine, Karolinska Institutet, Stockholm, Sweden

³⁷Allergy Center, CUF Descobertas Hospital, Lisbon, Portugal

³⁸Medical Faculty, Institute of Medical Statistics, and Computational Biology, University of Cologne, Cologne, Germany

³⁹CRI-Clinical Research International-Ltd, Hamburg, Germany

⁴⁰Rhinology Unit & Smell Clinic, ENT Department, Hospital Clínic, Clinical & Experimental Respiratory Immunoallergy, IDIBAPS, University of Barcelona, CIBERES, Barcelona, Spain

⁴¹Department of Allergy, Immunology and Respiratory Medicine, Central Clinical School, Monash University, Melbourne, Victoria, Australia

⁴²Alfred Health, Melbourne, Victoria, Australia

⁴³Center for Pediatrics and Child Health, Institute of Human Development, Royal Manchester Children's Hospital, University of Manchester, Manchester, UK

⁴⁴Allergy Department, 2nd Pediatric Clinic, Athens General Children's Hospital "P&A Kyriakou", University of Athens, Athens, Greece

⁴⁵Department of Otorhinolaryngology, University of Crete School of Medicine, Heraklion, Greece

⁴⁶Allergy Department, Athens Naval Hospital, Athens, Greece

⁴⁷Allergy and Clinical Immunology Unit, Institute of Immunology, Faculty of Medicine, Centro Hospitalar e Universitário de Coimbra, University of Coimbra, Portugal

⁴⁸Faculty of Medicine, ICBR - Coimbra Institute for Clinical and Biomedical Research, CIBB, University of Coimbra, Coimbra, Portugal

⁴⁹Department of Prevention of Environmental Hazards and Allergology, Medical University of Warsaw, Warsaw, Poland

⁵⁰PROMISE Department, University of Palermo, Palermo, Italy

⁵¹Allergy Service, Internal Medicine Department, University Hospital of Federal University of Santa Catarina (HU-UFSC), Florianópolis, Brazil

⁵²Department of Medicine, Surgery and Dentistry "Scuola Medica Salernitana", University of Salerno, Salerno, Italy

⁵³Department of ENT, Medical University of Graz, Graz, Austria

⁵⁴Pneumology and Allergy Department CIBERES and Clinical & Experimental Respiratory Immunoallergy, IDIBAPS, University of Barcelona, Spain

⁵⁵Department of Public Health, Vilnius University Institute of Clinical Medicine, Clinic of Children's Diseases, Institute of Health Sciences, Vilnius, Lithuania

⁵⁶European Academy of Paediatrics (EAP/UEMS-SP), Brussels, Belgium

⁵⁷Department of Lung Diseases and Clinical Immunology, University of Turku, Turku, Finland

⁵⁸Terveystalo Allergy Clinic, Turku, Finland

⁵⁹Unit of Geriatric Immunoallergology, University of Bari Medical School, Bari, Italy
**Correspondence**

Jean Bousquet, CHU Arnaud de Villeneuve,
371 Avenue du Doyen Gaston Giraud, 34295
Montpellier Cedex 5, France.

Email: jean.bousquet@orange.fr

**Funding information**

ARIA

Open access funding enabled and organized by
Projekt DEAL.

**Abstract**

**Background:** MASK-air® is an app that supports allergic rhinitis patients in disease control. Users register daily allergy symptoms and their impact on activities using visual analog scales (VASs). We aimed to assess the concurrent validity, reliability, and responsiveness of these daily VASs.

**Methods:** Daily monitoring VAS data were assessed in MASK-air® users with allergic rhinitis. Concurrent validity was assessed by correlating daily VAS values with those of the EuroQol-5 Dimensions (EQ-5D) VAS, the Control of Allergic Rhinitis and Asthma Test (CARAT) score, and the Work Productivity and Activity Impairment Allergic Specific (WPAI-AS) Questionnaire (work and activity impairment scores). Intra-rater reliability was assessed in users providing multiple daily VASs within the same day. Test-retest reliability was tested in clinically stable users, as defined by the EQ-5D VAS, CARAT, or "VAS Work" (i.e., VAS assessing the impact of allergy on work). Responsiveness was determined in users with two consecutive measurements of EQ-5D-VAS or "VAS Work" indicating clinical change.

**Results:** A total of 17,780 MASK-air® users, with 317,176 VAS days, were assessed. Concurrent validity was moderate-high (Spearman correlation coefficient range: 0.437-0.716). Intra-rater reliability intraclass correlation coefficients (ICCs) ranged between 0.870 (VAS assessing global allergy symptoms) and 0.937 (VAS assessing allergy symptoms on sleep). Test-retest reliability ICCs ranged between 0.604 and 0.878-"VAS Work" and "VAS asthma" presented the highest ICCs. Moderate/large responsiveness effect sizes were observed—the sleep VAS was associated with lower responsiveness, while the global allergy symptoms VAS demonstrated higher responsiveness.

**Conclusion:** In MASK-air®, daily monitoring VASs have high intra-rater reliability and moderate-high validity, reliability, and responsiveness, pointing to a reliable measure of symptom loads.

**KEYWORDS**

allergic rhinitis, mobile health, reliability, responsiveness, visual analog scales

# 1 | INTRODUCTION

Allergic rhinitis is a burdensome condition contributing to a substantial loss of work and school productivity, as well as to decreased quality of life.¹,² While there have been important advances on the treatment of allergic rhinitis, many patients remain poorly controlled.³ Mobile health-based approaches may contribute to addressing this problem.⁴ As an example, MASK-air® is a mobile app available in 25 countries, comprising a daily monitoring questionnaire assessing the impact that allergic symptoms have on the user each day.³,⁵⁻¹⁰

In MASK-air®, visual analog scales (VASs) are used for several questions on daily monitoring. Such VASs range from "not at all bothersome" (0) to "extremely bothersome" (100) and indicate the degree to which nose, eye, or asthma symptoms bother users during the day (or specifically impact their work or sleep activities). While the concurrent validity of some of these VASs has already been assessed,⁵ it has not yet been established for all VASs, particularly when taking all validated comparators into account. In addition, their reliability and responsiveness have not yet been evaluated. In fact, reliability estimates are needed to provide information on the

<footnote n="60">⁶⁰Department of Pulmonary Diseases, Faculty of Medicine, Celal Bayar University, Manisa, Turkey</footnote>

<footnote n="61">⁶¹Universitat Pompeu Fabra (UPF), Barcelona, Spain</footnote>

<footnote n="62">⁶²CIBER Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain</footnote>

<footnote n="63">⁶³IMIM (Hospital del Mar Research Institute), Barcelona, Spain</footnote>

<footnote n="64">⁶⁴Centre Hospitalier Universitaire, Montpellier, France</footnote>
stability of measures/inputs obtained at different times of the day from the same users (intra-rater reliability) or on different days in users considered clinically stable (test-retest reliability).¹¹ On the other hand, responsiveness estimates inform on the ability of daily monitoring VASs to change over a specific period of time in cases where changes in a reference measure of health status have occurred.¹²

An assessment of such properties is essential for determining whether daily monitoring VASs can actually be used as a reliable tool for measuring rhinitis control. Therefore, this study aimed to assess concurrent validity, intra-rater and test-retest reliability, as well as the responsiveness of MASK-air® daily monitoring VASs.

## 2 | METHODS

### 2.1 | Study design

We assessed the concurrent validity, reliability, and responsiveness of each daily monitoring VAS of MASK-air®. Reliability was evaluated by assessing intra-rater reliability (assessing the agreement between multiple values provided by the same users within the same day) and test-retest reliability (assessing the agreement between different daily VAS results provided by clinically stable patients). The main analysis concerned all countries in which MASK-air® is available. Sub-analyses using only data from European users were also performed.

### 2.2 | Setting and participants

MASK-air® has been available since 2015. It is currently being used in 25 countries and 19 languages (www mask-air com). We included the daily monitoring data of MASK-air® users aged 16–90 years and with a self-reported diagnosis of allergic rhinitis.

MASK-air® is used by people who find it on the Internet (namely on Apple App store or Google Play). Some of the users are patients who were asked by their physicians to use it. However, due to privacy rules, it is impossible to determine how patients come to use the app.

### 2.3 | Ethics

MASK-air® is CE1-registered but was not considered by the Ethical Committee of the Cologne Hospital (2017) as a medical device, given that it does not provide any recommendations concerning treatment or diagnosis. MASK-air® follows GDPR regulations. An independent review board approval was not required for this specific study, as all data had been anonymized prior to the study (including geolocation-related data) using k-anonymity, and users agreed to have their data analyzed in the terms of use (translated into all languages and first customized according to the legislation of each country, allowing the use of the results for research purposes).¹³,¹⁴

### 2.4 | Data sources and variables

We analyzed the MASK-air® daily monitoring data up to December 6, 2020. The daily monitoring of symptoms comprises six mandatory questions (addressing the period of 1 day), whose responses are provided by means of VASs:

i. How much the overall allergic symptoms bothered the user on that day ("VAS global allergy symptoms");

ii. How much nasal symptoms bothered the user on that day ("VAS nose");

iii. How much ocular symptoms bothered the user on that day ("VAS eyes");

iv. How much asthma symptoms bothered the user on that day ("VAS asthma");

v. How well the user slept on the previous night ("VAS sleep"); and

vi. How sleepy the user was during the day ("VAS sleepiness").

In addition, if users report that they are working on that day, they are asked how much their allergic symptoms affected work activities on that day ("VAS work").

When reporting daily VAS, users are asked to provide their daily medication using a scroll list customized for each country.¹⁵

When responding to the MASK-air® daily monitoring questionnaire, it is not possible to skip any of the questions, precluding missing data. While symptoms should be monitored on a daily basis, some users may have provided more than one daily input.

In addition to the daily monitoring of symptoms, MASK-air® users need to provide further clinical information (e.g., indicate their diagnosed allergies), and may respond to other questionnaires, including EuroQol-5 Dimensions (EQ-5D-5L), the Control of Allergic Rhinitis and Asthma Test (CARAT), Work Productivity and Activity Impairment: Allergic Specific (WPAI:AS), and the Epworth Sleepiness Scale (ESS). MASK-air® users can answer these questionnaires on any day they want to, and either before or after answering the daily monitoring questionnaire. EQ-5D-5L assesses the respondents' health status through five dimensions/questions (each with five levels) followed by a VAS assessing the general health status on that day.¹⁶,¹⁷ CARAT is a 10-item questionnaire assessing the control of allergic rhinitis and asthma in the previous 4 weeks, with four questions specifically concerning nasal symptoms.¹⁸,¹⁹ WPAI:AS is a 9-item questionnaire assessing the weekly impact of allergies on work and academic productivity, with three of its questions allowing estimation of the overall work impairment due to allergy, and the ninth question assessing the overall activity impairment due to allergy.²⁰,²¹ Finally, ESS evaluates respondents' sleepiness by assessing the chances of dozing in eight possible scenarios on that day.²²,²³

### 2.5 | Biases

Potential information biases were addressed by restricting our analyses to data from users with a self-reported diagnosis of allergic
rhinitis. Exclusion of data from users aged less than 16 years allowed us to address potential variability associated with age (i.e., differences between children and adults).

## 2.6 | Sample size

We did not perform sample size calculation, but rather analyzed all data from users meeting the eligibility criteria and with valid data. Nevertheless, analyses were not performed for situations/comparators in which the number of users providing data for at least one daily monitoring VAS never exceeded 50.

## 2.7 | Data analysis

Concurrent validity was assessed using statistical functions in Microsoft Excel 2016. All the other analyses were performed using software R (version 4.0, R Foundation for Statistical Computing, Vienna, Austria), with intraclass correlation coefficients (ICCs) being calculated with the "irr" package.²⁴

### 2.7.1 | Concurrent validity

Concurrent validity was assessed by computing Spearman correlation coefficients for the associations between three daily monitoring VASs ("VAS global allergy symptoms," "VAS Nose," and "VAS work") and EQ-5D VAS, CARAT (considering CARAT as a whole, as well as just the first four questions of CARAT—which concern the upper airways—and just the last six questions) and WPAI:AS (considering the "percentage of overall work impairment due to allergy" and question number 9—"percentage of activity impairment due to allergy"). In addition, correlations between different daily monitoring VASs were computed. Confidence intervals were estimated with alpha at 0.001 (indicating a 99.9% confidence level) and with standard deviation being Fisher's z-transformation of the Spearman correlation coefficient. We considered coefficients of 0.5 to 0.8 (or -0.8 to -0.5) to indicate moderate correlation, and of >0.8 (or <-0.8) to indicate strong correlation.

### 2.7.2 | Assessment of intra-rater reliability

The assessment of intra-rater reliability for daily monitoring VASs was estimated considering users providing multiple inputs within the same day. For each day of use (of the same user) with multiple inputs for the same VAS, we computed the difference between the first and the second inputs for each VAS. We computed the average difference for each VAS, as well as the frequency of cases in which (i) such same-day values were the same, (ii) the difference between such same-day values did not exceed 10 units, and (iii) the difference between such same-day values exceeded 10 units

(differences lower than 10 units in MASK-air® VAS point to low intra-individual response variability⁵). We calculated the ICC for assessment of intra-rater reliability (using two-way models estimating absolute agreement, based on average measurements²⁵,²⁶), also taking into account the first and second inputs within the same day by the same user. A sensitivity analysis was performed considering the first and last inputs for each VAS (instead of the first and second inputs).

### 2.7.3 | Assessment of test-retest reliability

We assessed test-retest reliability for each daily monitoring VAS. Assessment of test-retest reliability implies the identification of users with two measurements of a validated comparator indicating clinical stability. In this study, we used three different validated comparators for assessment of test-retest reliability: EQ-5D VAS, CARAT, and "VAS work" (despite being a daily monitoring VAS, the validity of "VAS work" was demonstrated in previous studies as well as in the assessment of its concurrent validity with WPAI:AS). WPAI:AS ("percentage of overall work impairment due to allergy") was also used to define clinical stability when assessing test-retest reliability for "VAS work." On the other hand, the ESS was used to define clinical stability when assessing test-retest reliability for "VAS sleep" and "VAS sleepiness."

Clinical stability was assumed whenever a user had two consecutive measurements less than 5 weeks apart, with results for validated comparators having a difference smaller than the minimal clinically important difference (MCID) value. Whenever the same user had more than two consecutive measurements (or more than one set of measurements) meeting the aforementioned criteria, the first two measurements were selected. Agreement was assessed by estimating ICCs using two-way models estimating absolute agreement, based on average measurements.²⁵,²⁶ We considered that ICCs of <0.5 indicate low reliability (both for test-retest reliability and intra-rater reliability), those of 0.5-0.75 indicate moderate reliability, and those of >0.75 indicate high reliability.²⁵

In the case of CARAT, differences ≤3 were considered to be lower than the MCID.²⁷ For the remaining comparators, no MCID for patients with allergic rhinitis has been defined. Therefore, such values were determined based on distribution-based methods—we considered the MCID to correspond to 0.5 × standard deviation of the baseline observations.²⁸ Based on such an approach, we estimated an MCID of 10 points for EQ-5D-VAS, of 11 points for "VAS work," and of 14% for WPAI:AS. For ESS, we considered clinical stability if the same category was observed for the different daily measurements.²⁹

### 2.7.4 | Responsiveness

We assessed responsiveness for each daily monitoring VAS. Assessment of responsiveness implies the identification of users with two measurements of a validated comparator indicating clinical change. In
<figure><img src="image_4.png" /><figcaption>FIGURE 1 Flow diagram illustrating the selection of users/days of use meeting eligibility criteria</figcaption></figure>

this study, validated comparators to indicate clinical change included the EQ-5D VAS and "VAS work." We were not able to use CARAT, WPAI:AS (to assess "VAS work" responsiveness), or ESS (to assess "VAS sleep" and "VAS sleepiness" responsiveness) as comparators, given that clinical change based on such measurements was observed in less than 50 users for every daily monitoring VAS.

Clinical change was assumed whenever a user had two consecutive measurements more than 5 weeks apart, with results for validated comparators having a difference equal to or higher than the MCID value. Following the impossibility of using CARAT (which assesses a period of 4 weeks) as a comparator, we performed a sub-analysis defining the EQ-5D VAS and "VAS work" clinical change based on periods more than 3 weeks apart. Whenever the same user had more than two consecutive measurements (or more than one set of measurements) meeting the aforementioned criteria, the first two inputs were selected.

Responsiveness was determined by calculating Cohen's effect size and the standardized response mean (SRM).¹² Cohen's effect size was calculated by dividing the mean difference between daily monitoring VASs by the standard-deviation of "baseline" VAS. The SRM was calculated by dividing the mean by the standard-deviation of the differences between the daily monitoring VASs. For each of these measures, values of 0.2-0.5 were considered to represent small effect sizes, 0.5-0.8 medium effect sizes, and >0.8 large effect sizes.¹²

## 3 | RESULTS

### 3.1 | MASK-air® data

Up until December 6, 2020, 39,810 people had used MASK-air®. Of these, 17,780 were aged 16-90 years, had self-reported allergic rhinitis, and had responded at least once to the daily monitoring VAS (Figure 1). These 17,780 users provided daily monitoring VAS data for 317,176 days, corresponding to an average of 17.8 days per user.

Daily monitoring VAS median values ranged from 0 ("VAS asthma") to 17 ("VAS sleep") (Table 1). For the EQ-5D VAS and CARAT, median values of 80 (interquartile range = 14) and 16 (interquartile range = 10) were respectively observed.

### 3.2 | Concurrent validity

Concurrent validity (Table 2) was calculated by mutually correlating nine different variables, including three daily monitoring VASs ("VAS global allergy symptoms," "VAS nose," and "VAS work") and six comparators. For VAS global allergy symptoms, correlations varied between 0.437 [95% confidence interval (CI) = 0.411-0.462] (CARAT -questions 5-10) and 0.663 [95% CI = 0.637-0.688] (WPAI:AS-question 9). Similar results were observed for VAS nose. For "VAS
<table><caption>TABLE 1 Number of users and days of use for which there is information on daily monitoring visual analog scales (VASs) and on validated questionnaires in the MASK-air® database</caption><thead><tr><th></th><th>N users</th><th>N days</th><th>Median value (Q1-Q3)</th></tr></thead><tbody><tr><td>Daily monitoring VAS</td><td></td><td></td><td></td></tr><tr><td>VAS global allergy symptoms<sup>a</sup></td><td>17,780</td><td>317,154</td><td>11 (2-29)</td></tr><tr><td>VAS eyes<sup>a</sup></td><td>17,780</td><td>317,176</td><td>4 (0-18)</td></tr><tr><td>VAS nose<sup>a</sup></td><td>17,780</td><td>317,176</td><td>12 (2-30)</td></tr><tr><td>VAS asthma<sup>a</sup></td><td>17,023</td><td>312,625</td><td>0 (0-10)</td></tr><tr><td>VAS work<sup>a</sup></td><td>11,980</td><td>149,732</td><td>8 (0-23)</td></tr><tr><td>VAS sleep<sup>a</sup></td><td>9009</td><td>151,011</td><td>17 (7-37)</td></tr><tr><td>VAS sleepiness<sup>a</sup></td><td>8911</td><td>142,444</td><td>16 (6-35)</td></tr><tr><td>EQ-5D VAS<sup>b</sup></td><td>2478</td><td>4324</td><td>80 (66-90)</td></tr><tr><td>CARAT<sup>c</sup></td><td>1086</td><td>2042</td><td>16 (11-21)</td></tr><tr><td>WPAI:AS<sup>d</sup></td><td>1411</td><td>2308</td><td>15 (2-44)</td></tr><tr><td>ESS<sup>e</sup></td><td>1081</td><td>2055</td><td>7 (4-11)</td></tr></tbody></table>

Note: The number of observations/days for each daily monitoring VAS is not always the same as, despite the impossibility of skipping questions when filling in each daily monitoring questionnaire, the MASK-air® daily monitoring questionnaire has evolved over time, with some questions having been added later than others. On the other hand, VAS Work can only be filled in for days when users report to be working.

Abbreviations: CARAT, Control of Allergic Rhinitis and Asthma Test; ESS, Epworth, Sleepiness Scale; Q1-Q3, first quartile-third quartile; WPAI:AS, Work Productivity and Activity Impairment Allergic Specific (median and quartiles for "percent overall work impairment due to allergy").

<sup>a</sup>Scale ranging from 0 to 100, with higher values indicating worse status.

<sup>b</sup>Scale ranging from 0 to 100, with higher values indicating better health status.

<sup>c</sup>Scale ranging from 0 to 30, with higher values indicating better allergic rhinitis and asthma control.

<sup>d</sup>Percent overall work impairment due to allergy-scale ranging from 0 to 100, with higher values indicating greater work impairment.

<sup>e</sup>Scale ranging from 0 to 24, with higher values indicating greater daytime sleepiness.

work," the strongest correlations were observed for WPAI:AS overall work impairment (0.716; 95% CI = 0.676-0.752) and for WPAI:AS question 9 (0.713; 95% CI = 0.675-0.747).

### 3.3 | Intra-rater reliability

Between 2412 ("VAS work") and 5827 ("VAS nose" and "VAS eyes") days with more than one daily monitoring VAS input provided by the same user were recorded. For all VASs, more than 50% of the days had no differences in the first and second values provided within the same day (Table 3). Differences between the first and second daily values differing by more than 10 units ranged between 11.2% ("VAS asthma") and 24.4% ("VAS nose"). ICCs varied between 0.870 ("VAS global allergy symptoms") and 0.937 ("VAS sleep"). Similar results were observed when analyzing data from MASK-air® European users, or when taking into account the first and last daily measurements (Tables 3 and 4).

### 3.4 | Test-retest reliability

Using clinical stability defined according to EQ-5D VAS, the test-retest reliability of daily monitoring VASs was based on data from 102 ("VAS work") to 270 ("VAS global allergy symptoms," "VAS nose," "VAS eyes," and "VAS asthma") users. The lowest ICC concerned "VAS sleep" (ICC = 0.675; 95% CI = 0.527-0.777) and "VAS sleepiness" (ICC = 0.686; 95% CI = 0.544-0.784), while the highest concerned "VAS work" (ICC = 0.823; 95% CI = 0.738-0.880) and "VAS asthma" (ICC = 0.857; 95% CI = 0.818-0.887) (Table 5).

Using clinical stability defined according to CARAT, the test-retest reliability of daily monitoring VASs was based on data from up to 134 users. The lowest ICC concerned "VAS sleepiness" (ICC = 0.550; 95% CI = 0.344-0.692) and "VAS sleep" (ICC = 0.604; 95% CI = 0.425-0.727), while the highest concerned "VAS work" (ICC = 0.823; 95% CI = 0.605-0.915) and "VAS asthma" (ICC = 0.840; 95% CI = 0.775-0.886) (Table 5).
<table><caption>TABLE 2 Concurrent validity of visual analog scales (VASs) for global allergy symptoms (versus six comparators), nose symptoms and impact on work, using data from MASK-air® users</caption><thead><tr><th></th><th>VAS global<br>allergy<br>symptoms</th><th>VAS nose</th><th>VAS work</th><th>EQ-5D VAS</th><th>CARAT</th><th>CARAT (Q1-4)</th><th>CARAT (Q5-10)</th><th>WPAI:AS<sup>a</sup></th><th>WPAI:AS (Q9)<sup>b</sup></th></tr></thead><tbody><tr><td>VAS global allergy symptoms</td><td>-</td><td>0.877 [0.875; 0.879] (317,154)</td><td>0.814 [0.811; 0.817] (149,726)</td><td>-0.512 [-0.533; -0.491] (4322)</td><td>-0.542<sup>c</sup> [-0.571; -0.512] (2042)</td><td>-0.497 [-0.525; -0.469] (2042)</td><td>-0.437 [-0.462; -0.411] (2042)</td><td>0.623 [0.591; 0.653] (2306)</td><td>0.663 [0.637; 0.688] (3279)</td></tr><tr><td>VAS nose</td><td>-</td><td>-</td><td>0.773 [0.769; 0.776] (149,732)</td><td>-0.481 [-0.502; -0.461] (4324)</td><td>-0.523<sup>c</sup> [-0.553; -0.492] (2042)</td><td>-0.530 [-0.561; -0.499] (2042)</td><td>-0.388 [-0.413; -0.362] (2042)</td><td>0.590 [0.559; 0.620] (2308)</td><td>0.613 [0.586; 0.638] (3281)</td></tr><tr><td>VAS work</td><td>-</td><td>-</td><td>-</td><td>-0.504 [-0.532; -0.475] (2324)</td><td>-0.359<sup>c</sup> [-0.393; 0.324] (965)</td><td>-0.249 [-0.274; 0.224] (965)</td><td>-0.358 [-0.392; -0.323] (965)</td><td>0.716 [0.676; 0.752] (1461)</td><td>0.713 [0.675; 0.747] (1656)</td></tr><tr><td>EQ-5D VAS</td><td>-</td><td>-</td><td>-</td><td>-</td><td>0.556 [0.515; 0.596] (1299)</td><td>0.371 [0.340; 0.401] (1299)</td><td>0.563 [0.522; 0.601] (1299)</td><td>-0.469 [-0.500; -0.438] (1783)</td><td>-0.489 [-0.515; -0.462] (2478)</td></tr><tr><td>CARAT</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>0.806 [0.776; 0.833] (2042)</td><td>0.882 [0.857; 0.902] (2042)</td><td>-0.559 [-0.613; -0.500] (641)</td><td>-0.599 [-0.643; -0.552] (1038)</td></tr><tr><td>CARAT (Q1-4)</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>0.451 [0.423; 0.479] (2042)</td><td>-0.350 [-0.392; -0.308] (641)</td><td>-0.443 [-0.481; -0.403] (1038)</td></tr><tr><td>CARAT (Q5-10)</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-0.556 [-0.610; -0.497] (641)</td><td>-0.530 [-0.572; -0.485] (1038)</td></tr><tr><td>WPAI:AS</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>0.770 [0.681; 0.836] (314)</td></tr><tr><td>WPAI:AS (Q9)</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

Abbreviations: CARAT, Control of Allergic Rhinitis and Asthma Test; CI, confidence interval; Q, question; WPAI:AS, Work Productivity and Activity Impairment Allergy Specific.

<sup>a</sup>Percent overall work impairment due to allergy.

<sup>b</sup>Question 9 of WPAI:AS assesses overall activity impairment due to allergy.

<sup>c</sup>Spearman correlation coefficients when considering the average VAS values for the 4 weeks prior to answering CARAT: -0.572 for the correlation between CARAT and VAS Global allergy symptoms; -0.559 for the correlation between CARAT and VAS Nose; and -0.466 for the correlation between CARAT and VAS Work.
<figure><img src="image_5.png" /><figcaption>TABLE 3 Results of intra-rater reliability for daily monitoring visual analog scales (VASs) using data from MASK-air® users where MASK-air® is available</figcaption></figure>

<table><thead><tr><th></th><th>N days with more than one daily value<sup>a</sup></th><th>Mean difference between first and second daily values (SD)</th><th>N (%) observations with no difference between first and second daily values</th><th>N (%) observations with difference between first and second daily values >10 VAS units<sup>a</sup></th><th>ICC</th></tr></thead><tbody><tr><td colspan="6"><strong>A. Data of users from all countries where MASK-air® is available</strong></td></tr><tr><td>VAS global allergy symptoms<sup>b</sup></td><td>5823 [1.8]</td><td>1.6 (17.3)</td><td>2809 (53.2)</td><td>994 (18.8) [0.3]</td><td>0.870 [0.863-0.877]</td></tr><tr><td>VAS eyes<sup>b</sup></td><td>5827 [1.8]</td><td>1.0 (14.5)</td><td>3918 (67.2)</td><td>1002 (17.2) [0.3]</td><td>0.894 [0.889-0.900]</td></tr><tr><td>VAS nose<sup>b</sup></td><td>5827 [1.8]</td><td>2.0 (16.8)</td><td>3219 (55.2)</td><td>1421 (24.4) [0.4]</td><td>0.890 [0.883-0.897]</td></tr><tr><td>VAS asthma<sup>b</sup></td><td>5700 [1.8]</td><td>0.7 (12.2)</td><td>4365 (76.6)</td><td>636 (11.2) [0.2]</td><td>0.906 [0.901-0.911]</td></tr><tr><td>VAS work<sup>b</sup></td><td>2412 [1.6]</td><td>1.4 (13.3)</td><td>1607 (66.7)</td><td>443 (17.3) [0.3]</td><td>0.902 [0.894-0.910]</td></tr><tr><td>VAS sleep<sup>b</sup></td><td>2732 [1.8]</td><td>0.6 (12.1)</td><td>1966 (72.0)</td><td>359 (13.1) [0.2]</td><td>0.937 [0.932-0.942]</td></tr><tr><td>VAS sleepiness<sup>b</sup></td><td>2567 [1.8]</td><td>1.3 (12.2)</td><td>1838 (71.6)</td><td>367 (14.3) [0.3]</td><td>0.931 [0.925-0.936]</td></tr><tr><td colspan="6"><strong>B. Data of European MASK-air® users</strong></td></tr><tr><td>VAS global allergy symptoms<sup>b</sup></td><td>3546 [1.6]</td><td>1.8 (17.4)</td><td>1571 (44.3)</td><td>1048 (29.6) [0.5]</td><td>0.871 [0.862-0.880]</td></tr><tr><td>VAS eyes<sup>b</sup></td><td>3550 [1.6]</td><td>1.4 (14.5)</td><td>2221 (62.5)</td><td>686 (19.3) [0.3]</td><td>0.903 [0.896-0.910]</td></tr><tr><td>VAS nose<sup>b</sup></td><td>3550 [1.6]</td><td>2.3 (16.7)</td><td>1849 (52.1)</td><td>923 (26.0) [0.4]</td><td>0.892 [0.883-0.901]</td></tr><tr><td>VAS asthma<sup>b</sup></td><td>3423 [1.6]</td><td>1.0 (13.1)</td><td>2492 (72.8)</td><td>457 (13.4) [0.2]</td><td>0.904 [0.897-0.910]</td></tr><tr><td>VAS work<sup>b</sup></td><td>1415 [1.3]</td><td>1.6 (13.4)</td><td>868 (61.3)</td><td>284 (19.0) [0.3]</td><td>0.903 [0.892-0.913]</td></tr><tr><td>VAS sleep<sup>b</sup></td><td>1666 [1.6]</td><td>0.6 (10.5)</td><td>1193 (71.6)</td><td>212 (12.7) [0.2]</td><td>0.952 [0.947-0.956]</td></tr><tr><td>VAS sleepiness<sup>b</sup></td><td>1588 [1.6]</td><td>1.3 (12.4)</td><td>1096 (69.0)</td><td>251 (15.8) [0.2]</td><td>0.928 [0.920-0.935]</td></tr></tbody></table>

Abbreviations: ICC, intraclass correlation coefficient; SD, standard deviation.

Using “VAS work” to define clinical stability, the test-retest reliability of daily monitoring VASs was based on data from 2078 (“VAS sleepiness”) to 5763 (“VAS nose,” “VAS eyes,” and “VAS global allergy symptoms”) users. As with the two previous scenarios, the lowest ICC was observed for VAS sleep (ICC = 0.685; 95% CI = 0.658-0.711) and VAS sleepiness (ICC = 0.748; 95% CI = 0.723-0.770), while the highest ICC was observed for VAS asthma (ICC = 0.878; 95% CI = 0.871-0.884) (Table 5).

Similar results were observed when analyzing data from MASK-air European users (Table 5).

## 3.5 | Responsiveness analysis

Using the EQ-5D VAS to define clinical change (based on observations at least 5 weeks apart), we assessed the responsiveness of daily monitoring VASs based on data from up to 85 users (“VAS global allergy symptoms,” “VAS nose,” “VAS eyes,” and “VAS asthma”). Moderate effect sizes were observed for “VAS global allergy

symptoms” (Cohen's effect size = 0.572; SRM = 0.530), “VAS sleepiness” (Cohen's effect size = 0.717; SRM = 0.566), and “VAS work” (Cohen's effect size = 0.735; SRM = 0.719) (Table 6). For the remaining daily monitoring VASs, estimated effect sizes were lower than 0.5 (i.e., small effect sizes).

Using “VAS work” to define clinical change, we assessed responsiveness based on data from between 450 (“VAS sleepiness”) and 1157 (“VAS nose,” “VAS eyes,” and “VAS asthma”) users. We observed moderate effect sizes for “VAS asthma” (Cohen's effect size = 0.573; SRM = 0.512), “VAS sleepiness” (Cohen's effect size = 0.625; SRM = 0.554) and “VAS eyes” (Cohen's effect size = 0.711; SRM = 0.611), and large effect sizes for “VAS nose” (Cohen's effect size = 0.877; SRM = 0.781) and “VAS global allergy symptoms” (Cohen's effect size = 0.994; SRM = 0.931) (Table 6).

When defining clinical change based on a period more than 3 weeks apart, similar results were observed when using “VAS work” as the comparator to define clinical change, while overall lower effect sizes were observed with EQ-5D as a comparator (Table 6).

<footnote><sup>a</sup>Values in square brackets correspond to the percentage of such observations in relation to the total number of available observations for each VAS.</footnote>

<footnote><sup>b</sup>Scale ranging from 0 to 100, with higher values indicating worse status.</footnote>
<table><caption>TABLE 4 Results of intra-rater reliability for daily monitoring visual analog scales (VASs) using the first and last daily inputs</caption><thead><tr><th></th><th>N days with more than one daily value<sup>a</sup></th><th>Mean difference between first and last daily values (SD)</th><th>N (%) observations with no difference between first and last daily values</th><th>N (%) observations with difference between first and last daily values >10 VAS units<sup>a</sup></th><th>ICC</th></tr></thead><tbody><tr><td>VAS global allergy symptoms<sup>a</sup></td><td>5823 [1.8]</td><td>1.7 (18.0)</td><td>2719 (46.7)</td><td>1615 (27.7) [0.5]</td><td>0.859 [0.851-0.867]</td></tr><tr><td>VAS eyes<sup>a</sup></td><td>5827 [1.8]</td><td>1.0 (15.0)</td><td>3835 (65.8)</td><td>1042 (17.9) [0.3]</td><td>0.886 [0.880-0.892]</td></tr><tr><td>VAS nose<sup>a</sup></td><td>5827 [1.8]</td><td>2.0 (17.4)</td><td>3140 (53.9)</td><td>1472 (25.3) [0.5]</td><td>0.881 [0.873-0.888]</td></tr><tr><td>VAS asthma<sup>a</sup></td><td>5700 [1.8]</td><td>0.7 (12.6)</td><td>4305 (75.5)</td><td>625 (11.0) [0.2]</td><td>0.900 [0.895-0.905]</td></tr><tr><td>VAS work<sup>a</sup></td><td>2412 [1.6]</td><td>1.4 (14.0)</td><td>1569 (65.0)</td><td>443 (18.4) [0.3]</td><td>0.892 [0.883-0.901]</td></tr><tr><td>VAS sleep<sup>a</sup></td><td>2732 [1.8]</td><td>0.8 (12.3)</td><td>1940 (71.0)</td><td>360 (13.3) [0.2]</td><td>0.935 [0.930-0.940]</td></tr><tr><td>VAS sleepiness<sup>a</sup></td><td>2567 [1.8]</td><td>1.4 (12.4)</td><td>1810 (70.5)</td><td>368 (14.5) [0.3]</td><td>0.929 [0.923-0.934]</td></tr></tbody></table>

Abbreviations: ICC, intraclass correlation coefficient; SD, standard deviation.

## 4 DISCUSSION

In this study, we observed that, overall, daily monitoring VASs presented with high intra-rater reliability and moderate-high concurrent validity, test-retest reliability and responsiveness. This is particularly relevant when taking into account the fact that, overall, the VAS has been shown to be a simple and sensitive instrument for measuring allergic rhinitis symptoms, having been used in both randomized controlled trials and observational studies.³⁰ The incorporation of VASs into a mobile app and the demonstration of their reliability and responsiveness supports their use as a tool to guide users in controlling disease activity and adapting medication. Previous studies have already assessed other properties of daily monitoring VASs, observing strong correlation between “VAS work” and other VASs (namely “VAS global allergy symptoms,” “VAS nose,” “VAS eyes,” and “VAS asthma”).⁵,³¹

The high intra-rater reliability observed across the different daily monitoring VASs suggests that values provided within the same day for such scales do not tend to be substantially different. This is further supported by the consistency of results observed when the first and last daily inputs are taken into account (instead of the first and second ones).

On the other hand, test-retest analysis results indicate that daily monitoring VASs remain reasonably stable when clinical stability is attested by other validated comparators (EQ-5D VAS, “VAS work” and CARAT). Overall, the presented ICCs are not dissimilar to those estimating test-retest reliability within the context of other measurement tools used in patients with rhinitis. For the Rhinitis Control Assessment Test, an ICC of 0.78 was observed,³² while for CARAT, the ICC was of 0.82.¹⁹ The lower ICC observed for sleep-related VAS suggests that several other factors can potentially affect sleep, that the question is too simple (simply asking whether the user had slept well in the previous

night or felt sleepy during the day), and/or that these VASs are not associated with the severity of allergy. However, there are several studies suggesting that sleep is impaired by rhinitis.³³⁻³⁶ “VAS sleep” was also found to be associated with lower responsiveness. Responsiveness measures the occurrence of change (in this case, of daily monitoring VASs) in cases where clinical change is attested by other validated comparators.¹² Responsiveness is typically assessed by effect size measures reporting on the magnitude of variation in relation to baseline or between-subject variability, with higher values indicating larger changes.¹² Therefore, in this study, we observed that daily monitoring VASs more strongly accompanied clinically relevant changes in “VAS work” than clinically relevant changes in EQ-5D VAS. This may be related to the fact that the latter is not specific to allergic diseases. Differences of methods used to assess responsiveness impair comparisons with other tools used in rhinitis patients, such as the Rhinitis Control Assessment Test,³² CARAT,¹⁹ WPAI:AS,²⁰ and ARIA-C.³⁷

This study has important limitations that are worth noting. Firstly, for some analyses (e.g., assessment of responsiveness in relation to the EQ-5D VAS and CARAT), the number of users/days meeting the required conditions was relatively small (although not too dissimilar to the sample sizes used in the assessment of test-retest reliability in CARAT¹⁹ and in some groups of Rhinitis Control Assessment Test³²), negatively affecting estimate precision. Such small numbers are explained not only by the conditions required to assess test-retest reliability and responsiveness, but also by the fact that, in MASK-air®, CARAT, EQ-5D, WPAI:AS, and ESS are not mandatory questionnaires within the context of daily monitoring. On the other hand, for each analysis, the number of users/days of use is not the same for each daily monitoring VAS. This is explained by the later introduction of certain VASs (namely “VAS sleep” and “VAS sleepiness”) in the daily monitoring questionnaire, as well as by the

<footnote><sup>a</sup>Values in square brackets correspond to the percentage of such observations in relation to the total number of available observations for each VAS.</footnote>

<footnote><sup>b</sup>Scale ranging from 0 to 100, with higher values indicating worse status.</footnote>
<table><caption>TABLE 5 Results for test-retest analysis of the different daily monitoring visual analog scales (VASs) for MASK-air® users from the 25 countries where MASK-air® is available</caption><thead><tr><th rowspan="2"></th><th colspan="2">EQ-5D VAS</th><th colspan="2">CARAT</th><th colspan="2">Work VAS</th></tr><tr><th>N users</th><th>ICC [95% CI]</th><th>N users</th><th>ICC [95% CI]</th><th>N users</th><th>ICC [95% CI]</th></tr></thead><tbody><tr><td colspan="7"><strong>A. Data of users from all countries where MASK-air® is available</strong></td></tr><tr><td>VAS global allergy symptoms<sup>a</sup></td><td>270</td><td>0.753 [0.662-0.817]</td><td>134</td><td>0.745 [0.637-0.821]</td><td>5761</td><td>0.848 [0.831-0.862]</td></tr><tr><td>VAS nose<sup>a</sup></td><td>270</td><td>0.772 [0.708-0.821]</td><td>134</td><td>0.770 [0.668-0.839]</td><td>5763</td><td>0.841 [0.829-0.852]</td></tr><tr><td>VAS eyes<sup>a</sup></td><td>270</td><td>0.747 [0.677-0.802]</td><td>134</td><td>0.740 [0.633-0.816]</td><td>5763</td><td>0.831 [0.822-0.840]</td></tr><tr><td>VAS asthma<sup>a</sup></td><td>270</td><td>0.857 [0.818-0.887]</td><td>134</td><td>0.840 [0.775-0.886]</td><td>5763</td><td>0.878 [0.871-0.884]</td></tr><tr><td>VAS work<sup>a,b</sup></td><td>102</td><td>0.823 [0.738-0.880]</td><td>35</td><td>0.823 [0.605-0.915]</td><td>-</td><td>-</td></tr><tr><td>VAS sleep<sup>a,c,d</sup></td><td>110</td><td>0.675 [0.527-0.777]</td><td>112</td><td>0.604 [0.425-0.727]</td><td>2257</td><td>0.685 [0.658-0.711]</td></tr><tr><td>VAS sleepiness<sup>a,c,d</sup></td><td>112</td><td>0.686 [0.544-0.784]</td><td>111</td><td>0.550 [0.344-0.692]</td><td>2078</td><td>0.748 [0.723-0.770]</td></tr><tr><td colspan="7"><strong>B. Data of European MASK-air® users</strong></td></tr><tr><td>VAS global allergy symptoms<sup>a</sup></td><td>217</td><td>0.742 [0.636-0.814]</td><td>103</td><td>0.792 [0.692-0.859]</td><td>4115</td><td>0.855 [0.839-0.869]</td></tr><tr><td>VAS nose<sup>a</sup></td><td>217</td><td>0.760 [0.683-0.818]</td><td>103</td><td>0.804 [0.709-0.867]</td><td>4117</td><td>0.842 [0.829-0.853]</td></tr><tr><td>VAS eyes<sup>a</sup></td><td>217</td><td>0.711 [0.620-0.780]</td><td>103</td><td>0.748 [0.628-0.829]</td><td>4117</td><td>0.839 [0.829-0.849]</td></tr><tr><td>VAS asthma<sup>a</sup></td><td>217</td><td>0.856 [0.812-0.890]</td><td>103</td><td>0.858 [0.791-0.904]</td><td>4117</td><td>0.875 [0.867-0.882]</td></tr><tr><td>VAS work<sup>a,e</sup></td><td>83</td><td>0.827 [0.734-0.888]</td><td>31</td><td>0.809 [0.533-0.915]</td><td>-</td><td>-</td></tr><tr><td>VAS sleep<sup>a,f,g</sup></td><td>92</td><td>0.649 [0.470-0.768]</td><td>87</td><td>0.616 [0.412-0.750]</td><td>1591</td><td>0.686 [0.653-0.715]</td></tr><tr><td>VAS sleepiness<sup>a,f,g</sup></td><td>95</td><td>0.677 [0.515-0.784]</td><td>87</td><td>0.561 [0.332-0.713]</td><td>1507</td><td>0.739 [0.709-0.766]</td></tr></tbody></table>

Abbreviations: CARAT, Control of Allergic Rhinitis and Asthma Test; CI, confidence interval; ICC, intraclass correlation coefficient.

<sup>a</sup>Scale ranging from 0 to 100, with higher values indicating worse status.

<sup>b</sup>ICC for VAS work in clinically stable patients as defined by the percent overall work impairment of Work Productivity and Activity Impairment Allergic Specific = 0.749 [95% CI = 0.594-0.845] (N = 68 users).

<sup>c</sup>ICC for VAS sleep in clinically stable patients as defined by the Epworth Sleepiness Scale = 0.620 [95% CI = 0.424-0.749] (N = 91 users).

<sup>d</sup>ICC for VAS day Sleepiness in clinically stable patients as defined by the Epworth Sleepiness Scale = 0.627 [95% CI = 0.435-0.753] (N = 91 users).

<sup>e</sup>ICC for VAS work in clinically stable patients as defined by the percent overall work impairment of the Work Productivity and Activity Impairment Allergic Specific = 0.857 [95% CI = 0.732-0.921] (N = 51 users).

<sup>f</sup>ICC for VAS sleep in clinically stable patients as defined by the Epworth Sleepiness Scale = 0.647 [95% CI = 0.441-0.777] (N = 76 users).

<sup>g</sup>ICC for VAS day sleepiness in clinically stable patients as defined by the Epworth Sleepiness Scale = 0.667 [95% CI = 0.471-0.790] (N = 75 users).

fact that "VAS work" is only answered on days when users report to be working.

Intra-rater reliability was assessed based on different same-day questionnaires by the same patient. However, within the same day (e.g., during the pollen season), a patient may experience changes in his/her symptoms. This potential limitation, however, results in an underestimation of intra-rater reliability. That is, real intra-rater reliability values may possibly be even higher than those we obtained. On the other hand, we used consecutive measurements less than 5 weeks apart in the definition of clinical stability. We cannot, however, exclude the possibility of "unstable periods" (i.e., clinically relevant changes) between these measurements. Nevertheless, the magnitude of such phenomenon is not expected to be particularly high, on account of the observed similarities in the test-retest ICC calculated based on CARAT (which assesses the previous month) and EQ-5D-VAS or "VAS work" (which assess a single day). Another important limitation concerns the main validated comparators used to assess concurrent validity, test-retest reliability and

responsiveness. In fact, the EQ-5D VAS is not specific for allergic diseases, measuring instead how good or bad the health of the respondent is on that day. On the other hand, while CARAT is specific for asthma and allergic rhinitis, it assesses allergic rhinitis (and asthma) control within the period of the last 4 weeks,³⁸ while only one single day (the day being assessed) is contemplated in daily monitoring VASs. In addition, we were not able to use it as a comparator when assessing responsiveness, due to an insufficient sample size.

Furthermore, no MCID for allergic rhinitis patients had been previously defined for the EQ-5D VAS and for "VAS Work." We determined the corresponding MCID values within the context of this study, based on distribution-based methods instead of anchor-based methods. While the latter are often preferable, they imply the existence of an anchor-based estimate,³⁹ which was unavailable in this context (i.e., there was no "gold-standard" variable to which the EQ-5D VAS or "VAS work" could be compared accurately).
<table><caption>TABLE 6 Results for responsiveness analysis of the different daily monitoring visual analog scales (VASs) for MASK-air® users from the 25 countries where MASK-air® is available</caption><thead><tr><th rowspan="2"></th><th colspan="3">EQ-5D VAS</th><th colspan="3">VAS work</th></tr><tr><th>N users</th><th>Cohen ES</th><th>SRM</th><th>N users</th><th>Cohen ES</th><th>SRM</th></tr></thead><tbody><tr><td colspan="7"><strong>A. Observations at least 5 weeks apart</strong></td></tr><tr><td>VAS global allergy symptoms<sup>a</sup></td><td>85</td><td>0.572</td><td>0.530</td><td>1156</td><td>0.994</td><td>0.931</td></tr><tr><td>VAS nose<sup>a</sup></td><td>85</td><td>0.448</td><td>0.380</td><td>1157</td><td>0.877</td><td>0.781</td></tr><tr><td>VAS eyes<sup>a</sup></td><td>85</td><td>0.417</td><td>0.411</td><td>1157</td><td>0.711</td><td>0.611</td></tr><tr><td>VAS asthma<sup>a</sup></td><td>85</td><td>0.366</td><td>0.413</td><td>1157</td><td>0.573</td><td>0.512</td></tr><tr><td>VAS work<sup>a</sup></td><td>29</td><td>0.735</td><td>0.719</td><td>-</td><td>-</td><td>-</td></tr><tr><td>VAS sleep<sup>a</sup></td><td>27</td><td>0.386</td><td>0.411</td><td>484</td><td>0.482</td><td>0.405</td></tr><tr><td>VAS sleepiness<sup>a</sup></td><td>28</td><td>0.717</td><td>0.566</td><td>450</td><td>0.625</td><td>0.554</td></tr><tr><td colspan="7"><strong>B. Observations at least 3 weeks apart</strong></td></tr><tr><td>VAS global allergy symptoms<sup>a</sup></td><td>108</td><td>0.467</td><td>0.451</td><td>1515</td><td>0.992</td><td>0.954</td></tr><tr><td>VAS nose<sup>a</sup></td><td>108</td><td>0.423</td><td>0.379</td><td>1517</td><td>0.893</td><td>0.810</td></tr><tr><td>VAS eyes<sup>a</sup></td><td>108</td><td>0.282</td><td>0.282</td><td>1517</td><td>0.694</td><td>0.598</td></tr><tr><td>VAS asthma<sup>a</sup></td><td>108</td><td>0.328</td><td>0.351</td><td>1517</td><td>0.578</td><td>0.513</td></tr><tr><td>VAS work<sup>a</sup></td><td>39</td><td>0.524</td><td>0.575</td><td>-</td><td>-</td><td>-</td></tr><tr><td>VAS sleep<sup>a</sup></td><td>37</td><td>0.345</td><td>0.399</td><td>659</td><td>0.489</td><td>0.418</td></tr><tr><td>VAS sleepiness<sup>a</sup></td><td>38</td><td>0.474</td><td>0.365</td><td>617</td><td>0.577</td><td>0.514</td></tr></tbody></table>

Abbreviations: ES, effect size; SRM, standardized response mean.

Finally, a selection bias may also be present on account of the representativeness of users and of days on which daily monitoring questionnaires were completed. In fact, it is expected that MASK-air® users may not be representative of allergic rhinitis patients—only 5% of the daily monitoring data concerned users aged more than 65 years, and only 7% concerned current smokers. This suggests that MASK-air® users are probably younger and more concerned about their health than overall allergic rhinitis patients. It is possible, however, that MASK-air® users are representative of those allergic rhinitis patients using apps in the management of their disease. On the other hand, and despite the low median values of daily monitoring VASs, daily monitoring may more often be performed when users feel bothered about their allergic symptoms. In other words, regarding the days on which daily monitoring VASs were used, there may be an overrepresentation of "more troublesome" days. This is all the more relevant taking into account that, in this study, the mean adherence to MASK-air® was found to be of 2.9% (with adherence/ intensity of use calculated as the number of actual reporting days divided by the reporting period—following the methods of Di Fraia et al.⁴⁰—computed as the period between December 6, 2020 and the date the user first used MASK-air®). While this low adherence may be motivated by the fact that, for most users, MASK-air® was not prescribed and promoted by their allergists, future studies should assess whether adherence patterns may have an impact on the validity and reliability of MASK-air® VASs.

This study also has important strengths. We analyzed real-world data from a diverse set of users with allergic rhinitis in 25 different

countries and 19 languages. The structure of the app precluded the existence of missing data within each response to the MASK-air® daily questionnaire. The validity, reliability and responsiveness of several VASs were assessed, with these VASs reflecting different types of allergic symptoms and different ways by which such symptoms can impact on users' activities. In our analyses, we used three different main validated comparators (along with ESS for sleep-related daily monitoring VASs), with consistent results being mostly obtained when identifying the best-performing and worst-performing daily monitoring VASs across the different analyses performed with different comparators. Finally, our results were robust to different sub-analyses or sensitivity analyses performed: similar results were observed when analyzing data restricted to European users or, for intra-rater reliability, when considering the first and last daily measurements (instead of the first and second ones).

In conclusion, in this study, we observed that daily monitoring VASs present high intra-rater reliability and moderate-high concurrent validity and test-retest reliability, with responsiveness being more variable across the different daily monitoring VASs and according to the chosen comparator. These results indicate that daily monitoring VASs are accurate instruments for measuring the daily impact of allergic rhinitis, possibly providing support for adapting medication and for controlling disease activity. Future research may focus on improving already existing scores or on developing new tools: (i) assessing daily rhinitis control, (ii) combining results from different daily monitoring VASs, (iii)

<footnote><sup>a</sup>Scale ranging from 0 to 100, with higher values indicating worse status.</footnote>
including information on medication use (that can be obtained using MASK-air®), and (iv) providing information on patients' demographic and clinical characteristics.

## ACKNOWLEDGMENTS

The study was funded by ARIA.

Open access funding enabled and organized by Projekt DEAL.

## CONFLICTS OF INTERESTS

Sinthia Bosnic-Anticevich reports grants from TEVA and personal fees from TEVA, AstraZeneca, Boehringer Ingelheim, GSK, Sanofi, Mylan. Jean Bousquet reports personal fees from Chiesi, Cipla, Hikma, Menarini, Mundipharma, Mylan, Novartis, Sanofi-Aventis, Takeda, Teva, Uriach, other from KYomed-Innov, and personal fees from Purina. Victoria Cardona reports personal fees from ALK, Allergy Therapeutics, LETI, Thermofisher, Merck, AstraZeneca, and GSK. Alvaro A Cruz reports grants and personal fees from GlaxoSmithKline, personal fees from Sanofi, AstraZeneca, Novartis, Chiesi, Boehringer Ingelheim, Mylan, and Eurofarma. Philippe Devillier reports personal fees from Mylan. Bilun Gemicioğlu reports grants from AstraZeneca, MSD, GSK, Deva, Novartis, and Abdi Ibrahim. Tari Haahtela reports personal fees from GSK, Mundipharma, Orion Pharma, Sanofi, Juan Carlos Ivancevich reports personal fees from Sanofi, Faes Farma, Laboratorios Casasco, Abbott, Ludger Klimek reports grants and personal fees from Allergopharma, LETI Pharma, MEDA/Mylan, and Sanofi, personal fees from Allergy Therapeut, Casssela med, and HAL Allergie, grants from ALK Abelló, Stallergenes, Quintiles, ASIT biotech, Lofarma, AstraZeneca, GSK, Inmunotk, and Membership: AeDA, DGHNO, Deutsche Akademie für Allergologie und klinische Immunologie, HNO-BV, GPA, and EAACI. Piotr Kuna reports personal fees from Adamed, AstraZeneca, Berlin Chemie Menarini, Boehringer Ingelheim, GSK, HAL Allergy, Lekam, Mylan, Novartis, and Teva. Désirée Larenas Linnemann reports personal fees from Allakos, Amstrong, AstraZeneca, DBV Technologies, Grunenthal, GSK, Mylan, Menarini, MSD, Novartis, Pfizer, Sanofi, Siegfried, UCB, Alakos, and Gossamer, grants from Sanofi, AstraZeneca, Novartis, Circassia, UCB, GSK, TEVA, and Purina Institute. Michael Makris reports personal fees from Novartis, AstraZeneca, GSK, Chiesi, Menarini, Sanofi Genzyme, and Pfizer. Joaquin Mullol reports personal fees and other from SANOFI-GENZYME & REGENERON, Genentech-Roche & NOVAR-TIS, grants and personal fees from VIATRIS (MEDA-MYLAN Pharma) and URIACH Group, and personal fees from Mitsubishi-Tanabe, Menarini, UCB, AstraZeneca, and GSK. Oliver Pfaar reports grants and personal fees from ALK-Abelló, Allergopharma, Stallergenes Greer, HAL Allergy Holding B.V./HAL Allergie GmbH, Bencard Allergie GmbH/Allergy Therapeutics, Lofarma, Anergis S.A., ASIT Biotech Tools S.A., Laboratorios LETI/LETI Pharma, and Glaxo Smith Kline, grants from Biomay, Circassia, Pohl-Boskamp, and Inmunotek S.L., and personal fees from MEDA Pharma/MYLAN, Mobile Chamber Experts (a GA²LEN Partner), Indoor Biotechnologies, Astellas Pharma Global, EUFOREA, ROXALL Medizin, Novartis, Sanofi-Aventis and Sanofi-Genzyme, Med Update Europe GmbH, streamedup! GmbH, John Wiley and Sons, AS, and Paul-Martini-Stiftung (PMS). Nikolaos G.

Papadopoulos reports personal fees from Novartis, Nutricia, HAL, MENARINI/FAES FARMA, SANOFI, MYLAN/MEDA, BIOMAY, AstraZeneca, GSK, MSD, ASIT BIOTECH, and Boehringer Ingelheim and grants from Gerolymatos International SA, and Capricare. Boleslaw Samolinski reports personal fees from Allergopharma, Polpharma, Viatris, TEVA, ADAMED, patient ombudsman, and Polish Allergology Society, grants from AstraZeneca, National Health Programme, and grants and personal fees from AstraZeneca. Ana Todo-Bom reports grants and personal fees from AstraZeneca, GSK (GlaxoSmithKline), Novartis, Sanofi, Teva, and Mundipharma, personal fees from Bial, and grants from Leti. Torsten Zuberbier reports personal fees from Bayer Health Care, FAES, Novartis, Henkel, Novartis, from Henkel, AstraZeneca, AbbVie Fee for talk, ALK Fee for talk, Almirall Fee for talk, Astellas Fee for talk, Bayer Health Care Fee for talk, Bencard Fee for talk, Berlin Chemie Fee for talk, FAES Fee for talk, HAL Fee for talk, Leti Fee for talk, Meda Fee for talk, Menarini Fee for talk, Merck Fee for talk, MSD Fee for talk, Novartis Fee for talk, Pfizer Fee for talk, Sanofi Fee for talk, Stallergenes Fee for talk, Takeda Fee for talk, Teva Fee for talk, UCB Fee for talk, Henkel Fee for talk, Kryolan Fee for talk, and L'Oréal Fee for talk.

## AUTHOR CONTRIBUTIONS

Jean Bousquet: writing-review & editing (equal); Bernardo Sousa Pinto: conceptualization (equal); Patrik Eklund: conceptualization (equal); Oliver Pfaar: conceptualization (equal); Ludger Klimek: conceptualization (equal); Torsten Zuberbier: conceptualization (equal); Wienczyłslawa Czarlewski: conceptualization (equal); Annabelle Bedard: conceptualization (equal); Carsten Bindslev-Jensen: conceptualization (equal); Anna Bedbrook: Supervision (equal); Sinthia Bosnic-Anticevich: conceptualization (equal); Luisa Brussino: conceptualization (equal); Victòria Cardona: conceptualization (equal); Alvaro Cruz: conceptualization (equal); Govert De Vries: conceptualization (equal); Philippe Devillier: conceptualization (equal); Wytske Fokkens: conceptualization (equal); José Miguel Fuentes Perez: conceptualization (equal); Bilun Gemicioglu: conceptualization (equal); Tari Haahtela: conceptualization (equal); Yunen Rocio Huerta Villalobos: conceptualization (equal); Juan Carlos Ivancevich: conceptualization (equal); Inger Kull: conceptualization (equal); Piotr Kuna: conceptualization (equal); Violeta Kvedariene: conceptualization (equal); Desiree Larenas-Linnemann: conceptualization (equal); Daniel Laune: conceptualization (equal); Michael Makris: conceptualization (equal); Erik Melen: conceptualization (equal); Mario Morais Almeida: conceptualization (equal); Ralph Mösges: conceptualization (equal); Joaquim Mullol: conceptualization (equal); Robyn O'Hehir: conceptualization (equal); Nikolaos Papadopoulos: conceptualization (equal); Ana Pereira: conceptualization (equal); Emmanuel Prokopakis: conceptualization (equal); Fotis Psarros: conceptualization (equal); Frederico Regateiro: conceptualization (equal); Sietze Reitsma: conceptualization (equal); Boleslaw Samolinski: conceptualization (equal); Nicola Scichilone: conceptualization (equal); Jane Da Silva: conceptualization (equal); Cristiana Stellato: conceptualization (equal); Ana Todo Bom: conceptualization (equal); Peter Tomazic: conceptualization (equal); Sanna Toppila
Salmi: conceptualization (equal); Antonio Valero: conceptualization (equal); Arunas Valiulis: conceptualization (equal); Erkka Valovirta: conceptualization (equal); Michiel van Eerd: conceptualization (equal); Maria Teresa Ventura: conceptualization (equal); Arzu Yorgancioglu: conceptualization (equal); Xavier Basagana: conceptualization (equal); Josep Anto: conceptualization (equal); João Almeida Fonseca: conceptualization (equal).

## ORCID

Bernardo Sousa-Pinto https://orcid.org/0000-0002-1277-3401

Oliver Pfaar https://orcid.org/0000-0003-4374-9639

Carsten Bindslev-Jensen https://orcid.org/0000-0002-8940-038X

Victoria Cardona https://orcid.org/0000-0003-2197-9767

Ralph Mösges https://orcid.org/0000-0002-1928-810X

Jean Bousquet https://orcid.org/0000-0002-4061-4766

## REFERENCES

1. Bousquet J, Antó JM, Bachert C, et al. Allergic rhinitis. *Nat Rev Dis Primers*. 2020;6(1):95.

2. Léger D, Annesi-Maesano I, Carat F, et al. Allergic rhinitis and its consequences on quality of sleep: an unexplored area. *Arch Intern Med*. 2006;166(16):1744-1748.

3. Bousquet J, Arnavielhe S, Bedbrook A, et al. Mask 2017: ARIA digitally-enabled, integrated, person-centred care for rhinitis and asthma multimorbidity using real-world-evidence. *Clin Transl Allergy*. 2018;8:45.

4. Bousquet J, Ansotegui IJ, Anto JM, et al. Mobile technology in allergic rhinitis: evolution in management or revolution in health and care? *J Allergy Clin Immunol Pract*. 2019;7(8):2511-2523.

5. Bédard A, Antó JM, Fonseca JA, et al. Correlation between work impairment, scores of rhinitis severity and asthma using the MASK-air® app. *Allergy*. 2020;75(7):1672-1688.

6. Bedard A, Basagana X, Anto JM, et al. Treatment of allergic rhinitis during and outside the pollen season using mobile technology. A MASK study. *Clin Transl Allergy*. 2020;10:62.

7. Bousquet J, Anto JM, Bachert C, et al. ARIA digital anamorphosis: digital transformation of health and care in airway diseases from research to practice. *Allergy*. 2021;76(1):168-190.

8. Bousquet J, Bedbrook A, Czarlewski W, et al. Guidance to 2018 good practice: ARIA digitally-enabled, integrated, person-centred care for rhinitis and asthma. *Clin Transl Allergy*. 2019;9:16.

9. Bousquet JJ, Schunemann HJ, Togias A, et al. Next-generation ARIA care pathways for rhinitis and asthma: a model for multimorbid chronic diseases. *Clin Transl Allergy*. 2019;9:44.

10. Bousquet J, Anto JM, Haahtela T, et al. Digital transformation of health and care to sustain planetary health: the MASK proof-of-concept for airway diseases-POLLAR symposium under the auspices of Finland's Presidency of the EU, 2019 and MACVIA-France, Global Alliance against Chronic Respiratory Diseases (GARD, WHO) demonstration project, reference site collaborative Network of the European Innovation Partnership on Active and Healthy Ageing. *Clin Transl Allergy*. 2020;10:24.

11. Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. *Am J Health Syst Pharm*. 2008;65(23):2276-2284.

12. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. *J Clin Epidemiol*. 2000;53(5):459-468.

13. Samreth D, Arnavielhe S, Ingenrieth F, et al. Geolocation with respect to personal privacy for the Allergy Diary app—a MASK study. *World Allergy Organ J*. 2018;11(1):15.

14. Laune D, Arnavielhe S, Viart F, et al. [Adaptation of the General Data Protection Regulation (GDPR) to a smartphone app for rhinitis and asthma (MASK-air(R))]. *Rev Mal Respir*. 2019;36(9):1019-1031.

15. Bousquet J, Devillier P, Arnavielhe S, et al. Treatment of allergic rhinitis using mobile technology with real-world data: the MASK observational pilot study. *Allergy*. 2018;73(9):1763-1774.

16. Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). *Qual Life Res*. 2011;20(10):1727-1736.

17. Devlin NJ, Brooks R. EQ-5D and the EuroQol group: past, present and future. *Appl Health Econ Health Policy*. 2017;15(2):127-137.

18. Fonseca JA, Nogueira-Silva L, Morais-Almeida M, et al. Validation of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. *Allergy*. 2010;65(8):1042-1048.

19. Fonseca JA, Nogueira-Silva L, Morais-Almeida M, et al. Control of Allergic Rhinitis and Asthma Test (CARAT) can be used to assess individual patients over time. *Clin Transl Allergy*. 2012;2:16.

20. Reilly MC, Tanner A, Meltzer EO. Work, classroom and activity impairment instruments. *Clin Drug Investig*. 1996;11:278-288.

21. Bousquet J, VandenPlas O, Bewick M, et al. The work productivity and activity impairment allergic specific (WPAI-AS) questionnaire using mobile technology: the MASK study. *J Investig Allergol Clin Immunol*. 2018;28(1):42-44.

22. Johns MW. A new method for measuring daytime sleepiness: the Epworth Sleepiness Scale. *Sleep*. 1991;14(6):540-545.

23. Johns MW. Reliability and factor analysis of the Epworth sleepiness scale. *Sleep*. 1992;15(4):376-381.

24. Gamer M. Package "IRR": Various Coefficients of Interrater Reliability and Agreement. Version 0.84.1; 2019. https://cran.r-project.org/web/packages/irr/irr.pdf

25. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. *J Chiropr Med*. 2016;15(2):155-163.

26. Qin S, Nelson L, McLeod L, Eremenco S, Coons SJ. Assessing test-retest reliability of patient-reported outcome measures using intraclass correlation coefficients: recommendations for selecting and documenting the analytical formula. *Qual Life Res*. 2019;28(4):1029-1033.

27. van der Leeuw S, van der Molen T, Dekhuijzen PNR, et al. The minimal clinically important difference of the Control of Allergic Rhinitis and Asthma Test (CARAT): cross-cultural validation and relation with pollen counts. *NPJ Prim Care Respir Med*. 2015;25:14107.

28. Mouelhi Y, Jouve E, Castelli C, Gentile S. How is the minimal clinically important difference established in health-related quality of life instruments? Review of anchors and methods. *Health Qual Life Outcomes*. 2020;18(1):136.

29. Epworth Sleepiness Scale. Accessed January 30, 2021. https://epworthsleepinessscale.com/about-the-ess/

30. Klimek L, Bergmann KC, Biedermann T, et al. Visual analogue scales (VAS): measuring instruments for the documentation of symptoms and therapy monitoring in cases of allergic rhinitis in everyday health care. *Allergo J Inter*. 2017;26(1):16-24.

31. Bousquet J, Bewick M, Arnavielhe S, et al. Work productivity in rhinitis using cell phones: the MASK pilot study. *Allergy*. 2017;72(10):1475-1484.

32. Meltzer EO, Schatz M, Nathan R, Garris C, Stanford RH, Kosinski M. Reliability, validity, and responsiveness of the rhinitis control assessment test in patients with rhinitis. *J Allergy Clin Immunol*. 2013;131(2):379-386.
33. Liu J, Zhang X, Zhao Y, Wang Y. The association between allergic rhinitis and sleep: a systematic review and meta-analysis of observational studies. *PLoS One*. 2020;15.e0228533.

34. Munoz-Cano R, Ribo P, Araujo G, Giralt E, Sanchez-Lopez J, Valero A. Severity of allergic rhinitis impacts sleep and anxiety: results from a large Spanish cohort. *Clin Transl Allergy*. 2018;8:23.

35. Leger D, Annesi-Maesano I, Carat F, et al. Allergic rhinitis and its consequences on quality of sleep: an unexplored area. *Arch Intern Med*. 2006;166(16):1744-1748.

36. Leger D, Bonnefoy B, Pigearias B, de La Giclais B, Chartier A. Poor sleep is highly associated with house dust mite allergic rhinitis in adults and children. *Allergy Asthma Clin Immunol*. 2017;13:36.

37. Valero A, Cuvillo AD, Navarro AM, Colás C, Sastre J, Mullol J. Validation of the ARIA items to assess allergic rhinitis control (ARIA-C). *Allergy*. 2020;75(11):2964-2966.

38. Azevedo P, Sousa JC, Bousquet J, et al. Control of Allergic Rhinitis and Asthma Test (CARAT): dissemination and applications in primary care. *Prim Care Respir J*. 2013;22(1):112-116.

39. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. *J Clin Epidemiol*. 2008;61(2):102-109.

40. Di Fraia M, Tripodi S, Arasi S, et al. Adherence to prescribed e-diary recording by patients with seasonal allergic rhinitis: observational study. *J Med Internet Res*. 2020;22(3).e16642.

**How to cite this article:** Sousa-Pinto B, Eklund P, Pfaar O, et al. Validity, reliability, and responsiveness of daily monitoring visual analog scales in MASK-air®. *Clin Transl Allergy*. 2021;e12062. doi:10.1002/clt2.12062