dataesr/scientific-paragraphs-categorization huggingface dataset:
We also tested the effect of the tolerance index from the World Value Survey database. Tolerance level was significantly associated with mask use for countries with low to intermediate tolerance values, but the association was negative for countries with high tolerance levels. This effect may be due to the fact that Scandinavian countries, which are characterized by high levels of tolerance, have implemented liberal policies on mask wearing. We also test if the trust in politicians and government can affect the mask wearing proportion in line with  5  but do not find a clear significant relationship: in other words, we do not find that a high proportion of people that do not trust in government 'at all' is associated to a weak proportion of mask wearing in the population but due to the limited number of observations, the inference work should be interpreted with cautious.
Data availability The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request and dat as well as main replication codes are available at https:// datav erse. harva rd. edu/ datav erse/ MaskC ovid19. A detailed SI Appendix is associated to the main manuscript with detailed informations about data and materials. The survey data come from the study performed by the University of Maryland and jointly conducted with the Facebook platform (see Fan et al. 29  , Barkay et al. 6  and Kreuter et al. 50 ) and are publicly available (see details in SI Appendix).
Work of Padrol and Philippe is supported by grants  ANR-17-CE40-0018  and  ANR-21-CE48-0020  of the  French National Research Agency ANR  (projects  CAPPS  and  PAGCAP ). Work of Padrol and Santos is supported by grant  PID2019-106188GB-I00 of MCIN/AEI/10. 13039/501100011033  and project  CLaPPo  ( 21. SI03. 64658 ) of  Universidad de Cantabria and Banco Santander. Funding Open Access funding provided thanks to the  CRUE-CSIC  agreement with  Springer Nature.
Acknowledgements: We acknowledge the contribution of the paediatric department, laboratory technical staff, the  BEHcl Team ,  Landel Verena  ( Hospices Civils de Lyon -proofreading ) and members of the VRS study group in Lyon ( Antoine Ouziel ,  Luc Panetta ,  Come Horvat ,  Dominique Ploin ,  Yves Gillet ,  Etienne Javouhey ,  Olivier Claris ,  Marine Butin ,  Muriel Doret-Dion ,  Pascal Gaucherand ,  Mona Massoud ,  Julie Fort-Jacquier ,  Sandrine Couray-Targe, Anne-Florence Myard Dury ,  Philippe Vanhems ,  Sylvie Bin ,  Stephanie Polazzi ,  Antoine Duclos ,  Mehdi Benchaib ,  Regine Cartier ,  Jean-Sebastien Casalegno ,  Florence Morfin ,  Marine Jourdain ,  Martine Valette ,  Bruno Lina ,  Michelle Ottmann ,  Sylvie Fiorini ,  Nathalie Rivat ,  Alexandre Gaymard ,  Yahia Mekki ,  Julie Haesebaert ,  Olivier Terrier ).
CONCLUSIONS The current global RSV situation emphasizes the need for plausible simulations that can be combined with the different streams of surveillance data and field expertise to timely allocate adequate local hospital resources to RSV epidemics. Further research is needed to define good modelling practices for the hospital setting. This represents an opportunity to strengthen the collaboration between modellers and hospital clinical management, who are both data providers and model users. Healthcare professionals could benefit from open access, easy-to-use, forecasting models that produce plausible RSV simulations with limited data set requirements, while modellers could benefit from direct access to comprehensive databases and the expertise of health care workers. Data availability: R-Code and raw data are joined.
Analysis period and data collection National COVID-19 surveillance systems are continually evolving and highly dynamic. For the purpose of this study, we limited the analysis to the surveillance systems as they existed in mid-May 2021. Study participants were asked to provide data reflecting their COVID-19 testing strategy and surveillance system at this time point. Data were gathered using multiple methods. Data on target testing populations, access to testing, types of tests used, testing strategy, data entry and management, statistical approaches to the calculation of case counts and incidence and case definitions were collected using a standard questionnaire followed by semi-structured interviews with participating country representatives. The questionnaire is available in the Supplementary material.
Questionnaires and interviews were completed between May and September 2021. Data on the number of COVID-19 tests and positive cases registered between week 13 (29 March to 4 April) and week 30 (26 July to 1 August) 2021 were collected from national public health organisation websites or the European Centre for Disease Prevention and Control (ECDC) and verified by country representatives. This period was chosen to represent the situation around mid-May 2021. These data, along with population figures from Eurostat  [14] , were used to calculate the average number of tests per 1,000 population per day, the test positivity rate (number of cases divided by the number of tests) and the average 7-day incidence per 100,000 population by week over the study period, and the proportion of RT-PCR among all tests.
Data management and missing data All studied countries had one central database for all testing results. For patient identification and elimination of double counting in the context of RT-PCR confirmation tests, all countries had a single patient identifier. Overall, countries indicated that unreported data for RT-PCR or antigenic testing was rare. The largest potential source of unreported came from at-home antigenic tests that were not recorded in national databases. However, all countries recommended a confirmatory RT-PCR test after a positive at-home antigenic test. No country was able to estimate the total number of at-home antigenic tests that were conducted or potentially missed in national databases or those that were followed by a confirmation RT-PCR. In England, individuals were required to report at-home antigenic test results on an online platform but the proportion of those that may have not done so is unknown.
Case definition All countries included persons presenting a positive test result (RT-PCR or antigenic) as a positive COVID-19 case. However, some differences did exist between countries during the study period regarding the definition of reinfection which can impact how multiple positive tests for an individual are recorded. In both Belgium and Italy, a period of 90 days between positive tests for the same individual was necessary for a new case to be counted. This period was 60 days in Italy if genotypic information was available showing that two infections were caused by different viruses. In France, this period was 60 days and in Sweden 12 months. England did not refer to a time limit and mentioned in its case definition that an individual could only count as a single positive case. Some of these definitions have changed since data were collected.
In England, frequent testing of non-symptomatic individuals using lateral flow devices was actively encouraged. The impact of this strategy is reflected in the substantially higher testing rate and lower positivity rate in England. Screening of asymptomatic individuals also had an important impact on case detection. Although no comparable information was available on the proportion of symptomatic vs non-symptomatic testing in the countries studied, Italy reported that over 20% of cases detected came via routine testing of asymptomatic individuals. A recent review study estimated the proportion of asymptomatic COVID-19 infections at over 40%  [15]. Testing rates and positivity rates may also be impacted by which tests are recorded as well as the population's adherence to guidelines. Outside of the recording of lateral flow devices conducted at home in England, at-home antigenic tests were not recorded in national databases.
A country which used the more sensitive RT-PCR test such as Sweden may have a higher positivity rate vs another country with a higher proportion of antigenic testing. In England, the high proportion of antigenic testing may in part explain the relatively low positivity rate, although testing strategy and testing rate likely played a more important role. Overall, data systems and data management were similar across countries and likely created only small hurdles to comparability. One notable exception was in England where all results for at-home antigenic tests were recorded in national databases. Case definitions were generally similar, although some variation did exist between countries in the definition of a reinfection. At the time of this study, cases of reinfection were uncommon so these differences were likely to have had little impact on comparability.
While our study identified several important differences in surveillance systems, it also had several limitations. First, our study collected data for a single time point around May 2021. Surveillance systems can be highly dynamic and our study was not able to consider these changes. Second, our study focused on the systematic features of surveillance systems but did not consider other possibly important factors in system performance such as population adherence to recommendations. Adherence can be differentially impacted by cultural or socioeconomic factors, country or region, symptom severity or other factors which may create important biases in testing results. Other factors outside the surveillance systems that were not measured include current control measures in place such as school closings and the physical availability of tests.
A positive test within 12 months of another for the same person is not considered a new case. Data logistics Data logistics RT-PCR and antigenic tests are entered into the same RT-PCR and antigenic national database. A unique tests are included in an national person ID is used individual surveillance to remove multiple records database. A patient number for the same patient. created for this database is used, along with other checks, to remove multiple test results. RT-PCR and antigenic test results are entered into the same national database. A unique national person ID RT-PCR and antigenic tests is used to remove multiple are entered into the same records for the same national database. A unique patient. confirmation number created for this database is used to remove multiple records for the same patient.
RT-PCR and antigenic test results are entered into the same national database. A pseudonym using name and date is used to remove RT-PCR and antigenic tests multiple records for the are entered into the same same patient when counting national database. A unique new cases. national person ID is used to combine multiple records for the same patient. Missing tests Missing tests Results of at-home antigenic tests generally are not Results of at-home recorded in the antigenic tests are not national database. recorded in national databases. Lateral flow test results should be recorded by the person/organisation performing them. An 95% of RT-PCR and unknown proportion antigenic tests are of these tests are not registered in national reported. databases. Results of at-home antigenic tests are not recorded in national databases.
Some antigenic Some privately tests may be not be performed antigenic recorded by health tests data may not professionals and no be included in the at-home antigenic national database tests are recorded (e. g. workplace nationally. screening). At-home antigenic tests are not recorded in the national database. Positive at-home antigenic tests should have a confirmation RT-PCR test. Testing data entry and exhaustivity Testing data entry and exhaustivity Antigenic tests At-home antigenic tests Antigenic tests At-home antigenic tests Positive at-home antigenic tests Testing data are entered may be reported to the national database if they accompany a positive first into local databases RT-PCR confirmation test. Positive by laboratory personnel or at-home antigenic tests may also be GPs then sent electronically included in the national database if a to one central data physician reports it as a valid antigen warehouse.
test. Otherwise, these tests are not Testing and case data are provided by the regions and aggregated at the national level. reported. At-home antigenic tests are not recorded in national databases. Depending on where the test is done, this can be done manually by the If any individual conducts a individual who self-lateral flow test at home, they are administered the test, responsible for recording the result the school, workplace, in the national database via an online healthcare facility or platform. testing site where testing Testing data are recorded by laboratory personnel, medical centres, or GPs then uploaded to the took place, via an online national database or platform. entered directly into the At-home antigenic tests are not recorded in national databases. This information is not entered national database by the public health authority in the national database directly.
personnel. RT-PCR confirmation tests following positive at-home antigenic test Manual entry into national include free text for a specific code which indicates that the RT-PCR database is done by health is following a positive at-home professionals (nurses, GPs, antigenic test. This information is not pharmacists, dentists). analysed at this point. A website is Testing data are entered very recently available for results of into the national database at-home antigenic tests for entry by by either the treating GP or individuals but this is not linked to a the diagnostic laboratory. national database. At-home antigenic test are not recorded in national databases. RT-PCR tests RT-PCR tests Testing data are Testing data are recorded by the person entered first into conducting the test local databases by (usually pharmacy or laboratory personnel laboratory).
These then sent electronically results are then to one central data centralised at the local, warehouse. regional and national level. Case data follow the same pathway but start with a physician or primary health Testing data are provider. entered into national Testing data are database by laboratory recorded by laboratory personnel. personnel then uploaded to the national database or entered directly into the national database by the public health authority personnel. Testing data are entered into local databases by laboratory personnel and sent electronically Testing data are to a national database. entered into the national database by the diagnostic laboratory. Country Country Belgium Italy England Romania France Sweden -19 testing data entry, exhaustivity of data, data logistics and case definition, six European countries, 2021 GP: general practitioner; LFD: lateral flow device; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2.
Forecast targets and models We sought forecasts for the incidence of COVID-19 as the total reported number of cases and deaths per week. We considered forecasts for 32 countries in Europe, including all countries of the European Union, European Free Trade Area, and the United Kingdom. We compared forecasts against observed data reported for each country by Johns Hopkins University (JHU,  Dong et al. , 2020). JHU data sources included a mix of national and aggregated subnational data. We aggregated incidence over the Morbidity and Mortality Weekly Report (MMWR) epidemiological week definition of Sunday through Saturday. Teams could express their uncertainty around any single forecast target by submitting predictions for up to 23 quantiles (from 0. 01 to 0. 99) of the predictive probability distribution.
Teams could also submit a single point forecast. At the first submission, we asked teams to add a pre-specified set of metadata briefly describing the forecasting team and methods (provided online and in supplementary information). No restrictions were placed on who could submit forecasts. To increase participation, we actively contacted known forecasting teams across Europe and the US and advertised among the ECDC network. Teams submitted a broad spectrum of model types, ranging from mechanistic to empirical models, agent-based and statistical models, and ensembles of multiple quantitative or qualitative models (described at European  Covid-19 Forecast Hub, 2023a). We maintain a full project specification with a detailed submissions protocol  (European Covid-19 Forecast Hub, 2023c). We collected forecasts submitted weekly in real time over the 52-week period from 08 March 2021 to 07 March 2022.
Teams submitted at latest 2 days after the complete dataset for the latest forecasting week became available each Sunday. We implemented an automated validation programme to check that each new forecast conformed to standardised formatting. Forecast validation ensured a monotonic increase of predictions with each increasing quantile, integer-valued non-negative counts of predicted cases, as well as consistent date and location definitions. Each week we used all available valid forecasts to create a weekly real-time ensemble model (referred to as 'the ensemble' from here on), for each of the 256 possible forecast targets: incident cases and deaths in 32 locations over the following one through 4 weeks. The ensemble method was an unweighted average of all models' forecast values, at each predictive quantile for a given location, target, and horizon.
Forecast evaluation In this study, we focused only on the comparative performance of forecasting models relative to each other. Performance in absolute terms is available on the Hub website  (European Covid-19 Forecast Hub, 2023b). For each model, we assessed calibration and overall predictive performance. We evaluated all previous forecasts against actual observed values for each model, stratified by the forecast horizon, location, and target. We calculated scores using the scoringutils R package  (Bosse et al. , 2023). We removed any forecast surrounding (both the week of, and the first week after) a strongly anomalous data point. We defined anomalous as where any subsequent data release revised that data point by over 5%. To investigate calibration, we assessed coverage as the correspondence between the forecast probability of an event and the observed frequency of that event.
Results For 32 European countries, we collected, visualised, and made available online weekly COVID-19 forecasts and observed data  (Sherratt, 2022). Over the whole study period, we collected forecasts from 48 unique models. Modellers created forecasts choosing from a set of 32 possible locations, four time horizons, and two variables, and modellers variously joined and left the Hub over time. This meant the number of models contributing to the Hub varied over time and by forecasting target. Using all models and the ensemble, we created 2139 forecasting scores, where each score summarises a unique combination of forecasting model, variable, country, and week ahead horizon (Figure  1 ). Of the total 48 models, we received the most forecasts for Germany, with 29 unique models submitting 1-week case forecasts, while only 12 models ever submitted 4-week case or death forecasts for Liechtenstein.
The model used here is supported by previous work  (Cramer et al. , 2021b ), yet previous evaluation in a similar context has suggested that choice of baseline affects relative performance in general  (Bracher et al. , 2021b) , and future research should be done on the best choices of baseline models in the context of infectious disease epidemics. Our assessment of forecast performance may further have been inaccurate due to limitations in the observed data against which we evaluated forecasts. We sourced data from a globally aggregated database to maintain compatibility across 32 countries  (Dong et al. , 2020). However, this made it difficult to identify the origin of lags and inconsistencies between national data streams, and to what extent these could bias forecasts for different targets.
Collating time-stamped predictions ensures that we can test true out-of-sample performance of models and avoid retrospective claims of performance. Testing the limits of forecasting ability with these comparisons forms an important part of communicating any model-based prediction to decision makers. For example, the weekly ECDC Communicable Disease Threats reports include the specific results of this work by qualitatively highlighting the greater uncertainty around case forecasts compared to death forecasts. This study raises many further questions which could inform epidemic forecast modellers and users. The dataset created by the European Forecast Hub is an openly accessible, standardised, and extensively documented catalogue of real time forecasting work from a range of teams and models across Europe  (European Covid-19 Forecast Hub, 2023b) , and we recommend its use for further research on forecast performance.
In the code developed for this study, we provide a worked example of downloading and using both the forecasts and their evaluation scores (covid19-forecast-hub-europe, 2022). Future work could explore the impact on forecast models of changing epidemiology at a broad spatial scale by combining analyses of trends and turning points in cases and deaths with forecast performance, or extending to include data on vaccination, variant, or policy changes over time. There is also much scope for future research into methods for combining forecasts to improve performance of an ensemble. This includes altering the inclusion criteria of forecast models based on different thresholds of past performance, excluding or including only forecasts that predict the lowest and highest values (trimming)  (Taylor and Taylor, 2023) , or using alternative weighting methods such as quantile regression averaging  (Funk et al.
, 2020). Exploring these questions would add to our understanding of real time performance, supporting and improving future forecasting efforts. We see additional scope to adapt the Hub format to the changing COVID-19 situation across Europe. We have extended the Forecast Hub infrastructure to include short term forecasts for hospitalisations with COVID-19, which is a challenging task due to limited data across the locations covered by the hub. As the policy focus shifts from immediate response to anticipating changes brought by vaccinations or the geographic spread of new variants (European Centre for Disease  Prevention and Control, 2023) , we are also separately investigating models for longer term scenarios in addition to the short term forecasts in a similar framework to existing scenario modelling work in the US  (Borchering et al.
AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by  Ministry of Science and Higher Education of Poland  with grant  28/WFSN/2021  to the  University of Warsaw. BRe, CPe ,  JLAz  funded by  Ministerio de Sanidad/ISCIII. BT, PG funded by  PERISCOPE European H2020  project, contract number  101016233. CP, DL, EA, MC, SA funded by  European Commission -Directorate-General for Communications Networks, Content and Technology  through the contract  LC-01485746 , and  Ministerio de Ciencia ,  Innovacion y Universidades  and  FEDER , with the project  PGC2018-095456-B-I00. DE. , MGu funded by  Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by  Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL)  under project number  20200700ER.