Gaining control of an infectious disease outbreak can require difficult decisions to be made.
Children are often in close physical proximity at school, have less than perfect hygiene behaviours and have low prior immunity to many infections, leading them to be dubbed 'super-spreaders' of influenza [1] . For this reason, school closures are often mooted as one way of delaying the spread of infection [2] . There is evidence to suggest that social contacts should reduce when schools are closed. For example, it has been reported that students have contact with fewer people at weekends [3] and that the number of contacts children have with others approximately halves during the holidays [4, 5] . Several studies have also examined illness transmission rates during planned school closures, reporting a reduction in illness during school holidays [6, 7, 8 ] and teacher strikes [9] .
School closure is not a step that can be taken lightly, however. Clearly, closures can have an impact on the education of the children involved. But they can also have an impact on the healthcare system and wider economy if large numbers of the workforce stay home to look after their children, on household incomes, on social policies implemented at school and on the likelihood of children engaging in other risky behaviours if they must be left unattended at home [10] . Indeed, the secondary economic and social effects of school closures are potentially huge [11] .
Understanding whether the efficacy of school closure in terms of reducing the spread of disease is sufficient to justify these impacts is therefore important. One of the key unknowns that prevents us from resolving this issue is what happens to children after a school is closed.
The optimum answer, from an epidemiological perspective, is that they remain in their bedrooms for the duration of the closure, never coming into contact with another person [12, 13] . However, this is impractical and front-line experience of outbreak management is full of tales of children continuing to congregate after being sent home from school and sometimes engaging in behaviour likely to increase the risks of infection spreading [14, 15] . Any full assessment of the impact of school closures must take this into account.
A related issue is the extent to which the quality rather than quantity of social interaction changes following a school closure. For example, the role of grandparents in caring for children following school closure may be a cause for concern if this increases the likelihood of older adults coming into contact with the infectious disease in question.
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020. 03.17.20037457 doi: medRxiv preprint Finally, given that school closures are often accompanied by advice to parents to limit the contact their children have with others, understanding what practical or attitudinal factors affect the likelihood of children mixing during a closure may also be helpful in improving the advice that is given out. 
This work was carried out as a rapid evidence review in response to the COVID-19 outbreak in December 2019, which has led to policy-makers worldwide discussing how best to minimise the spread of the disease. As there are no specific guidelines for rapid reviews, the PRISMA checklist has not been completed.
We used the following search strategy to search MEDLINE®, PsycINFO® and EMBASE®:
1. school* ADJ3 close* OR ADJ3 closure* OR ADJ3 closing* OR ADJ3 dismiss* 2. nurser* ADJ3 close* OR ADJ3 closure* OR ADJ3 closing* OR ADJ3 dismiss* 3. kindergar* ADJ3 close* OR ADJ3 closure* OR ADJ3 closing* OR ADJ3 dismiss* 4. playgroup* ADJ3 close* OR ADJ3 closure* OR ADJ3 closing* OR ADJ3 dismiss* 5. play-group* ADJ3 close* OR ADJ3 closure* OR ADJ3 closing* OR ADJ3 dismiss* 6. 1 OR 2 OR 3 OR 4 OR 5 7. behaviour* OR behavior* OR contact* OR mix* OR social* OR targeted layered containment 8. 6 AND 7
We repeated the same search on Web of Science using NEAR instead of ADJ3.
To be included in the review, studies had to: i) report on primary research; ii) be published in peer-reviewed journals; iii) be written in English or Italian (languages spoken by our team);
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint and iv) report on social activities of children during unplanned temporary school closures (because mixing behaviour will likely be different during closures with plenty of notice, when parents have more time to plan what to do). We excluded papers based on intentions, hypothetical scenarios or simulations.
One author (SKB) ran the search strategy on all databases and downloaded all resulting citations to EndNote© version X9 (Thomson Reuters, New York, USA). All titles and abstracts were screened for relevance according to the inclusion criteria by at least two authors (SKB, LES, RKW, DW, or LW). The authors compared which texts they had chosen for inclusion and discrepancies were resolved through discussion with the wider team. Full texts of all remaining citations were obtained and reviewed by one author (SKB), excluding any which did not meet all inclusion criteria. Finally, the reference lists of remaining papers were hand-searched for any additional relevant studies. A flow chart of the screening process is presented in Figure 1 .
We designed spreadsheets to extract the following data from papers: authors, publication year, country of study, design, participants (including n and demographic information), reason for school closure, length of school closure, and key results. Data extraction was carried out by one author (SKB).
Nineteen papers were included, eighteen of which used a cross-sectional design using questionnaires to assess difficulties during the school closures, activities outside the home during the closures and/or who provided childcare during the closures. The remaining paper used a qualitative design. The majority (n=10) were from the USA; four papers were from Australia, and the remaining papers from Argentina, Japan, Russia, Taiwan and the UK (one for each country). Most papers reported on school closures due to the H1N1 pandemic (n=11) or other influenza or influenza-like outbreaks (n=7); one paper reported on a school closed in preparation for a hurricane. The duration of school closures ranged from one day to two weeks. The size of the quantitative studies ranged from 35 households (representing 67 children) to 2,229 households (representing 4,171 children). Table 1 provides a summary of   activities outside the home found in the reviewed studies, while Table 2 summarises . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint childcare arrangements involving non-household members during school closures.
Supplementary Appendix 1 provides more detail on the results of each study.
Participation in activities and interactions with others did appear to decrease during school closures compared to regular school days [16, 17, 18] ; for example, one study [16] reported that school closure was associated with a 65% reduction in the mean total number of contacts for each student. However, social contact was still common. All nineteen studies showed that at least some (and in some cases, most) children took part in activities outside of the home during school closures, even despite health recommendations to remain indoors and isolated from others.
Several studies suggested that children who reported illness during a school closure were less likely to take part in activities outside the home [16, 19, 20, 21] . For example, Effler et al. [19] reported a significant difference for the proportion of cases (students testing positive for H1N1 virus), students who had been in close physical proximity to cases, and peers who did not meet case or contact criteria who reported leaving the home more than once during the closure period (42%, 66% and 92% respectively). Cases reported an average of 0.8 out-ofhome activities per student per week, compared with 2.9 for contacts and 5.6 for peers. Other studies reported that children who reported illness or lived in households in which influenzalike illness was reported did not participate in the majority of activities reported by other students [20, 21] and that their contact with others was reduced [16] .
However, other studies reported few differences in out-of-home activities between symptomatic and asymptomatic children [18, 22, 23, 24, 25] . In fact, one study [18] found that students with illnesses were more likely to report an increase in travel plans, for reasons that are not clear. Two studies found that children with an influenza-like illness were more likely to have visited a healthcare provider, but no other statistically significant differences in out-of-home activities were found between students with and without symptoms [23, 24].
Age . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. Three studies noted more activities and contacts among older children [18, 22, 26] . In particular, Grade 12 students (16-18yrs) [18] had significantly more contacts than other grades during closures, particularly late in the week. Because many Grade 12 students were not regularly attending classes at the school prior to the outbreak, they may have felt that they or their friends had not been exposed to the infection. One study [22] found that children aged twelve and over were significantly more likely to go to fast food restaurants and parties but less likely to go grocery shopping than children under twelve.
Conversely, one study [25] found that younger children were more likely to leave the home during a closure, with 53.2% of kindergarten pupils, 42.5% of primary school pupils, 30.3% of junior high school pupils and 33.2% of high-school pupils reporting that they left the house at least once. Primary school pupils were significantly more likely to leave the home to visit a supermarket or convenience store (likely with their parents), while junior high school pupils and primary school pupils were significantly more likely to leave the home to attend extracurricular studies compared with pupils in other school categories. It is unclear why this study shows the reverse of many others, but as this was the only study from Japan it may relate to cultural differences.
There is a small amount of evidence that location can affect the out-of-home activities children take part in during school closures, from a study of behaviour in children from two different school districts [27] . In this study, children in Jujuy were more likely to attend religious events, use public transport, and go to plazas and recreation areas than children in Ushuaia; children in Ushuaia were more likely to go to the movie theatre and restaurants than children in Jujuy. Socio-economic differences may well be the reason for this: Ushuaia has one of the lowest poverty rates in the country, whereas Jujuy has one of the highest.
One study [24] found that if all adults in the home were employed, ill children were less likely to leave the home. A baseline household had a 34% probability of a child visiting any other venue; if all adults were employed, the probability of children leaving the home decreased to 24%. This is an unexpected finding in that we would expect that children living only with employed adults might have to leave the house for childcare arrangements; the . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint authors did not offer reasons for the association between employed adults and reduced likelihood of children leaving the home.
Two studies found that parental opinion about the appropriateness of the school closure was significantly correlated with student participation in activities outside the home [19, 25] .
Students of parents who thought the school closure was not appropriate reported a mean of 4.7 out-of-home activities, compared with a mean of 4.3 activities for students of parents who were unsure and 2.8 for students of parents who thought the closure was appropriate [19] .
This pattern persisted when the analysis was restricted to the 202 students who were asymptomatic. Similarly, Mizumoto et al. [25] found that proportionately fewer children left the home in households that believed the closure was appropriate: 38.8%, compared to 53.2% of children in households who felt the closure was inappropriate.
One study [25] found that extent of the school closure was significantly associated with the frequency of children leaving the home: closure of the entire school, closure of a single grade or single class suspension were associated with 47.8%, 32.2% and 40.3% of children leaving the home, respectively.
One study, with high compliance rates with advised quarantine [28] found that children stayed at home for more than 94% of the days they were advised to be in quarantine and this figure was not associated with the length of quarantine nor did it fluctuate over the course of the quarantine period.
In one study [28] contact rates of uninfected students at the end of the week were lower than at the beginning. Contacts substantially increased for older children (grades 11 and 12) on Friday and Saturday.
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. concern that school athletic events were still held on days that school was closed [29] .
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. Reassuringly, our review found that relatively few children required special arrangements to be made for their childcare that might actively increase the risk of disease transmission, such as being placed into a semi-formal childcare arrangement with other children or being looked after by grandparents. The proportion of children left home alone unsupervised, however, while low, is of concern. If school closures are considered in the future, public health officials should consider how best to support parents and prevent this from occurring.
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint Parental attitudes associated with agreeing or disagreeing with school closures were similar to those seen in relation to other preventive health behaviours for infectious diseases [36, 37] .
In particular, two of the studies included in this review suggested there was a strong association between allowing children to socialise outside the home and disagreeing with the school closure [19, 25] . Ensuring parents understand why school closure is important will be a key factor determining the success of the measure in any future disease outbreak. In this regard, it was concerning that two studies appeared to highlight a lack of clarity in terms of advice about children's social activities and knowing what children were and were not advised to do [29, 31] . Advice from schools should be consistent with public health advicehosting extra-curricular activities and sporting events during a closure sends mixed messages to parents and can be confusing or detrimental [14] .
We found unclear evidence about the majority of the other predictors of out-of-home activities which we reviewed. In particular, there was mixed evidence about whether children showing symptoms of illness or who have been ill during the closure will take part in similar out-of-home activities to children who are not ill. Clearly, it would be particularly concerning if even symptomatic children are participating in out-of-home activities.
Different studies found that both older age and younger age were associated with leaving the house during school closures. It may be that the direction of findings depends on the activity in question. For example, younger children seem to be more likely to go grocery shopping (likely because they are too young to be left at home alone when their caregiver goes to the shops) whereas older children are more likely to take part in social activities like parties and going to restaurants.
In terms of how our findings fit with the wider literature, one particular discrepancy is worthy of note. Evidence from studies in which people are asked how they would react to a hypothetical school closure often find that parents believe they would co-operate with public health advice. For example, one study involving a hypothetical scenario of schools closing for three months due to an influenza pandemic found that 85% of parents responsible for children aged between 5-17 believed they would be able to keep their children from taking public transport, going to public events, and gathering outside the home during this lengthy school closure period [38]; another found that 96.7% of parents claimed they would keep . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint 1 3 their children away from others for a month if schools and child-care facilities were closed [39] . Despite these good intentions, our review of real school closures suggests parents are less likely to achieve this, even when schools are closed for much shorter periods of time.
Regardless of the conviction with which people answer questions about their likely future actions, substantial caution is needed in using such data. The duration of planned closure of schools is perhaps critical here too; short closures of up to a couple of weeks may be manageable by parents as seen in the studies found but longer closures required for curtailing pandemic waves of the order of months may provide more challenge to them.
Further research is needed to identify how best to ensure that children are incentivised to stay at home during a school closure. The relatively sparse research conducted to date (limited by the real-world occurrence of school closures and the feasibility of conducting rapid research when these do occur) do not allow us to provide a ready answer to this question, but improved communication with both parents and children is likely to be required.
In terms of limitations for this review, the generalisability of the individual studies we identified is unclear. In particular a lot may depend on the cultural context, perceptions of the illness in question, the length of the closure, the socio-economic status of the families that are affected and the information or instructions that are given to them by public health authorities.
With relatively few studies in this field, it is difficult to disentangle these effects.
Recent reviews of the incorporation of human behaviour into infectious disease models have advocated the use of appropriate, detailed, real-world behavioural data within infectious disease modelling [40, 41] . It is to be hoped that our identification of real-world data concerning social contact and mixing behaviour during unexpected school closures will help improve existing models and also promote more rigorous quantitative research in this area.
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. Left with a relative or family friend (82% / 88% depending on region), hired nanny (13% / 5%), other special arrangements (3% / 4%), left alone (2% / 1%). Special childcare arrangements including grandparents, other relatives, other adults, taking the child to work, having older siblings watch them or using childcare programs (10%), one or more night spent outside the household (3%).
Households with influenza: Adult from outside the home (44.4% for households that complied with advice to remain in home vs 2.4% for noncompliant households). Households without influenza: Adult from . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. Old enough to care for themselves (11.6%), went to work with parents (5.3%), childcare program (2.6%), left alone without supervision (2.5%).
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint This supplementary material is hosted by Eurosurveillance as supporting information alongside the article 'The impact of unplanned school closure on children's social contact: Rapid evidence review', on behalf of the authors, who remain responsible for the accuracy and appropriateness of the content. The same standards for ethics, copyright, attributions and permissions as for the article apply. Supplements are not edited by Eurosurveillance and the journal is not responsible for the maintenance of any links or email addresses provided therein. 67% of children visited public places at least once; 45% left the house several times. Approximate percentages taking part in outof-home activities (Ushuaia region / Jujuy region respectively): ~45% / ~35% visited the supermarket; ~20% / ~40% visited plazas / recreation areas; ~28% / ~30% visited shopping malls; ~28% / 22% attended indoor friend gatherings; ~5% / ~30% attended religious events; ~10% / ~28% used public transport; ~20% / ~3% visited movie theaters; ~10% / ~12% attended indoor sports practice; ~15% / ~8% visited restaurants; ~3% / ~2% participated in group activities.
Childcare arrangements (Ushuaia region / Jujuy region respectively): 82% / 88% were left with a relative or family friend; 13% / 5% were with a hired nanny; 3% / 4% had 'other special arrangements'; 2% / 1% of children were left alone. 554 households; Median age of children: 8 years; Schools closed for 5-7 days due to 30% of students visited at least one locale outside their homes. For children without an influenza-like illness: ~20% visited a playground or park; ~5% went to a grocery store / mall / ran . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint H1N1 errands; ~3% visited family; ~3% visited library, ~3% went to a restaurant; ~3% attended a religious function or service; ~2% attended a sports practice or game; ~2% went to a movie theatre; and friend's house. ~1% went to a friend's house; ~1% visited a doctor. Children who were sick with an influenza-like illness before or during the closure were only slightly less likely to visit a playground or park; their second most common venue was a doctor visit which they were significantly more likely to visit than a non-sick child; and they were slightly more likely to visit family than non-sick children. Braunack-Mayer et al. (2013), Australia [14] 4 school principals, 25 staff, 14 parents, 13 students; Students ranged in age from 12-17; Schools either partially or fully closed due to H1N1 (length of closure unclear)
Qualitative study indicating most people adhered to advised quarantine, but in the absence of clear instructions, many invented their own rules.
Some parents quarantined their children to avoid being seen as irresponsible. However, many were home alone and so it was unclear whether children complied with this. Others reported seeing the closure as ineffective and did not quarantine their children. One student reported meeting friends regularly even though his parents believed he was at home. 69% visited at least one other location. The most common activities outside of the home were: 'other'; shopping, sports activities, restaurants, visiting family, visiting healthcare provider, visiting friends, going to a movie or concert, visiting the library, . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted March 20, 2020. 20.5% left the home for non-essential reasons. Of the 351 students who left the home: 40.9% went shopping at a supermarket or convenience store; 19.3% went shopping at a department store or large shopping centre; 18.7% attended extra classes, prep school, or English school; 9.5% visited an outdoor playground or pool; 8.7% participated in a sports activity; 8.2% ate at restaurants; 2.0% participated in a concert, drawing, or other artistic activities; 1.0% had an overnight stay away from home; 0.5% participated in religious activities; 0.3% participated in an outdoor group activity; 0.3% visited the cinema; 0.1% went to a party; 11.2% took part in other activities outside the house.
28.5% children looked after themselves; 64.3% were looked after by another household member; 7.3% required a special arrangement such as parental absence from work. Russell et al. 77% of children went outside the home or visited with a non-household member, participating in a mean of two activities (IQR . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint 1 Students in prekindergarten up to 12 th grade; School closed for 4 days due to influenza-like illness 1-4): 54% visited a mall or department store; 51% visited a grocery store; 32% attended religious services; 32% visited family; 24% went to restaurants or sports activities.
Childcare involving contact with nonhousehold members: 20% were looked after by a non-household adult; 1% were in a childcare program. Ages not reported; Childcare centres and schools closed due to H1N1 -10% were closed for 1 day, 19% for 2 days, 29% for 3 days, 15% for 4 days, 17% for 5 days, 9% for more than 5, 2% didn't know 56% reported their child participated in at least one activity involving people outside the household: 30% visited other children's houses; 30% went grocery shopping; 23% went to fast food restaurants; 17% went to public events such as movies, sporting events or concerts; 15% visited large shopping areas or malls; 6% attended social events such as parties or dances. 81% were cared for by an adult in the household, 20% by a family member outside the household, 1% by a friend/neighbour, 3% by a professional care provider, and 10% stayed home alone. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. 99 students with laboratory confirmed pH1N1; Age 6-17; 49% were aged 14-15; 33% were aged 16-17; 9% were aged 12-13; 5% were aged 10-11; 4% were aged 6-7; Seven schools closed for a range of 3-9 days (not including weekends) due to H1N1 26% (21/81) who reported usually taking part in extra-curricular activities (not sports or religious activities) continued to take part in extra-curricular activities; 23% (14/62) continued to participate in sports activities; 20% (2/10) continued to take university classes; 11% (2/18) continued to attend part-time work; 5% (1/20) continued to take part in religious activities. 2,229 households representing 4,247 students; Kindergarten -Grade 12; Schools closed for 4 days in preparation for Hurricane Isaac 5.3% of children went to work with their parents; 2.6% of children were put in a childcare program. 11.6% were old enough to care for themselves, while 2.5% were left at home without supervision.
. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted March 20, 2020. . https://doi.org/10.1101/2020.03.17.20037457 doi: medRxiv preprint