WHO guideline on school health services. Web Annex F. Systematic reviews of the effectiveness and acceptability of comprehensive school health services: GRADE evidence profiles and evidence-to-decision table# Contents

# SREA systematic reviews of effectiveness and acceptability
A glossary of terms used throught the guidance and its web annexes is provided in the guidance document.

# Systematic reviews of the effectiveness and acceptability of comprehensive school health services: GRADE evidence profiles and evidence-to-decision table
Web Annex F

# F.1 SREA evidence-to-decision process
Following the GRADE process for assessment for evidence to recommendations (GRADEpro 2020) and the Evidence-to-Decision framework (Interactive Evidence-to-Decision Framework 2020) (2), Table F.1 sets out the questions and evidence profiles that were considered by the SHS Guideline Development Group (GDG), as well as their final judgments for each question (see Table 5 in the SHS guideline for a summary of GDG discussion points for each of the 12 questions). This process took place in a global virtual meeting that involved sessions being repeated for a western hemisphere subgroup and an eastern hemisphere subgroup. To avoid biases, information about what happened in one subgroup related to the formulation of recommendations was not shared with the other subgroup until both had evaluated the evidence and made judgements. If a GDG member participated in both subgroups, she or he only contributed to discussion and participated in the decision-making in the first subgroup. GDG members primarily drew on the systematic review findings in making judgements, but also drew on their knowledge and expertise. Web Annex F provides the Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence profiles and evidence-to-decision (ETD) table from the systematic reviews of the effectiveness and acceptability (SREA) of comprehensive school health services (SHS) (1). The Convention on the Rights of the Child (where "child" includes people less than 18 years old) includes the right to the highest attainable standard of health (3).

# F.2 SREA ETD outcome tables
Problems that arise during the second decade of life affect later health and development. WHO has stated that there are global goals and targets directed to young people (4)(5)(6).
In addition to worldwide recognition of child and adolescent health as a priority, individual countries such as the United States of America have also stated that it is a national priority to support the health and education of students (7) and have specific priorities such as reducing suicide among young people (8). The non-randomized controlled studies (NCSs) found evidence of a benefit of comprehensive SHS on the critical effectiveness outcomes of reduction in suicide planning (one study: 1 994 participants), hospitalization for asthma (one study: 273 participants), emergency department (ED) visits for asthma (five studies: 273, 1 994, 6 664, 5 866 and 2 369 participants, respectively; total 17 166), school absence/attendance (three studies: 6 664, 3 181 and 2 305 participants, respectively; total 12 150) and academic progress (one study: 2 305 participants).

# Suicide outcomes
In a study by Hutchinson et al. (9), the raw data reported were: SHS versus no SHS: suicidal ideation: males: 44/425 (10.4%) versus 34/386 (8.7%); females: 71/578 (12.2%) versus 83/605 (13.7%); suicide planning: males: 30/425 (7.1%) versus 30/386 (7.7%); females: 50/578 (8.6%) versus 66/605 (10.9%). Estimated marginal effects for school-based health centre (SBHC) versus non-SBHC students were calculated using kernel matching with regression adjustment. Males were less likely to have undertaken suicide planning in the last 12 months (p < 0.01); there was no effect among females on this measure or on suicide ideation in last 12 months among males or females (9). Hospitalization
In one study, the raw data for hospitalization with asthma were 12/196 (6.1%) versus 10/77 (13.0%). Based on results from the generalized estimating equation analysis of repeated measure Poisson regression, the relative risks of hospitalization for asthma were higher for non-SBHC students than SBHC students after the SBHCs opened for intervention students; relative risk 3.403, 95% CI: 1.536, 8.473, p < 0.05 (10).

# ED visits
In one study, the raw data for ED visits for asthma were SHS 40/196 (20.4%) versus 30/77 (39.0%). The relative risks of ED visits for asthma decreased 33.5% after the SBHCs opened for intervention students. The risk of ED visits for children in SBHC schools after the SBHC programme was 43% lower than children in non-SBHC schools (10).
In a second study, reported ED visits for all reasons were: intervention schools: 110/990 (11%); comparison schools: 172/1499 (11%) and for asthma 15/990 (1.5%) versus 40/1499 (2.7%) (11).
In a third study, there was no significant difference in ED use in the past year: intervention schools: Year 1: 456/1377 (33.1%); intervention schools Year 2: 454/1 455 (31.2%); comparison schools: Year 1: 295/992 (29.7%); comparison schools: Year 2: 329/1 044 (31.5%); p = 0.182 (12).
In a fourth study, the raw data were: SHS versus no SHS: 301/1 003 (30%) of students in SBHC schools had been to the ED in the last 12 months versus 348/991 (35.1%) in non-SBHC schools (p = 0.019; this was not significant after kernel matching with regression adjustment) (9). The fifth study found an increase in ED visits of only 1.2 visits per 100 person years in intervention schools compared to an overall increase of 12 visits per 100 person years in the control school districts without SBHCs over the 18-year study period (13).

# School absence
In one study, students in intervention schools were less likely than students in control schools to miss one or more days of school due to illness (2097/2877 (72.9%) versus 2477/3204 (77.3%); p <.05) (11).
In a second study, students enrolled in an SBHC were significantly more likely to return to class or not to be dismissed early from school than students not enrolled in an SBHC (absence: 11/351 (3.1%) versus 23/322 (7.1%); p = 0.013) (14).
In a third study, propensity score analysis was used and SBHC users had lower attendance rates than non-users at baseline (p <.001). Initially, attendance rates dropped for SBHC users but over time increased at a greater rate than non-users (15).

# GDG subgroup judgement
Research evidence from the systematic reviews of the effectiveness and acceptability of SHS, including GRADE tables

# Academic progress
In one study, propensity score analysis was used and there were overall increases in grade-point average over time for both groups and the more rapid increase for SBHC users than non-users (p < 0.05) (15).  Results came from a single study so inconsistency could not be evaluated.
c Estimated marginal effects for SBHC versus non-SBHC students were calculated using kernel matching with regression adjustment. Using this, the difference for males was stated to be significant, p < 0.01. Risk of bias; downgraded twice. The overall response rate was 76.2% of the enrolled school populations. Some differences in significance between raw data and after kernel matching with regression adjustment; 1 994 participants (9). The study did not differentiate between students who were treated by the SBHCs and students in the SBHC schools who were not treated; 273 participants (10). The parent survey involved analysis of 990 questionnaires from parents in intervention schools (35% response rate) and 1 499 from the comparison schools (response rate 48%); 6 664 participants in all (11). Limited information on exclusions and missing data; 5 866 participants (5). Only approximately 42% of the eligible students completed useable surveys (12).
g I 2 = 75%. h
The study did not differentiate between students who were treated by the SBHCs and students in the SBHC schools who were not treated; 273 participants (10). The parent survey involved analysis of 990 questionnaires from parents in intervention schools (35% response rate) and 1 499 from the comparison schools (response rate 48%); 6 664 participants in all (11).
The overall response rate was 76.2% of the enrolled school populations. Some differences in significance between raw data and after kernel matching with regression adjustment; 1 994 participants (9). Only approximately 42% of the eligible students completed useable surveys (12).
j I 2 = 65%. k
The parent survey (N = 990 in intervention schools; 35% response and 1 499 in comparison schools; 48%); 6 664 participants in all (11). There were substantial baseline differences between intervention and comparison schools for race (p = 0.001), age (p <.001) and gender (p = 0.005); 3 181 participants (14). The authors selected users in their first semester of ninth grade (n = 444) but excluded young people using the SBHC later (n = 993). Compared to the excluded group, SBHC users demonstrated significantly lower grade-point average, lower attendance rates, higher discipline rates, more single-parent or other guardianship, greater percentage of African-American or Native American race, more likely to be free-lunch eligible and more likely to be female; 2 305 participants (15).  Results came from a single study so inconsistency could not be evaluated.
c Downgraded twice for imprecision: very small study (n = 31) (16).
The NCSs also found evidence of a benefit of SHS on the important effectiveness outcomes in terms of students who had carried a weapon (one study; 1 994 participants), been in a fight (one study; 1 994 participants), ever had sex (one study; 1 994 participants), taken exercise at least four times a week (one study; 1 994 participants), ever drunk alcohol (one study; 1 994 participants) or ever used marijuana (one study; 1 994 participants) and on their quality of life (two studies; 109 and 1 360 participants, respectively; total 1 469), the student's academic expectations (one study; 416 schools (number of participants not stated)) and school engagement (one study; 416 schools (number of participants not stated)). Risk of bias; downgraded once. The overall response rate was 76.2% of the enrolled school populations. Some differences in significance between raw data and after kernel matching with regression adjustment; 1 994 participants (9).
Results came from a single study so inconsistency could not be evaluated.
Other important outcomes from NCSs that could not be assessed using GRADE were as follows.

# Health complaints
In one study, health complaints stayed stable among students who were offered the school nursing service, but increased among those with no such service, although the difference was not significant (multivariate analyses of one-year changes in score: F score = 3.63; p = 0.061) (17).

# Quality of life
In one study, student SBHC users showed significant improvement over time versus comparison schools on physical health-related quality of life (HRQOL) (p < 0.05). Compared to SBHC users (reference category), regression coefficients, representing unit changes in the paediatric quality of life measure over time, were 2.1 (nonsignificant) for SBHC non-users in SBHC schools and 8.0 (p < 0.05) for students in non-SBHC schools (18).
In a second study, whereas the initial quality of life was significantly lower among SBHC users than non-users in SBHC schools or students in non-SBHC schools (p < 0.05), the average HRQOL score increased among SBHC users and decreased among non-users and comparison school students over time, so that there was no significant difference between the groups at three years (19).

# School engagement
Engagement consisted of 12 questions such as, "The adults at my school look out for me," "My school offers a wide enough variety of activities or courses to keep students engaged at my school," and "School leaders encourage collaboration among teachers." Students in schools with SBHCs rated school engagement significantly higher than students in the comparison group (6.5 versus 6.3; p = 0.018). Parents in schools with SBHCs also rated school engagement significantly higher (6.2) than parents in the comparison group (6.0; p = 0.002). There was no significant difference between how teachers in schools with SBHCs rated school engagement versus teachers in the comparison group (20). Important outcomes: observational studies studies See above GRADE tables -the GDG discussed the certainty ratings, which ranged from very low to moderate. Given that the direction of the estimate of effect for all critical outcomes was in favour of comprehensive SHS, with some statistically significantly in favour and no signals of clear harms, and that many of the studies were large and diverse in terms of geographic area and type of health outcomes assessed, the overall certainty of evidence was moderate.    studies Four studies (273, 109, 6 664 and 658 participants, respectively; total 7 704) provided moderate strength of evidence that comprehensive SHS were cost-saving. Three studies (5 056, 1 430 and 477 163 participants, respectively; total 483 649) showed cost-benefits.

# Of the cost savings studies 
In the first study, after the opening of the SBHC, the relative risks of hospitalization and ED visits in the SBHC group decreased 2.4-fold and 33.5%, respectively. Due to reduced numbers of hospitalizations after the SBHC opened, the cost of hospitalization per child decreased significantly (p = 0.044) from US$ 1 150 to US$ 180 per child after controlling covariates in the intervention group, while the cost of hospitalization per child was relatively unchanged from US$ 583 to US$ 606 per child in the comparison group before and after the SBHCs opened. Due to reduced numbers of ED visits after the SBHC opened, the costs of ED visits per child were US$ 303 in both the intervention and comparison groups before the SBHCs opened, then decreased to US$ 275 per child in the intervention group and increased to US$ 331 per child in the comparison group after the SBHCs opened (p < 0.0001 between groups) (10).
- In the second study, conducted in the same schools as the first study in the same time period but focusing on 109 students with mental health problems, after the SBHC programme, the proportions of students accessing mental health-care services for urban and rural SBHC intervention schools increased 5.6% (p < 0.0001) and 5.9%
(p < 0.0001), respectively, compared with increases of 2.6% (p = 0.1023) and 0.2% (p = 0.9361) for urban and rural non-SBHC schools, respectively. Based on Medicaid claims, the study found SBHC students had significantly lower total health-care costs (p = 0.005) and lower costs of mental health services (p = 0.010) compared with non-SBHC students (18).
- In the third study, conducted between 2007 and 2009, mean absenteeism due to illness decreased when full-time nurses were added to intervention schools, but increased in comparison schools with only part-time nurses, so students in intervention schools missed fewer days due to illness than students in comparison schools (3.03 days/student versus 3.51 days/student, p <.05). The California Department of Education determines the amount of funding a school district receives by its average daily attendance, that is, the proportion of the total number of full-day student attendance to the total number of days per school year (180 days), leading to increased revenue for schools. Additionally, children in intervention schools were less likely to require ED visits for asthma-related events than children in comparison schools (15.15 visits/1 000 children versus 26.68 visits/1 000 children), leading to savings in parental wages. Average daily attendance and parental wage savings were estimated to be US$ 75 700, while the savings in ED visits was US$ 30 263 per 1 000 students (11).
- In the fourth study, conducted in 1985-1986, an SHS visit cost the Division of Public Health US$ 20.26. At the SHS, a student waited no more than five minutes for care, giving the student's lost school time cost of US$ 1.24 and a total cost of US$ 21.50 per visit. The cost for care at a private physician's office averaged US$ 23.94, plus the lost wages for a parent of US$ 14.30 per visit, plus school costs incurred whether or not the student was in class of US$ 4.59 giving a total cost of US$ 42.83, which is about twice the cost of the SHS visit (21).

# Of the cost-benefit studies 
In the first study, conducted in 1997-2003, net benefits of the SBHC programme in four school districts was estimated to be US$ 1 352 087 over three years, based on total costs of US$ 1 998 659 and total benefits of US$ 3 350 746. The researchers estimated that the SBHCs could have saved the health insurance scheme (Medicaid) at least US$ 35 per student per year (22).
In the second study, conducted in 1979-1980, the cost of a school nurse practitioner was calculated as US$ 9 044 and the nurse practitioner had 708 health encounters, giving a cost per encounter of US$ 12.74. Benefits were counted using the costs of the encounters if they had been provided by other providers, such as physicians or counsellors, and a benefit : cost ratio of 1.33 : 1 was calculated (23).
- In the third study, a total of 477 163 students in 933 schools in 78 school districts received SHS during the 2009/2010 school year. At a cost of US$ 79.0 million, the SHS programme prevented an estimated US$ 20.0 million in medical care costs, US$ 28.1 million in parents' productivity loss (associated with student early dismissal and medication administration) and US$ 129.1 million in teachers' productivity loss (associated with addressing student health issues). As a result, the programme generated a net benefit of US$ 98.2 million to society. For every dollar invested in the programme, society would gain US$ 2.20 (24). From the studies with high-quality (NCS) research designs (all of which were conducted in high-income countries (HIC)) that evaluated the acceptability of comprehensive SHS, there was moderate strength of evidence of a benefit of SHS on the critical acceptability outcome of user satisfaction in HIC.
- In the first study, among SBHC users, 605/637 (95%) supported having a health centre in their school compared with non-users in the SBHC school (619/728 (85%)) and students in the comparison school (491/711 (69%)); p < 0.001 for users versus comparison school and non-users versus comparison school (25).
- In the second study (416 schools (number of participants not stated)), linear regression was used to determine the relationship between being from a SBHC school and overall satisfaction with the learning environment when controlling for poverty, enrolment, school type, special education and English-language learners. Thirty-six per cent of the variation in satisfaction was accounted for by SBHC and the control variables. The slope suggested that students from the SBHC school rated the learning environment better by approximately 0.8 points compared to those in schools without SBHCs (p  0.05) (20). Risk of bias; downgraded once. The authors point out the difficulty in matching public high schools on key demographic factors that may also influence use of the SBHC. Although they did control for these statistically, high schools often have unique histories and leadership that are difficult to measure or control. The authors measured the impact of the SBHC across the entire student body in the intervention school, whether students had used the SBHC or not. This dilutes measured impact, as many students continue to use providers outside of the school; 2 076 participants (25).
Results came from a single study so inconsistency could not be evaluated. Individual studies found the following.
- Student scoring of health-care provider performance at their last routine physical, based on a seven-point Likert scale for "Explained things in an understandable way" was: control school: 5.14; intervention school SHS non-user: 5.50; intervention school SHS user: 5.84; non-users versus control: p < 0.001; user versus control: p < 0.001 (25).
- On a 0-10 scale, students in schools with SBHCs rated communication higher (5.7) than students in the comparison group (5.5); however, the difference was not statistically significant (t = 1.85, df = 301; p =.066). Parents in schools with SBHCs rated communication significantly higher (7.1) than parents in the comparison group (6.9) (t = 2.68, df = 414; p =.008). There was no significant difference between how teachers in schools with SBHCs rated communication versus teachers in the comparison group (20).
- Student scoring of health-care provider performance at their last routine physical, based on a seven-point Likert scale for "Had respect for student's concerns" was: control school: 5.86; intervention school SHS non-user: 6.21; intervention school SHS user: 6.44; non-users versus control: p < 0.001; user versus control: p < 0.001 (25).
- On a 0-10 scale, students in schools with SBHCs rated safety and respect higher (6.0) than students in the comparison group (5.9), however, the difference was not statistically significant (t = 1.69, df = 301; p =.092). There was no significant difference in the rating of safety and respect between SBHC schools and the comparison group for parents or teachers (20).
- Risk of bias; downgraded once. The authors point out the difficulty in matching public high schools on key demographic factors that may also influence use of the SBHC. Although they did control for these statistically, high schools often have unique histories and leadership that are difficult to measure or control. The authors measured the impact of the SBHC across the entire student body in the intervention school, whether students had used the SBHC or not. This dilutes measured impact, as many students continue to use providers outside of the school; 2 076 participants (25). The overall response rate was 76.2% of the enrolled school populations. Some differences in significance between raw data and after kernel matching with regression adjustment; 1 994 participants (9).
High unexplained heterogeneity. The participants in the two studies were of similar ages (grade 9-12 and age 14-18), although the majority of students in the study by Gibson et al. (25) were Latino and most in the study by Hutchinson et al. (9) were African-American.
Risk of bias; downgraded once. The authors point out the difficulty in matching public high schools on key demographic factors that may also influence use of the SBHC. Although they did control for these statistically, high schools often have unique histories and leadership that are difficult to measure or control. The authors measured the impact of the SBHC across the entire student body in the intervention school, whether students had used the SBHC or not. This dilutes measured impact, as many students continue to use providers outside of the school; 2 076 participants (25).
Results came from a single study so inconsistency could not be evaluated.
The observational (cross-sectional) studies from the LMIC that evaluated the acceptability of comprehensive SHS suggested that users were less satisfied with their SHS than for HIC, although there was no information about how satisfied they would have been with any realistic alternative or no SHS (two studies; 625 and 1 121 participants, respectively; total 1 746).
- In one study, the overall level of satisfaction of the high-school students with the SHS offered was addressed by the question, "Are you satisfied with the school health services?" Overall, 54.9% reported that they felt dissatisfied (26).
- In another study, on a three-point satisfaction scale that scored from 1 (dissatisfied), 2 (neutral), to 3 (satisfied), although the total mean (SD) satisfaction score was slightly higher for private school students than public school students (2.15 (0.5) and 2.05 (0.4), respectively), both were only slightly better than neutral and the difference was not statistically significant (27).
The 10 observational (cross-sectional) studies from LMIC that addressed the important acceptability outcomes showed that access to SHS (when measured by utilization) was variable (four studies; 1 577, 3 005, 360 and 830 participants, respectively; total 5 772). Access ranged from almost all students (97.1%) using the SHS in one study to only 5.4% of adolescents seeking health care through a school clinic in another study.
One observational (cross-sectional) study from LMIC (625 participants) suggested that some students were disappointed in terms of confidentiality (61.6%), respected privacy (57.0%), listening (85.3%), understanding (82.6%), dialogue (82.4%), support (79.0%), information (51.0%), empathy (43.0%), respect (53.4%) and exam time (70.9%). Three observational (cross-sectional) studies from LMIC provided variable results in terms of provider or other professional satisfaction (148, 720 and 60 participants, respectively; total 928):
- in one study, the school principals' overall satisfaction was rated as "satisfied to some extent"
- in a second study, 337 (93.6%) teachers in private schools and 338 (93.9%) teachers in public schools believed that school nurses were needed, for example for first-aid, providing health education to students and promoting good health habits, and evaluating students with behavioural problems
- in a third study, the percentage of participants (health trainers, training managers and teachers working in elementary schools) agreeing that standards were met was: 93.85% for first-aid and screening functions of nurses; 69.29% for consultations with colleagues, peers and mental health consultants; 71.92% for health education; 76.31% for diagnosis; 62.29% for managing health planning with educational plans; but only 39.47% for using a suitable evaluation system for the plan for students, parents and staff.
Particular attributes of SHS highlighted included that they were:
- comprehensive - convenient - necessary - an opportunity for gaining information and learning that may be beneficial in maintaining health. Potential benefits noted were to identify health problems (such as dental, visual and nutritional problems (like obesity), delayed vaccination status and social risk conditions)
and the integration of health, school and family.