Nonresponse bias in ratings of mental health problems 1 Predicting nonresponse bias from teacher ratings of mental health problems in primary school children

The impact of nonresponse on estimates of mental health problems was examined in a prospective teacher screen in a community survey of 9155 seven-nine year olds. For 6611 of the children parents consented to participation in the actual study (Responders) and for 2544 children parental consent was not obtained (Nonresponders). The teacher screen involved assessment of a broad set of problems and symptoms (emotional, conduct/oppositional defiance, hyperactivity, inattention, peer relationships, social functioning, obsessive compulsive disorder (OCD), tics and also an assessment of possible functional impairment (impact). Calculations of non-response coefficients, a function of effect sizes and non-response proportion, revealed only ignorable non-response bias for both mean scores and correlations between subscale scores. However, the results from binary logistic regressions revealed that moderate symptom scores on the teacher screen were associated with increased risk of children not obtaining parental consent to participate. This was most frequent among children with no impact, but it was a consistent finding that children with high scores of attention problems were more likely not to obtain parental consent to participate. These findings suggest that effect size estimates of nonresponse bias should be accompanied by a logistic regression approach to estimate if characteristic of the child is related to study participation. Nonresponse bias in ratings of mental health problems 3 Introduction Nonresponse is a challenge in all epidemiologic research. It introduces a possible bias when there are substantial and systematic differences between the research sample and the target population. This is a threat to the generalizability of the research findings. Such a bias may arise for a number of reasons; sample frame exclusion, differential mortality or greater reluctance to participate. Surveys of mental health problems in children are often thought to be particularly susceptible to nonresponse bias since the children’s caregivers, who consent to participation, may be reluctant to report such problems (e.g Owens et al, 2002). This is also based on the observation that many children with mental health problems never seek treatment (e.g. McMiller & Weisz, 1996; Pavuluri, Luk, & McGee, 1996) As a consequence, estimates of mental health problems from populationbased surveys may be inaccurate, with possible serious consequences for the tailoring of appropriate health services. Differences in the magnitude of non-response bias may also be one of the reasons for the considerable differences in prevalence estimates for child mental health problems (see Roberts et al. 1998). Response proportion is often used as an indicator of the likelihood of nonresponse bias, but a high response proportion is neither a necessary nor a sufficient condition for unbiased estimates (e.g. Stang, 2003). In fact, substantial differences between Responders and Nonresponders has been Nonresponse bias in ratings of mental health problems 4 found in surveys of mental health problems in children with low nonresponse proportion (e.g. Noll et al. (1997), wheras high nonresponse proportion might be associated with little bias, as in French and Waas (1985). An explanation for a possible inverse relation could be that a large nonresponse proportion comprise multiple reasons for nonresponse and that these level each other out, while when nonresponse proportion is low it may be more directly linked to the subject of the study, and thereby creating more bias. Noll et al. (1997) found that the small group of children who did not participate in a study of social skills were judged to be less socially accepted by their peers. However, to substantiate that amount of nonresponse bias is linked to the reason for nonresponse, one would need information from the Nonresponders about their reasons to not participate, which most surveys find difficult to provide. Surveys on mental health problems in children instead typically evaluate the impact of nonresponse by comparing various child and family background variables (e.g. Weinberger et al, 1990). However, even if there are differences between the two groups, they do not necessarily imply that this causes bias. To draw such a conclusion, one would need to substantiate that characteristics of nonresponders are associated with the outcome measures; something which is rarely done. Gerrits et al. (2001) argued that the magnitude of nonresponse bias could be estimated by the use of standardized mean scores (or effect sizes) Nonresponse bias in ratings of mental health problems 5 between Responders and nonresponders adjusted for nonresponse proportion. Gerrits et al. (2001) used the data from an original survey of social development to demonstrate that even for data sets with substantial nonresponse proportions, there were only neglible non-response bias (ranging from .00 to +/-.12), due to small differences between the Responders and Nonresponders. However, effect size estimates of overall group comparisons may be of limited interest, since most mental health problems are relatively low frequent in children. What is interesting is thus not only if a response and nonresponse group differ in estimates of mental health problems at a group level, but if the number of children with high scores are overor underrepresented among the nonresponders. A way to accommodate this would be to examine the frequency distribution of scores in Responders and Nonresponders and calculate their risk for being high scorers on the screening questionnaire. To understand nonresponse bias in surveys of mental health problems in children, both Responders and Nonresponders should be assessed prospectively within the same study to ensure that all children are screened on the same items and under the same circumstances. This was achieved in the first wave of the Bergen Child Study, a total population community survey of primary school children, where teachers filled in the questionnaire for all their pupils independent of parental consent. Teacher Nonresponse bias in ratings of mental health problems 6 screen for children with parental consent were subsequently made identifiable (Responders), while the teacher screen for the children without parental consent remained anonymous (Nonresponders). Nonreponse bias was evaluated against a broad range of symptoms/problem areas: emotional symptoms, conduct problems, hyperactivity/inattention, problems with peer relations, problems with social functioning, tics, obsessive compulsive disorders (OCD) and Oppositional Defiance Disorder (ODD). Teachers also rated the impact of the symptoms/problems, to yield an estimate of the child’s functional impairment. Two different statistical approaches were used to answer the question of nonresponse bias. The first approach comprised comparisons of mean scores and correlations using bias estimates and effect size calculations according to Gerrits et al.’s (2001) suggestions. The second approach comprised comparisons of the frequency distributions of subscale scores between the two groups, and logistic regression to evaluate the extent to which the teachers’ report of mental health problems in the children predicted the likelihood of the children to take part in the study.