Assessing Emotional and Behavioural Problems in Children: Factors Associated with Multiple Informant Consistency in New Zealand

Behaviour checklists using multiple informants are commonly used in the assessment of childhood behavioural and emotional problems. Characteristics of both the informants and children may effect such ratings, and in turn, this may effect inter-rater consistency and the confidence by which accurate conclusions can be drawn. This study investigates the extent to which selected demographic and clinical factors affect the consistency of ratings between informants on the commonly used Child Behaviour Checklist and the Teacher Report Form. Data were collated from a cohort of 200 cases aged 6-10 years referred to a New Zealand specialist outpatient clinic. Significant effects were found in relation to two problem areas. First, in regard to anxiety/depression problems, informants reported significantly greater consistency of agreement when the identified case was male, or when both informants were female. Second, in regard to the rating of thought problems, informants reported significantly greater consistency of agreement when initial assessment suggested the presence of a psychiatric diagnosis. Both the observation and expression of anxiety/depressive symptoms appear particularly affected by gender issues. These results reinforce the need for multiple sources of information when assessing emotional and behavioural problems in children. ********** Clinicians assessing the presence and extent of childhood emotional and behavioural disorders often employ a two-stage procedure to identify problematic conditions. In conjunction with detailed initial interviews with the child and relevant adults, behaviour surveys/checklists are frequently used as an economical way to survey multiple domains of functioning and help evolve clinical formulations. Such checklists are also used to obtain 'cut off' scores enabling children to be categorised as potential 'cases' for research or clinical purposes, and to assess meaningful changes in psychopathology arising from treatment (Visser et al., 2003). Surveys or checklists are typically completed by parents, teachers, or children, or any combination of such informants. Indeed, capturing information on children's specific behaviour across and within situations is considered important (American Psychiatric Association, 2000). An important aspect of the assessment, treatment and research of children's behavioural or emotional problems is deciding who should rate such behaviour. However different informants differ with respect to their sensitivity, perception, and relationship to aspects of a child's behaviour. As such, gathering information from a number of sources poses some unique problems regarding the extent to which accurate conclusions can be drawn for diagnostic or research purposes. This is an issue for both 'clinical' and 'nonclinical' populations, and for this reason, the issue of inter-rater consistency has long been discussed, especially in regard to one of the most common rating scales the Child Behaviour Checklist (CBCL) and it's parallel informant supplements (Achenbach, 1991a, , 1991b; Achenbach & Edelbrock, 1991). For example, Rutter, Tizard, Yule, Graham and Whitmore (1977) found that mothers and teachers agreed on the presence of behaviour disorders in only 7% of the disturbed children. Other studies echo concerns about poor correlations (Fergusson & Horwood, 1987a, 1987b, 1989), leading some to conclude that clinicians simply should expect that significant disagreement between informants will be the norm (Lee et al., 1994). The implications of this extend beyond clinical situations where assessment and treatment may be the focus. Sawyer, Baghurst and Clark (1992) noted that differences between informant reports could seriously distort prevalence estimates in epidemiological studies. Low agreement between different informants for any measures may be interpreted negatively and thereby cast doubt on one or both informants. Another interpretation is that, rather than reports being invalid, different informants contribute different information simply on the basis that symptoms or behaviours vary from one situation to another (Achenbach, McConaughy et al. …