Comments on the papers by Gelman et al. and Wakefield and Salway

Lianne Sheppard (University of Washington, Seattle) As both the papers by Gelman et al. and Wakefield and Salway emphasize, a deeper understanding of ecologic studies is vital to their application. There is a fundamental misalignment in many ecologic studies because the level of analysis, the group, does not match the level of inference, the individual. In their stimulating papers, Gelman and his colleagues and Wakefield and Salway address this misalignment by concentrating on features of ecologic regressions under fundamentally different frameworks. Gelman and his colleagues restrict their attention to binary or categorical covariates and cast their models in terms that are familiar in the political science literature. Within this structure they suggest some workable approaches to identifying violations of model assumptions. They rightly highlight the fundamental inability to evaluate all the assumptions that are necessary in an ecologic regression with the data at hand. Wakefield and Salway focus on the more general challenge of coping with continuous covariates in a group level setting. They present their framework for ecologic studies in the context of environmental epidemiologic applications. In all applications, group level studies, unlike multilevel studies, cannot separately estimate covariate effects that operate differently through groups and individuals. Since the additional effects of group membership (often called contextual effects (Firebaugh, 1978)) are much more likely to derive from the social rather than the physical environment, the intended application is crucial to the potential for valid inference from group level studies. It is natural to assume that environmental epidemiologic applications will require fewer untestable assumptions than ecologic studies in the social and behavioural sciences. Wakefield and Salway emphasize the importance of the within-area joint distribution of the exposure variables. Of particular value, they clarify the variance of an aggregate outcome (conditional on the exposure) induced under a range of assumptions about the exposure. They restrict their detailed treatment to cases where the within-area outcomes are independent. In many applications this assumption will be too strong and consequently there will be additional dependences in the variance models. A second feature of ignoring the within-area exposure distribution is its influence on the bias of the relative risk parameter estimates for all but the additive risk model. For instance, in their epidemiologic example, Gelman and his colleagues substitute average covariate measurements for individual values in a non-linear model. This is common practice in epidemiology (see for example Guthrie and Sheppard's (2000) discussion of Greenland and Robins's (1994) treatment of a similar radon model). The fitted ecologic model is clearly misspecified when inference is directed at the individual. The implications of this misspecification depend on the full within-area exposure distribution, not just the mean, features which are not included in the ecologic model. Gelman and his colleagues describe ramifications that are specific to radon exposures, whereas Wakefield and Salway give a more general perspective. These papers, as well as others in this issue, give fresh insights into many important considerations for ecologic studies.