Relevance of the Hierarchical Linear Model to TIMSS Data Analyses.

With more than 40 nations participating and five grade levels assessed in two subject areas, the Third International Mathematics and Science Study (TIMSS) is the largest and most comprehensive project in comparative education (Martin & Kelly, 1996). Secondary analyses of the TIMSS data may enrich information on the U.S. condition of education in a cross-nation context (Peak, 1996). Because TIMSS data have a hierarchical structure with students nested in schools and schools nested in countries, some researchers attempted to adopt the hierarchical linear model (HLM) for international comparisons. Based on the eighth grade science data, researchers of the TIMSS International Center reported, "An unconditional, three level analysis of science achievement indicates that the variance can be approportioned 26% within school, 35% between schools, [and] 39% between countries" (Gregory & Shen, 1999, p. 8). Meanwhile, Gregory and Shen (1999) cautioned that the lack of randomization at the country level may not need the variance partitioning in HLM. Since the choice of research methodology inevitably affects results of TIMSS data analyses, the purpose of this study is to examine features of the TIMSS design, and specify proper fixed and random factors within the hierarchical data structure. This investigation may help clarify relevance of HLM for TIMSS investigations. Fixed and Random Factors in Hierarchical Modeling Multilevel data can be analyzed by several software packages, such as HLM, MlwiN, and SAS (Wang, 1997). Accordingly, the terminology varies across the different computer software applications. Bryk and Raudenbush (1992) acknowledged in their book for HLM that, The models discussed in this book appear in diverse literatures under a variety of titles. In sociological research, they are often referred to as multilevel linear models [cf. Goldstein, 1987; Mason et al., 1983]. In biometric applications, the terms mixed-effects models and random-effects models are common [cf. Elston & Grizzle, 1962; Laird & Ware, 1982]. (p. 3) At any levels of the hierarchy, factors can be either fixed or random, depending on the research design. Casella and Berger (1990) defined: A factor is a fixed factor if all the values of interest are included in the experiment. A factor is a random factor if the values of interest are not included in the experiment and those that are can be considered to be randomly chosen from the values of interest. (p. 529) Milliken and Johnson (1984) added that "a model is called a mixed or mixed effects model if some of the factors in the treatment structure are fixed effects and some are random effects" (p. 213). In TIMSS, research interest is confined in around 40 participating nations. Thus, country can be treated as a fixed factor, and the research findings are not designed for generalization over all countries in the world. On the other hand, schools and students are sampled randomly within each nation. Factors at these levels can be treated as random factors, and the sample statistics will be used to represent the condition of education in each nation. Therefore, the hierarchical model contains both fixed and random factors. The HLM computing, as a method of mixed-effects modeling, should be relevant to the TIMSS data analyses. Dummy Factor Regression Before TIMSS, the International Association for the Evaluation of Educational Achievement (IEA) sponsored the first and second round of international mathematics and science studies. International comparisons were made using a dummy variable approach in the regression analyses (personal communication with John Keeves, April 13, 1998). For simplicity, a two level model is illustrated in this proposal to articulate relations between the dummy variable regression and the hierarchical linear modeling. According to Bryk and Raudenbush (1992), the hierarchical model can be presented as: Level 1 (e. …