metadata: authors: 0: author: Lockheed, Marlaine E.
1: author: Burns, Barbara
count: Brazil
docna: 0: docna: School effects on achievement in secondary
            mathematics and Portuguese in Brazil
docty: Policy Research Working Paper
subtopic: Labor Markets,Labor Standards,Economics of Education,Educational Sciences,Educational Institutions & Facilities,Education Finance,Effective Schools and Teachers,Rural Labor Markets
teratopic: Gender
abstracts: cdata!: The authors use a multilevel modeling
            procedure to explore 1) the percentage of variance in
            secondary school achievement in Brazil that could be
            attributed to the types of school attended, 2) differences
            between schools in students' achievement in mathematics
            and Portuguese, and 3) differences between schools in
            reducing achievement differences based on students'
            socioeconomic status. The authors identified the following :
            students in Brazil's federal technical schools
            outperformed students in other schools in both mathematics
            and Portuguese. Important factors were class size
            (achievement was higher in larger classes), the number of
            hours math was taught (the more the better), the
            school's organizational complexity, average family
            social class background, and the number of hours students
            spent working.
geo_reg_key: 119228,517191
repnme: repnme: School effects on achievement in secondary
            mathematics and Portuguese in Brazil
display_title: School effects on achievement in
            secondary mathematics and Portuguese in Brazil
listing_relative_url: /research/1990/10/700157/school-effects-achievement-secondary-mathematics-portuguese-brazil
topicv3: Teachers Management m1326119 1523,Working Hours m1326490 1672,Teacher Training m1326786 1532,Education m1327048 470,Private Education m1327271 1490
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Gender
origu: Policy, Research & Ect Affairs
no_of_pages: 30
model_extractions: **Policy, Research,'and External Affairs**

<!-- Start of picture text -->
*  _S  1<br>tEducaion  and Employment<br>WVPS 525<br><!-- End of picture text -->

- This paper a product of the Education and Employment Division, Population and Human Resources - Department is part of a larger effort in PRE to understand differences in educational effectiveness. Copies are available free from the World Bank, 1818 H Street NW, Washington DC 20433. Please contact Cynthia Cristobal, room S6-033, extension 33640 (26 pages).

Lockheed and Bruns use a multilevel modeling procedure to explore (I) the percentage of variance in secondary school achievement in Brazil that could be attributed to the types of school attended, (2) differences between schools in students' achievement in mathematics and Portuguese, and (3) differences between schools in reducing achievement differences based on students' socioeconomic status.

Students in federal technical schools outperformed students in general secondary, SENAI,* and teacher training schools in both mathematics and Portuguese, after holding constant for gender, age, family size, and the number of hours the student spent working. This could reflect differences in students' entry-level performance as admission to federal technical schools in Brazil is highly selective.

other schools, Lockheed and Bruns explored variations in their organization, quality, and social composition.

Factors significantly related to average mathematics achievement were class size (achievement was higher in larger classes) and the number of hours math was taught (the more time, the higher average achievement), and the school's average student socioeconomic status (family sociai O.Aass background), suggesting that student selection into the schools accounted for much of the observed difference.

Factors significantly related to average achievement In Portuguese were the school's organizational complexity, the ,.verage sociocconomic status, and the average number of hours students spent working.

For mathematics only, students in private schools outperforned those in public schools.

To explain why students in federal technical and private schools outperformed students in Performance was not different for schools paying higher salaries, day schools, high-fee schools, or schools where teachers attended university.

*SENAI secondary schools are financed by the federal govcrnmerit but administered by the National Confederation of Industry (a private association of industrial employers).

The PRE Working Paper Series disseminates the findings of work under way in the Bank's Policy, Research, and Extemal AffairsComplex. Anobjectiveoftheseries is to getthcse findingsoutquickly, even if presentations are less than fullypolished. The findings, interpretations, and conclusions in these papers do not necessariiy represent official Bank policy.

Produced by **the** PRE Dissemination Center
# Table of Contents

|Introduction<br>................................................................<br>2|
|---|
|Method<br>......................................................................<br>3|
|Sample<br>................................................................<br>4<br>|
|Instruments<br>...........................................................<br>5<br>|
|Analytic sample<br>.......................................................<br>5|
|Analytic Models and Results<br>.................................................<br>6<br>|
|Models<br>................................................................<br>6|
|School and individual contribution<br>to variance in achievement<br>.........<br>8<br>|
|School type effects on achievement and social class<sup>differentiation.</sup><br>.10|
|Student background effects on achievement<br>............................<br>11|
|Average achievement<br>............................................<br>12|
|SES-achievement<br>gap<br>............................................<br>14<br>|
|Differences between schools<br>..........................................<br>15|
|School organizational<br>and quality factors<br>............................<br>17<br>|
|Average achievement<br>............................................<br>17<br>|
|Social class differentiation<br>...................................<br>18|
|School compositional<br>factors<br>.........................................<br>19<br>|
|Average achievement<br>............................................<br>20|
|Social class differentiation<br>...................................<br>20|
|Final reduced model<br>..................................................<br>20|
|Conclusion<br>.................................................................<br>22|
|Annexes<br>....................................................................<br>25|
|References<br>.................................................................<br>26|

The World Bank does not accept responsibility for the views expressed herein, which are those of the authors and should not bB<sup>attributed to the World Bank</sup> or to its affiliated organizations. The findings, interpretations, and conclusions are the results of research or analysis supported<sup>by the Bank;</sup> they do not necessarily represent official policy of<sup>the Bank. The authors</sup> gratefully acknowledge the contributions of Mr. Qinghua Zhau for computational assistance and of Dr. Heraldo Vianna of the Carlos Chagas Foundation<sup>for</sup> designing and administering the test and making the data available<sup>to us.</sup> Support for this research was provided by RPO 674-84.
# - 2 -

# 1. Introduction

1. The differential effect of schools on student achievement has been a matter of considerable debate over the past quarter century. While early studies, typically utilizing cross-sectional designs, sought to identify the relative contribution of family background versus school characteristics to achievement, more recent research, often employing panel data, has sought to identify those elements of schools that explain (i) differences in student growth in achievement (e.g. Raudenbush and Bryk, 1986; Lockheed and Longford, 1989; Riddell, 1989), or (ii) differences in schools' ability to reduce the effects of family background on achievement (e.g. Lee and Bryk, 1988). Recent research has also differed from earlier work insofar as it employs multi-level modelling techniques appropriate for hierarchically organized data (see Aitken and Longford, 1986; Raudenbush, 1988 for reviews).

2. Key variables that have been examined include school sector (public or private, see Coleman, Hoffer and Kilgore, 1982); religious affiliation (Catholic or lay, see Lee and Bryk, 1988); and a variety of school-level inputs and processes (particularly peer, organizational, and teaching process variables). While a number of studies in more developed countries have erployed both panel designs and multi-level modelling techniques, comparable research on school effects on achievement in developing countries is, to the
-3.

best of o%r knowledge, limited to one study in Zimbabwe (Riddell, 1989) and one in Thailand (Lockheed and Longford, 1989); even research employing crossectional designs occurs infrequently. The two studies that examined school effects in developing countries reached similar conclusions, however: that student background and prior achievement have significantly greater impact on present achievement than do school factors, and that different schools are relatively consistent in their transformation of prior achievement into present achievement.

3. This study contributes to the literature by providing evidence for a Latin American country and by exploring a larger variety of school types than previously examined in either developed or developing countries. The present study examines the relative effect of four types of secondary schools (technical, SENAI, teacher training, and general secondary) and two sectors (public and private) on student achievement in mathematics and Portuguese in four cities in Brazil.

# 2. Method

4. The Ministry of Education and the World Bank commissioned the Carlos Chagas Foundation to design and administer a standardized test of mathematics and Portuguese to a sample of secondary students in four cities (Fortaleza, Salvador, Sao Paulo and Curitiba). The achievement test was designed for students at the end of their third (and final) year of secondary school, and comprised test items that had appeared previously on entrance examinations for Brazilian universities (yestibular). The test was accompanied by a background questionnaire for each student and a questionnaire about the school that was completed by the school director. Both tests and questionnaires were
-4-

administered in November, 1988. The overall purpose of the study was to explore differences in the relative effectiveness of various types of school in enhancing student achievement and in reducing performance differences between students from different social class backgrounds.

# Sample

5. For each city, a stratified random sample of schools was identified, with replacements; stratification was based on school type (Federal Technical, SENAI schools,1' teacher training [magisterio] schools, and general secondary schools), time of shift (day or night) and ownership (public or private); it was desigred to be representative of the actual distribution of schools and students in each state. Within each school, all students present on the day of test administration were requested to complete the test and accompanying questionnaire, although participation was voluntary.

6. The voluntary nature of participation may have affected the overall estimates of achievement for individuals, schools and cities. Private school and day shift students were slightly oversampled in Fortaleza but undersampled in Sao Paulo. In Salvador, public schools were oversampled. Strong collaboration from the State Secretary of Education in Parana resulted in the most representative sample for this city.

I/ SENAI secondary schools are financed by the federal government but administered by the National Confederation of Industry (a private association of industrial employers). The 17 SENAI secondary schools in Brazil offer a four-year program that combines a full secondary education with technical specialization.
-5-

# Instruments

7. Four instruments were developed: a mathematics test, a Portur *se test, a student background questionniaire, and a school questionnaire. The mathematics and Portuguese tests were designed to measure understanding of the basic secondary school curriculum in these areas, and included one or more item for each area of content: for example, -.erb tenses and reading comprehension in Portuguese and linear functions and trigonometry in mathematics. Items were selected from a pool of items used on vestibular tests on the basis of "item facility", a statistical measure of item discrimination. Reliabilities for the total tests were high, with Cronbach's - alpha .84 for mathematics and .75 for Portuguese. However, reliabilities calculated separately for students in teacher training programs (nmagisterio" students) were substantially lower, .30 for mathematics and .58 for Portuguese.

## Analytic sample

8. Data were obtained from 2648 students and 66 schools; after cleaning the data (range and logic checks) and making necessary corrections, usable data were obtained from 2611 students and 62 schools. Three schools were deleted for insufficient student-level data (9, 3 and 10 cases respectively) and a fourth school was deleted for insufficient school-level data (all fields were blank).
- 6 -

# 3. Analytic models and results

9. When longitudinal data are not available, an alternative approach is to disaggregate the variance in achievement into a student-level and a grouplevel component; it is then possible to model the group level variance in achievement while statistically controlling for student level characteristics. To do this, a multi-level modelling package, HIM, is used (Bryk, Raudenbush, Seltzer and Congdon, 1988). One advantage of the HLM procedure over ordinary least squares (OLS) is that is correctly estimates the standard errors for the school-level coefficients. This in turn means that the statistical significance of school-level variables is correctly estimated. A second advantage of the HIM procedure is that it can analyze the factors that influence differences in student achievement within schools (e.g. it models within school relationships, such as the rel onship between social class and achievement) at the same time as it evaluates the factors that influence student achievement across schools.

# Models

10. The multi-level modelling approach used in these analyses enables us to model two elements of the social distribution of achievement within schools. The student outcomes considered in these analyses are senior-year mathematics and Portuguese test scores. The within-school model holds constant sex, age, family size and the number of hours the student works, and regresses mathematics and Portuguese achievement for student i within school j as a function of family social class background (SES) which is a composite
-7.

index comprising father's occupation, father's education, mother's education and family income; Annex A reports the factor loadings for the index: MATHACNH - jo + pal SES + e

# PORTACHj - P' + Pjl SES + ej

The social distribution of achievement in each school is characterized in terms of two parameters: an intercept and one regression slope. SES is a continuous variable centered around its school mean. Tne two parameters m y be interpreted as follows: dR - Mean mathematics (Portuguese) achievement for students in school j.

- fjl The degree to which social class differences among students relate to achievement (the social class differe.atiation effect).

11. The school that is effective in equalizing the distribution of achievement would be characterized as simultaneously having a high average level of achievement, pp and a weak differentiation effect with regard to social class (i.e., a small value for _Pj1)._ These eff..cts are hypothesized to vary acrors schools as a function of school-level differences in management, organizational structure and peer group influence (compositional effects).
-8-

# School and individual contribution to variance in achievement

12. The first step in the HLM estimation process involves fitting an unconditional, or random regression model. We do this in two steps. In the first step, we partition the variance in the mathematics and Portuguese test scores into their between unit (school) and within unit (individual) components. The results are presented in Table 1; they show that school level factors account for nearly two-thirds of the variance in mathematics achievement but only about one third of the variance in Portuguese achievement. This result is consistent with research from other countries showing that, as one might expect, the more abstract the subject matter, the more important formal schooling is for developing student achievement.

Table 1: Results of variance component analysis: Brazil secondary, 1988

|Score|School|Individual|
|---|---|---|
|Mathematics|62.38|37.62|
|Portuguese|36.36|63.64|

13. In the second step, we fit an unconditional model that includes the SES-achievement gap. Table , presents the results. The average mathematics score is 14.29 (out of a maximum possible score of 45) and the average Portuguese score is 17.36 (out of a maximum possible score of 35). The average social class differentiation (the average within-school social classachievement slope) is .14 for mathematics and .31 for Portuguese. All are significant at probability levels less than .05. The school average achie.vement estimates are highly reliable for both mathematics and Portuguese, but the social class differentiation estimates are considerably less so. This
.9means that much of the observed variability<sup>in regression coefficienti is</sup> sampling variance and as a result unexplainable by school factors. Sufficient variability across schools on both the<sup>mean achievement and the SES</sup> differentiation effect does exist, however, to proceed.<sup>(The Chi-square chart</sup> indicates that all but the Portuguese<sup>social class differenti&vion</sup> effect estimated parameter variances are significantly different from zero).

Table 2: The _HIM_ unconditional modeil

||Gamma|||
|---|---|---|---|
|Estimated Effects|coefficients|Standard<br>||
|||error<br>t-stat|p-value|
|School mean Achievement||<br>||
|Mean mathematics|14.29|.75<br>19.04<br> <br>|.000<br>|
|Mean Portuguese|17.36|.41<br>42.39|.000|
|Social Class Differentiatio|n<br>|<br>||
|Mean mathematics|.29<br>|.15<br>1.99<br> <br>|.046<br>|
|Mean Portuguese|.31|.12<br>2.54|.011|
|The Chi Square Table||||
|Parameter|Estimated<br>D/F<br>|Chi-square|p-value|
||value|||
|Mean Achievement||||
|Mathematics|34.28<br>61<br>|4028.2<br>|.000<br>|
|Portuguese|9.87<br>61|1424.0|.000|
|SES Differentiation||||
|Mathematics|0.39<br>61<br>|90.41<br>|.009<br>|
|Portuguese|0.16<br>61|68.84|.229|
|<br>Reliability of School-Level<br>|<br>Random Effect|||
|Mean Achievement||||
|Mathematics-<br>.982||||
|Portuguese-<br>.948||||
|SES Differentiation||||
|Mathematics-<br>.223||||
|Portuguese-<br>.125||||
* 10 -

School tvoe effects on achievement and social class differentiation 14. The major purpose of this study was to explore differences in the mathematics and Portuguese achievement of students attending different types of secondary schools. Comparisons were made, therefore, between the scores of those attending federal technical, SENAI, teacher training and general secondary schools; Table 3 presents the average achievement of studonts attending variot's types of schools.

Table 3: Mean achievement scores for students in different types of secondary school, Brazil, 1l88

|School type|Mathem<br>Mean|atics<br>S.D.|Portue<br>Mean|uese<br>S.D.|
|---|---|---|---|---|
|Federal technical|22.60|2.18|21.11|0.88|
|SENAI|1275|0.53|16.79|0.82|
|Teacher training|11.24|1.56|16.79|2.04|
|General secondary|13.70|5.79|17.00|3.28|

15. On average, students in federal technical schools scored significantly higher on both mathematics (about 10 points) and Portuguese (about 5 points) than students in any of the other types of schools. Students in teacher training schools scored lowest in mathematics, w.th students in SENAI schools scoring about 1.5 points higher and students in general secondary schools about 2.5 points higher than students in teacher training schools. In Portuguese, performance at all types of schools other than the federal technical schools was equivalent. Average differences between schools, however, do not mean that the schools are responsible for the differences. Di£'farences in recruitment practices can also account for differences in achievement. Where some types of schools, such as the federal
11 -

-

technical schools, SENAI schools and the best private schools are selective and others are not, it is likely that the average ability of students will differ across schools.

# Student Background Effects on Achievement

16. In fact, the data clearly indicate that the different types of schools attract different types of students (Table 4). Students in federal technical schools and SENAI secondary schools are disproportionately male and come from higher social class backgrounds (SES is a composite index comprising father's occupation, father's schooling, family income and mother's schooling; Annex A reports the factor loadings for this index, which is standardized with a mean of 0), while students in teacher training schools are disproportionately female and come from lower social class backgrounds. The average socioeconomic level for students in general secondary schools in this sample was in the mid-range, but it should be recalled that this average is composed of private (generally high SES) and public (generally lower SES) school students. Other differences are that students in general secondary schools spend more hours a week working (which presumably detracts from time available for study), and students in federal technical schools spend fewer hours working than either SENAI or teacher training students.
- 12 -

Table 4: Average characteristics of students attending different types of secondary schools, Brazil 1988

|||School|type||
|---|---|---|---|---|
|Student characteristic|Federal<br>technica|l SENAI|Teacher<br>training|General<br> <br>secondary|
|Sex (% female)|26.0|6.8|96.1|58.7|
|Age in years|18.5|19.4|18.9|18.7|
|SES (factor score)|.18|.19|-.29|.02|
|Family size|4.7|4.2|4.4|4.5|
|Hours working weekly|5.8|8.6|8.0|14.5|
|Sample size|192<br>|118|309|1992|

17. To take into account these background differences between students attending different schools, we entered five student characteristic variables into the model: social class background, sex, age, family size, and number of hours per week that a student worked. All background characteristics were allowed to vary within schools, but a random effect was observed for SES only. The other four variables were "fixed" and the residual variances set to zero. Because general secondary and teacher training schools are found in both the public and private sector, we also included a dummy variable to test for sector effect on achievement.

18. Average achievement. The results of these analyses are presented in Table 5 (general secondary schools and public schools are the two omitted categories); since we are modelling the within-school SES-achievement relationship, the school average SES is entered into the achievement equation as a control variable. The results show that older students performed less well on both mathematics and Portuguese tests than did younger students, and girls outperformed boys in Portuguese, but boys outperformed girls in mathematics. Students who worked less outperformed those than students who
- 13 -

worked more hours per week. Family size was unrelated to achievement in either mathematics or Portuguese, and was dropped for subsequent analyses. Students from higher social class backgrounds outperformed those from lower social class backgrounds, on average, but the relationship differed between schools.

19. With student characteristics held constant, students in federal technical schools scored significantly higher in both mathematics (by about 9 points, or 1.5 standard deviations) and Portuguese (by about 3 points, or one standard deviation) than students in general secondary schools; previously observed modest differences betweeui students in general secondary schools and those in both teacher training schools and SENAI schools were entirely due to differences in student background. Controlling for school type and student background, students in private schools outperformed students in public schools on the mathematics test, but not in Portuguese. School type explained 69.7% of the between school variance in mathematics achievement and 64.9% of the between school variance in Portuguese achievement.
14 -

-

Table 5: School type effects on achievement and social class differentiation

|<br>|Mathema<br>|tics<br>|Portug<br>|uese<br>|
|---|---|---|---|---|
|Independent<br>variable|Coeff.|t-stat|Coeff.|t-stat|
|Fixed|||||
|Female|-1.48***<br>|<br>-7.06<br> <br>|0.58**<br>|3.02<br> <br>|
|Age|-0.37***|<br>-5.84|-0.48**<br>|<br>-8.24<br>|
|Family size|0.05|0.27<br>|0.05<br>|1.11<br> <br>|
|Working hours|-0.03***|<br>-4.65|-0.02**|<br>-3.62|
|Mean achievement||||<br>|
|Intercept|13.92***|<br>22.29<br>|16.81***<br>|<br>43.71<br> <br>|
|Average SES|4.71***|<br>4.74|3.82***<br>|<br>6.42<br>|
|Federal technical|9.28***|<br>5.20<br>|3.68**<br>|3.44<br>|
|SEtNAI|-1.52|-0.63|-0.84|-0.58<br>|
|Teacher training|-1.37|-1.06<br>|-0.09<br>|-0.12<br>|
|Private|3.20*|2.12|-0.20|-0.22|
|SES achievement gap|||||
|Base|0.11|0.54<br>|0.21<br>|1.26<br>|
|Federal technical|0.68<br>|1.31<br>|-0.25<br>|-0.57<br>|
|SENAI|-0.69|-1.07<br>|-0.37<br>|-0.70<br>|
|Teacher training|-0.35|-0.80<br>|-0.28<br>|-0.74<br>|
|Private|0.02|0.06|0.22|0.72|

* p < .05, ** p < .01, *** p < .001

20. SES-achievement gap. Although the SES-achievement relationship was significantly different from zero for both mathematics and Portuguese (Table 2), neither school type nor sector explained this relationship.<sup>However, for</sup> mathematics, the direction of the effect was positive for federal technical schools, suggesting that they increased social class<sup>differentiation,</sup> while it was negative for both SENAI and teacher training schools,<sup>suggesting that they</sup> may have ameliorated the effect. For Portuguese, effects of all three types of schools were negative, but statistically insignificant. Private schools increased social class differentiation, but weakly and statistically nonsignificantly.
- 15 -

# Differences Between Schools

21. For this analysis, we classified school characteristics into two groups: (i) school organization and quality and (ii) school composition. Four school organization variables were examined: the school's organizational complexity (a composite index comprising number of sessions (shifts), number of classes, and number of students in the school; Annex B reports the factor loadings for this index, which is standardized with a mean of 0), public or private sector (an indicator of more decentralized administration), day or night session, and average class size. Four school quality variables were examined: average teacher salary, average number of hours of mathematics instruction, average number of hours of Portuguese instruction (indicators of the opportunity to learn), and whether or not the school charged a high tuition fee. In addition to the average social class background of students in the school, two other school composition variables were explored: the average hours worked by students in the school and the percent of students who were female.

22. There were substantial differences among school types in these variables (Table 6). On average, federal technical schools were more organizationally complex, paid higher teacher salaries, but offered fewer hours of mathematics and Portuguese instruction than other schools. SENAI schools were less organizationally complex, offered more hours of instruction and had smaller classes than other schools. Schools also differed in terms of the composition of their student bodies.
- 16 -

# Table 6: Organizational quality and compositional characteristics of different types of secondary schools, Brazil 1988

|||Sch|ool Tvpe|
|---|---|---|---|
|School characteristic|Federal<br>technical|<br>SENAI|Teacher<br>General<br>training<br>secondary|
|Organization||||
|Complexity (factor score)|0.86|-1.40|0.20<br>-0.05|
|Number of classes|12.0|6.0|10.25<br>9.8|
|School size|420.0|180.0|400.0<br>377.4|
|Number of shifts|3.0|2.0|2.4<br>2.1|
|Private (%)|0|0|25.0<br>27.1|
|Day school (%)|100.0|100.0|87.5<br>58.3|
|Class size|35.0|25.0|37.5<br>38.8|
|Quality||||
|Teacher salaryaj<br>CZ2<br>(US|50,000 CZ25<br>$425) (US$|0,000 CZ<br>425) (U|112,100<br>CZ121,329<br>S$191)<br>(US$206)|
|Hours of Portuguese|1.75|3.0|2.63<br>2.58|
|Hours of mathematics|2.0|3.0|2.38<br>2.29|
|High fee private (%)|0|0|12.5<br>20.8|
|ComPosition||||
|Average SES (factor score)|<br>.11|.08|-.20<br>.04|
|Average working hours|4.98|13.13|8.82<br>14.53|
|Percent female|19.3|11.9|96.6<br>56.5|
|Sample size (schools)|4|2|8<br>48|

a/ November 1988 Cruzeiros. The average exchange rate for that month was CZ588.1 - US$1.

23. We next sought to explain the school type and sector effects on achievement, by examining the extent to which differences in school organization, quality or composition influenced the social distribution of achievement.
- 17 -

School Organizational and Ouality Factors 24. Our first hypothesis was that differences in the management, organization or quality of the schools accounted for the differences observed. Four school organization and quality variables were examined: the number of hours of mathematics or Portuguese instruction offered, the average teacher's salary, and average class size. Table 7 presents the results of these analyses.

Table 7: School organization and quality effects on secondary achievement and social class differentiation, Brazil 1988

||Mathem|atics|Portug|uese|
|---|---|---|---|---|
|Independent<br>variable|Coeff.|t-stat|Coeff.|t-stat|
|Fixed|||||
|Female|-1.51***|<br>-7.24|0.57**|2.99|
|Age|-0.37***|<br>-5.90|-0.48***|-8.24|
|Working hours|-0.03***|<br>-4.64|-0.02**|<br>-3.58|
|Mean achievement|||||
|Intercept|2.73|1.01|12.52***|<br>8.14|
|Average SES|5.43***|<br>5.66|3.79***|<br>7.93|
|Federal technical|8.87***|<br>4.73|3.34*|2.66|
|Private|1.66|1.08|-||
|Organizational<br>complexit|y 0.72|1.53|0.63*|2.54|
|Class size|0.15*|2.57|0.05|1.52|
|Hours of math/Port.|2.21**|3.73|0.72|1.91|
|Teacher salary|0.05|0.69|0.05|0.88|
|SES achievement gap|||||
|Base|0.08|0.08|0.23|0.29|
|Federal Technical|-0.10|-0.15|-|-|
|SENAI|-1.14|-1.49|-|-|
|Organizational<br>complexit|y 0.15|0.84|- 0.09|-0.63|
|Teacher salary|0.05|1.69|0.01|0.41|
|Class size|-0.01|0.47|- 0.18|-0.91|
|Hours of math/Port.|-0.11|-0.40|0.01|0.46|

* p < .05, ** p < .01, *** p < .001

25. Average achievement. For mathematics, two school organization and quality factors were associated with higher achievement: class size and the
> - 18 -

number of hours that mathematics was taught in the school. Students in schools with larger classes and in those that taught mathematics for more hours scored higher on the mathematics test; neither organizational complexity nor teacher quality (as indicated by teacher salary) had any impact on average achievement. For Portuguese achievement, however, organizational complexity was strongly and significantly related to achievement; students in larger schools with more sessions and classes outperformed students in schools that were less organizationally complex. However, the number of hours Portuguese was taught, class size and teacher salary were unrelated to achievement.

26. The introduction of school organization and quality variables into the model did not diminish relationship between federal technical schools and achievement; these variables did, however, reduce the private school effect, suggesting that the private school difference in mathematics achievement could in some part be attributed to more hours of mathematics instruction and larger classes. This latter result may reflect the fact that the lower-achieving SENAI schools also had substantially smaller classes than all other schools. School organizational and quality factors added about 9% additional variance explained for mathematics achievement (for a total of 78.3% of variance explained), and about 7% additional variance explained for Portuguese achievement (for a total of 72.3% of variance explained).

27. Social class differentiation. For mathematics only, school organization factors seemed to explain why federal technical schools exaggerated this effect; these factors did not, however, account for the reduction of social class differentiation in SENAI schools.
- 19 -

School ComRositional Factors '!8. A second hypothesis was that differences between schools was accounted for by peer effects: the composition of the peer group differed between the types of schools. Three school compositional variables in addition to average student SES were examined: percent female students, average number of hours students worked, and average school achievement in mathematics or Portuguese (for social class differentiation only). The results of these analyses are presented in Table 8.

Table 8: School composition, organization and quality effects on Secondary achievement and social class differentiation, Brazil 1988

||Mathem|atics|Portug|uese|
|---|---|---|---|---|
|Independent<br>variable|Coeff.|t-stat|Coeff.|t-stat|
|Fixed|||||
|Female|-1.48***|-7.05|0.55**|2.86|
|Age|-0.38***|-5.95|-0.49|-8.31|
|Working hours|-0.03***|-4.60|-0.02**|-3.58|
|Mean achievement|||||
|Intercept|7.15*|2.17|17.30***|<br>11.74|
|Average SES|5.05***|<br>C 66|3.06***|<br>6.20|
|Federal Technical|7.64***|<br>4.24|2.55*|2.19|
|Organizational complexity|<br>-|-|0.56*|2.10|
|Class size|0.17**|3.69|-|-|
|Hours of math/Port.|1.67*|2.69|0.55|1.57|
|Percent female|-3.18|-1.71|-1.31|-1.1?|
|Average working hours|-0.09|-1.94|-0.09**|-3.07|
|SES achievement gap|||||
|Base|-1.62|-1.40|1.10|0.80|
|Salary|0.04|1.36|-|-|
|Percent female|1.06|1.39|0.40|0.77|
|Average SES|-0.33|-0.93|0.50|1.58|
|Average working hours|-0.00|-0.03|-0.01|-0.32|
|Average math/Port. test|0.04|0.93|-0.06|-0.93|

* p < .05, ** p < .01, *** p < .001
- 20 -

# 29.

Average achievement. School composition effects on average achievement were very significant. As noted previously, students in schools where the average SES of their peers was higher outperformed students in schools where the average SES of their peers was lower; schools in which the average SES of the stud6nts was one standard deviation above the average SES scored 5 points higher in mathematics and 3 points higher in Portuguese, other things constant. Students in schools where few students worked outperformed students in schools where more students worked. And in mathematics, students in schools with a lower proportions of female students outperformed students in schools with higher proportions of female students. School composition factors added an additional 1% of explained variance in mathematics achievement (for a total of 79.2% of variance explained) and an additional 3% of explained variance in Portuguese achievement (for a total of 75.4% of variance explained).

# 30. Social class differentiation. School composition variables

completely washed out the effects of school organization on the social classachievement gap in mathematics, but themselves contributed nothing to explaining the parameter variance.

# Final Reduced Model

31. We next excluded non-significant variables from the models in Table 8. A final, simplified model presents the school and individual level variables that explain average achievement among the schools in this sample (Table 9).
- 21 -

Table 9: Reduced model, Brazil secondary achievement, 1988

||Mathem|atics|Portu|guese|
|---|---|---|---|---|
|Independent<br>variable|Coeff.|t-stat|Coeff.|t-stat|
|Fixed|||||
|Female|-1.47***|<br>-7.04|0.52*|2.72|
|Age|-0.37***|<br>-5.87|-0.48***|<br>-8.26|
|Number of hours working|-O.03***|<br>-4.63|-0.02**|-3.61|
|Mean achievement|||||
|Intercept|7.18*|2.19|17.92***|<br>41.84|
|Average SES|.02***|<br>6.66|3.17***|6.74|
|Federal technical school<br>|7.31***|<br>4.09|2.68**|2.87|
|Class size|0.17**|3.71|-||
|Hours of mathematics|1.71*|2.77|-||
|Percent female|-3.39|-1.84-|||
|Average working hours|-0.09|-1.92|-0.08*|-3.42|
|Organizational Complexity|<br>-|-|0.40|1.55|
|SES Achievement Gap|||||
|Base|0.10|0.67|0.18|1.43|

*p <.05, **p <.10, ***p <.001

32. These reduced models explain as much of the variance in average achievement as was explained by the larger models (Table 10). At this point, we also tested, for possible inclusion, three other variables for their effect on average mathematics or Portuguese achievement (day school, high fee private school, and number of teachers currently attending university); none were significantly related to average achievement.
- 22

Table 10: Percent between-school variance in secondary achievement explained by school type, school organization and quality, and school composition, Brazil 1988

||Mathe|matics|P|ortuguese|
|---|---|---|---|---|
|Model|tau|Percent<br>variance<br>explained|<br>tau|Percent<br>variance<br>explained|
|Unconditional|34.28|<br>-|9.87|<br>-|
|School type|10.40|<br>69.7%|3.46|<br>64.9%|
|School organization<br>and quality|7.45|<br>78.3%|2.73|<br>72.3%|
|School composition|7.12|<br>79.2%|2.43|<br>75.4%|
|Final reduced|7.13|<br>79.2%|2.54|<br>74.3%|

# 4. Conclusior

33. We found that a significant portion of the variance in achievement in secondary schools in Brazil could be attributed to the types of school attended. Students in federal technical schools outperformed students in general secondary, SENAI and teacher training schools in both mathematics and Portuguese, after holding constant sex, age, family size and the number of hours the student spent working. In mathematics, students in private schools outperformed those in public schools. We also found between-school differences in the SES-achievement gap for mathematics.

34. In seeking to explain the federal technical and private school effects on average achievement, we explored variations in school organization, quality and social composition. Two school quality factors were significantly related to average mathematics achievement: class size (achievement was higher in larger classes) and the number of hours mathematics was taught (the more
- 23 -

time spent on mathematics, the higher the average achievement). But the average SES of students in the school was also significantly related to average achievement, suggesting that student selection into the schools accounted for much of the observed differetnces. Variables unrelated to average mathematics achievement included teachers' salary levels, whether or not the school charged a high fee, enrolled students in day only classes, employed teachers who attended university or were organizationally complex. Once student background, school quality and compositional effects were taken into account, there was still a significant difference between the average performance of students at federal technical schools and those at other schools that we were unable to explain. It is plausible that this difference reflects differences in students' entry level performance, as admission to federal technical schools in Brazil is highly selective.

35. Average achievement in Portuguese was also higher at federal technical schools. One school organization variable (organizational complexity) and two compositional variables (average SES and average number of hours student spent working) were significantly related to average achievement. Again, however, even after student background, school quality and compositional effects were taken into account, there remained significant unexplained differences in the achievement of students at federal technical schools and those at other schools. Again, performance was not different for schools paying higher salaries, day schools, high fee schools, schools where teachers attended university, or those with smaller classes.
-

24 36. We were able to explain about 80% of between-school differences in average mathematics achievement and 75% of between-school differences in Portupuese achievement.
- 25 -

# Annex A: SES Factor

|Variable|Factor 1|Commonality|Factor Score<br>Coefficient|
|---|---|---|---|
|Father's occupation|-.497|.247|-.240|
|Father's schooling|.859|.738|.414|
|Family income|.651|.424|.314|
|Mother's schooling|.816|.665|.393|
|Eigenvalue|2.075|||

# Annex B: Organizational Complexity Factor

|Variable|Factor 1|Commonality|Factor Score<br>Coefficient|
|---|---|---|---|
|Number of shifts|.822|.675|.353|
|Number of classes|.960|.921|.413|
|Number of students|.854|.730|.367|
|Eigenvalue|2.326|||