# The Effects of Training School Staff for Utilizing Student Monitoring System Data

Laura Staman, Adrie J. Visscher, Hans Luyten

<figure><img src="image_1.png" /></figure>

## To cite this version:

Laura Staman, Adrie J. Visscher, Hans Luyten. The Effects of Training School Staff for Utilizing Student Monitoring System Data. 10th Next Generation of Information Technology in Educational Management (ITEM), Aug 2012, Bremen, Germany. pp.3-14, 10.1007/978-3-642-38411-0_1 . hal-01468475

HAL Id: hal-01468475

https://inria.hal.science/hal-01468475

Submitted on 15 Feb 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# The Effects of Training School Staff for Utilizing Student Monitoring System Data

Laura Staman, Adrie J. Visscher and Hans Luyten

Department of Educational Sciences, University of Twente, The Netherlands
a.j.visscher@utwente.nl

**Abstract.** The Dutch School Inspectorate defines achievement-oriented work (AOW) as the maximization of student performance in a systematic and goal-oriented way. Research by the Inspectorate shows that students in achievement-oriented schools perform better than students in schools that meet the criteria for achievement-oriented work less. The University of Twente has developed a training course in which school teams learn to work in an achievement-oriented way. Parallel to the training activities, the effects of training schools are studied. The research findings show that the training course has a positive effect on attitudes towards AOW, as well as on knowledge and skills relevant for AOW of school staff.

**Keywords.** Achievement-oriented work; student monitoring system; professionalization; data use; evaluation.

## 1 Introduction

Growing attention for achievement-oriented work (AOW) in Dutch primary education is caused by the supposed declining performance of Dutch students in the core subjects: arithmetic, language and reading. Research by the Dutch School Inspectorate [1] shows that students in AOW-schools perform better. The assumption is, therefore, that performance in Dutch schools in the core subjects can be improved if Dutch schools in general adopt the AOW approach. Achievement-oriented schools are supposed to use the results of evaluations of student performance (by means of student monitoring systems) for maximizing student performance by working in a more goal-oriented way and by adapting instruction in such a way that it meets students' needs as much as possible [2]. The research by the Inspectorate shows that the degree of AOW varies between schools. Only 30% of all primary schools operate in an AOW-way, and the majority of schools can improve its AOW-skills a great deal. This especially applies to analyzing student performance and diagnosing causes of underperformance, formulating challenging goals for all students, and adapting instructional activities to students' educational needs. Promoting AOW requires teacher professional development, and the development of an achievement oriented school organization [3, 4, 5, 6]. Based on the research of the Inspectorate with regard to the level of AOW, the conclusion can be drawn that in most Dutch primary schools AOW is not a reality yet. The Ministry of Education supports, however, various
initiatives for promoting AOW. One of these initiatives is the development of a
training course for primary schools in which schools learn to work in a systematic and
achievement-oriented manner. This so-called ‘Focus-project’ aims to improve student
performance in Dutch primary education by means of a two-year training course in
which schools learn to utilize student monitoring system data, formulate challenging
performance goals, and to provide instruction that is adapted to students’ instructional
needs.

## 2 Theoretical framework

Achievement-oriented work is equivalent to what in other countries is called ‘data-
driven teaching’ (i.e. [7]). In many countries student performance data for improving
student performance is promoted by the accountability contexts of schools [6].
Schools are held responsible for the performance of their students, and are supposed
to monitor and optimize student performance systematically [8]. Analyzing student
data in depth can point to specific problems of schools, teachers and groups of
students [9]. According to Carlson, Borman and Robinson [10] an achievement-
oriented approach includes collecting, interpreting, and distributing data, which could
support school improvement initiatives. The literature on school improvement and
school effectiveness also points to the central role of using student assessment data
within school improvement processes [11, 12]. In the view of Perie, Marion and Gong
[13] test data can be used in three ways:

* as a tool for teachers in adapting instruction and the curriculum to the needs of students;

* for evaluating and improving instruction; and

* for predicting students' test scores on future tests.

Using test data can also lead to a more professional school culture and more
cooperation within schools [14]. As a result, communication and knowledge within
schools increase, and data use can improve teachers’ attitudes towards providing
instruction for students [15]. Visscher and Ehren [4] explain the mechanisms through
which AOW at the school level can promote better education and student results. A
good image of the starting situation of the students, formulating challenging goals
based on that information, and choosing a route for accomplishing the goals set at the
school, grade and student level are all required. Achievement-oriented work as such
touches the whole school organization (see Figure 1; note that in this Figure, board
level indicates the school board level).
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Achievement-oriented work at various levels.</figcaption></figure>

**2.1 Achievement-oriented work at the school and classroom level**

Goertz, Olah and Riggan [5] show that leadership and an achievement-oriented school culture in combination with support from local government are critical factors in assisting teachers with AOW. The literature on school improvement stresses the positive relationship between distributed leadership and school innovation. Hulpia [16] describes distributed leadership from three different perspectives: as a distribution of leadership functions; as cooperation between the members of a management team; and as participative decision-making by teachers. The role of teachers in decision-making processes and the contribution of strong, collegial relationships are mentioned as important prerequisites for school improvement [17]. In the view of Earl [18] and Sutherland [19] distributed leadership in addition to supporting AOW can contribute to school improvement. School leaders can fulfill an important model function here. They should stimulate and support their school staff for AOW [19]. In an achievement-oriented school culture, high performance of all students in the core subject is considered important. Furthermore, achievement-oriented work is influenced by the degree to which school staff are offered opportunities to cooperate within the school. Additionally, a school leader is supposed to promote a shared vision, norms and school goals, as a climate in which data are used for school improvement (instead of using the data for evaluating teacher
performance). A school leader can also facilitate the work of his/her team members by providing time for, for example, collecting, analyzing and interpreting data. Conrad and Eller [20] note that the implementation of AOW requires intensive professional development interventions for teachers and school leaders. Finally, a combination of pressure and support is supposed to promote AOW [6]. Diamond and Spillane [21] and Slavin [22] stress that external pressure, for example through the governmental accountability policy in combination with support, provides the best opportunity for innovating the school organization. In the view of Visscher and Ehren [4] this also applies to the role of the school leader. Within AOW the school leader ideally monitors the AOW innovation process and stimulates all team members to do their best to implement the three components of AOW at the classroom level (see Figure 1), and by supporting them where necessary.

**Component 1: an accurate picture of students' starting situations.** According to Visscher and Ehren [4], the first important activity in an achievement-oriented approach is obtaining a good image of the actual performance level of students. Schools for that purpose have to collect test data at the school level on how the school functions. These test data can be compared with previous test data on the performance of the school as a whole, about grades, and individual students. Comparisons with national average performance are also possible. In addition to this, analyses can be made of which specific components of subject matter are mastered by students or not [23]. Test data should also lead to further analysis of causes of specific student results. A student monitoring system can support these activities. Performance feedback can be an important tool for school improvement. Research by Chen, Heritage and Lee [24] shows that school information systems can assist school staff in timely finding of students at risk which can help in adapting instruction to students' instructional needs, and higher student achievement levels [25, 26]. The positive effects of performance feedback on performance improvement have been reported repeatedly [27, 28, 29]. However, Hattie and Timperley [28] point to the fact that feedback effects are not always positive. The effects depend, for example, on the context in which the feedback is provided and the way in which it is provided. In order to have a positive effect, the feedback should be corrective (showing what is wrong and why) and show how the feedback recipient can improve his/her performance. About 90% of Dutch primary schools use one or more components of the CITO student monitoring system [3]. By means of a student monitoring system, achievement can be monitored on a longitudinal basis across the whole primary school at the level of individual students, grades and schools. The basis of the system is provided by high quality standardized tests for primary school students which are taken twice (in most cases: at the end of a school year, and half way through the school year) or once a school year. The system supports the analysis of the performance of individual students, student cohorts (for one test, or across all tests taken for a cohort), or classes (for one test taken, or across all tests taken for this class). The system not only supports the analysis of the performance level of students in comparison with the national benchmark but also the analysis of which components of the subject matter a student masters and which not (which is very important for designing instruction). As student performance as measured by means of all the tests taken can be expressed on one and the same scale, the monitoring system allows the
added value between two or more measurement moments to be determined. This is
important for analyzing how much a student learns in a specific period of time (e.g. a
school year) as a student may perform at a relatively high level but not have grown
much in that period of time. Similarly, it is also valuable knowing that a student who
performs at a relatively low level has grown a lot since the previous test taken.

**Component 2: setting goals for all students.** According to Visscher and Ehren [4] the second component of AOW includes formulating desired performance levels in terms of explicit and clear goals for individual students and the school as a whole. Locke and Latham's goal setting theory [30] states that explicit, specific and challenging goals lead to higher performance. Difficult goals require more effort and dedication than easy goals. Such goals steer behavior, and in general hard goals motivate people to accomplish those goals. In addition, definitions of success vary less if goals are formulated explicitly and clearly as the goals in that case indicate precisely when they are accomplished and when not. Locke and Latham [30], however, also stress the importance of self-efficacy in the context of goal-setting. Higher self-efficacy levels are proven to lead to higher goals while more goal-ownership leads to more searching for goal attainment strategies [31]. The combination of goal-setting and providing feedback proves to be more effective than the sum of the effects of each of these two activities. Based on the feedback, the strategies applied for accomplishing the goals set can be evaluated and improved.

**Component 3: determining the route for goal accomplishment.** The third step in the model of Visscher and Ehren (see Figure 1) includes taking decisions with respect to the instructional strategies that will be used for accomplishing the goals set. Research by Heritage et al. [32] shows that teachers are more capable of interpreting assessment data than they are of taking decisions on the desired nature of instruction based on assessment data. What is needed at this stage is deliberate practice: ideally teachers carefully and professionally take decisions on how instruction will look, based on student progress information (which will vary between students) instead of working in a routine way regardless of student progress details. Next to deciding on how instruction should look, teachers should evaluate the effects of instructional decisions taken. According to Massell [15], using assessment data for taking instructional decisions is a very complex task and in most cases teachers have not been prepared for taking such decisions.

## 2.2 The practice of achievement-oriented work

Based on the research on AOW, the conclusion can be drawn that AOW has not been
implemented in many schools yet [3] and that the expertise of school leaders, school
internal coaches and teachers regarding AOW is limited. Especially lacking are the
skills and knowledge for analyzing and interpreting assessment data, as are the taking
of decisions based on the assessment data with regard to how to adapt instruction to
students' educational needs. There is also much room for improvement regarding
formulating specific measurable goals at the school and classroom level.
Schildkamp and Kuiper [6] and Wayman [33] conclude that AOW can be
promoted by means of training and supporting school staff. School staff themselves
also indicate the need of support in the context of AOW [34]. Teachers' attitudes
regarding the effectiveness of interventions prove to be very important for the
effectiveness of school improvement initiatives [35, 36]. Their attitudes influence
their efforts and enthusiasm. More specifically, various authors stress the importance
of a positive attitude towards school performance feedback [37]. Training school staff
can have a positive effect on the attitudes of school staff towards school performance
feedback, and on the knowledge and skills for using the feedback [38].

## 3 Research question

In 2010 a longitudinal study was started, looking at the effects of the Focus training course for AOW on the basis of school performance feedback in 86 Dutch primary schools. The effects of the training course on the knowledge and skills and attitudes of school staff with respect to AOW have been investigated.

The data acquired have been used to answer the following research question: What effects does the Focus training course have on school staff's knowledge and skills with respect to the student monitoring system they have, as well as on their attitudes towards the use of the student monitoring system and AOW?

## 4 Research method

Schools were approached for participation in the Focus-project and all but one school was located in the Twente/Salland region in the Eastern part of the Netherlands. Schools participated in the Focus training project in the school years 2010 to 2011 and 2011 to 2012. Data from the pre-test and post-test were available with regard to knowledge and skills and attitudes concerning AOW for the participants in the school year 2010 to 2011 (54 school leaders, 55 internal coaches and 578 teachers in grades 1 up to and including grade 5). In the school year 2011-2012, school leaders, internal coaches and teachers in grades 6, 7 and 8 participated in the training course. A number of schools had an auxiliary branch which could be considered to be a school in itself, as a result of which the total number of experimental schools was 48.

**4.1 Instruments for data collection**

Instruments used for measuring knowledge and skills, and attitudes to AOW were:

* a test measuring skills to interpret student monitoring system data;

* a test measuring knowledge about various ways to use the student monitoring system;

* an instrument measuring general attitude towards AOW;

* an instrument measuring attitude towards Focus components of AOW; and
* an instrument measuring attitude towards student monitoring system use.

## 4.2 The intervention

As mentioned in previous sections, Visscher and Ehren [4] distinguish between three AOW components: obtaining an accurate image of the students’ starting situation; setting goals for all students; and choosing the route for accomplishing the goal set. The schools were trained to transform analyzed test data into an action plan (a group plan including learning and performance goals for all students as well as the didactical approaches for accomplishing the goals set). The Focus cycle for AOW included the analysis and diagnosis of test data, formulating action plans (group plans), and monitoring and evaluating the execution of this plan. The various components of the cycle had been specified in a protocol which included twelve steps. This protocol was filled out by schools twice a year (after the end-of-school-year test and the test taken after half a school year). Figure 2 presents the various components of the cycle, including the various steps in the protocol. In steps 1 - 6 teachers/schools are shown the various forms of performance feedback the student monitoring system can provide: data on the performance of students and student groups at one or more measurement moments. Thereafter, causes for underperformance are diagnosed using information about learning progression as well diagnostic conversations with individual students. Based on all the information available, teachers draw up group plans for their student groups in which they specify how each student in the group performs, how each student ideally will perform when the next test is taken, and which didactical approach will be used for each student to transform actual performance to a desired performance (step 10). Next, the group plans are carried out (step 11) and after some time the results are evaluated to determine whether the didactical strategy works. If not, the group plan is adapted to increase the probability of success.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** The Focus AOW cycle showing activities in the protocol at each stage.</figcaption></figure>
## 4.3 Data collection procedure

Knowledge and skills about the student monitoring system, and attitudes towards AOW were measured during Focus training sessions. In the case of knowledge and skills about the student monitoring system, respondents were asked to answer multiple choice questions (developed by the researchers based on their knowledge of the student monitoring system) about the types of data analysis the monitoring system allows, and about what the data mean.

Respondent's attitudes were measured by means of statements about the value of achievement-oriented work, and the use of a student monitoring system. The respondents were asked to indicate to what extent(s) they agreed with each of the statements.

## 4.4 Data analysis

Because of their nested structure (students nested in groups and teachers nested in schools) the data were analyzed by means of multilevel analysis techniques. In the case of measuring respondents' knowledge and skills about the student monitoring system, the number of correct answers on the pre-test and the post test was computed and compared to analyze the effects of the training activities. Differences between attitudes towards achievement-oriented work and the student monitoring system on the pre-test and the post- test were also computed. Only the data for respondents participating in the pre-test as well as the post-test were included in the analysis.

## 4.5 Results

The results point to a remarkable difference within and between schools with respect to the degree to which staff had acquired required knowledge and skills for AOW, and attitudes (motivation) towards AOW. Table 1 presents results of multi-level analyses.

Statistically significant differences between the post-test and the pre-test were found for ‘Knowledge student monitoring system’ (the correct interpretation of the data) (t = 11.46, p < 0.001), and ‘Knowledge student monitoring system’ (the various analyses that can be made by means of the system)’ (t = 16.28, p < 0.001). Furthermore, school leaders differed significantly from teachers on the pre-test with regard to the variable ‘Knowledge student monitoring system’ (interpretation) (t = 2.87, p < 0.010) and ‘Knowledge student monitoring system’ (possibilities) (t = 3.20, p < 0.010). On the pre-test, school leaders proved to know more about how the student monitoring system could be and should be (with correct interpretation) used. On the post-test, school leaders differed significantly from teachers in terms of their growth in knowledge with regard to how the student monitoring system output should be interpreted (‘Knowledge student monitoring system) (t = -2.06, p < 0.050).
<table><caption>Table 1. The growth in knowledge and skills, and attitudes towards AOW.</caption><thead><tr><th colspan="11">Fixed Effects</th><th colspan="11">Random Effects</th></tr><tr><th rowspan="2">Dependent variable</th><th colspan="2">Intercept</th><th colspan="2">School leader</th><th colspan="2">School leader * Time</th><th colspan="2">Internal Coach</th><th colspan="2">Internal Coach * Time</th><th colspan="2">Time</th><th colspan="2">Level 3 (school)</th><th colspan="2">Level 2 (respondent)</th><th colspan="2">Level 1 (measurement)</th></tr><tr><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th><th>Effect Estimates</th><th>S.E.</th></tr></thead><tbody><tr><td>Knowledge SMS (interpretation)</td><td>47.80</td><td>2.44</td><td>14.82**</td><td>5.16</td><td>-12.67*</td><td>6.15</td><td>14.61***</td><td>4.06</td><td>-8.75</td><td>5.15</td><td>23.33***</td><td>2.03</td><td>68.73*</td><td>31.57</td><td>41.03</td><td>23.07</td><td>168.14***</td><td>23.38</td></tr><tr><td>Knowledge SMS (possibilities)</td><td>47.22</td><td>1.52</td><td>10.66**</td><td>3.33</td><td>-7.09</td><td>4.26</td><td>7.45**</td><td>2.72</td><td>-6.31</td><td>3.37</td><td>24.73***</td><td>1.52</td><td>31.97*</td><td>12.45</td><td>49.55**</td><td>17.10</td><td>183.53***</td><td>17.47</td></tr><tr><td>General attitude AOW</td><td>40.25</td><td>.48</td><td>2.45*</td><td>1.00</td><td>.40</td><td>1.23</td><td>2.09*</td><td>.93</td><td>-.64</td><td>1.13</td><td>1.40**</td><td>.49</td><td>3.02*</td><td>1.26</td><td>3.76*</td><td>1.70</td><td>19.42***</td><td>1.87</td></tr><tr><td>Attitude Focus components AOW</td><td>13.94</td><td>.54</td><td>1.42</td><td>1.27</td><td>-4.02*</td><td>1.82</td><td>.48</td><td>1.16</td><td>-1.18</td><td>1.60</td><td>-1.79*</td><td>.72</td><td>1.03</td><td>.18</td><td>-</td><td>-</td><td>43.84***</td><td>2.93</td></tr><tr><td>Attitude SMS use</td><td>15.93</td><td>.39</td><td>1.26</td><td>.94</td><td>-1.36</td><td>1.28</td><td>1.01</td><td>.86</td><td>.35</td><td>1.14</td><td>1.53**</td><td>.51</td><td>.46</td><td>.60</td><td>1.64</td><td>2.07</td><td>21.88***</td><td>2.36</td></tr></tbody></table>

<footnote>* significant at *p* <.05, ** significant at *p* <.01, *** significant at *p* <.001</footnote>

<footnote>SMS = student monitoring system</footnote>
School leaders grew less than teachers but already had a higher score on the pre-test. Internal coaches also proved to differ significantly from teachers on the pre-test. They seemed to know more already about AOW: ‘Knowledge student monitoring system (interpretation)’ (t = 3.60, p < 0.001); and ‘Knowledge student monitoring system (possibilities)’ (t = 2.74, p < 0.010). Internal coaches did not grow significantly less or more than teachers on these variables.

The scores on the post-test for the variables 'General attitude towards AOW' (t = 284, p < 0.010), 'Attitude towards the Focus AOW components' (t = -2.48, p < 0.050), and 'Attitude towards student monitoring system use' (t = 2.98, p < 0.010) proved to differ significantly from the attitudes measured by means of the pre-test. The general attitude towards AOW and the attitude towards the student monitoring system use improved significantly whereas the attitude towards the Focus AOW components was lower on the post-test.

The results differed somewhat between the various types of school staff. School leaders and internal coaches were more positive towards AOW than teachers on the pre-test (for school leaders t = 2.45, p < 0.050; for internal coaches t = 2.25, p < 0.050). Moreover, school leader attitudes towards the Focus AOW components were lower on the post-test than teachers' attitudes towards the Focus components of AOW (t = -2.22, p < 0.050).

# 5 Conclusion and discussion

The results show that Focus training influences attitudes towards, and knowledge and skills for AOW in a positive way. This is in line with Branderhorst [38] who also found a positive effect of training activities on the attitudes of school staff towards school performance feedback. The fact that school leaders and internal coaches had a more positive general attitude towards AOW on the pre-test than teachers might have been caused by the fact that school leaders and internal coaches already knew more about AOW (which was proven in this study). As a result of this they might already have been more capable of seeing the importance of and need for AOW. Moreover, the more positive attitude of school leaders and internal coaches might also have been caused by the fact that they were involved in the process in which schools were asked to participate in Focus training, during which they heard more about the Focus training course and AOW, and its importance.

The growth in knowledge and skills for AOW varied considerably between the various types of school staff. School leaders and internal coaches scored better on the pre-test than teachers which might explain that they grew relatively little between the pre-test and the post-test. In other words, the Focus training activities professionalized teachers most with regard to how the student monitoring system could be used (the various types of analyses) and how the results of the analyses should be interpreted. As teachers were the central actors in the process of producing school results, this finding is very promising for improving the results of schools. One would expect that schools in which teachers learned how to use their student monitoring systems, and in which teachers used the information retrieved from their student monitoring systems for improving the quality of instructional processes in the long-run will have better
results. However, choosing the most promising instructional strategies based on information regarding where the students are in the learning process is not easy [32].

## References

1. Inspectie van het Onderwijs: Opbrengstgericht werken in het basisonderwijs; een onderzoek naar opbrengstgericht werken bij rekenen-wiskunde in het basisonderwijs. Inspectie van het Onderwijs, Utrecht (2010)

2. Inspectie van het Onderwijs: De staat van het Onderwijs: onderwijsverslag 2007/2008. Inspectie van het Onderwijs: Utrecht (2009)

3. Ledoux, G., Blok, H., Boogaard, M., Krüger, M.: Opbrengstgericht werken over de waarde van meetgestuurd onderwijs. SCO-Kohnstamm Instituut., Amsterdam (2009)

4. Visscher, A., Ehren, M.: De eenvoud en complexiteit van opbrengstgericht werken. Accessed on 23 January 2012 at: http://www.rijksoverheid.nl/documenten-en-publicaties/rapporten/2011/07/13/de-eenvoud-en-complexiteit-van-opbrengstgericht-werken.html, (2011)

5. Goertz, M.E., Olah, L.N., Riggan, M.: From testing to teaching: The use of interim assessments in classroom instruction. University of Pennsylvania, Philadelphia, PA (2009)

6. Schildkamp, K., Kuiper, W. (2010). Data-informed curriculum reform: which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26, 482-496, (2010)

7. Fullan, M.: The new meaning of educational change. Teacher College Press, New York, NY and London (2007)

8. Kerr, K.A., Marsh, J.A., Ikemoto, G.S., Darilek, H., Barney, H.: Strategies to promote data use for instructional improvements: actions, outcomes, and lessons from three urban districts. American Journal of Education, 112, 496-520, (2006)

9. Kennedy, E.: Raising test scores for all students: An administrator's guide to improving standardized test performance. Corwin, Thousand Oaks, CA (2003)

10. Carlson, D., Borman, G.D., Robinson, M.: A Multistate District-Level Cluster Randomized Trial of the Impact of Data-Driven Reform on Reading and Mathematics Achievement. Educational Evaluation and Policy Analysis, 33 (3), 378-398, (2011)

11. Scheerens, J., Bosker, R.: The foundations of educational effectiveness. Pergamon, New York (1997)

12. Earl, L., Katz, S.: Leading schools in a data-rich world. Harnessing data for school improvement. Corwin Press, Thousand Oaks, CA (2006)

13. Perie, M., Marion, S., Gong, B.: Moving toward a comprehensive assessment system: A framework for considering interim assessments. Educational Measurement: Issues and Practice, 28, 5-13, (2009)

14. Feldman, J., Tung, R.: Whole school reform: How schools use the data-based inquiry and decision making process. Paper presented at the American Educational Research Association conference, Seattle, WA (April 10-14, 2001)

15. Massell, D.: The Theory and Practice of Using Data to Build Capacity: State and Local Strategies and Their Effects. In Fuhrman, S.H. (ed.) From the Capitol to the Classroom: Standards-Based Reform in the States. University of Chicago Press, Chicago, IL (2001)

16. Hulpia, H.: Distributed leadership and organizational outcomes in secondary schools. Proefschrift, Universiteit Gent, Gent, Belgium (2009)

17. Harris, A.: Distributed Leadership: What We Know. In Harris, A. (ed.) Studies in Educational Leadership. Institute of Education, London (2009)

18. Earl, L.: From accounting to accountability: Harnessing data for school improvement. Paper presented at the ACER research conference, Melbourne, Australia (2005)
19. Sutherland, S.: Creating a culture of data use for continuous improvement: a case study of an Edison project school. The American Journal of Evaluation, 25 (3), 277-293, (2004)

20. Conrad, W.H., Eller, B.: District data-informed decision making. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL (2003)

21. Diamond, J.B., Spillane, J.P.: High-stakes accountability in urban elementary schools: challenging or reproducing inequality. Teachers College Record, 106 (6), 1145-1176, (2004)

22. Slavin, R.: Sands, bricks, and seeds: school change strategies and readiness for reform. In Hargreaves, A., Lieberman, A., Fullan, M., Hopkins, D. (eds.) International Handbook of Educational Change, Vol. 5, pp. 1299-1313. Kluwer Academic Publishers, Dordrecht/Boston/London (1998)

23. Lubbe, M. van der: Pupil Monitoring System (PMS) for Primary Education. Cito, Arnhem, The Netherlands (2008)

24. Chen, E., Heritage, M., Lee, J.: Identifying and monitoring students' learning needs with technology. Journal of Education for Students Placed at Risk, 10 (3), 309-332, (2005)

25. Coe, R.: Evidence on the role and impact of performance feedback in schools. In Visscher, A.J., Coe, R. (eds.) School improvement through performance feedback, pp. 3-26. Swets & Zeitlinger, Lisse, The Netherlands (2002)

26. Kluger, A.N., DeNisi, A.: The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119 (2), 254, (1996)

27. Black, P., William, D.: Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5 (1), 7-74, (1998)

28. Hattie, J.A.C., Timperley, H.: The power of feedback. Review of Educational Research, 77 (1), 81-112, (2007)

29. Hattie, J.A.C.: Visible learning: a synthesis of over 800 meta-analyses relating to achievement. Routledge, Abingdon (2009)

30. Locke, E.A., Latham, G.: Building a Practically Useful Theory of Goal Setting and Task Motivation. The American Psychologist, 57 (9), 705-717, (2002)

31. Locke, E.A., Latham, G.P.: A theory of goal setting and task performance. Prentice Hall, Englewood Cliffs, NJ (1990)

32. Heritage, M., Kim, J., Vendlinski, T., Herman, J.: From Evidence to Action: A Seamless Process in Formative Assessment? Educational Measurement: Issues and Practice, 28, 24-31, (2009)

33. Wayman, J.C.: Involving Teachers in Data-Driven Decision Making: Using Computer Data Systems to Support Teacher Inquiry and Reflection. Journal of Education for Students Placed at Risk, 10 (3), 295-308, (2005)

34. Verhaeghe, G.: School performance feedback systems: Design and implementation issues. PhD thesis, University of Gent, Gent, Belgium (2011)

35. Fullan, M.: The new meaning of educational change. Teachers College Press, New York, NY (1991)

36. Borman, G.D., Hewes, G.M., Overman, L.T., Brown, S.: Comprehensive School Reform and Student Achievement. A Meta-Analysis. John Hopkins University, Baltimore, MD (2002)

37. Bosker, R.J., Branderhorst, E.M., Visscher, A.J.: Improving the Utilization of Management Information Systems in Secondary Schools. Schools Effectiveness and School Improvement, 18 (4), 451-467, (2007)

38. Branderhorst, E.M.: Wat is er mis met MIS? Ontwerp en evaluatie van een training in het gebruik van managementinformatie binnen het voorgezet onderwijs. Proefschrift, Universiteit Twente, Enschede, The Netherlands (2005)