# Datafication in Education: A Multi-Level Challenge for IT in Educational Management

Andreas Breiter

► To cite this version:

Andreas Breiter. Datafication in Education: A Multi-Level Challenge for IT in Educational Management. 1st International Conference on Stakeholders and Information Technology in Education (SAITE), Jul 2016, Guimarães, Portugal. pp.95-103, 10.1007/978-3-319-54687-2_9. hal-01625383

HAL Id: hal-01625383

https://inria.hal.science/hal-01625383

Submitted on 27 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Datafication in Education: a Multi-Level Challenge for IT in Educational Management

Andreas Breiter

University of Bremen, Institute for Information Management, Germany
abreiter@uni-bremen.de

**Abstract.** While data-driven decision-making has become a new paradigm for school development and accountability, research on the underlying ICT infrastructures and the ICT management processes have been less prominent. With the trend of datafication, educational management gains new options but also requires adequate controlling mechanisms to take care of the data and to account for privacy and security. The information management cycle can be used to define all relevant aspects of the management process. This is adopted to the specific situation of educational institutions and furthermore extended to account for the different levels of educational governance. This leads to a new concept of educational technology governance as a necessary frame for supporting datafication processes.

**Keywords.** Educational governance · ICT management · datafication · information technology governance

## 1 Datafication: a Social Trend

In the past decade, an increasing trend to measure social life in numbers can be observed as part of the so-called ‘audit society’ [1]. With this turn, almost all aspects of social life have become measured and quantified. Originating from the business sector, large-scale data have attracted the attention of many education scientists as a basis for decision-making aiming at education improvement. Decision support systems in the field of education have started to emerge in which student data are used in a feedback loop to improve student performance. Datafication raises expectations concerning increasing transparency for the public, accountability and civic participation but also associated fears with respect to surveillance and privacy [2]. This is also addressed in recent critical studies on ‘big data’ [3–6].

Educational decision-making processes are increasingly linked to the collection, processing and visualization of data [7–9]. These data practices relate to schools’ performances and student achievements, which are compared on a national and international scale. They affect salaries of teachers and school managers, which are adjusted according to test scores as well as decision-making of parents for school choice or control of teachers. The general idea is twofold: data can be used for school improvement [8–10], but also for accountability [11–14]. This is part of a larger output-orientation of education measured by standardized achievement tests (high
stakes), which started in the U.S., Australia and the UK and has a growing importance all over the world [15]. This output-orientation goes hand in hand with increased control (governance) of school systems [11], also based on neo-liberal reforms for ‘new public management’ [16]. Selwyn [17] points out that data are being generated and processed in increased volume, velocity and variety, resulting in what is known as large-scale data or big data. This trend has increasingly led to discussions on data mining and analytics in business, science and government. Williamson [18] speaks of ‘governing software’ and the ‘emergence of “digital governance” in public education’ (p.83). He argues that educational decision-making is increasingly being delegated to database-driven analytics software and states that ‘software has now become a significant social actor that can govern and shape people’s lives’ [18].

Within the educational context, more data and more heterogeneous data are being generated—deliberately—for monitoring, surveillance or evaluation purposes but also—automatically—through routine operations of digital devices and systems [17]. They range from computer-based tests to learning analytics on large-scale data in complex information systems. They allow the ‘recording, storage, manipulation and distribution of data in digital form’ [17, emphasis in original]. Digital data are distinct from pre-digital forms as they may be exhaustive in scope, highly detailed and can be combined in a flexible manner. Datafication has to be understood in different dimensions. As more and more media are based on computer devices and software, we generate ‘digital traces’ ([19]) by using them. This can be aggregated and processed automatically. A rather new trend is learning analytics and educational data mining. While some embrace the new opportunities for improving teaching and learning [20–22], others refer to the challenges of an education system which is defined algorithmically [23–25].

## 2 Datafication and Educational Governance

Eynon [26] points out that the use of big data observed in the educational sector is not a new phenomenon, as it has been widely applied in the commercial sectors, where data sets are combined to understand markets, as well as in the field of natural sciences. In the business sector, customer satisfaction is usually utilized to measure how well a company performs. The more satisfied the customers are with the service provided by a company, the more profits are likely to be gained. Likewise, in the education sector, students’ performance is usually used to measure the quality of education, i.e., the better the students perform, the better the quality of education is usually perceived to be. Education itself brings profit to a country from the perspective of improvement of human capital. In this context, educational government bodies can be seen as the main organizations that provide services to society in the form of education. Especially in education, the governance structure of the education system has a strong explanatory power to understand changes from the classroom level (micro) to the organizational level (meso) to the policy level (macro). ICT has to be embedded in each of these levels and has its impact across the levels. We follow the framework model of educational governance [27, 28] and contextualize it to the role of ICT in schooling [29]. The three-tier model of educational technology
governance allows a broader view on the intertwined processes on the macro, meso and micro level.

On the macro level, in most countries, public pressure on changing education policy enforced by international non-governmental organizations (like OECD or IEA) can been observed since the publication of PISA results in the 1990s. Martens et al. [15] explained different reactions of nation states to these pressures – from adoption of achievement tests in the national education policy, to ignoring it. They even identified a convergence of educational systems to follow the logic of the test systems [30].

Other studies in education policy highlight numbers and evidence-based policy as the mode of educational governance in Australia [31]. They point out that from a sociological point of view, it is necessary to be aware of the social construction behind the numbers, which come to constitute education policy, as an understanding of the socially constructed nature of data leads to the identification of its weaknesses and strategies to improve. On the meso level of the district (or region), comprehensive information systems have been built in order to cope with increasing data [32–34]. This level is rather under-researched as intermediaries are less in focus but also the differences of educational governance across countries are particularly strong. Depending on the level of autonomy of the schools, the centralization or decentralization of the national school system as well as the size of the school system to be attractive for commercial software companies varies. Hence, the micro level of the school according to data practices and school information systems has been studied extensively. This ranges from technology adoption of school information systems [35, 36] by administrators and teachers to different uses of information systems for performance feedback [37–39] to learning analytics [20, 25]. In this paper, we argue that all information systems and the underlying infrastructures need to be managed, i.e. planned, organized and controlled.

# 3 Educational Technology Governance for a Datafied Education System

## 3.1 Educational Information Management

Data are a carrier of information that can be turned into knowledge when they are interpreted in a particular context by humans. This knowledge may lead to action by individuals or organizations when it is perceived as referring to a certain context [10]. Data, information and knowledge can be seen as a continuum where information management is rooted in an efficient and reliable infrastructure for data processing [40]. In the enterprise world, the purpose is to make an enterprise as intelligent as possible by valuing its knowledge assets for success. Choo [40] defines an intelligent organization as one, which is able to learn from its internal and external environments, by detecting and correcting errors between outcome and expectation. Internal environment in this context refers to any available data or information that originates from within the organization. Applying this to the educational sector,
student performance can be improved by learning from data or information that resides in the information system the educational institution operates.

While in general information systems can be understood as technologies and associated people and processes, Visscher [41] defines an information system in the context of education as any computer or set of computers with data bases that enable users to record, process, retrieve and distribute data. Breiter et al. [42] make a clear distinction of the following types of school information systems: (1) Assessment information system, (2) Learning management system, (3) Administrative school information system, and (4) Electronic school directory. An assessment information system is aimed at presenting the evaluation results of students and schools. Drawing on self-evaluation tests, classroom-based assessments or standardized tests, this information system is accessed by students, teachers, parents, and decision-makers. Learning management systems are about teaching and learning and are addressed at teachers and students, such as e-learning or learning platforms. Administrative school information systems are used for administrative or management purposes of school staff. Electronic school directories cover school master data, such as school profiles or addresses, which are usually made available to the public.

Information systems are means to the end for better decision-making, based on an ICT infrastructure (networks, databases etc.) which itself needs to be managed (fig. 1). Anagnostopoulos et al. [12] define the underlying information infrastructure as an assemblage of people, technology and policy. This infrastructure is described as being similar to the physical infrastructure of cities or states, which includes canals, traffic and street intersections. For an efficient transportation across countries or regions, this infrastructure needs people to design, build and maintain it continuously, including its rules, norms, and underlying assumptions about its use. The term infrastructure itself is defined as something big, layered, and complex, where adjustment involves aspects dependent on the context applied [43, 44]. Information infrastructures can be regarded as institutions or formative contexts on the basis that they ‘constitute the background condition for action, enforcing constraints, giving direction and meaning, and setting the range of opportunities for undertaking action’ [45]. The implication is that as they are ‘infused with value’, they become more taken for granted and less expendable. When it comes to decision support systems in the education area, Anagnostopoulos et al. [12] define the underlying information infrastructure as an assemblage of people, technology and policy, using the famous term of Latour [46]. This infrastructure is described as being similar to the physical infrastructure of cities or states, which includes canals, traffic and street intersections. For an efficient transportation across countries or regions, this infrastructure needs people to design, build and maintain it continuously, including its rules, norms, and underlying assumptions about its use. The information infrastructure itself is pointed out to have economic, social, political and cultural impacts. It is more than just technology, as it is (re)shaped through the interaction of individual, social, political and technological forces.

As in business, we can find additional cross-sectional educational management processes on the leadership level such as strategic development, human resources and staff development, organizational issues and controlling. On the top level, the availability of data to make informed decisions as well as the demand for this data has to be managed. Further research suggests that this is not a static concept but has to be regarded as an ongoing process across the levels. As we know from generic
technology acceptance models [47, 48], as well as from research on school information systems [41], perceived usefulness and ease of use are predictors for integrating technology in everyday activities.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Information management in education (adaptation from [49])</figcaption></figure>

According to Choo [40], the information management cycle consists of a continuous loop with six distinct steps. It starts with (1) the identification of information needs, which basically consists of answering the questions of what kind of data are needed for which purposes. This is then followed by (2) information acquisition (data collection), (3) information organization and storage (data storage), (4) development of information products and services and (5) data distribution, which is executed by data organizers with the help of ICTs, and then (6) information use.

From an educational management perspective, input data in this system are not limited to student achievement data only, but the system can also be fed with other kinds of related data. Student achievement assessments have been widely accepted as the standard to evaluate student performance for class, school or international levels of comparison. Additionally, budget data, process data, students' e-portfolios or computer-generated data like logfiles [24] or from learning analytics ([20, 25]) can be added to build a heterogeneous data set with different formats, time span, from structured to non-structured and in different aggregation levels. These are typically all characteristics of “big data” ([50]).

## 3.2 Educational Technology Governance

The process character of the management cycle and the relevance of data for decision making on the different levels of educational governance illustrates the necessity for an overall plan and control mechanism. As this is already in place following the
Deming-cycle of plan, do, check and act (PDCA) in IT Governance, the next step is to account for the special aspects of IT in educational management. Usually, the term information technology governance is used in two different directions: First, as a description of how decisions on ICT are made in complex organizations in order to align IT and business [51]. Weill and Ross [52] define information technology governance as follows: ‘Specifying the decision rights and accountability framework to encourage desirable behavior in the use of IT” (p.8). They make a distinction between different forms of organizational structures: from IT or business monopoly in which one defines the action of the other (i.e. IT drives business or vice versa) to duopolies and to anarchy. Empirical research based on quantitative studies research shows a positive and significant linkage between information technology governance mechanisms and strategic alignment and, further, between strategic alignment and organizational performance [53]. In educational governance research, there is no correspondent research, yet.

Second, IT governance is used as a definition of meta-processes for ‘good governance’ to support strategic business development. The currently most accepted model is COBIT in its version 5 [54]. In COBIT 5, information technology governance is ‘... the system by which the current and future use of ICT is directed and controlled’ (p.3). The standard identifies three core elements: (1) Evaluate: checking for current and future use of IT; (2) Direct: defining directives to prepare and implement strategies, plans and guidelines to secure IT business alignment; (3) Monitor: controlling the implementation and the capabilities. COBIT encompasses altogether 37 sub-processes, which are defined with activities, metrics and input-output relations. Furthermore, a maturity level can be assigned to each process in order to monitor quality for continuous improvement.

Transferred to educational technologies, this would mean to implement governance processes on both the strategic and the operational level, focusing on teaching and learning as well as on educational management.

## 4 Conclusions

As we have highlighted, the rise of data for decision-making on all levels of educational governance has not only implications for preventing unintended consequences but also for processes of ICT management. The information management cycle needs to be reflected on all three levels of educational governance. It is part of educational policy making, requiring adequate data provision, which allow national and international comparisons. This demand is built into information systems, which are based on ICT infrastructures (e.g. national indicators, ranking systems, databases by state and non-governmental organizations, state-wide information systems). On the meso level of the school district, district-wide information systems are in place, serving for the needs of administrators and policy makers. On the micro level of the school, building management needs data, implemented in school information systems and based on ICT infrastructure. Additionally, there are cross-sectional processes on each level and, as a distinct characteristic of the school system, also across levels. Not only the data provision is
bottom-up from the school to the policy level (e.g. as reports and achievement test
results) but also from school to district and from district to school, and vice versa.
Furthermore, specific requirements to process sensible educational data such as
grades, performance levels or attendance need to be embedded into an overall policy
of privacy and security.

As we concluded, the strategic processes need to be implemented, managed and
monitored. In order to cope with the challenge of datafication on a large scale,
strategic processes of governance for educational technologies are necessary. This is
relevant for both the administrative data practices and teaching and learning.
Education systems and policy makers could save time and energy by adopting
existing models from business. This would lead education and educational technology
to the next level of professional management.

References

1. Power, M.: The Audit Society. Rituals of Verification. Oxford University Press, Oxford (1999).

2. Hope, A.: Governmentality and the “selling” of school surveillance devices. Sociol. Rev. 840–857 (2015).

3. Kitchin, R.: The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. SAGE, Thousand Oaks, CA (2014).

4. Borgman, C.L.: Big data, little data, no data: scholarship in the networked world. The MIT Press, Cambridge, Massachusetts (2015).

5. Gitelman, L. ed: "Raw Data" Is an Oxymoron. The MIT Press, Cambridge, Massachusetts ; London, England (2013).

6. Boyd, D., Crawford, K.: Critical questions for big data. Inf. Commun. Soc. 15, 662–679 (2012).

7. Piety, P.J.: Assessing the educational data movement. Teachers College Press, New York, NY (2013).

8. Mandinach, E.B., Honey, M.: Data-Driven School Improvement. Linking Data and Learning. Teacher College. Columbia University, New York (2008).

9. Schildkamp, K., Lai, M.K., Earl, L.: Data-based Decision Making in Education: Challenges and Opportunities. Springer, Dordrecht (2013).

10. Breiter, A., Light, D.: Data for school improvement: Factors for designing effective information systems to support decision-making in schools. J. Educ. Technol. Soc. 9, 206–217 (2006).

11. Behn, R.D.: Rethinking Accountability in Education. How Should Who Hold Whom Accountable for What? Int. Public Manag. J. 6, 43–73 (2003).

12. Anagnostopoulos, D., Rutledge, S.A., Jacobsen, R.: The infrastructure of accountability: Data use and the transformation of American education. Harvard Education Press, Cambridge, MA (2013).

13. Sellar, S.: Data infrastructure: a review of expanding accountability systems and large-scale assessments in education. Discourse Stud. Cult. Polit. Educ. (2014).

14. Wong, K.K.: Politics and Governance: Evolving Systems of School Accountability. Educ. Policy. 27, 410–421 (2013).

15. Martens, K., Nagel, A.-K., Windzio, M., Weymann, A.: Transformation of Education Policy. Palgrave, Basingstoke (2010).

16. Pollitt, C., Bouckaert, G.: Public Management Reform: A Comparative Analysis. Oxford University Press, Oxford (2000).
17. Selwyn, N.: Data entry: towards the critical study of digital data and education. Learn. Media Technol. 40, 64–82 (2015).

18. Williamson, B.: Governing software: networks, databases and algorithmic power in the digital governance of public education. Learn. Media Technol. 40, 83–105 (2015).

19. Karanasios, S., Thakker, D., Lau, L., Allen, D., Dimitrova, V., Norman, A.: Making sense of digital traces: An activity theory driven ontological approach. J. Am. Soc. Inf. Sci. Technol. 64, 2452–2467 (2013).

20. Berland, M., Baker, R., Blikstein, P.: Educational Data Mining and Learning Analytics: Applications to Constructionist Research. Technol. Knowl. Learn. 19, 205–220 (2014).

21. Papamitsiou, Z., Economides, A.A.: Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence. Educ. Technol. Soc. 17, 49–64 (2014).

22. Greller, W., Drachsler, H.: Translating Learning into Numbers: A Generic Framework for Learning Analytics. Educ. Technol. Soc. 15, 42–57 (2012).

23. Reimann, P., Markauskaite, L., Bannert, M.: e-Research and learning theory: What do sequence and process mining methods contribute? Br. J. Educ. Technol. 45, 528–540 (2014).

24. Schulz, A.H., Breiter, A.: Monitoring User Patterns in School Information Systems Using Logfile Analysis. In: Passey, D., Breiter, A., and Visscher, A.J. (eds.) Next Generation of IT in Educational Management. pp. 94–103. Springer, Berlin (2013).

25. Ferguson, R.: Learning analytics: drivers, developments and challenges. Int. J. Technol. Enhanc. Learn. 4, 304–317 (2012).

26. Eynon, R.: The rise of Big Data: what does it mean for education, technology, and media research? Learn. Media Technol. 38, 237–240 (2013).

27. Sergiovanni, T.J., Burlingame, M., Coombs, F.S., Thurston, P.: Educational governance and administration. Prentice-Hall, Englewood Cliffs, NJ (1987).

28. Altrichter, H.: Theory and Evidence on Governance: conceptual and empirical strategies of research on governance in education. Eur. Educ. Res. J. 9, 147–158 (2010).

29. Kozma, R.B.: Comparative analyses of policies for ICT in education. In: Voogt, J. and Knezek, G. (eds.) International handbook of information technology in primary and secondary education. pp. 1083–1096. Springer, Berlin (2008).

30. Bieber, T., Martens, K., Niemann, D., Teltemann, J.: Towards a Global Model in Education? Interdisciplinary Perspectives on International Student Assessments and their Impact on Policies and Institutions. In: Hamilton, M., Maddox, B., and Addey, C. (eds.) Literacy as Numbers. Researching the Politics and Practices of International Literacy Assessment Regimes. pp. 165–186. Cambridge University Press, Cambridge (2015).

31. Lingard, B., Creagh, S., Vass, G.: Education policy as numbers: data categories and two Australian cases of misrecognition. J. Educ. Policy. 27, 315–333 (2012).

32. Wayman, J.C., Jimerson, J.B., Cho, V.: Organizational considerations in establishing the Data-Informed District. Sch. Eff. Sch. Improv. 23, 159–178 (2012).

33. Cho, V., Wayman, J.C.: Districts' Efforts for Data Use and Computer Data Systems: The Role of Sensemaking in System Use and Implementation. Teach. Coll. Rec. 116, (2014).

34. Coburn, C.E., Talbert, J.E.: Conceptions of evidence-based practice in school districts: Mapping the terrain. Am. J. Educ. 112, 469–495 (2006).

35. Passey, D., Breiter, A., Visscher, A.J. eds: Next Generation of IT in Educational Management. Springer, Berlin (2013).

36. Tatnall, A., Kereteletswe, Omponye Coach, Visscher, A.J. eds: Information Technology and Managing Quality Education. Springer, Berlin (2011).

37. Breiter, A., Stauke, E.: Assessment Information Systems for Decision Support in Schools. In: Tatnall, A., Okamoto, T., and Visscher, A.J. (eds.) Knowledge Management for Educational Innovation. pp. 9–18. Springer, Berlin (2007).
38. Verhaeghe, G., Vanhoof, J., Valcke, M., Van Petegem, P.: Using School Performance Feedback: Perceptions of Primary School Principals. Sch. Eff. Sch. Improv. 21, 167–188 (2010).

39. Visscher, A.J., Coe, R.: School Improvement Through Performance Feedback. Swets & Zeitlinger, Lisse (2002).

40. Choo, C.W.: Information Management for an Intelligent Organization: The Art of Environmental Scanning. 3rd Edition. Information Today, Medford, NJ (2002).

41. Visscher, A.J.: Information technology in educational management as an emerging discipline. Int. J. Educ. Res. 25, 291–296 (1996).

42. Breiter, A., Stauke, E., Büsching, N., Lange, A.: Educational Management Information Systems - Case Studies from 8 Countries. Shaker, Aachen (2006).

43. Star, S.L.: The Ethnography of Infrastructure. Am. Behav. Sci. 43, 377–391 (1999).

44. Star, S.L., Ruhleder, K.: Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces. Inf. Syst. Res. 7, 111–134 (1996).

45. Ciborra, C.U., Hanseth, O.: From tool to Gestell. Inf. Technol. People. 11, 305–327 (1998).

46. Latour, B.: Reassembling the social: An introduction to Actor-Network-Theory. Oxford University Press, Oxford (2007).

47. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340 (1989).

48. Venkatesh, V., Morris, M., Davis, G.B.: User acceptance of information technology: Toward a unified view. MIS Q. 27, 425–478 (2003).

49. Krcmar, H.: Information Management. Springer, Berlin (2004).

50. Mayer-Schonberger, V., Cukier, K.: Big Data: A Revolution that will Change How We Live, Work and Think. John Murray, London (2013).

51. Wulf, V., Stiemerling, O., Pfeifer, A.: Tailoring groupware for different scopes of validity. Behav. Inf. Technol. 18, 199–212 (1999).

52. Weill, P., Ross, J.W.: IT Governance. How Top Performers Manage IT Decision Rights for Superior Results. Harvard Business School Press, Boston, MA (2004).

53. Wu, S.P.-J., Straub, D.W., Liang, T.-P.: How Information Technology Governance Mechanisms and Strategic Alignment Influence Organizational Performance: Insights from a Matched Survey of Business and IT Managers. MIS Q. 39, 497–518 (2015).

54. ISACA: COBIT 5 - A Business Framework for the Governance and Management of Enterprise IT. Information Systems Audit and Control Association (2012).