A NLP-based Analysis of Reflective Writings by Italian Teachers
Authors: Giulia Chiriatti, Valentina Della Gala, Maria Chiara Pettenati, Maria Teresa Sagri, Felice Dell'Orletta, Simonetta Montemagni, Giulia Venturi (INDIRE, CNR-ILC, 2018)
Topic: Learner Corpora and Language Acquisition

Abstract
This paper reports ﬁrst results of a wider study devoted to exploit the potentialities of a NLP-based approach to the analysis of a corpus of reﬂective writings on teaching activities. We investigate how a wide set of linguistic features allows reconstructing the linguistic proﬁle of the texts written by the Italian teachers and predicting whether are reﬂective.

Introduction
Since 2014, the “National Institute for Documentation, Innovation and Educational Research” (INDIRE) manages for the Ministry of Education (MIUR) the induction program of the Italian Newly Qualiﬁed Teachers (NQTs), i.e. the induction phase of teachers professional development that aims to support teachers in their transition from their initial teacher education into working life in schools. Experimented for the ﬁrst time in 2014, it became effective starting in 2015 with the DM 850/2015.1 The program involves all new hiring teachers from primary to secondary school for a total of 130,000 NQTs committed in the last 3 years. The underlying theoretical framework developed by INDIRE, MIUR and University of 1http://neoassunti.indire.it/2018/ﬁles/DM 850 27 10 2015.pdf Macerata is based on the alternation of laboratorial and traditional classroom activities with documentation and reﬂection activities. The purpose is “to inﬂuence practices through a process that alternates between moments of immersion and distancing, which are actualised in When I teach and When I reconsider my teaching to think of what happened” (Magnoler et al., 2016). An on-line environment developed and managed by INDIRE2 was set up to support teachers to reﬂect about and document their educational and professional activities (see Figure 1) during the induction program. All evidences of the instructional tasks (surveys, writing tasks, lesson plans, instructional materials, etc.) are collected in the e-portfolio and printed by the teachers for the ﬁnal exam. An yearly monitoring of teachers activities is carried on by INDIRE to assess the effectiveness of the whole induction program, as well as of the single instructional tasks. It is aimed to modify, whenever needed, the program in order to improve stakeholders’ scaffolding to the newly qualiﬁed teachers and lastly teachers’ professional development. Figure 1: The on-line environment collecting the e-portfolio of the newly qualiﬁed teachers. In this paper, we report ﬁrst results of an ongoing study devoted to investigate the potentialities offered by Natural Language Processing methods and tools for the analysis of the NQTs e-portfolio. We consider in particular the documents written by the 26,526 teachers hired in the 2016/17 school year. Many protocols (or models) have been proposed to assess reﬂection in teachers writing, e.g. (Sparks-Langer et al., 1990; Hatton and Smith, 1995; Kember et al., 2008; Larrivee, 2008; Harland and Wondra, 2011). These models rely on features that suggest either different levels of reﬂection (means focused on the depth of reﬂection) or content of reﬂection (focused on the breadth of reﬂection), and usually they have found to mix features of both classes (depth and breadth) (Ullmann, 2015). We rather focus here on the analysis of the form to study which are the main linguistic phenomena, distinguishing reﬂective from non reﬂective writings. Speciﬁcally, we devised a methodology devoted to investigate whether and to which extent a wide set of linguistic features automatically extracted from texts can be exploited to characterize NQTs’ reﬂective writings. Our contribution: i) we collect a corpus of reﬂective writings manually annotated by experts in the learning science domain and classiﬁed with respect to different types of reﬂectivity; ii) we detect a wide set of linguistically phenomena, characterizing the collected writings; iii) we report the ﬁrst results of an automatic classiﬁcation experiment to assess which features contribute more in the automatic prediction of reﬂexivity.

Conclusion
We reported ﬁrst results of a on-going study devoted to reconstruct the linguistic proﬁle of a corpus of reﬂective writings by Italian newly recruited teachers that we collected for the speciﬁc purpose of this paper. We are currently enlarging the corpus with new manually annotated data to improve the accuracy of the automatic classiﬁcation of different types of reﬂectivity.