# Social Learning Analytics to Study Teachers’ Large-Scale Professional Networks

Riina Vuorikari, Santi Scimeca

<figure><img src="image_1.png" /></figure>

## To cite this version:

Riina Vuorikari, Santi Scimeca. Social Learning Analytics to Study Teachers' Large-Scale Professional Networks. 1st Open and Social Technologies for Networked Learning (OST), Jul 2012, Tallinn, Estonia. pp.25-34, 10.1007/978-3-642-37285-8_3. hal-01349398

HAL Id: hal-01349398

https://inria.hal.science/hal-01349398

Submitted on 27 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Social Learning Analytics to study Teachers' Large-scale Professional Networks

Riina Vuorikari, Santi Scimeca

European Schoolnet, Brussels, Belgium

{Riina.Vuorikari, Santi.Scimeca}@eun.org

**Abstract.** A growing interest in research focuses on teachers' large-scale socio-technical networks. Social learning approaches such as social constructivist theory is well established, however, the current challenges lie in creating reliable methods to gather evidence of how and under which conditions social learning takes place in such socio-technical networks and how does it support teachers' lifelong learning goals. The field of Learning Analytics (LA) addresses the issue of individual learners, whereas Social Learning Analytics (SLA) addresses that of groups' processes in knowledge construction.

The eTwinning action is used as a case study for applying the concepts of Social Learning Analytics. Our interest is on teachers' co-operation behaviour and patterns within a socio-technical network and how that can support teachers' continuous professional development. The eTwinning platform currently hosts more than 160000 European teachers. We first introduce the underlying pedagogical and lifelong learning related assumptions regarding teachers' co-operation. To better understand the type of activities that teachers undertake in eTwinning, they are classified according to the OECD's indices for teachers' co-operation. This creates the core of the eTwinning Analytics framework, which operationalises activities and allows them to be better measured and monitored. A snapshot of data from the platform is used as a proof of concept.

**Keywords.** Socio-technical networks, teachers' professional development, social learning analytics, eTwinning, Tellnet

## 1. Introduction

Learning networks are technology-supported communities through which participants share knowledge with one another and jointly develop new knowledge [1]. They potentially enrich the experiences and form a viable setting for professional development and lifelong learning. They can also be called socio-technical networks as they enable knowledge and expertise to be shared indirectly as well, for example, when a message is read by someone other than the person to whom it was originally addressed [2]. Examples of such networks for teachers' professional development are communities who want to improve their work through exchange of experiences and collaboration, e.g. Tapped-in [3], Teachernet [4], eTwinning [5].

The underlying assumption is that teachers' networks, like other learning networks, can offer participants informal ways to support competence building and personal and
*professional development*. Such competences include 21st century key competences, e.g. teachers gaining new digital competences, learning new methods to use ICT to support teaching and learning, learning to communicate in foreign languages, as well as gaining in other areas of personal development such as intercultural dialogue and social competence.

The second important assumption is that teachers’ activities in collaboration networks can also have a relation to their teaching and instructional practices in the classroom. The OECD’s TALIS study indicates to the direction that teachers’ professional development may be effective in instructing and inspiring teachers to use modern and multifaceted practices, especially *student-oriented practices* (e.g. students work in small groups to come up with a joint solution to a problem or task; different work is given to the students that have difficulties learning and/or to those who can advance faster) and *enhanced activities* (e.g. students work on projects that require at least one week to complete; students make a product that will be used by someone else) [6]. Especially activities that take place at regular intervals and involve teachers in a rather stable social and collaborative context (i.e. networks or mentoring) have a significantly stronger association with teaching practices than regular workshops and courses [6, p. 101]. Thirdly, according to the same study, *co-operation* among teachers can also create opportunities for social and emotional support, exchange of ideas and practical advice that can thus enhance professionalism, feelings of self-efficacy and prevent stress and “burnout”.

With the above-listed benefits, teachers’ networks also offer challenges. How can evidence be gathered of how and under which conditions teachers gain new competences in such socio-technical network? Do certain activities support better teachers’ lifelong learning goals and professional development? How do the activities undertaken in such networks eventually translate into actions in classroom (e.g. student-oriented and enhanced activities)? The first step towards this challenge is to create and test a framework that allows defining, operationalising, measuring and monitoring activities teachers undertake in socio-technical networks.

In this paper, we present a case study on the teachers’ professional network in eTwinning. In section 2, a general description of eTwinning is offered focusing on the motivations behind this research. Section 3 offers a concept of a framework to define and operationalise teachers’ activities in networks so that they eventually can be measured and better monitored. In section 4, a proof of concept is offered elaborating on a small number of research questions. The following section reflects on results and methods, and offers direction for future work.

## 2. eTwinning

eTwinning is defined as “The community for schools in Europe” [5]. It promotes school collaboration through the use of Information and Communication Technologies (ICT). In August 2012, 32 countries in Europe participate in eTwinning and the platform hosts more than 170000 registered members. Since 2005, eTwinning has been one of the most successful actions of the school education programme (Comenius) under the European Union’s Lifelong Learning programme.
The main actors of the platform are eTwinners, i.e. teachers from 32 different European countries that currently participate in the initiative. The platform is also used by the National Support Services (NSS) that function in each participating country and support local users in eTwinning. The platform is managed by the Central Support Service (CSS), who is responsible for the implementation and considered as a service provider. The Central Support Service, run by European Schoolnet [7], is in the position to collect and process data according to the data protection rules defined in the eTwinning privacy statement<fnref n="1" />. It also observes all communications and interactions between eTwinners (data processor). The platform is run under a service contract for the European Commission, whereas the ultimate data controller is the Education, Audiovisual & Culture Executive Agency.

eTwinning offers teachers three main streams of activities:

1. Teachers can find schools from other countries to run school collaboration projects using Information and Communication Technologies (ICT) provided by the platform<fnref n="2" />.

2. Various formal and informal professional development (PD) activities are offered. These include online Learning Events<fnref n="3" />, a distance course for teachers, and more informal PD activities such as online interest Groups<fnref n="4" /> and Teachers Rooms on topics of interest.

3. Additionally, the participating teachers have a set of social networking tools available, these include a profile page with personal and professional information<fnref n="5" />, a display of connections to other members (i.e. contacts) and posting on a personal journal (e.g. status updates), but also on contacts' journals.

The development of eTwinning and its community building aspects are described in [8] and [9]. We call these micro-level studies focusing on the actions of individual teachers and students. These studies focus on active members and interesting cases that may not represent the larger community. As argued by [2] "such accounts serve to show the field what is attainable under particular conditions, interventions, or context, but we are still unable to rigorously measure their value, much less predict, guide, or replicate results reliably or at scale".

On the other hand, there are studies on eTwinning on the macro-scale: the growth of the network and its spread within each country is presented in [10] using measurements such as eTwinning Reach. Synergies between eTwinning and national teachers' professional development schemes are elaborated in [11]. These macro-scale studies also include the use of Social Network Analysis (SNA) and information visualisation to study eTwinning (e.g. [12], [13], [14]). In addition to the two approaches explained above, the CSS also has a need to understand eTwinning at the meso-level. This middle-layer allows zooming on social structures that emerge from activities such as interactions among members and their interactions with content and

<footnote n="1">¹ http://www.etwinning.net/en/pub/misc/privacy_statement.htm</footnote>

<footnote n="2">² http://www.etwinning.net/en/pub/tools/twinspace_tools.htm</footnote>

<footnote n="3">³ http://www.etwinning.net/en/pub/professional_development/learning_events.htm</footnote>

<footnote n="4">⁴ http://www.etwinning.net/en/pub/professional_development/etwinning_groups.htm</footnote>

<footnote n="5">⁵ http://www.etwinning.net/en/pub/tools/desktop_tools.htm</footnote>
tools. But instead of focusing on individuals, the meso-level is interested in the emerging behavioural patterns as a unit of study.

### 3. The eTwinning Analytics framework

eTwinning Analytics focus on the measurement, collection, analysis and reporting of data about eTwinners and their contexts, for purposes of understanding and optimising their co-operation and the environment in which it occurs. The goal of eTwinning Analytics, therefore, is to offer eTwinning stakeholders better tools to monitor the action through emerging trends and patterns upon which they can better base their decisions.

<table><caption>Table 1. Mapping between TALIS activities to eTwinning activities.</caption><thead><tr><th>Category</th><th>TALIS: examples of activities</th><th>eTwinning: examples of activities</th></tr></thead><tbody><tr><td>Index of Exchange and co-ordination for teaching</td><td><ul><li>Discuss and decide on the selection of instructional media (e.g. textbooks, exercise books).</li><li>Exchange teaching materials with colleagues.</li><li>Attend team conferences for the age group taught.</li><li>Ensure common standards in evaluations for assessing student progress.</li><li>Engage in discussion about the learning development of specific students.</li></ul></td><td><ul><li>Participate in a Group to discuss and exchange on</li><li>a topic of interest</li><li>pedagogical practices</li><li>ICT tools and practices</li><li>teaching materials</li><li>Comments and ratings on Kits</li><li>Posts and comments on Project Cards</li><li>Applications for Quality Labels</li></ul></td></tr><tr><td>Index of Professional collaboration</td><td><ul><li>Teach jointly as a team in the same class.</li><li>Take part in professional learning activities (e.g. team supervision).</li><li>Observe other teachers' classes and provide feedback.</li><li>Engage in joint activities across different classes and age groups (e.g. projects).</li><li>Discuss and co-ordinate homework practice across subjects.</li></ul></td><td><ul><li>Take part in Learning Lab (A short intensive online event offered on a number of themes. They are led by an expert and include active work and discussion among teachers across Europe.)</li><li>Engage in a cross-country project using ICT. These projects can be either within or across disciplines or age groups.</li></ul></td></tr></tbody></table>

The definition follows that of Learning Analytics in general [15], more particular, eTwinning Analytics are part of Social Learning Analytics focusing on building knowledge together in cultural, social and technological settings [16].

As explained previously, teachers' co-operation carries many promises for lifelong learning opportunities. The aim of the eTwinning Analytics is therefore to
operationalise the *construct* of teachers' co-operation in eTwinning to allow it to be monitored and measured. Teachers' co-operation implies *teachers working together in groups or teams to improve educational processes and outcomes*, Table 1 shows TALIS examples of such activities across the two indices: "teachers' exchange and co-ordination activities" and "professional collaboration" [6]. Similar to this division, a mapping between TALIS' activities and their counterpart in eTwinning is presented.

Table 2 gives brief details of eTwinning Analytics framework version 1. Two additional categories have been added to the core of eTwinning Analytics Framework, namely "social networking activities" which includes the use of personal profile, social networking tools such as writing status updates, collection of contacts and receiving their status updates. A category for general statistic is created to allow monitoring trends in engagement with the activity.

<table><caption>Table. 2. Elements of the eTwinning Analytics framework (v1)</caption><thead><tr><th colspan="2">eTwinning Analytics Framework</th></tr></thead><tbody><tr><td>General</td><td><ul><li>Logging on to the Desktop</li><li>Sending messages</li></ul></td></tr><tr><td>Social networking activities</td><td><ul><li>Journal posts, comments, "I like it!"</li><li>Adding Contacts</li><li>Participation in Teachers' rooms</li></ul></td></tr><tr><td>Teachers' coordination and exchange activities</td><td><ul><li>Participation in Groups</li><li>Commenting: on Kits, other people's Project Diary, Project Card, Quality Label applications</li></ul></td></tr><tr><td>Teachers' professional collaboration</td><td><ul><li>Participation in Learning Events</li><li>Project collaboration and Twinspace</li><li>Writing Project Diary</li></ul></td></tr></tbody></table>

Various requirements have been gathered from eTwinning stakeholders for the framework; they wish to monitor the action by the country from which the eTwinning originates, the year of registration and whether the eTwinning has engaged in any collaborative cross-border school projects within eTwinning. Although school collaboration projects are not a mandatory part of the action, in terms of professional development gains they offer great advantages (i.e. construct "teachers' professional collaboration"). Additionally, understanding how various tools under the three main streams of activities are used is an asset (as explained in section 2: project collaboration, formal and informal PD, social networking).

Interest in these emerging trends and behavioural patterns can be turned into simple research questions (RQ) which we will answer in the following section. These questions are as following: RQ1: Is there evidence of eTwinners remaining engaged with the action over a long period of time (i.e. 6 years)? RQ2: There are eTwinners who run school collaboration projects and the ones who do not: do these two groups use the Social networking tools in the same way? RQ3: Over a long period of time (i.e. 6 years), are there any trends that emerge in teachers' co-operation activities?

To answer to our research questions, authentic data from the eTwinning platform is used. The *SteerCom*-Desktop tool, which offers a set of pre-determined queries on the
data within a certain period (e.g. from 2005 to 2011), was used. Additionally, some
custom queries were made where the data was extracted on Nov 2011. In the
following section we provide results to our research questions, discussion of which is
offered in the last section with future work.

4. Demonstrating the use of eTwinning Analytics

**RQ1: eTwinning retention rate.** Is there evidence of eTwinners remaining engaged with the action over a long period of time (i.e. since its start in 2005)?

"Retention rate" is a term used in Web Analytics to indicate to the percentage of users
who sign up for the service and come back within a period of time.

<figure><img src="image_3.png" /><figcaption>Fig. 2. eTwinning retention rate in 2011. Data on returning eTwinners by the year of their registration: “0 year” refers to eTwinners who registered in 2011; “1 year” = in 2010, etc.</figcaption></figure>

Retention rate for eTwinning refers to the percentage of eTwinners who have
registered on eTwinning since its beginning in 2005, and who still return to log-in to
the eTwinning platform in 2011. Figure 2 shows the eTwinning retention, the x-axis
represents the number of years since registration on eTwinning and the vertical one
represents the percentage of eTwinners.

We can observe that in year 0, the retention rate is 89%. This means that 89% of
users who registered on eTwinning in 2011 returned to login onto eTwinning at least
once during that year. In year 1, the retention rate is 42%, indicating that 42% of those
who signed up in 2010 returned to eTwinning at least once in 2011. The retention
curve is closely fitted with the power law curve (dashed line).

Finally, the retention curve shows evidence that among early eTwinners (i.e.
registered in '05 and '06), about 1 in 6 still remains engaged in 2011. We were not
able to find similar retention rates available for other professional learning networks
or social networking sites, but the annual version of eTwinning Analytics will keep
track of the trend in the future.
**RQ2: Use of Social networking tools.** There are eTwinners who engage in "professional collaboration" (i.e. run school collaboration projects) and the ones who do not: do these two groups use Social networking tools in the same way?

Figure 3 shows the usage of four different Social networking tools available for eTwinners (Contact, Profile picture, Journal Wall posts and Teachers' rooms) divided by the two groups. As a general trend, eTwinners using social networking tools are more involved in projects (average 64%) than the ones not engaged in projects (36%). This indicates that between these two groups, there are different ways in which they engage using the available Social networking tools. It is noteworthy to mention that the data does not give any indication of the amount of activities in which these two groups engage, nor of the causality of the usages (e.g. does social networking precede project collaboration).

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** eTwinners' use of social networking tools, divided by eTwinners with projects and without. Snapshot of data extracted from *SteerCom*-Desktop tool (Feb 2012).</figcaption></figure>

**RQ3: Teachers' co-operation activities.** *Over a long period of time (i.e. 6 years), are there any trends that emerge in teachers' co-operation activities?*

Figure 4 shows the percentage in which eTwinners have engaged in various co-operation activities, namely in *professional collaboration activities*, e.g. participation in cross-border school collaboration and in *Social networking activities*, e.g. adding Contacts and participating in Teachers' rooms. The x-axis represents the year since registered on the portal. Two patterns can be observed:

1. Professional collaboration: eTwinners in their early years (registered in year 0 and 1) seem to be less engaged in project collaboration (line: triangular) than those who have been on the platform for 3 years or more (average 18% vs. 30%).

2. Social networking: eTwinners in their early years (registered in year 0 and 1) seem slightly more involved in Teachers Rooms than others. On the other hand, even in the early years, Contacts feature seems to be well used by almost half of the eTwinners (45%), and the use seems to intensify after that.
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** eTwinners engagement on the portal disaggregated by the year of registration.
"0 year" refers to eTwinners who registered in 2011; "1 year" to those registered in 2010, etc.</figcaption></figure>

**5. Discussion and Future work**

This paper presents the concept of eTwinning Analytics that allows observing emerging behaviours and patterns within a large-scale teacher network. eTwinning was used as a case study of a network aimed at teachers' professional development where building knowledge takes place in a cultural, social and technological setting. The focus of analysis is on the meso-level using the emerging behavioural patterns as the unit of analysis. Three research questions were used to demonstrate the usage of such analytics. In the following, we offer some discussion on the results and the method, as well as new directions for this work.

From the RQ1 it becomes clear that eTwinning has a potential to engage its users over a long period of time (e.g. over 5 years). Evidence was found that *one eTwinner in six who registered on the platform between 2005 and 2006 still returns to it*. More studies are needed to understand whether eTwinning serves the evolution of teachers' competences; how well it can serve their lifelong learning goals over such a long period of time; and how to make sure that more “mature” eTwinners can contribute back to the community, for example, by mentoring newcomers.

The RQ2 looked at the use of Social networking tools on the platform by two groups: eTwinners engaged in project collaboration and the ones not. About 2/3 of the users of Social networking tools were active in project collaboration, showing that this group of active users use a large variety of tools and engage in many activities. However, *one third of the users of Social networking tools are eTwinners with no project experience engaging in a positive activity for building weak ties in the network*.

Weak ties are important for connecting people and bringing new ideas so that information can travel through a network [17]. More studies are needed to understand whether there is a sequence in using the different tools, e.g. social networking precedes project collaboration or whether they are complementary and overlapping activities. Such sequential interaction models would allow better insights
into supporting teachers' competence building over time, as well as to offer better support for new comers.

The RQ3 shows that to experience a full range of professional development activities in eTwinning (i.e. teachers' co-operation as defined by TALIS), and therefore to gain full advantage of it, a substantial time investment is needed. However, it is not clear whether we can infer that there is "an eTwinning life-cycle" with an assumption that it takes around two full school years for most eTwinners to "mature" and get involved in school collaboration - a topic needing a further investigation.

In this paper we have demonstrated the concept of eTwinning Analytics highlighting new insights into teachers' activities within a socio-technical network aimed at professional development. Even if some limitations with the available data were shown, the results are encouraging. They allow new hypothesis being created for further investigation on how teachers' co-operation takes place within a large-scale socio-technical network, and eventually take steps towards understanding when such networks better support teachers' personal and professional continuous development.

The future research proposes to investigate how the Uptake Analysis framework by [18] could offer a suitable methodological framework to further develop eTwinning Analytics and advance the study on teachers' large-scale socio-technical networks. More specifically, we are interested in investigating sequential interaction models and artefact-mediated collaborative activities within the eTwinning network that potentially lead us to gather better evidence of how, when and why socio-technical networks support lifelong learning. Evidence from the previous research [18] indicates towards the direction that the Uptake model offers an interesting methodological framework, especially thanks to its ability to achieve media independence, which has been deemed problematic for the studies in the past (e.g. [19]).

**Acknowledgements.** The paper has been made possible thanks to collaboration and funding through the LLP project called Teachers' Lifelong Learning Network [20] and eTwinning [5].

## References

1. Sloep, P., Berlanga, A.: Learning Networks, Networked Learning. Comunicar. 19, 37, 55–64 (2011)

2. Schlager, M. S., Farooq, U., Fusco, J., Schank, P., & Dwyer, N.: Analyzing Online Teacher Networks: Cyber Networks Require Cyber Research Tools. Journal of Teacher Education. 60, 1, 86–100 (2009)

3. Tapped-in, http://tappedin.org

4. Teachernet: webarchive.nationalarchives.gov.uk/*/*http://www.teachernet.gov.uk/*/

5. eTwinning, http://www.etwinning.net

6. Creating Effective Teaching and Learning Environments: First results from TALIS. OECD (2009)

7. European Schoolnet (EUN), http://www.europeanschoolnet.eun.org
8. Crawley, C., Gerhard, P., Gilleran, A., & Joyce, A. (eds.): eTwinning 2.0 Building the community for schools in Europe. European Schoolnet (2010)

9. Crawley, C., Gilleran, A., Scimeca, S., Vuorikari, R., & Wastiau, P.: Beyond School Projects, A report on eTwinning 2008-2009. Central Support Service for eTwinning (CSS), European Schoolnet (2009)

10. Vuorikari, R., Berlanga, A. J., Cachia, R., Cao, Y., Fetter, S., Gilleran, A., Klamma, R., et al.: ICT-Based School Collaboration, Teachers’ Networks and their Opportunities for Teachers’ Professional Development - A Case Study on eTwinning. In: Leung, H. et al. (eds.) Advances in Web-based Learning. pp. 112–121 Springer-Verlag, Hong Kong, China, December 8-10, 2011. (2011a)

11. Vuorikari, R., Gilleran, A., & Scimeca, S.: Growing beyond Innovators – ICT-Based School Collaboration in eTwinning. In: Kloos, C.D. et al. (eds.) Towards Ubiquitous Learning. pp. 537–542 Springer Berlin Heidelberg, Berlin, Heidelberg (2011b).

12. Breuer, R., Klamma, R., Cao, Y., & Vuorikari, R.: Social Network Analysis of 45,000 Schools: A Case Study of Technology Enhanced Learning in Europe. Learning in the Synergy of Multiple Disciplines. pp. 166–180 (2009)

13. Song, E., Petrushyna, Z., Cao, Y., & Klamma, R.: Learning Analytics at Large: The Lifelong Learning Network of 160,000 European Teachers. In: Kloos, C.D. et al. (eds.) Towards Ubiquitous Learning. pp. 398–411 Springer, Berlin, Heidelberg (2011)

14. Berlanga, A.J., Vuorikari, R. eds: Symposium - Learning Networks for Professional Development: Current Research Approaches and Future Trends. Proceedings of the 8th International Conference on Networked Learning 2012. The Netherlands (2012)

15. Siemens, G. (2011). http://www-learninganalytics.net/?p=126

16. Ferguson, R., Buckingham Shum, S.: Social Learning Analytics: Five Approaches. 2nd International Conference on Learning Analytics & Knowledge. ACM Press: New York, Vancouver, BC, Canada (2012)

17. Haythornthwaite, C.: The strength and the impact of new media. System Sciences, 2001. Proceedings of the 34th Annual Hawaii International Conference on. (2001)

18. Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R.: A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning. 5, 1, 5–42 (2010)

19. Ley, T. et al.: Implicit and Explicit Memory in Learning from Social Software: A Dual-Process Account. In: Kloos, C. et al. (eds.) Towards Ubiquitous Learning. pp. 449–454 Springer Berlin / Heidelberg (2011)

20. Teachers’ Lifelong Learning Network, http://www.tellnet.eun.org