# Understanding the Stakeholder Perspectives on Assessing Educators’ Digital Competence

Linda Helene Sillat, Kairit Tammets, Mart Laanpere

► To cite this version:

Linda Helene Sillat, Kairit Tammets, Mart Laanpere. Understanding the Stakeholder Perspectives on Assessing Educators’ Digital Competence. IFIP World Conference on Computers in Education (WCCE), Aug 2022, Hiroshima, Japan. pp.663-674, 10.1007/978-3-031-43393-1_59. hal-05533287

HAL Id: hal-05533287

https://inria.hal.science/hal-05533287v1

Submitted on 2 Mar 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Understanding the Stakeholder Perspectives on Assessing Educators’ Digital Competence

Linda Helene Sillat✉️, Kairit Tammets ID, and Mart Laanpere ID

Tallinn University, 10120 Tallinn, Estonia

{sillat,kairit,martl}@tlu.ee

**Abstract.** Digital competence of educators is one of the key factors affecting wide-scale digital transformation of education and is considered as one of the major aims in the European Digital Education Action Plan (2021–2027). To plan, conduct and report progress in digital competence development requires the ability to measure this competence. Hence, the importance of valid, reliable and usable digital competence assessment has been increasing among researchers, policy makers and teacher educators in recent years. While there exist a number of different instruments for assessing digital competence, there are not many comparative studies that would inform different stakeholders how to choose a suitable instrument to match specifically their context and goals. Furthermore, there is a lack of understanding of needs and expectations of various stakeholders when they consider assessment of educators’ digital competence. This paper summarizes a study that explored the stakeholder perspectives in digital competence assessment and related trade-offs.

**Keywords:** Digital Competence · Assessment · Teacher Education · Instrument Validation

## 1 Introduction

Digital transformation in schools and teacher education is necessary to prepare children and young people to act in the future knowledge society [1]. One of the key skills of knowledge society citizens is adaptation to fast technological change, rapidly growing knowledge, as well as global competition. Knowledge society development is closely related to the strategies that guide the embedding of information and communication technology in schools [2]. This means that the educators and school leaders need to be digitally competent to provide quality education while implementing digital technologies. To analyze the level and context of digital competence of educators we need valid and reliable assessment instruments. While there exists a multitude of tests, self-assessment scales, rubrics, portfolios and other instruments for measuring teachers’ digital competence, only a few of these have been properly validated by researchers [3]. Furthermore, there are no comparative studies that contrast the quality and suitability of various alternative assessment instruments, considering different needs of various stakeholders. We may safely assume that one instrument, even a scientifically validated one, is not equally suitable for all potential user groups.
This paper introduces the results of a mixed method study which focused on deep-
ening the understanding of stakeholder needs and perspectives in implementing digital
competence assessment instruments. The instruments described in the study were based
on the European framework of digital competence of educators DigCompEdu [4] and the
aim of the research was to understand the characteristics of the instruments based on the
stakeholder experiences and how they support the digital competence assessment pro-
cess. Based on the variety of stakeholders involved in the study we aim to determine the
relevance and meaningfulness of different assessment methods for the different actors
involved in the assessment of teachers' digital competence. The study was guided by a
single research question: What are the main characteristics of assessment instruments
that increase the perceived usefulness of the teachers' digital competence assessment
process for different stakeholder groups?

## 2 Theoretical Implications

The concept of competence has been defined as an ability to use the knowledge, skills and attitudes in the context of professional practice which are presented in the appropriate level of generality [5]. In this paper, the concept of competence is narrowed down to ‘digital competence’. Based on the European Commission [6] description, digital competence is the ability to confidently and critically use Information Society Technologies for work, leisure and communication. The same definition is used as a base level when designing DigCompEdu, the European Framework for Digital Competence of Educator. This framework defines teachers’ digital competence as a construct with six dimensions: the use of digital technology in professional engagement, learning resources, teaching and learning, assessment, empowering learners, and facilitating learners’ digital competence [4]. However, despite the efforts to design frameworks and instruments for assessing teachers’ digital competence, it has been described in research as a hard-to-measure construct. Hard-to-measure constructs mean that the interaction and dependencies with other types of skills and knowledge not intended to be investigated in the construct influence the assessment [7]. Constructs can be hard to measure due to different reasons: there might be a lack of clear definition of the construct, subjectivity of assessment and scores, multidimensionality of the construct etc. Although there are some constructs which are hard-to-measure because of their complex nature and theoretical background, it is still possible to draw conclusions about an individual’s level of certain competence based on what they say or how they act in certain situations [8].

It is widely accepted that a survey is the method applied the most for measuring digital
skills and competence, which might be useful for studying large samples and their self-
perceived skills, but it is also known that self-reported measures face issues with validity
due to the misalignment between perceived and actual skills [9]. Pinto et al. [10] explain
that digital competence includes mixed perspectives that combine both perceptions and
evidence, meaning that during the assessment process both subjective (attitudes and
motivations) and objective (knowledge and skills) values are evaluated. Although it can
be argued that attitudes and motivations should not drive the assessment process itself,
they are still necessary to understand and consider when choosing the suitable assessment
instrument. Digital competence is often defined as a multidimensional construct or latent
variable that reflects a complex relationship between general cognitive and technical skills [11]. Law et al. [12] explain that multidimensional constructs often exist at a deeper level than their dimensions and thus need to be considered as a latent trait. Also, Cartelli et al. [13] have pointed out that digital competence is a complex multidimensional construct that integrates cognitive, relational and social abilities and skills. They claim that it cannot be measured by a single test, it is tightly connected with other competences like reading, problem-solving etc. And finally - it is sensitive to the socio-cultural context, because the meaning of digital competence can change over time depending on the context where individual operates and develops daily (ibid). Clearly different approaches for assessing digital competence need to be argued.

There are different ways to assess hard-to-measure constructs and three major trends are knowledge-based testing, self-assessment and portfolio-based formative assessment approaches. Often all these approaches are built on common frameworks and models. One such approach highly accepted by the community is the DigCompEdu framework, which models' teachers' digital competence as a multidimensional construct into six dimensions or factors. The framework proposes six proficiency levels: newcomer, explorer, integrator, expert, leader and pioneer. There are many ways to develop assessment instruments based on the same competence model. As for any other latent variable, the digital competence is inferred indirectly from observed variables that are measured by a specifically designed instrument, such as an online self-rating scale or multiple-choice test that consists of many test items. Traditionally, researchers have preferred the latter type of measurement to any other, as it allows unbiased and objective data collection in a controlled environment. However, what is preferable for researchers, does not have to be so for teachers or school leaders. Next, we describe a few alternative assessment instruments that were both designed in line with the DigCompEdu framework.

## 2.1 DigCompEduSAT: Knowledge-Based Online Test

The DigCompEduSAT instrument was developed in collaboration with the European Commission Joint Research Centre (JRC) in 2019 and was the first instrument designed and developed based on the DigCompEdu framework. Although the questionnaire is presented as a self-assessment instrument, it was designed as a diagnostic knowledge-based test, which included 47 items divided into six competence dimensions and in three difficulty levels – easy, medium, hard. During the design stage it was agreed that each of the item inputs to the overall score or the competence area equally, meaning if Area 1 – professional development consisted of 4 items each of the items was valued at 25% of the total area score. Thus, it was considered that the construct of digital competence could indirectly be derived from a single item reliability. Based on the results, calculated by percentages, the test divides participants into 6 different proficiency levels: newcomer (A1), explorer (A2), integrator (B1), expert (B2), leader (C1), pioneer (C2). As a restriction of the online survey platform used for implementing the test, the test consisted of multiple-choice items that had one correct response out of four options (see Fig. 1 below).
## 2.2 SELFIE for Teachers: Online Self-reflection Scale

The next assessment instrument, DigCompEduSELF or alternatively branded as SELFIE for Teachers was developed by JRC and a group of international experts in 2021. This self-reflection questionnaire includes 32 statements that cover six dimensions of the DigCompEdu framework and it was piloted with more than 3200 teachers in Estonia, Italy, Lithuania and Portugal. The instrument is designed for use by individual teachers as well as groups, providing both individual and group-based reports that were highly appreciated by a large majority of participants in the pilot study. The self-reflection scale includes six statements that correspond to six digital competence proficiency levels suggested by the DigCompEdu framework and are also used in the DigCompEduSAT instrument.

## 2.3 Portfolio-Based Assessment Instruments

In addition to the self-assessment and diagnostic testing approaches, the community has widely accepted the portfolio-based assessment of digital competence as one of the alternatives and more formative approaches. For decades portfolios have been considered as a potential tool for the development and assessment of teachers' competence [14] and also as a tool that could make teachers' competence transparent for the teacher and the wider audience [15]. Different competence models can be used in portfolio-based assessment, however, the research in this sense is rather scarce and often based on localized frameworks. A study involving student teachers indicated that using portfolios for competence development and assessment helps them to accomplish personal growth [16]. However, it was also emphasized that such an assessment approach needs scaffolding: examples of good practices to illustrate competence and clear assessment criteria are needed.

Based on the above, we can see that there are a variety of approaches developed for assessing teachers' digital competence and also ways to make it meaningful for the teachers.

# 3 Research Design and Methods

The research was carried out in the two phases. First, a large-scale national web-based self-assessment study of teachers' digital competence was carried out, with the aim to understand how teachers perceive the experience of assessing their digital competence. In this phase, the self-assessment methodologically enabled engagement with a large number of teachers in authentic assessment situations. In the second phase, semi-structured interviews with typical representatives of four of the most common stakeholder groups were carried out to collect a deeper understanding on the stakeholder perspective, motivation, experiences and needs in the digital competence assessment process.

## 3.1 Sampling

In the first phase of the study, 1125 Estonian primary and secondary school teachers were recruited in the national level study to assess their own digital competence. 92%
of the teachers were female and 8% of the teachers were male. 8% of the respondents were younger than 30 years, 19% of them were aged between 31–40 years, 25% were 41–50 years old, and 48% were older than 51.

In the second phase of the study, four representatives of the key stakeholders were chosen to be interviewed. Based on our previous studies on identifying stakeholders in teachers' digital competence assessment, we came up with four personas to guide our research design and sampling. Personas are fictional characters that are used to inform the empathic design process mainly for two purposes: (1) for writing contextualized narrative scenarios, and (2) sampling the relevant and representative participants for interviews or participatory design sessions. We created a purposive sample based on the four personas, inviting one interviewee per each persona. The stakeholders included:

1. A teacher with a low digital experience (T1) – 26 years of work experience with the main focus on teaching music and natural sciences.

2. A teacher with a rich and deep digital experience (T2) – 4 years of work experience with additional training in implementing technology in teaching and student learning.

3. An educational technologist (ET) – 7 years of experience in supporting schools and kindergartens in supporting teachers in implementing technology in student learning and facilitating digital competence development.

4. A teacher educator in a university (TT) – 15 years of experience in teaching courses related to technology enhanced learning with the focus on supporting teacher students' digital competence and professional development.

## 3.2 Data Collection and Analysis

In the first phase, teachers were asked to self-assess their digital competence through a web-based questionnaire, which was designed based on the DigCompEdu Framework, localized by an Estonian expert working group and validated in 2019. Questionnaire items were based on six competence dimensions: Professional Engagement, Digital Learning Resources, Teaching and Learning, Assessment, Empowering Students and Developing Students' Digital Competence. The questionnaire consisted of 25 items and teachers were asked to assess themselves on the scale of 0 (not applicable) to 5 (leader). Additionally, teachers were asked to provide open answers regarding the perceived usefulness of self-assessing their digital competence. In this study we report only the results of the open answers, due to the scope of this research. Content analysis was carried out to analyze teachers' responses regarding the usefulness of assessing their digital competence. Answers were categorized as: self-assessment was not useful; self-assessment was useful; feedback about the instrument and 'other' - when respondents provided answers which were not relevant for the self-assessment process.

In the second phase, the data was collected through a semi-structured interview which was guided by five broad questions. Questions were focused on understanding stakeholder motivation behind digital competence assessment, their previous experience with the DigCompEdu questionnaires and needs for digital competence assessment. The questions also elaborated on the process of digital competence assessment and the outcome of the questionnaires. We used an inductive content analysis approach to understand the differences in stakeholder perspectives, meaning we coded the data
based on the leading interview questions and highlighted the commonalities. Based on
the analysis we propose trade-off scales in digital competence assessment.

4 Results

By analyzing the stakeholder responses to understand the perceptions towards the digital competence assessment we produced an initial trade-off model to describe the constructs related to motivation and needs. We also described the commonalities between the stakeholders.

4.1 Teachers’ Experiences with the Self-assessment Process

During the first phase of the study 523 teachers (46% of the respondents) filled in the open question where we asked them to reflect on how relevant and useful the digital competence self-assessment process was. 68% out of 523 teachers reported that the process was useful for them, 17% considered the process not useful, 8% were not sure and 7% provided irrelevant answers for this question.

Close to 90 teachers who considered the process useful did not provide any explanations for their response but the rest - almost 300 teachers - provided different justifications. The majority of teachers admitted that the self-assessment process made them think about their professional development gaps and needs ("*The questions made me understand my own level more clearly*"; "*I learned that I am not a digitally competent person*"; "*made me think about how I can develop myself as a teacher and a learner and about what my students need*"). Teachers also pointed out some more concrete gaps in their practice ("*I realized that I don't focus enough on ethical aspects in my teaching*"). Regarding the instrument, teachers valued the concrete examples that were provided ("*The examples were thought-provoking*"). 17% of teachers who considered the self-assessment process as not useful pointed out the following main reasons: instrument design and the culture of self-reflection. About 30 teachers argued that the instrument was too long ("*it was too long a process*"), items were complicated ("*statements need to be more explained*"), the scale was not appropriate ("*I missed the option "can but don't use"* "). A significant number of teachers however reported that the process of self-assessment was not useful for them, because they already knew their level of competence ("*It did not help much, I already know what I can do and what I can't* "). Few teachers mentioned that focusing only on assessment of digital competence is not meaningful for them ("*Not helpful, in our school we analyze our development as a whole picture* "). Finally, about 30 teachers also pointed out that such self-assessment needs to provide feedback for the teachers ("*After such a questionnaire, you might get an idea if there is room for improvement. But not very much in terms of what should change* ").

Such results clearly indicate that the self-assessment process makes teachers think
about their professional development gaps and needs, but there is a need to think how
to embed self-assessment in the other processes of professional development. We learnt
that a significant number of teachers think that they don't need self-assessment, because
they already know their level of competence, however it might not always be the most
subjective feeling. And finally, it is important to think about the feedback loops that
self-assessment would be more forward-looking and it helps teachers to understand how to fill in the gaps in their level of competence.

## 4.2 Stakeholder Motivation for Digital Competence Assessment

During the second phase of the study we focused on getting a deeper understanding of the teachers' feedback on digital competence self-assessment. During this phase we involved four stakeholders representing different personas to get a wider view on the digital competence assessment needs. This meant that to understand their perceptions towards digital competence assessment we first needed to understand their motivation behind the processes. All participants emphasized that digital competence assessment is highly important due to the focus on continuous professional development. Both teachers stated that it is becoming quite difficult to prepare teaching and learning materials without using digital technologies and they need to understand their own digital competence before facilitating the students. They also brought out that digital competence assessment provides them with an overview of their digital competence and helps to plan additional training (T1: "I need to assess my digital competence to understand what are my weakest areas and plan additional training based on that"). Similarly, the educational technologist stated that motivating and supporting all teachers in the school to evaluate their digital competence helps to get an overview of the state of competence and plan future training, infrastructure updates and collaboration between teachers. The teacher trainer brought out that having student teachers assess their digital competence gives an opportunity for personalized learning paths and also engages student teachers in setting their professional development goals. She also considered digital competence assessment results as means for learning analytics and getting feedback from the students (TT: "I ask the students to analyze their digital competence so they plan their personal goals in the long run and plan activities accordingly").

The participating teachers both stated that they would need external motivation to carry out digital competence assessment meaning they would prefer for the school leader or educational technologist to guide the process. At the same time, they pointed out that it is important for them to be aware where the assessment data and results are stored and who has access to it. Still both agreed that they would prefer if the assessment results are used as input for teaching staff training and the school leader is aware of their competence level (T1 & T2: "I wouldn't mind if my school leader sees the results if they plan or send me to trainings based on the results"). The teachers also stated that external motivation in the form of higher income or new technologies, i.e. updated school computers or robotics tools, would motivate them. The beginner teacher deemed important that the school leader is aware of the level of digital competence for providing support and limiting expectations meaning that the responsibilities would be appropriate based on the competence level (T1: "I think it's important my school leader sees the results. I don't want him to give me job assignments that I'm not able to perform").

While teachers pointed out that trust towards the data collecting – human access or safe storage was a weighing aspect the teacher trainer and educational technologist also stated that gaining trust of the assessment participants is an important factor. They stated that without the participant's trust it is close to impossible to get reliable results (ET: "Before I ask the student teachers to assess their competence I need to explain the use and
access of the data and results. I don't expect them to freely give out their results – although that would already reflect their digital competence"). Because digital competence is a central competence in teacher qualification standards, the teacher trainer and educational technologist believed that regular self-assessment and development goal setting is important. They stated that not only self-assessment is needed but also giving evidence on the digital competence is necessary which would explain the weak-spots of educators digital competence (TT: "They might assess themselves as an advanced technology user but when the evidence is just a link to an online website it tells me opposite information. Thus, it's not only necessary to carry out regular self-assessment but we also need to inform educators on what is applicable evidence"). All participants suggested that digital competence assessment should be a means for structured evidence-based decision making and centrally monitored process with clear outcomes like updated infrastructure, additional teacher training, continuous professional development or personalized learning paths.

During the study all participants had access to the DigCompEduSAT and DigCompE-
duSELF instruments prior to the interviews. Both teachers had previous experience in
assessing their digital competence with the DigCompEduSELF questionnaire and a stan-
dardized self-assessment survey based on the DigCompEdu Framework. They stated
that DigCompEduSELF provided them with a clear overview of the framework and
its dimensions making it easy to follow the questionnaire. It was also important that
the instrument provided examples and terminology explanations where necessary (T2:
"Some other surveys that I have used rarely give me examples of technology use and
also don't include explanations on foreign words or specific terminology"). Because the
DigCompEduSELF was easy to follow the teachers agreed that giving evidence on their
digital competence would be easy as the questionnaire items explain real-life situations
in schools (T1: "The question about creating digital learning resources would be easy
to provide evidence because while reading the question statement I right away started
thinking about the digital worksheets I created on Live Worksheet platform"). Teachers
also thought that the questionnaire influenced them to think about digital competence in
a wider sense and realized that they are actually doing a lot more than they previously
believed. They also brought out that although DigCompEduSELF took them longer
to finish it was more informative and reflective. Meaning that although time plays an
important factor for the teachers they consider quality feedback more important than
the survey length or completion time. The item length was also noted as a positive fac-
tor in DigCompEduSELF as all stakeholders believed that lengthy item statements are
confusing and often reflect on the participant's motivation. However, the teacher trainer
believed that a longer item statement which includes real-life examples could provide
the educators with a learning possibility. The teacher trainer and educational technol-
ogist brought out that the DigCompEduSELF reflects well on the framework and also
that the items reflect the educational reality well. It was also important for them that the
technical solution allows raw data to be downloaded and to get a holistic view on their
participant group results (TT: "I think the best thing about DigCompEduSELF is that I
can create my own participant group and although the platform gives me a really good
overall report and visuals it also allows me to download raw data. The raw data format
however needs some work").
DigCompEduSAT was mainly redeemed unusable as it didn't reflect the educational reality and the assessment scale was difficult to follow. The teachers stated that although it would be interesting to see which competence level they fit into it is difficult to assess themselves as many questions do not relate to their everyday work experience. Advanced teachers also believed that assessments based on testing make it easier for school leaders to compare teachers' results which often reflects back on their attitude towards teachers with lower digital competence. Teachers also stated that although some of the test items did not relate their daily work or experiences it was easy to guess the right answer (T2 "I'm not sure what the right answer is based on my own work but I can guess the right one because all other possibilities seem completely unbelievable"). Educational technologists thought that although it would be convenient to use an instrument which divides educators based on the competence level it was still difficult to relate some of the DigCompEduSAT items to teachers' daily work. It was also clear that the test design was based on an average sized (500–1000+ students) general school (1–12 grades) and does not consider small schools (under 200 students) or basic schools (grades 1–6(9)), thus making some of the questions redundant. Because DigCompEduSAT does not require participants to give evidence or always reflect on the educational reality, the teacher trainer felt that it was not usable among the student teachers as the test items might give a distorted overview of digital competence. It was also stated by the teacher trainer and educational technologist that because DigCompEduSAT does not provide access to the overall report it is nearly impossible to support participants in goal setting or reflection.

## 4.3 Stakeholders’ Needs for Purposeful Digital Competence Assessment

During the data analysis of both research phases we realized that the stakeholders' needs presented repetition meaning that there are characteristics that relate back to each stakeholder and digital competence assessment method (Fig. 1). We first focused on understanding the stakeholder teacher (T1) needs which indicated that the teachers evaluate the usefulness of the assessment instrument and tool based on the transparency, time demand, anonymity, feedback quality and accessibility. It was also evident that the decision making while choosing a self-assessment instrument came mainly from external motivators which included compulsory self-assessment administered by the school leader. Although the teacher with rich experience (T2) in implementing digital technologies mainly valued the same aspects it was also clear that they understood the need for self-assessment and thus can be considered as internally motivated. They also regarded both personalized and generalized feedback important as it would help them to understand their development needs in comparison of other teachers.

Opposite ends of the scale points reflect the needs indicated by the educational technologist (ET) and teacher trainer (TT). They stated the decision making when choosing a digital competence assessment instrument is strongly guided by the evidence and data which can be used to support teachers' professional development but also improve their teaching practices. Additionally, the question of instrument sustainability was considered as important as based on the stakeholder experiences there is a constant change in the assessment instruments which makes it difficult to focus on repeatability and comparison the assessment results.
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Trade-off model for teachers’ digital competence assessment instrument design.</figcaption></figure>

Based on the teachers experiences and explanations of the self-assessment usefulness and stakeholder needs for digital competence assessment we described the trade-off scales (Fig. 1).

The trade-off model visualizes the differences between perspectives of four stakeholder groups regarding nine opposite value pairs that can be considered when designing or choosing a suitable assessment instrument, or combining two or more different instruments. The opposite ends of each scale depict user values that are incompatible and suggest a need for trade-off.

# 5 Discussion and Conclusions

A noteworthy result of our national self-assessment process indicates that teachers actually consider the self-assessment of their digital competence to be a rather useful experience, because it helps them to understand their professional development gaps and needs. However, there are some implications that need to be considered: self-assessment should be embedded into other assessment processes, the instrument needs to be simplified and most importantly, feedback loops are needed.

Additionally, one of the main outcomes of the study was the overview of contrasting stakeholder needs regarding teachers’ digital competence assessment in the future. Based on the inductive content analysis of the differences in stakeholder needs gave a clear indication towards understanding the scope and dimension of different preferences that can be represented in a form of trade-off scales (Fig. 1). This first version of the trade-off model and the scales help us understand the need for different assessment instrument depending on its usage contexts and users. While a scientifically valid and reliable test might be a preferable option for a researcher, it will not be as helpful for a single teacher
struggling with motivation to develop her basic digital competence. Figure 1 gives an overview of the digital competence trade-off model and describes the relations and importance of the scales based on stakeholder needs which gives us a way for structured decision making in the digital competence assessment process.

This study explored the differences in stakeholder perspectives regarding teachers’ stakeholders, we identified nine aspects that can be used as binary, contrasting value pairs forming a trade-off model for designing, selecting or combining different digital competence assessment instruments depending on the prioritized target groups and goals of assessment. The current study suggests an initial version trade-off model and we consider that it has to be validated with a larger sample of participants and also a larger set of instruments in the future.

**Acknowledgements.** The research presented in the article has received partial funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No. 856954.

## References

1. Ahonen, A.K., Kankaanranta, M.: Introducing assessment tools for 21st century skills in Finland. In: Griffin, P., Care, E. (eds.) Assessment and Teaching of 21st Century Skills. EAIA, pp. 213–225. Springer, Dordrecht (2015). https://doi.org/10.1007/978-94-017-9395-7_10

2. Ilomäki, L., Kankaanranta, M.: The information and communication technology (ICT) competence of the young. In: Tan Wee Hin, L., Subramaniam, R. (eds.) Handbook of Research on New Media Literacy at the K-12 Level: Issues and Challenges, pp. 101–118. IGI Global (2009)

3. Lucas, M., Bem-Haja, P., Siddiq, F., Moreira, A., Redecker, C.: The relation between in-service teachers’ digital competence and personal and contextual factors: what matters most? Comput. Educ. **160**(3), 104052 (2021)

4. Redecker, C., Punie, Y.: Digital Competence of Educators DigCompEdu. EUR 28775 EN. Publications Office of the European Union, Luxembourg (2017). https://doi.org/10.2760/178382

5. Biggs, J.: Enhancing teaching through constructive alignment. High. Educ. **32**, 347–364 (1996). https://doi.org/10.1007/BF00138871

6. Brande, L., Carretero, S., Vuorikari, R.: DigComp 2.0: the digital competence framework for citizens. Publications Office. Joint Research Centre (European Commission) (2017)

7. Barton, K., Schultz, G.: Using technology to assess hard-to-measure constructs in the common core state standards and to expand accessibility: English language arts. In: Invitational Research Symposium on Technology Enhanced Assessments, pp. 1–17 (2012)

8. Shute, V., Ke, F., Wang, L.: Assessment and adaptation in games. In: Wouters, P., van Oostendorp, H. (eds.) Instructional Techniques to Facilitate Learning and Motivation of Serious Games. AGL, pp. 59–78. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-39298-1_4

9. van Laar, E., van Deursen, A.J., van Dijk, J.A., de Haan, J.: Measuring the levels of 21st-century digital skills among professionals working within the creative industries: a performance-based approach. Poetics **81**, 101434 (2020)
10. Pinto, M., Fernandez-Pascual, R., Puertas, S.: Undergraduates' information literacy competency: a pilot study of assessment tools based on a latent trait model. Libr. Inf. Sci. Res. 38(2), 180–189 (2016)

11. Markauskaite, L.: Exploring the structure of trainee teachers' ICT literacy: the main components of, and relationships between, general cognitive and technical capabilities. Educ. Technol. Res. Dev. 55, 547–572 (2007). https://doi.org/10.1007/s11423-007-9043-8

12. Law, K.S., Wong, C.-S., Mobley, W.H.: Toward a taxonomy of multidimensional constructs. Acad. Manag. Rev. 23(4), 741–755 (1998)

13. Cartelli, A., Dagienė, V., Futschek, G.: Bebras contest and digital competence assessment: analysis of frameworks. Int. J. Digit. Lit. Digit. Competence 1, 24–39 (2010)

14. Struyven, K., Blieck, Y., De Roeck, V.: The electronic portfolio as a tool to develop and assess pre-service student teaching competences: challenges for quality. Stud. Educ. Eval. 43, 40–54 (2014)

15. Korhonen, A.-M., Lakkala, M., Veermans, M.: Identifying vocational student teachers' competence using an ePortfolio. Eur. J. Workplace Innov. 5(1), 41–60 (2019)

16. Korhonen, A.-M., Ruhalahti, S., Lakkala, M., Veermans, M.: Vocational student teachers' self-reported experiences in creating ePortfolios. Int. J. Res. Vocat. Educ. Training 7(3), 278–301 (2020)