# Touch, See and Talk: Tangibles for Engaging Learners into Graph Algorithmic Thinking

Andrea Bonani, Andreas Bollin, Rosella Gennari

<figure><img src="image_1.png" /></figure>

## To cite this version:

Andrea Bonani, Andreas Bollin, Rosella Gennari. Touch, See and Talk: Tangibles for Engaging Learners into Graph Algorithmic Thinking. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.630-651, 10.1007/978-3-030-85610-6_36. hal-04215518

HAL Id: hal-04215518

https://inria.hal.science/hal-04215518v1

Submitted on 22 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Touch, See and Talk: Tangibles for Engaging Learners into Graph Algorithmic Thinking

Andrea Bonani¹[0000-0001-5891-8496], Andreas Bollin²[0000-0003-4031-5982], and Rosella Gennari¹[0000-0003-0063-0996]

¹ Free University of Bozen-Bolzano, Italy, andrea.bonani@scuola.alto-adige.it and gennari@inf.unibz.it
https://www.unibz.it/en/faculties/computer-science/

² Universität Klagenfurt, Austria, Andreas.Bollin@aau.at, https://aau.at/en/informatics-didactics/

**Abstract.** Algorithmic Thinking (AT) is at the core of Computational Thinking (CT). A number of initiatives target CT, few of them focus on AT and even less deal with Graph Algorithmic Thinking (GAT) with younger learners. This paper reports on tangibles’ design for GAT, appealing to different senses so as to engage learners actively. It presents a field study with GAT tangibles and 14–15 years old high-school learners, divided into two groups: one group used tangibles, the other used traditional means, namely, paper and pencils. The study results show that tangibles were more engaging than in the traditional GAT setting, and differences among groups are statistically significant. The paper concludes by discussing the study results and advancing suggestions for future interventions related to engagingly teaching GAT.

**Keywords:** Multimodal · Tangible · Algorithmic Thinking · Computational Thinking · Children · Teens.

# 1 Introduction

## 1.1 Background and Rationale

In parallel with the rapid growth of digital technology and its pervasiveness in everyday life, all citizens must have a range of new Computer Science related skills and competences, which schools need to teach in order to help grow tomorrow’s citizens, starting from the early years of primary schools [13,17,41]. A number of educational initiatives have been developed to promote such skills, in various contexts. Among others, *Computational Thinking (CT)* initiatives have gained popularity in different countries, e.g., in Italy, with “Programma il Futuro” [35]. CT represents a universally applicable attitude, and a set of Computer Science skills needed by everyone, not just by computer scientists [55].

At the core of CT skills are *Algorithmic Thinking (AT)* skills. Strictly related with problem solving, AT is concerned with representing a problem in a suitable abstract manner, and planning its resolution through a step-by-step strategy.
This paper stems from research which promotes AT at school in a novel manner: it abandoned frontal lectures or computer screens, and it designed tangibles for AT which promote diverse interaction modalities (briefly, *tangibles*).

Such tangibles are designed primarily for 9–15 years old children. Reference theories for the their design were, mainly, socio-constructivism and multi-modal learning, as explained in the following. Socio-constructivism holds that children actively construct their knowledge through diverse interactive experiences. In line with that, tangibles are designed for different learning scenarios, according to the envisioned learning goal. Moreover, tangibles are designed intentionally screen-less so as to enable social interactions as head-up games do [52]. By leveraging on multi-modal learning, such tangibles aim at appealing to different senses and making immediately perceivable otherwise too abstract reasoning forms for 9–15 years old children [31]. In particular, Moreno and Meyer defined guidelines and recommendations for multi-modal learning, which have been considered in the design of tangibles for AT so as to best combine different sensory stimuli (e.g., audio and tactile) [36].

In summary, the research at the basis of this paper designed AT tangible for fostering an engaging experience, as in multi-modal learning and socio-constructivism.

## 1.2 Focus and Contributions

This paper reports on a field study with tangibles for AT with specific data structure: graphs. Example problems with graphs are found in social networks, computer networks, traffic networks. The formal definition of a graph, subsumed in the rest of the paper, is as follows: a(n undirected) graph is an ordered pair $G = (V, E)$, with $V = \{v_1, ..., v_n\}$ the set of nodes, and $E = \{{v_i, v_j} | v_i, v_j \in V\}$ the set of edges between pairs of nodes.

The paper deals with *Graph AT* (GAT). GAT is relevant and applicable in different learning contexts; it is possible to find concrete examples taken from real life contexts, and hence to contextualise problems that learners are asked to tackle. Imagine a traffic network context, with islands to connect via bridges. Islands can be modelled as graph nodes, and edges between nodes represent connecting bridges. See a graphical representation of such a situation in Fig. 1, which has four islands (A, B, C, D) and four connecting bridges to construct. Therein, a sequence of connecting bridges from the island A to the island C gives a so-called path, e.g., through the bridges from A to B and then from B to C.

GAT in this context can mean tackling diverse problems: Can an island connect to all the others, and how? Is there a way of connecting all the islands together, so that there is exactly one path from any one island, to any other? And, in case bridges have different costs, is there a way of minimising the total cost? Answering the first question means deciding if the graph is connected. Answering the second means deciding whether there exists a spanning tree. The answer to the third question considers the costs of bridges, and selects those leading to a minimum cost. By referring to Fig. 1, this would be the path
with all bridges except the one connecting D and C. That amounts to finding a
so-called minimum spanning tree.

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Typical intervention with a graphical representation of islands (A, B, C, D) and connecting bridges with costs (e.g. 10 and 50 cents). The GAT tangibles represent nodes and edges of the graph.</figcaption></figure>

At the same time, GAT is a topic rarely dealt within schools; no children
involved in the study of this paper knew of graphs or GAT, which enabled the
study to test engagement in GAT through tangibles without children having
prior experience of it. Teaching GAT means asking learners to:

1. understand what graphs are,

2. model a problem with graphs,

3. design general strategies to solve the problem and similar problems by relying on graphs,

4. explore general strategies for doing so.

In the study reported in this paper, learners were randomly divided in two
groups: one group used GAT tangibles; the other group used pencils and paper.
All learners, however, tackled the same scenarios for GAT at school (briefly, *GAT
scenarios*), so as to have a comparable GAT experience in terms of learning goals
and tasks. Initial scenarios started with goals and tasks at the basis of GAT,
that is, understanding graphs and modelling a problem with graphs. Subsequent
scenarios gradually moved learners to more advanced goals and through the
related tasks, such as exploring the Prim Algorithm for constructing a minimum
spanning tree for a given graph.

Data related to learners’ engagement in the GAT experience were gathered
with different instruments and analysed. The comparison of engagement results
shows differences across the two groups in relation to the overall experience with
GAT: engagement is statistically significantly higher in the group who worked
with tangibles. Further analyses related to the group with tangibles help unveil
reasons for the superior engagement results: tangibles seem to engage learners
into watching, touching and talking for collaborating around a GAT scenario,
thereby soliciting diverse interaction modalities.
## 1.3 Novelty and Outline

Compared to related work, to the best of our knowledge, the study reported in this paper is the first that introduces 14–15 years old children to GAT via tangibles. The study leveraged on a series of past actions, conducted by following an action-research paradigm [34,12]. Notice that past actions concerning GAT tangibles focussed on the usability of the tangibles and the co-creation of preliminary learning scenarios for GAT. The refined tangibles and scenarios, resulting from past research, are employed in the study reported in this paper. The study, reported in this paper, was never described in other publications prior to this. The results of the study pave the way for work related to the design of multi-modal interactive solutions for (G)AT, which stimulate different senses and engage learners.

The paper is organised as follows. Section 2 presents relevant related work. Section 3 describes the field study design. Section 4 presents its results. Section 5 discusses the results of the study, along with reflections related to limitations and future work.

# 2 Related Work

CT is relevant for nowadays’ children. Different interventions bring CT into schools, but relatively few activities are related to AT, and even fewer are those that employ tangibles. The next subsection briefly reports on some of them. The goal of the reported study is to explore engagement in a GAT experience with tangibles, which stimulate different senses—visual, tactile, auditory and kinesthetic. The follow-up subsection outlines how engagement in an activity can be defined and assessed.

## 2.1 Computational Thinking and Algorithmic Thinking

The twenty-first century is arguably the century of computing. Computing is increasingly involved in transforming work, commerce, and everyday life: big data, the internet of things, cloud computing, voice, and facial recognition, robotics, and more. Social media are changing how and where people work, collaborate, communicate, shop, travel, receive news and entertainment. Information technology is also transforming and innovating every discipline, becoming an integral and transversal tool for every activity [30].

All that inevitably demands to prepare younger generations to CT to make them able to interpret the world they are in and help shape it [54]. CT initiatives in compulsory education curricula are being undertaken in Europe and various other parts of the world [8]. The New Skills Agenda (European Commission, 2016) explicitly invites the Member States to develop “coding / Computer Science” in education [20]. Indeed, more and more new projects aim at establishing CT, and AT in basic education [39,42].

In the latest decade, the school world has shown a growing interest for introductory activities on basic elements of Computer Science, in many cases carried
out through CT activities. The European Schoolnet published a report survey-
ing the current situation in 20 European countries [21]. In 13 of these countries,
programming has been (or soon will be) introduced in K-9 education. In seven
of these, among them Estonia, England and Greece, programming is included
as a compulsory part of the curriculum [4,53]. In non-European countries (e.g.
Israel, USA, Australia, New Zealand), elements of computer science have been
introduced into the basic school curricula [5,18]. More relevant for this paper is
research by Bollin et al.: they introduced a way to analyse and compare cur-
ricula, education standards and competency models, by using a graph-based
representation form and several graph-theoretical metrics [40].

Recent research reflects on the importance of making learners concentrate not
on the technology itself or programming per se, but on core CT skills, and espe-
cially AT skills [9]. AT, in particular, is promoted via three main approaches: (1)
without a computer such as CS unplugged [6]; (2) with computers for coding with
specific programming environments for younger learners, such as Scratch [33];
(3) with tangibles. This last approach is the one followed in this study.

Several research studies highlight the benefits that tangibles play in problem
solving tasks [50]. Well-known examples gravitating around various tangibles
are instructional activities with robotics, and others requiring different physi-
cal computing devices, which have been used in educational settings for many
decades [25]. Specifically, there are several activities which rely on educational
robotics [32]. An example is TangibleK, which used robotics and a tangible
interface as tools for teaching an engineering design process [7]. Teaching AT
through tangibles or physical objects has received increasing attention in re-
cent years albeit examples are few [26]. AT has been introduced, for instance,
with a haptic model [14,25]. That turned out to be helpful in different learning
contexts [19]. Another example is a tangible computational drum kit with pro-
grammable behaviour, or other tangibles that show how learners can successfully
develop AT [3,43,48,56]. A research highlighted that learners who used physical
manipulation (e.g., through the use of flashcards) were better at strategy design
than learners who did not use physical manipulation [1].

## 2.2 Engagement

Engagement is a debated construct in education, to the point that there is great
variety in the ways engagement is defined and operationalized [47]. Although the
researchers’ interest in engagement has increased in the last years, its distinction
from other constructs such as motivation remains subject to debate [2,15].

Many researchers consider engagement as a meta-construct that includes
other sub-constructs [47]. Fredericks described what have become the three com-
mon sub-constructs of engagement in learning, with or without technology [23]:
behavioural, emotional, cognitive engagement. This paper focuses on behavioural
and emotional engagement, so as to assess whether children are engaged in GAT,
with or without tangibles.

In technology-mediated learning experiences, behavioural and emotional en-
gagement have been measured in different manners. The study in this paper
considers two main instruments for assessing engagement, namely, observations and self-report questionnaires, which are often combined in the design of exploratory and evolving learning technologies [27,28]. The original instruments are explained, separately, in the following.

**Questionnaire.** Non-observable aspects of engagement, such as emotional engagement, are often assessed with self-report questionnaires at the end of an experience [2]. Self-reports have the advantage of representing the most direct way of having insight on people's emotional world [46]. For instance, learners are asked to reflect on the various aspects of their commitment, and select the answer that best describes them [24].

As suggested by Read and MacFarlane, such questionnaires for children need to phrase questions in a language clear for children and have a specific format, adequate to the considered age range [45]. In this research, the again-and-again instrument of the standardised Fun Toolkit by Read and MacFarlane was used to assess participants’ engagement in the entire experience. This consists of a single question, asking participant children whether they “would do it again” or whether they would suggest the experience to others. Younger learners answer by choosing one out of usually 3 possible items on a discrete Likert type scale, from “absolutely not”, to “absolutely yes”. The scale was changed to 5 items with older children, as in the study reported in this paper.

**Observations.** Observations are most reliable when all learners are present in the same location, and when they perform similar tasks, as in the case of the field study reported in this paper with GAT tangibles. Observations can be direct by an observer present in class, or indirect observations such as videos [22]. This study employed videos, which enable for more reliably coding observations. The BROMP observation protocol can be employed in observations so as to detect and code diverse indicators of engagement [38]. Indicators, in general, are markers or descriptive parts inside the target construct [51]. The coding can be later used to perform quantitative or qualitative analyses in relation to engagement, as in the study reported in the remainder. The BROMP indicators are related to tasks with a technology, and can be grouped into two broad categories, as follows:

- behavioural indicators, related to being acting as supposed (e.g., using GAT tangibles to find a spanning tree), or off-task, for a learner who is working neither on his/her own nor interacting with others for the task at hand (e.g., using GAT tangibles to play);

- affective indicators, related to either (1) positive emotional states, such as “delight”, for learners who are smiling or otherwise indicating that they are having a pleasurable experience, (2) or negative emotional states, such as “boredom”, for learners who appear to find the task dull or tedious.

The field study, reported in this paper, considered them all and slightly revised behavioural indicators to the context of usage of GAT as explained in details in the next section.
# 3 Field Study

The field study, reported in this paper, took place in a technical school in Merano. It involved 14–15 years old learners and their teacher. It took place at the end of the school year.

This study leverages on prior work by Bonani et al., which studied and improved on the usability of tangibles besides on the co-creation of learning scenarios, e.g., [10,11]. Specifically, by following an action research paradigm, actions involved teachers and learners of various age groups, from 9 to 14 years old. Actions led to the co-design of GAT scenarios with teachers. In turn, scenarios with teachers stirred the design of more and more usable tangibles. The final revised tangibles and scenarios are employed in the study reported in this paper.

This section starts with the research questions of the study. Then it reports on the participants and how they were involved in the field study. The section describes the adopted GAT scenarios, each with its own learning goal and related tasks, for all participants in the study. It concludes with an outline of the GAT tangibles and information on the data collection instruments.

## 3.1 Research Questions

The study revolved around two main research questions:

(RQ1) Are there differences in the engagement of learners with tangibles, over the engagement of those with traditional paper-and-pencil means?

(RQ2) In case so, what could be task-related engagement indicators?

## 3.2 Participants

Before the study, parents or guardians were informed and asked to state their agreement with their child's research participation with a consent form approved by the ethical board of the organising university.

A total of 28 learners participated in the study. Their teacher randomly divided them into two groups. All tackled the same scenarios, as follows:

1. 20 learners tackled GAT scenarios with paper and pencil;

2. 8 learners tackled GAT scenarios with GAT tangibles.

Their participation lasted a total of 6 hours, split across different days.

The division of the groups into 20 and 8 was dictated for operational reasons. The students who worked with tangibles operated in working groups of 4 students. The limited time available due to school organisational constraints, as well space-related constraints imposed by the tangibles did not allow for the creation of additional working groups.

Before the study, all participants already knew the word *algorithm* and knew of CT, in that they had studied the basics of programming using the programming language C#. None of the learners, however, had ever studied algorithms, and they had no knowledge of graphs.
<figure><img src="image_5.png" /><figcaption>Fig. 2. Students with GAT tangibles.</figcaption></figure>

## 3.3 Scenarios

GAT shares several similarities with problem-solving in educational contexts. In the research described in this paper, GAT in educational contexts is supported by GAT scenarios, co-designed with school teachers over time.

GAT scenarios link the contents of learning to learners’ everyday life situations, and target the following main GAT skills:

(a) *understanding*: making learners understand certain graphs;

(b) *modelling*: making learners model certain problems with certain graphs;

(c) *designing*: making learners design strategies for solving certain problems with certain graphs;

(d) *exploring*: making learners explore strategies for solving certain problems with certain graphs.

All GAT scenarios follow the same structure. They share characteristics with traditional learning scenarios [16]. They come with a learning goal, which is related to one out of the four aforementioned ones (understanding, modelling, designing or exploring). Moreover, they turn the goal into measurable objectives for teachers, which are related to assessment instruments that teachers can use in class. The scenarios also document the relevant information concerning the environment in which they take place and the used technology, in case any is used. GAT scenarios also share characteristics with scenarios of interaction design, in that they focus on tasks of learners or teachers [44]. The description of is mainly a narration of tasks in GAT, linked to the learning objectives of the scenario and the primary learning goal. An example scenario is in Table 1.

The GAT study, reported in this paper, centred around GAT scenarios listed in Table 3. Based on the main goal to be achieved, scenarios are grouped into three main areas:

- building graphs,
<table><caption>Table 1. A scenario, related to building simple graphs</caption><tr><td><strong>Title:</strong></td><td>Simple graphs.</td></tr><tr><td><strong>Participants:</strong></td><td>9-15 years old children; their teachers; possibly, a researcher as observer or for technical assistance.</td></tr><tr><td><strong>Goal:</strong></td><td>To understand simple graphs, and to model problems with simple graphs.</td></tr><tr><td><strong>Objectives:</strong></td><td>To understand the properties of a simple graph, namely, (P1) it has no self-loops, and (P2) it has no parallel edges; to model a problem with simple graphs so that P1 and P2 are satisfied.</td></tr><tr><td><strong>Assessment instruments:</strong></td><td>Question-answering during the activity; post-activity questionnaire.</td></tr><tr><td><strong>Environment:</strong></td><td>A classroom (with a table for placing GAT tangibles, if present); learners are free to move around and collaborate.</td></tr><tr><td><strong>Description:</strong></td><td>The teacher introduces a problem which can be modelled as a simple graph, e.g., islands to connect as in Fig. 1. The teacher introduces mistakes on purpose and asks learners to reflect on them (according to the feedback of GAT tangibles, if used). Mistakes are related to properties of simple graphs: (P1) self-loops; (P2) parallel edges. Then, the teacher introduces other similar example problem and asks learners to model them as simple graphs (with GAT tangibles, if present). An example for teachers is in Table 2. The teacher invites learners to reason upon their choices. Finally, the teacher stirs the conversation so as to abstract properties P1 and P2 from the examples, in order to foster the understanding of simple graphs.</td></tr></table>

- finding a spanning tree,

- finding a minimum spanning tree.

Each area is preparatory to the next one: a teacher starts from scenarios pertaining to building graphs and gradually moves towards the others. Specifically, a teacher starts asking students to build a graph for real-life situations, e.g., see the simple-graph scenario in Table 1. Next, the teacher asks learners to find first a tree, then a spanning tree and finally a minimum spanning tree, always referring to real-life situations, e.g., how to connect all islands with connecting bridges by minimising the costs of bridges (find a minimum spanning tree). See Fig. 1.

## 3.4 Tangibles

GAT tangibles are physical objects, made interactive and smart through programmable microelectronics. Examples of GAT tangibles of the study are in Fig. 2, whereas a possible educational setting has already been depicted in Fig. 1.

From an architecture viewpoint, GAT tangibles are part of a distributed client-server system, with server and clients communicating through WiFi. The server, usually the teacher's laptop, coordinates clients, displays the interface for
Graph

Desks are located as shown in the image. During the whole year, the teacher recorded
the classmates of each learner:

learner classmates during the year

Anna Paul, Markus

Martina Hans, Markus

Hans Martina, Markus

Paul Anna, Miriam

Miriam Paul

Markus Hans, Martina, Anna

Julia

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

(a) Can you represent this situation with a graph?

(b) What do nodes represent? And what do edges stand for?

(c) What are the properties of this graph?

<caption>**Table 2.** An example for teachers related to the simple graph scenario</caption>

teachers, verifies graph properties and hence enables for GAT with tangibles, ac-
cording to what is specified in GAT scenarios. See Fig. 3 for the teacher interface
as well as Fig. 1 for the overall setting.

<figure><img src="image_10.png" /><figcaption>**Fig. 3.** The teacher interface.</figcaption></figure>

GAT tangibles are the clients of the system. See Fig. 4. They are used by
learners to construct and reason with graphs and graph algorithms.

GAT tangibles interact with the server and learners through microelectronic
components. They represent parts of a graph, such as nodes and edges, and addi-
tional components such as the so-called confirmation button. They are explained
briefly in the following.

**Nodes.** Nodes of a graph are 3D-printed hemispheres with a Raspberry Pi 3 micro-computer, a PowerBank, several micro-electronic components such as LED coloured strips or simple red LEDs, 4 USB sockets for cables, representing
<table><caption>**Table 3.** List of scenarios</caption><thead><tr><td>Scenario</td><td>GAT skill</td></tr></thead><tbody><tr><td>Nodes and Edges</td><td>Understanding a graph, modelling a problem using a graph</td></tr><tr><td>Simple Graph</td><td>Understanding a graph, modelling a problem using a graph</td></tr><tr><td>Connected Graph</td><td>Understanding a graph, modelling a problem using a graph</td></tr><tr><td>Spanning Tree</td><td>Understanding a tree, modelling a problem using a tree, designing a strategy</td></tr><tr><td>Spanning Tree</td><td>Understanding a tree, modelling a problem using a tree, exploring a strategy</td></tr><tr><td>Minimum Spanning Tree</td><td>Understanding a tree, modelling a problem using a tree, designing a strategy</td></tr><tr><td>Minimum Spanning Tree</td><td>Understanding a tree, modelling a problem using a tree, exploring a strategy</td></tr><tr><td>Minimum Spanning Tree</td><td>Understanding a tree, modelling a problem using a tree, exploring a strategy (Prim)</td></tr><tr><td>Minimum Spanning Tree</td><td>Understanding a tree, modelling a problem using a tree, exploring a strategy (Prim)</td></tr></tbody></table>

<figure><img src="image_11.png" /><figcaption>**Fig. 4.** Schema of GAT tangibles.</figcaption></figure>

edges. A button on top of the hemisphere is used to select or deselect the node, e.g., in case learners have to select components of a spanning tree. See Fig. 5.

**Edges.** Edges of a graph are electrical cables ending with USB connectors, terminating in nodes. Similarly to nodes, edges have a button on a small 3D-printed box, positioned halfway through the cable, enabling edge selection or deselection, e.g., when learners are building a spanning tree. Near the button, small LEDs in a small box serve to indicate the edge's weight, e.g., when learners are working with weighted graphs. This small box also contains a Raspberry Pi 0, a lithium-polymer battery and several electronics components. See Fig. 5.

**Confirmation button.** The confirmation button is a cylinder having on top a large LED blue button, pressed for giving specific feedback, typically for checking whether a task is complete, according to the GAT scenario under con-
sideration. It consist in a Raspberry Pi 0 with a PowerBank and several electronic
components. See Fig. 5.

<figure><img src="image_12.png" /><figcaption>**Fig. 5.** GAT tangibles: two nodes, one edge and its confirmation button.</figcaption></figure>

**3.5 Data Collection**

Data were collected in relation to the main research questions: (RQ1), which
investigates differences in engagement between those using tangibles and those
using traditional means; (RQ2), which explores task-related engagement with
tangibles.

By following a mixed-method approach, data were collected with two instru-
ments, explained in Section 2, one per research question. The again-and-again
instrument of Read and MacFarlane was used in order to tackle (RQ1). In case
of (RQ2), indirect observations (videos) were used, considering engagement in-
dicators of the BROMP protocol with tasks of GAT scenarios.

The again-and-again instrument was administered to learners at the end of
the study. It had a single question: *would you suggest the GAT experience to
another class?*. Learners answered by choosing one out of 5 possible items on a
discrete Likert type scale, from “absolutely not”, to “absolutely yes”.

The BROMP protocol was employed by researchers trained in coding indi-
cators of engagement in videos, at the end of the study.

4 Results

4.1 Engagement Differences (R1)

Both the group with tangibles and the group with pencil-and-paper reported
about their engagement in GAT via the again-and-again instrument. Their an-
swers were aggregated into: non-positive, indicating lack of engagement, in case
of “absolutely not”, “probably not” and “neuter”; positive, indicating engagement, in case of “probably yes” and “absolutely yes”. Table 4 shows the proportions of answers per item of the scale, divided per group, and the proportions of positive answers.

<table><caption>Table 4. Again-and-again question answers</caption><thead><tr><th>Answers</th><th>pencil-and-paper</th><th>tangibles</th></tr></thead><tbody><tr><td>absolutely not</td><td>1</td><td>0</td></tr><tr><td>maybe not</td><td>0</td><td>0</td></tr><tr><td>neuter</td><td>10</td><td>0</td></tr><tr><td>maybe yes</td><td>4</td><td>3</td></tr><tr><td>absolutely yes</td><td>0</td><td>5</td></tr><tr><td>Total</td><td>15</td><td>8</td></tr><tr><td>Positive</td><td>4</td><td>8</td></tr></tbody></table>

Then 95%-Confidence Intervals (95%-CIs) were calculated by using the Adjusted Wald method around proportions of positive answers out of all given answers. Results are different per group:

- 95%-CI[0.1, 0.52] for the pencil-and-paper-group,

- 95%-CI[0.7, 1] for the tangible-group.

Since the CIs of the positive answers between the two groups do not overlap, there is 95% confidence about a significant difference between the two groups' engagement [49].

Given such strong engagement results for the group of learners using GAT tangibles, two researchers analysed the videos of the learners' tasks with GAT tangibles, in order to confirm, or not, the self-report findings, and to explore further opportunities for engaging learners in GAT. How they proceeded and results of the analysis are reported next.

## 4.2 Task-related Engagement (R2)

Videos were segmented and annotated by considering tasks of GAT scenarios and the BROMP engagement indicators. The usage of BROMP was briefly motivated in Section 2. Its usage in the study is explained in details as follows.

An on-task behaviour refers to a student who is doing what he or she is supposed to be doing, whereas an off-task behaviour refers to the opposite case. It is also possible to make other distinctions and those relevant for the study are listed in Table 5. BROMP generally uses a dual coding scheme, which records the behaviour simultaneously, but separately from affect. The affective indicators commonly used are recapped in Table 6.

Two researchers, both trained in coding, worked independently on a set of videos of learners, not in the tangible-group, with GAT tangibles. For each
<table><caption>**Table 5.** BROMP behavioural indicators of (dis)engagement</caption><thead><tr><th>On-task</th><th>Description</th></tr></thead><tbody><tr><td><em>Conversation</em></td><td>who is working toward his assignment while having conversation with another learner or teacher about the learning task.</td></tr><tr><td><em>Help seeking</em></td><td>who has paused work, but only because she/he is seeking help from another learner/teacher.</td></tr><tr><td><em>Proactive Remedia-<br>tion</em></td><td>teacher receives information on learner progress and intervenes to work with the learners.</td></tr><tr><th>Off-task</th><th>Description</th></tr><tr><td><em>Aggression</em></td><td>who is off task and behaving in a threatening manner towards another learner.</td></tr><tr><td><em>Passive</em></td><td>who is off task but not interacting with anybody.</td></tr><tr><td><em>Social</em></td><td>who interacting with peers not about the task.</td></tr></tbody></table>

learner and task, they coded behavioural and affective indicators, simultaneously but independently.

They compared their results, and agreed on the “work” indicator for a learner interacting directly with the GAT material (e.g., tangibles), or observing other learners doing it. They also merged “conversation”, “help seeking” and “proactive remediation” into “conversation”, given that they were not easy to distinguish. All other indicators were maintained as in Tables 5 and 6.

<table><caption>**Table 6.** BROMP affective state indicators of (dis)engagement</caption><thead><tr><th>Positive</th><th>Description</th></tr></thead><tbody><tr><td><em>Engaged concentration</em></td><td>who is paying focused attention to his/her primary current task.</td></tr><tr><td><em>Delight</em></td><td>who is smiling or otherwise indicating that they are having a pleasurable experience.</td></tr><tr><td><em>Surprise</em></td><td>when posture, facial expressions, or vocal expressions indicate that a previous affective state was interrupted unexpectedly.</td></tr><tr><th>Negative</th><th>Description</th></tr><tr><td><em>Confusion</em></td><td>who looks like they are having difficulty understanding the learning material/tasks.</td></tr><tr><td><em>Boredom</em></td><td>who appears to find the task dull or tedious.</td></tr><tr><td><em>Frustration</em></td><td>who presents feelings of distress or annoyance.</td></tr></tbody></table>

Thereby the two researchers proceeded, independently, to code all videos of the tangible-group of the field study. Finally, they compared their results. Their percentage agreement was calculated (83%) and differences were resolved through discussion with a third researcher. No off-task behaviour or negative affective indicators emerged. All on-task behaviour indicators emerged. As for the affective indicators, “engaged concentration” and “delight” emerged.
Table 7 reports the coding of all tasks with GAT tangibles by means of engagement indicators, behavioural and affective, and the related percentages of occurrence. In particular, 39.17% of tasks were coded as “work”. The majority of tasks, namely, 57.50%, were coded specifically as “work with conversation”, a novel indicator which emerged with GAT tangibles. This denotes a collaborative conversation-based activity, e.g., a group of learners spontaneously started a discussion on the strategies for creating a spanning tree. Moreover, conversation opportunities were coded in almost all modelling, design or exploration tasks, whereas they did not emerge in understanding tasks, which apparently required more individual reflections.

As for affective indicators, 84.17% of tasks were coded as “engaged concentration”, and 15.83% as “delight”. Delight indicators were most difficult to trace in videos. However, they emerged mainly when feedback through LEDs was given.

<table><caption>**Table 7.** Indicators for coded tasks with GAT tangibles and occurrence percentages</caption><thead><tr><td>Indicator for coded tasks with GAT tangibles</td><td>Percentage</td></tr></thead><tbody><tr><td>work</td><td>39.17%</td></tr><tr><td>work with conversation</td><td>57.50%</td></tr><tr><td>engaged concentration</td><td>84.17%</td></tr><tr><td>delight</td><td>15.83%</td></tr></tbody></table>

Other interesting observations were traced by researchers in their own notes while coding. In particular, it emerged that, when learners were designing and exploring strategies for building trees, the teacher sometimes had to stop them: the novelty and ease of use of tangibles tended to make learners rush through tasks of scenarios without deeply reflecting on strategies. Moreover, learners were able to see all nodes and edges arranged on tables and, based on this global view, they tended to construct a minimum spanning tree, without reflecting about a step-by-step strategy, unless invited by the teacher to do so.

# 5 Discussion

This section reflects on results of the mixed-method research study, reported above, in relation to the two research questions, gravitating around engagement in GAT: (RQ1) differences in engagement among groups, with either pencil and paper, or tangibles; (RQ2) task-related engagement indicators and what may be reasons for them.

Results concerning engagement in the entire GAT experience and differences among groups (RQ1) were collected through a self-report standardised instrument, asking whether learners would repeat the experience. Statistical analyses show that engagement was different between the pencil-and-paper group and the tangible group. See Table 4. Note that only 4 out of 15 in the pen-and-paper group were positive (“maybe yes”), whereas 8 out of 8 of the tangible were positive
or very positive (“maybe yes”, “absolutely yes”). As remarked by an anonymous reviewer of this paper, it is also interesting that only 1 out of 15 in the pencil-and-paper, and none in the tangible group were absolutely negative (“absolutely no”). Such result suggests that the GAT topic per se was interesting and the usage of tangibles seems to have played a role in engaging learners differently.

In particular, there is a statistically significant difference among the two groups, with positive results for the tangible-group. In other words, GAT turned out to be significantly more engaging with tangibles than with pencil-and-paper, in spite of the period in which it was proposed—the end of the school year, right before the start of a long summer break.

Given such results, videos of the tangible-group were analysed and coded, so as to trace reasons for the superior engagement results and answer the second research question (RQ2). Coding looked for (dis)engagement indicators, both behavioural and affective, recapped in Table. The video analysis of the tangible-group confirmed results about engagement self-reports. It also indicates what might have mostly engaged learners, as discussed in the following.

First of all, the video analysis highlighted that the use of GAT tangibles enabled learners to reflect on the tasks they were carrying out, mainly through conversations with others, in line with socio-constructivism for learning which GAT tangibles refer to. For instance, learners spontaneously started a discussion on the strategies for creating a spanning tree. Moreover, conversations were coded in modelling, design or exploration tasks, whereas they did not emerge in understanding tasks, which apparently relied on individual reflections.

The video analysis also highlighted that learners from the tangible-group were mostly engaged with concentration, and show delight expressions especially when LED or sound feedback was given in response to their actions. However, notes by coders also reveal an aspect which deserves attention in a learning context: when learners were designing and exploring strategies for trees, the teacher sometimes had to stop them, because they tended to rush through scenarios and manipulate tangibles. Such results deserves reflections for the future design of tangibles for GAT, which are presented in the conclusions.

# 6 Conclusions

This paper reports on research related to the usage of tangibles for fostering AT, part of CT, in schools. The research in the paper focuses on AT for graphs, namely, GAT. Grounded on socio-constructivism and soliciting a multi-modal experience, it supports a novel approach to GAT for schools, which requires the usage of GAT tangibles with companion scenarios for different learning goals. It stems from past action-based research, which helped design GAT scenarios and develop GAT tangibles of the study.

The study was conducted with 14–15 years old learners, from the first year of a technical high school, and their teacher. No learners knew of graphs or GAT before participating in the study, albeit they understood the concept of algorithm. All learners performed the same GAT scenarios, which consider different
GAT skills, ranging from the understanding of basic graph properties to the exploration of strategies for solving problems with graphs. However, the participant learners were split into two groups, one using GAT tangibles and the other instead using pencil-and-paper material. Data processing adopted a mixed-method research approach so as to investigate the main research question, that is, what learners' engagement in GAT was. Data were processed by means of two different instruments: the standardised again-and-again self-report instrument, asking whether learners would like to repeat the GAT experience; observations in videos of learners in GAT, lately coded by adopting and slightly adapting engagement indicators, standard in the technology-education literature.

Self-reported data were analysed with SPSS and compared between groups, with and without GAT tangibles. Results show differences between groups. The group working with tangibles tended to have higher levels of engagement with the experience than the other group: differences for engagement are statistically significant. Such results were then triangulated with results of the analysis of observations, related to learners' engagement in tasks with GAT tangibles, so as to understand possible reasons for the strongly positive engagement results for GAT with tangibles. It emerged that scenarios with GAT tangibles spontaneously led students to move and engage in conversations, whereas paper-and-pencil based scenarios did not naturally lead students to interact and collaborate. That can partly explain differences in engagement.

The paper concludes acknowledging the limitations of the reported research, related to learners’ engagement with GAT, and with lessons for future work related to GAT, both from the education researchers’ and practitioners’ perspective, and the perspective of Human Computer Interaction (HCI).

## 6.1 Limitations of the Work

The contextual nature of the reported work, and the small number of involved participants affect the generality of its results. However, the intervention at school was described with contextual factors, so as to make it possible to replicate it in other different contexts. Further detailed information on the study protocol, and the data collection instruments are publicly available in a report [12].

Furthermore, the employed tangibles per se might have also placed constraints. The limited number of available tangibles affected the number of learners using them in the study. The use of tangibles, and the time spent every time before using them (e.g., connecting and activating edges) also constrained the time-span of the GAT intervention within the educational context.

However, results of learners with GAT tangibles indicate their strong engagement with respect to results of learners using pencil and paper, advocating for interactive multi-sensory experiences for teaching GAT, as discussed in the remainder of this paper.
## 6.2 Recommendations for Future Work

In view of the results concerning the group using GAT tangibles, in the future, education researchers and teachers could consider collaborative approaches to teaching GAT through tangible artefacts, which stimulate conversations, physical manipulations and movement, as GAT tangibles do. In fact GAT tangibles seem to have played a role in engaging learners in work-related conversations in all tasks which go beyond pure understanding of basic concepts (e.g., graphs and trees), namely, in modelling, designing or exploring (e.g., modelling a problem with a graph and designing a strategy concerning how to find a spanning tree).

HCI researchers could consider the fact that the number of GAT tangibles made learners approach tasks related to (spanning) trees without thinking strategically. Future work may consider adopting a larger number of nodes and edges. Such a choice might induce learners to appreciate the importance of a step-by-step general strategy, as opposed to intuitive choices on an ad-hoc basis.

Although this paper did not focus on learning, learning and engagement are often correlated. For instance, Gennari et al. show that a high level of engagement, related to self-reported affective indicators, significantly correlated to high learning performances, related to the quality of children's products, part of the learning activity [29]. On-going work of authors of this paper is considering learning and engagement results, reported in this study, and studying their correlations.

Last but not least, this paper considered observable indicators of engagement with technology-mediated tasks, part of the technology education literature. The original indicators were slightly adapted and made evolve, as documented in the paper, by considering the specific context of tangibles for GAT, which is collaborative and relying on physical objects to move. Future work may apply such indicators in similar contexts. Interestingly, as pointed out by an anonymous reviewer of this manuscript, similar engagement indicators were considered in the HCI literature, in the work by Nasir and colleagues [37]. Their results point to at least two distinct multi-modal behavioural patterns which indicate “high learning in constructivist, collaborative activities”. Future work can investigate whether their specific patterns are correlated to engagement indicators found across GAT tasks, reported in this paper.

**Acknowledgements.** Authors would like to thank anonymous reviewers, who helped reflect on the paper and improve on it. They are grateful to V. del Fatto, for helping with coding, G. Mahlknecht, the teacher of the school in Merano, who also collaborated on the co-creation of scenarios, besides all participant learners.

## References

1. Aggarwal, A., Gardner-McCune, C., Touretzky, D.S.: Evaluating the effect of using physical manipulatives to foster computational thinking in elementary school. In: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer
Science Education. pp. 9–14. SIGCSE ’17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3017680.3017791

2. Appleton, J.J., Christenson, S.L., Kim, D., Reschly, A.L.: Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology 44(5), 427 – 445 (2006). https://doi.org/https://doi.org/10.1016/j.jsp.2006.04.002, motivation

3. Athanasiou, L., Topali, P., Mikropoulos, T.A.: The use of robotics in introductory programming for elementary students. In: International Conference EduRobotics 2016. pp. 183–192. Springer (2016)

4. Barendsen, E., Mannila, L., Demo, B., Grgurina, N., Izu, C., Mirolo, C., Sentance, S., Settle, A., Stupurienė, G.: Concepts in k-9 computer science education. In: Proceedings of the 2015 ITiCSE on Working Group Reports. pp. 85–116. ITICSE-WGR ’15, ACM, New York, NY, USA (2015). https://doi.org/10.1145/2858796.2858800

5. Bargury, I.Z., Muller, O., Haberman, B., Zohar, D., Cohen, A., Levy, D., Hotoveli, R.: Implementing a new computer science curriculum for middle school in israel. In: Frontiers in Education Conference (FIE), 2012. pp. 1–6. IEEE (2012)

6. Bell, T. Alexander, J., Freeman, I., Grimley, M.: Computer science without computers: new outreach methods from old tricks. In: Proc. of the 21st Annual Conference of the National Advisory Committee on Computing Qualifications (2008)

7. Bers, M.U.: The tangiblek robotics program: Applied computational thinking for young children. Early Childhood Research & Practice 12(2), n2 (2010)

8. Bocconi, S., Chioccariello, A., Dettori, G., Ferrari, A., Engelhardt, K., Kampylis, P., Punie, Y.: Developing computational thinking in compulsory education. European Commission, JRC Science for Policy Report (2016). https://doi.org/10.2791/792158

9. Bocconi, S., Chioccariello, A., Dettori, G., Ferrari, A., Engelhardt, K., Kampylis, P., Punie, Y.: Developing computational thinking: Approaches and orientations in k-12 education. In: EdMedia: World Conference on Educational Media and Technology. pp. 13–18. Association for the Advancement of Computing in Education (AACE) (2016)

10. Bonani, A., Del Fatto, V., Dodero, G., Gennari, R., Raimato, G.: First Steps Towards the Design of Tangibles for Graph Algorithmic Thinking, pp. 110–117. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-60819-8_13

11. Bonani, A., Del Fatto, V., Dodero, G., Gennari, R., Raimato, G.: Participatory Design of Tangibles for Graphs: A Small-Scale Field Study with Children, pp. 161–168. Springer International Publishing, Cham (2018). https://doi.org/10.1007/978-3-319-61322-2_16

12. Bonani, A. and Del Fatto, V. and Gennari, R.: Interactive Objects for the Scaffolding of Graph Algorithmic Thinking at School. https://doi.org/10.13140/RG.2.2.34628.37760, (Last accessed: 14 January 2021)

13. Brondino, M., Dodero, G., Gennari, R., Melonio, A., Pasini, M., Raccanello, D., Torello, S.: Emotions and inclusion in co-design at school: Let's measure them! Advances in Intelligent Systems and Computing **374**, 1–8 (2015). https://doi.org/10.1007/978-3-319-19632-9_1

14. Capovilla, D., Krugel, J., Hubwieser, P.: Teaching algorithmic thinking using haptic models for visually impaired students. In: Learning and Teaching in Computing and Engineering (LaTiCE), 2013. pp. 167–171. IEEE (2013)
15. Christenson, S.L., Reschly, A.L., Wylie, C.: Handbook of research on student engagement. Springer Science & Business Media (2012)

16. Collaborative Education Lab: Learning scenarios, available at: http://colab.eun.org/learning-scenarios (Last accessed: 14 January 2021)

17. Corradini, I., Lodi, M., Nardelli, E.: Conceptions and misconceptions about computational thinking among italian primary school teachers. In: Proceedings of the 2017 ACM Conference on International Computing Education Research. pp. 136–144. ICER ’17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3105726.3106194

18. Duncan, C., Bell, T.: A pilot computer science and programming course for primary school students. In: Proceedings of the Workshop in Primary and Secondary Computing Education. pp. 39–48. ACM (2015)

19. Eisenberg, M., Elumeze, N., MacFerrin, M., Buechley, L.: Children’s programming, reconsidered: settings, stuff, and surfaces. In: Proceedings of the 8th International Conference on Interaction Design and Children. pp. 1–8. ACM (2009)

20. European Commission: New Skills Agenda for Europe. Available at: https://ec.europa.eu/social/main.jsp?catId=1223, (Last accessed: 14 January 2021)

21. European Schoolnet: Computing our future. European Schoolnet (2015), available at: http://fcl.eun.org/documents/10180/14689/Computing+$+our+$+future_final.pdf

22. Figg, C., Jamani, K.J.: Exploring teacher knowledge and actions supporting technology-enhanced teaching in elementary schools: Two approaches by pre-service teachers. Australasian journal of educational technology **27**(7) (2011)

23. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: Potential of the concept, state of the evidence. Review of Educational Research **74**(1), 59–109 (2004). https://doi.org/10.3102/00346543074001059

24. Fredricks, J.A., McColskey, W.: The Measurement of Student Engagement: A Comparative Analysis of Various Methods and Student Self-report Instruments, pp. 763–782. Springer US, Boston, MA (2012). https://doi.org/10.1007/978-1-4614-2018-7-37

25. Futschek, G., Moschitz, J.: Learning algorithmic thinking with tangible objects eases transition to computer programming. In: Informatics in Schools. Contributing to 21st Century Education, pp. 155–164. Springer (2011)

26. Futschek, G.: Algorithmic thinking: the key for understanding computer science. In: Informatics education–the bridge between using and understanding computers, pp. 159–168. Springer (2006)

27. Gennari, R., Melonio, A., Rizvi, M.: The participatory design process of tangibles for children’s socio-emotional learning. In: Proc. of the 6th International Symposium on End-User Development, IS-EUD 2017. vol. 10303 LNCS, pp. 167–182 (2017). https://doi.org/10.1007/978-3-319-58735-6_12

28. Gennari, R., Melonio, A., Rizvi, M.: From turntalk to classtalk: the emergence of tangibles for class conversations in primary school classrooms. Behaviour & Information Technology pp. 1–20 (2019)

29. Gennari, R., Melonio, A., Raccanello, D., Brondino, M., Dodero, G., Pasini, M., Torello, S.: Children’s emotions and quality of products in participatory game design. International Journal of Human-Computer Studies **101**, 45–61 (2017)

30. Grover, S., Pea, R.: Computational thinking: A competency whose time has come. Computer Science Education: Perspectives on Teaching and Learning in School p. 19 (2018)
31. Hourcade, J.P.: Interaction design and children. Foundations and Trends® in Human-Computer Interaction 1(4), 277-392 (2008). https://doi.org/10.1561/1100000006

32. Lee, I., Martin, F., Denner, J., Coulter, B., Allan, W., Erickson, J., Malyn-Smith, J., Werner, L.: Computational thinking for youth in practice. ACM Inroads 2(1), 32–37 (2011)

33. Maloney, J., Resnick, M., Rusk, N., Silverman, B., Eastmond, E.: The scratch programming language and environment. ACM Transactions on Computing Education (TOCE) 10(4), 16 (2010)

34. Mascio, T., Gennari, R., Tarantino, L., Vittorini, P.: Designing visualizations of temporal relations for children: Action research meets hci. Multimedia Tools Appl. 76(4), 4855-4893 (Feb 2017). https://doi.org/10.1007/s11042-016-3609-6

35. Miur & CINI: Programma il futuro. Available at: https://programmailfuturo.it/chi-siamo, (Last accessed: 14 January 2021)

36. Moreno, R., Mayer, R.: Interactive Multimodal Learning Environments. Educational Psychology Review 19(3), 309-326 (2007). https://doi.org/10.1007/s10648-007-9047-2

37. Nasir, J., Bruno, B., Dillenbourg, P.: Is there 'one way' of learning? a data-driven approach. In: Companion Publication of the 2020 International Conference on Multimodal Interaction. p. 388–391. ICMI '20 Companion, ACM, New York, NY, USA (2020). https://doi.org/10.1145/3395035.3425200

38. Ocumpaugh, J., Baker, R., Rodrigo, M.: Monitoring protocol (BROMP) 2.0 technical & training manual. New York, NY: Teachers college (2015)

39. Parmentier, Y., Reuter, R., Higuet, S., Kataja, L., Kreis, Y., Duflot-Kremer, M., Laduron, C., Meyers, C., Busana, G., Weinberger, A., et al.: Piaf: Developing computational and algorithmic thinking in fundamental education. In: EdMedia+Innovate Learning. pp. 315–322. Association for the Advancement of Computing in Education (AACE) (2020)

40. Pasterk, S., Bollin, A.: Digital literacy or computer science: Where do information technology related primary education models focus on? In: 2017 15th International Conference on Emerging eLearning Technologies and Applications (ICETA). pp. 1–7 (2017). https://doi.org/10.1109/ICETA.2017.8102517

41. Pasterk, S., Sabitzer, B., Demarle-Meusel, H., Bollin, A.: Informatics-lab: Attracting primary school pupils for computer science. In: Proc. of LACCEI Int. Multi-Conference for Engineering, Education, and Technology, San José, Costa Rica (2016)

42. Pawlowski, J., Clements, K., Dimitrakopoulou, D., Idzik, M., Muilu, M., Pilv, M., Sotiriou, S.: Computational thinking and acting: An approach for primary school competency development. In: CEUR Workshop Proceedings. RWTH Aachen (2020)

43. Peng, H.: Algo.rhythm: Computational thinking through tangible music device. In: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction. pp. 401–402. TEI ’12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2148131.2148234

44. Preece, J., Sharp, H., Rogers, Y.: Interaction design: Beyond human-computer interaction. Wiley (2019)

45. Read, J.C., MacFarlane, S.: Using the fun toolkit and other survey methods to gather opinions in child computer interaction. In: Proceedings of the 2006 Conference on Interaction Design and Children. pp. 81–88. IDC ’06, ACM, New York, NY, USA (2006). https://doi.org/10.1145/1139073.1139096
46. Reinhard Pekrun, M.B.: Self-Report Measures of Academic Emotions, chap. 28. Routledge (2014). https://doi.org/10.4324/9780203148211.ch28

47. Reschly, A.L., Christenson, S.L.: Jingle, Jangle, and Conceptual Haziness: Evolution and Future Directions of the Engagement Construct, pp. 3–19. Springer US, Boston, MA (2012). https://doi.org/10.1007/978-1-4614-2018-7_1

48. Root, E., Steinkamp, M., Coldewey, B., Poloczek, C., Scharnowski, F., Kettner, M., Koelle, M., Ananthanarayan, S., Willms, M.: Grasping algorithms: Exploring toys that teach computational thinking. In: Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia. pp. 387–392. MUM ’17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3152832.3156620

49. Sauro, J., Lewis, J.R.: Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann (2016)

50. Schneider, B., Jermann, P., Zufferey, G., Dillenbourg, P.: Benefits of a tangible interface for collaborative learning and interaction. IEEE Transactions on Learning Technologies 4(3), 222–232 (July 2011). https://doi.org/10.1109/TLT.2010.36

51. Skinner, E.A., Pitzer, J.R.: Developmental Dynamics of Student Engagement, Coping, and Everyday Resilience, pp. 21–44. Springer US, Boston, MA (2012). https://doi.org/10.1007/978-1-4614-2018-7_2

52. Soute, I., Markopoulos, P., Magielse, R.: Head up games: combining the best of both worlds by merging traditional and digital play. Personal and Ubiquitous Computing 14(5), 435–444 (2010). https://doi.org/10.1007/s00779-009-0265-0

53. Syslo, M.M., Kwiatkowska, A.B.: Introducing a new computer science curriculum for all school levels in poland. In: International Conference on Informatics in Schools: Situation, Evolution, and Perspectives. pp. 141–154. Springer (2015)

54. Wang, D., Wang, T., Liu, Z.: A tangible programming tool for children to cultivate computational thinking. The Scientific World Journal 2014 (2014). https://doi.org/10.1155/2014/428080

55. Wing, J.M.: Computational thinking. Communications of the ACM 49(3), 33–35 (2006)

56. Yu, J., Zheng, C., Tamashiro, M.A., Gonzalez-millan, C., Roque, R.: Codeattach: Engaging children in computational thinking through physical play activities. In: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 453–459 (2020)