# An Ontology to Formalize a Creative Problem Solving Activity

Chloé Mercier

<figure><img src="image_1.png" /></figure>

## To cite this version:

Chloé Mercier. An Ontology to Formalize a Creative Problem Solving Activity. IEEE Transactions on Cognitive and Developmental Systems, 2022, pp.1-1. 10.1109/tcds.2022.3210234. hal-03791482

HAL Id: hal-03791482

https://inria.hal.science/hal-03791482

Submitted on 29 Sep 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# An Ontology to Formalize a Creative Problem Solving Activity

Chloé Mercier

*Abstract*—Our study is set in an educational context: to better teach and assess twenty-first century skills, such as computational thinking or creative problem solving, we propose to formalize a specific activity that involves these competencies. This activity, referred to as #CreaCube, is presented as an open-ended problem which consists of assembling a set of robotic cubes into an autonomous vehicle. We not only anchor our formalization in classical learning science frameworks but we also propose to draw on neuro-cognitive models to describe the observed behaviors of learners engaged in this activity. The chosen formalism is symbolic and is aligned on upper ontologies to ensure that the vocabulary is well specified. This allows for a better communication between the summoned research fields, namely learning science, cognitive neuroscience and computational modeling. Beyond this specification purpose, we suggest performing inferences using available reasoners to better guide the analysis of the observables collected during the experiments. This operationalization of a creative problem-solving activity is part of an exploratory research action. In addition, an effective proof of concept is described in this study.

*Index Terms*—Cognitive Neuroscience, Ontology, Problem-Solving, Learning Science, Computational Thinking

## I. INTRODUCTION

We propose to formalize creative problem solving using symbolic knowledge representation to set up the basis of learner modeling in the precise context of a specific activity, by representing shareable and reusable knowledge across three domains: learning science, cognitive neuroscience and computational modeling. The task that is reviewed here is presented as an open-ended problem that aims to initiate the participants to computational thinking and invites them to solve a problem appealing to creativity using tangible artifacts. Let us first review the context and research objectives, to define the competencies we aim to better understand and justify the choice of the activity before highlighting the related challenges of such a formalization.

*Educational Context:* Transversal competencies, which are sometimes referred to as “twenty-first century skills”, include (but are not limited to) creativity, problem solving and computational thinking [1]. Integrating these skills in the K-12 curricula is a key challenge in today’s learning science field [1], [2]. We postulate that teaching and assessing transversal competencies could benefit from a better understanding of the learners’ behaviors in specific activities that require these competencies. These skills are intrinsically related to each other, which leads to interdependent definitions. In our context, **creativity** refers to the process of producing something new and adequate with regard to the task being performed (e.g., a problem being solved) [3]–[5]. **Problem solving** has been

formalized at a computational level [6] and defined as finding a path from an initial state (the current situation) to a final state (the goal), while taking into account some constraints (e.g., task requirements). However, problem solving in everyday life takes many different forms, depending on the task and context. Realistic problems are most often ill-defined, as we will further clarify in this study. **Computational thinking** uses tools inspired by the field of computer science (e.g., algorithms, information coding and data representation) to (for example) solve problems or design systems [7] (see also [8] for a recent review). By extension, computational thinking also includes technology literacy. Furthermore, techno-creative activities [9] have the potential to develop computational thinking, while they should also engage learners in critical thinking [2]. We have chosen to focus on an activity involving robotic artifacts, as developed in the sequel.

These transversal competencies also need to be considered in relation to (meta-)cognitive processes. The awareness of one’s own thought processes during learning is key not only to solve ill-defined problems but also to be able to transfer skills from a given problem-solving task to another [10]. Here, we hypothesize that better formalizing and understanding the learner upstream to meta-learning may contribute to better pedagogical resource design and help teach this critical awareness of one’s own learning process.

*Ill-defined Problem Solving:* Research on problem solving has focused on a diversity of tasks but most of all on well-defined problems, such as the chess game or the Tower of Hanoi, which allows us to define task models and problem-solving methods. In short, the research objective in problem solving is to represent the problem space, including the positions of the initial state and of the desired goal (possibly a region of the space if the goal is not defined by a unique position but specified through instructions), and to define a path between the positions. This can be done in both forward and backward directions, where knowledge can provide rules for navigating the space while taking constraints into account, and requires exploration within this space when considering open problems, as already stated in the pioneering work of [6]. This also can be solved at a numerical level which is biologically more plausible (see, e.g., [11]). Nonetheless, problem solving in everyday life is typically ill-defined in the sense that the goal is only partially defined or may have to be changed throughout the resolution, the initial state is only partially observable, and the problem constraints are discovered on the way towards the solution. Consequently, solving these problems requires us to consider not only prior knowledge but also the contextual knowledge that is built throughout the task process based on the different operations
that the subject performs [12]. Moreover, the exploration of
the means given to solve the problem should facilitate the
generation of new stimuli, which can then help the learner to
make progress towards the goal by fulfilling sub-goals and
validating or invalidating hypotheses. This is exactly what
we would like to consider here. In particular, we aim to
consider the problem-solving process instead of considering
only performance scores of the task. Therefore, we need to
take into account the set of knowledge required to solve a
particular task and how it is managed. We hypothesize that
ontology representation is a powerful and relatively accessible
formalism to this end.

**Learning with Tangible Objects:** At a more concrete level, we aim to study creative problem solving through screen-less activities, also known as unplugged activities [13]; for instance, using tangible artifacts, such as pedagogical robots or connected objects [14], or even every day objects. This paradigm has demonstrated its efficiency, especially for computational thinking initiation [13]. However, in contrast to online activities on a computer, a non-trivial setup is required to track the learning process through the activity. One of the challenges is the use of efficient measurement devices of reasonable cost. It also appears that a precise model of the task allows us to define specific observables that are thus more robust to estimate [14]. Another obstacle is the bias that is induced by the fact that the learner is observed and monitored. One solution is to take this as a chance to involve the learner in their own learning process, which is the case with Open Learner Models [15]. This is yet to be properly studied in this context, see [16] for a discussion on these points. Ethical issues of collecting human data in a way that ensures data privacy also needs to be considered, to ensure that only required data is effectively collected. For all these reasons, the construction of a precise model of the task and of the learner involved in this task is a lever to take up this challenge. Using tangible material allows us to observe behavioral patterns [14] which, if properly characterized, may help us infer cognitive processes.

**The Observation Relevance Challenge:** Properly characterizing observable behavioral patterns can be difficult because we are working on a relatively small batch of data (around a hundred individuals, compared to the thousands of data used in classical statistical methods). For the results to be meaningful, we propose to introduce prior information upstream to the analysis of the observables, which results in highly structured data. This has led us to choose ontology modeling, which is an appropriate tool to describe structured knowledge. This prior information can come from existing theoretical models in the learning sciences.

**The Learner Modeling Challenges:** This model is intended to be applicable to the observables and learning analytics, and to allow us to interpret them. Indeed, phenomenological models already exist — for example, the activity theory [17] provides a framework that is appropriate to the context of this study: it specifies an activity as performed by a subject through the mediation of a tool to achieve a certain outcome. Based on such phenomenological frameworks, our present study is positioned at an operational level. We also hypothesize that such an operationalization could benefit from a multi-

disciplinary perspective and we thus draw on behavioral mod-
els taken from neuro-cognitive science.

The finality of developing a computational model of a computational thinking task [1] aims to contribute, as a long-term perspective, to a better understanding, at a very concrete level, of the fundamental notions regarding natural versus artificial so-called intelligence, as studied in [2], including a better understanding of our human intelligence. We focus on a specific learning task instead of considering human learning at a general level to offer a precise and operational formalization that is translatable to other learning tasks.

This paper is an extension of a work that was originally
reported in Proceedings of the 2021 IEEE International Con-
ference on Development and Learning (ICDL) [18].

II. RELATED WORK

*Ontologies to Model Learning Activities:* A recent sys-
tematic review [19] of ontology use in education has observed
a growing trend in the contribution of ontologies to educational
systems in the last five years, and has identified the following
common use cases:

* Describing learning domains,

• Describing learner data,

* Modeling and managing curricula,

• Describing learning services.

In our case, we are particularly interested in works that attempt
to model both the learning domain and the learner data. This
has been done especially in intelligent tutoring systems and
adaptive learning systems, as reviewed in [20], where semantic
inference is performed on learning analytics and the learning
content to tune the resources that are presented to the learner
(e.g., in [21]). These studies are inspiring but they differ from
our approach in some important aspects.

Most reviewed approaches attempt to give a general model
the learner (i.e., in a diversity of learning tasks). From that
perspective, learner modeling is often directed towards classi-
fication of learners into several learner profiles with regard
to a set of characteristics, and by doing so to select the
more adequate resources for a given learner (e.g., according
to learning styles in [22]). This is also the case for knowledge
representation beyond ontology approaches, such as those
related to the xAPI and game-based learning profiles [23].
The cognitive trajectories introduced by [24] are an interesting
approach to better understand the learner behavior, at a general
level or in a specific task.

Another point is that a large majority of these tasks are well
defined (e.g., well defined knowledge or how-to acquisition),
whereas we consider here a more open ill-defined problem
solving task. We can, however, cite the work of the Ludo Game
Model Ontology (LUDO) [25] to describe serious games.
These activities can be presented as open problems, and
(despite being not really reusable here, because it does not
involve tangible artifacts) the ontology itself includes both a
model of the game (task) and a model of the player (learner)
behavior.

Eventually, the goal of the existing ontologies is to improve
the learner's performance [20], which is less easy to define in
our problem solving context (solving the problem faster does not mean that we learn a lot from it) and is not our main purpose, which is to better *understand* how we solve such an open-ended problem.

With respect to these works, we introduce here computational and cognitive neuroscience knowledge in the modeling process, which does not seem to be a current practice, and we also consider open-ended problem solving, which is seldom addressed.

*Ontologies in Neuroscience and Cognitive Science:* The Cognitive Atlas<fnref n="1" /> [26] is a collaborative initiative that aims to unify the vocabulary used in cognitive science and indexes cognitive tasks described in the literature. It includes over 800 cognitive concepts but does not encompass much logic, aiming for a more flexible approach. Its task model is partially based on CogPO [27], which is another ontology that focuses on the description of experimental paradigms used in neuroscience (mainly fMRI) tasks. While CogPO might be slightly too specific and medical-oriented for our problem, the Cognitive Atlas encloses many of the cognitive concepts that we may need to refer to in subsequent work. It also seems that the concepts we target here are not (to the best of our knowledge) addressed at the level of ontology modeling.

Beyond ontology-based methods, a wide spectrum of approaches, out of the scope of the present work, have been recruited to model ill-defined creative problem solving; for instance, using description logic, extended with probabilistic approaches [28]. This example is interesting because the authors have made the distinction between different types of knowledge and memories, and considered its integration in a cognitive architecture [29].

*Considering Foundational Ontologies:* A good practice in ontology modeling is to base our model on a foundational ontology to better specify the design choices by relating our concepts to well-established formalized semantics and facilitate the integration with other ontologies. Neuroscience ontologies often rely on recommendations set by the Open Biological and Biomedical Ontology (OBO) Foundry, which is itself built on the principles of Basic Formal Ontology (BFO), as is the case for CogPO. However, the ontological choices made in BFO are meant to represent real-world elements and not, for example, mental states. Integration to biomedical ontologies is not necessarily a requirement for us, and the Cognitive Atlas, in that matter, is rather agnostic; therefore, a more appropriate choice for us would likely be the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) [30] which is general enough to be linked with both the cognitive and learning science aspects of our formalization work. In brief, it is a descriptive ontology of particulars that “aims at capturing the ontological stands that shape natural language and human cognition, based on the assumption that the surface structure of natural language and the so-called commonsense have ontological relevance” [30], which may better fit which our paradigm.

### III. MATERIAL AND THEORETICAL FRAMEWORK

#### A. *The #CreaCube Activity*

This task has been designed considering the cognitive science literature regarding problem solving tasks [6], [31], but addressing here considerations regarding ill-defined problems.

The #CreaCube study<fnref n="2" /> focuses on problem-solving strategies using modular robotic cubes, targeting children between 8 and 12 years old. The task is explained to the participants as “building an autonomous vehicle, composed of four items, able to move from a point A to another point B”, while a set of four modular robotic cubes that differ in their appearances (e.g., different colors) and features (e.g., wheels, switch etc.) is given. The learner is expected to understand that the four items are the four cubes on the table, to discover their behavior, and to assemble these cubes into a certain configuration that will be able to move autonomously. It is worth noting that this problem does not have a unique solution because several configurations may satisfy the goal requirements. Furthermore, additional goals may emerge, such as understanding the cubes, exploring alternative outcomes with those items, preserving the ego with respect to the task achievement, experiencing playful pleasure, and so on. The task is relatively easy in the sense that most participants come up with a solution in less than 15 minutes but complex to model because of the large problem space and the lack of specification (e.g., the possible actions, i.e operators on the problem space states, are not clearly stated) [12], which makes it an ill-defined problem.

In this activity, the participant has likely no prior experience of the modular robotic cubes. To complete the task, there is thus a need for exploration, defined here from a learning science perspective, as a way to gather information from the environment [32]. Throughout the task, the participant moves towards the goal by mobilizing prior knowledge that appears to make sense in relation to their experience with the material being manipulated and the requirements defined by the guideline. This includes understanding the artifacts, their physical and technological characteristics, leading to formulate hypotheses on the material and define sub-goals to solve the problem. This process is better explained using the notion of affordances [33], as detailed in section III-D.

The problem is modeled based not only on the knowledge needed to solve it but also on the initial states of the hardware and the final state for its success, as detailed in Fig. 1. We can see the different observables taken into account (i.e., the possible configurations of the cubes—disassembled or assembled into a certain shape), the identification and conceptualisation of affordances (i.e., the practical possibilities offered by the cubes, as we will further develop), the outcomes of actions performed (e.g., at the motion level), and also elements regarding the subject's emotions or attitude (perseverance, abandonment) towards the task. These observables will serve as a basis for modeling the activity in the form of structured knowledge.

The data model that is generated from this interface has been developed as a hierarchical data structure (in JSON syntax at the file format level), with both the raw and computed data, as

<footnote n="1">¹See http://www.cognitiveatlas.org/ for a browsable version, and on BioPortal for a reusable ontology version.</footnote>

<footnote n="2">²See https://creamaker.wordpress.com/2019/02/06/publications-within-the-creamaker-project for full details</footnote>
<figure><img src="image_3.png" /><figcaption>**Fig. 1:** Interface for the identification of observables, this figure describes a part of the system states to which is added, for example, the identification of each cube (recognizable by its color: dark blue battery, black sensor, white motor, red inverter) and the states of the cubes (e.g., "connected/disconnected" or "on/not on wheels"). This sub-ensemble of the possible states corresponds to the observables that have been chosen to analyse the activity (see http://aide-line.inria.fr/public/doc/vid.mp4 for a video of the experiment.)</figcaption></figure>

well as the description of each type of information and its relationships to other types. This allows formal manipulation and representation of the collected information, as a first structuring step towards an ontology representation, and integrated as an interactive platform to facilitate the ergonomics of manual video analysis [34]. Each video for the #CreaCube activity is encoded as a temporal sequence of states, corresponding either to a configuration of the activity material or to a state of the learner engaged in the activity, as explained above (Fig. 1). This is done manually by the experimenter using a dedicated interactive interface. An example of encoded data generated from the interface follows:

```json
{
  "idVideo": "v0001",
  "nbParticipants": "1",
  "events": [
    {
      "time": "12",
      "tclicks": [
        "U00",
        "START1"
      ]
    },
    {
      "time": "58",
      "tclicks": [
        "AF01"
      ]
    }
  ],
  "time": "73",
  "tclicks": [
    "AF04"
  ]
},
{
  "time": "77",
  "tclicks": [
    "AF03"
  ]
},
{
  "time": "95",
  "tclicks": [
    "AF02"
  ]
},
{
  "time": "121",
  "tclicks": [
    "U00",
    "S02"
  ]
},
{
  "time": "146",
  "tclicks": [
    "F030-BSWI-F",
  ]
}
```
```json
  "P02"
],
[...]
{
  "time": "232",
  "tclicks": [
    "F030-BIWS-F",
    "P02"
  ]
},
{
  "time": "268",
  "tclicks": [
    "F030-BIWS-T"
  ]
},
{
  "time": "274",
  "tclicks": [
    "T03"
  ]
}
],
"idParticipant": "p0001",
"age": 11
}
```

The methodology to generate an organized set of learning analytic log from video analysis has been developed from a learning sciences perspective through the ANR CreaMaker<fnref n="3" /> project to study problem solving activities. Beyond the analysis of observables for research purposes, these videos can also be used to train the teachers who would be willing to put this activity into practice within their classes. [35].

To allow for the analysis of the learner's activity in the context of the task, a formalization of the learning activity is developed in the form of an ontology in the computational sense that we will now define.

## B. The Ontology Approach

**Terminology:** Ontologies allow us to represent structured knowledge by defining concepts, as well as relationships and hierarchies between these concepts. In the following, we use the Web Ontology Language (OWL) terminology, as used in the Semantic Web:

(i) **Individuals** represent atomic, real-world objects.

(ii) **Classes** represent concepts; a class may therefore be a collection of individuals (which are called instances of the class).

(iii) Individuals may be linked by relationships, which are labeled by **properties**. Properties may have a domain (i.e., the class that they can be applied to) and a range (i.e., the class that they can take values from).

OWL also defines both class hierarchy and property hierarchy. This formalism allows us to structure and specify data that can be provided in a standard such as the Resource Description Framework (RDF).

OWL specifications are based on description logics, providing in some cases<fnref n="4" /> computational completeness and decidability (a didactic introduction can be found in this guide [36]). This makes it possible to perform logical reasoning to validate the model, as well as logical inference to find all of the assertions that can be deduced from the user-defined statements.

This representation could be improved using a numerical weighting of RDF statements based on the notions of possibility and necessity (in the sense of [37]) as proposed in [38], but we shall not discuss this any further here.

**Methodology:** [39] describes a simple methodology to build an ontology in seven steps:

(a) Determine the domain and scope of the ontology.

(b) Consider reusing existing ontologies.

(c) Enumerate important terms in the ontology.

(d) Define the classes and the class hierarchy.

(e) Define the slots (i.e., the properties shared by a class).

(f) Define the facets of each slot (i.e., the possible values that a property can take).

(g) Create instances.

Considering (a) the scope defined in the previous paragraph, we have chosen to (b) rely on a lightweight version of the DOLCE foundational principles (DOLCE+DnS UltraLite) and reuse concepts defined in the Cognitive Atlas. This step (b) had been overlooked in the first version of our work [18], which we have thus revised to better integrate existing ontologies. In the following, we describe (c,d,e,f) the terms used to define the concepts and properties in our ontology (the slots refer here to the properties shared by a class, and the facets to the possible values that a property can take). Finally, we explicitize (g) how we instantiated those classes.

Based on these elements, the ontology has been designed at two levels. At the task material level, all cubes configurations and other physical elements of the setup have been identified and encoded, factorizing equivalent configurations (e.g., up to a rotation), and reducing the state space to a few qualitative relevant values. At the learner modeling level, considering the previously published studies [12], [14], [40], our work has been to formalize what has been experimentally determined by learning scientists and education experts, including the chosen pertinent observables. In order to do so, we have attempted to rely on theoretical learning science and cognitive science frameworks, and to identify the concepts that were the most relevant for our context. Observing experiment courses and discussing with the learning experimenters allowed us to capture those different aspects, both at the task level and at the learner level, and specify them as ontology entities.

## C. Borrowing from Learning Sciences: Learner Modeling Framework and the Activity Theory

As previously mentioned, the growing use of intelligent tutoring systems (ITS) led to a need for the development of consistent learner models. Dillenbourg and Self [41] proposed

<footnote n="4">⁴Several OWL “profiles” are available, each of them providing a different trade-off between expressivity and reasoning power. The OWL-DL profile fully adheres to description logic principles and is thus decidable.</footnote>

<footnote n="3">³https://creamaker.wordpress.com</footnote>
a general framework for learner modeling in problem-solving activities, considering two dimensions: on the one hand, they distinguished the system from the learner and defined concepts regarding the system's representation of the learner; and on the other hand, they established a distinction between **behavior**, **behavioral knowledge**, and **conceptual knowledge**. Behavioral knowledge refers to a set of rules and primitives that can be used to infer some behavior for a given problem P, whereas conceptual knowledge contains the definition of the concepts underlying the behavioral knowledge. This representation allows us to precisely define, for example, misconceptions (i.e., the difference between the learner's and the system's conceptual knowledge) or bugs (i.e., the difference between the learner's and the system's behavioral knowledge). In our case, even if we are not implementing a tutoring system, we are in the situation of a system observing the learner's behavior and inferring their knowledge. Moreover, the problem is ill-defined from the point of view of the learner, but it can be well-specified from the point of view of the system to guide the analysis of the observables, which is what we are trying to achieve here.

This framework does not really address the use of tangible artifacts in problem-solving activities. The activity theory [17] fills in the blank by introducing the notion of tools that play the role of mediator for the activity, which enables the learner (subject) to pursue a goal (object). Fig. 2 shows how we incorporate these notions into the learner modeling framework proposed by Dillenbourg & Self. In #CreaCube, the tools are the cubes and the waypoints, and the main object is to solve the problem (e.g., build a vehicle). In what follows, we will see how this main goal can be broken down into sub-goals (which can be in concurrence). We will also explicitize how the aforementioned conceptual and behavioral knowledge can be understood in light of the distinction that is made in cognitive neuroscience between different types of memory.

## D. Borrowing from Cognitive Sciences

**Memories in Interaction:** The role of memory in learning, and more precisely the interactions between memories that are of several types [42], is of primary importance at the neuroscience cognitive level. Traditionally, a distinction has been made between short-term (i.e., working) memory and long-term explicit (or declarative) and implicit memories, with regard to how long the information was allegedly stored in the brain. **Working memory** is considered not simply as a system for the transitory storage of information but also as a processing system [43]). Incoming stimuli perceived by sensory modalities are encoded in a sensory “buffer” (sometimes called sensory memory, e.g. [44]) where they persist very briefly. After being filtered through attentional mechanisms, some of them can be temporarily retained in working memory for future use and manipulation. For example, when solving a problem, working memory makes it possible to store traces useful to the processes involved in carrying out the task, as well as updating them as the task progresses. Because these processes often require prior information, working memory also enables retrieval from longer-term memories. **Procedural memory**

is an example of implicit memory: it stores know-how and integrated procedures that have been automatized, such as motor skills. These procedures are gradually learned through a transfer from declarative and working memories. Declarative memories can themselves be distinguished between semantic and episodic. **Semantic memory** stores general knowledge about facts and concepts, while **episodic memory** refers to the subjective experience of the individual: it stores traces of past events that are contextualized in time and space, and associated to emotional data. They are stored as sequences of events called episodes, which can later be replayed forwards or backwards or rearranged, thus providing off-line learning. Eventually, recurrent patterns in these episodes can be consolidated into more generalized traces in semantic memory or procedural memory. Episodic memory is therefore believed to play a crucial role in human learning [45]. These concepts form the basic architecture of the learner cognitive modeling proposed here.

**Goals and Related Executive Cognitive Functions:** Solving a problem requires one to set some goals. The main goal is to complete the task, but in order to do so, this task usually needs to be broken down in several sub-tasks, thus yielding several sub-goals. In the context of #CreaCube, the main goal given by the guideline translates into the concrete realization of a vehicle, using the four cubes presented on the table, that is able to move autonomously from the red point to the black point. Assembling the cubes into a valid configuration is a goal in itself, while making it move in the right direction is another. These high-level goals yield new sub-goals on the sensorimotor level—to assemble two cubes, one needs to localize them and mobilize specific motor skills to grasp them and put them together.

We consider here a framework of cognitive architecture in which these goals can be organized around four fundamental questions [46] that relate to the learner's cognitive functions: (i) *What* is the object of the goal? (ii) *Why* does it answer some current motivation or specification? (iii) *Where* is it located and how can it be accessed? And, (iv) *how* can it be manipulated and more generally addressed by a skill? This description of the goals represent an operational way to deal with all semantic, motivational and sensorimotor aspects of a goal to be considered in problem solving, consistent with what is known about associative neural circuits.

We also wish to consider goal orientation [47], distinguishing between mastery goals and performance goals. Participants concerned with *mastery* are oriented to develop task-related self-improvement [48], while those focusing on *performance* aim to achieve the task objective using “known ways to quickly implement knowledge and skills that have already been mastered” [49].

**Creative Use of Affordances:** Building with modular bricks is based on visuo-spatial constructive play objects [50] used for engaging learners into tangible programming brick games [51], [52] with modular educational robots [40], [53]. In the context of an ill-defined problem solving task [31], the affordances of the robotic elements will influence the actions of the participant [54] towards the objects (observing, grasping, assembling). Indeed, as opposed to a well-defined
<figure><img src="image_4.png" /><figcaption>**Fig. 2:** Framework chosen for the analysis of the observables, adapted from Dillenbourg & Self [41] completed by the activity theory [17]. This framework corresponds to the learning science concepts taken into account for the construction of the ontology. The observables are the ones described in Fig. 1. See text for more details.</figcaption></figure>

problem, the legal operators that can be considered to act on the problem states are not specified here. Instead, they are suggested by the physical features of the artifacts: for example, the blue cube has a switch on one of its faces, appealing to be activated. The exploration of one's environment (as developed in the next paragraph) aims to discover such affordances, as defined by Gibson [33] as “opportunities of actions” offered by the physical properties of this environment. Through their interactions with the task material, the learner relates the perceived physical properties of the artifacts to prior knowledge stored in long-term memories (e.g., a switch can be flipped to activate a device) and generates new hypotheses. Affordances are a way to make sense of stimuli, and a key point in the creative process is to be able to think of different appropriate ways to use the material to solve the problem, using mechanisms such as analogy to transfer knowledge from a task to another [3], [4], [10].

**Exploration vs Exploitation:** While the notion of exploration and exploitation is well known in cognitive science and in machine learning at the level of reinforcement learning, they are two distinct conceptualizations that we will now make explicit. In problem-solving activities, subjects alternate between two main strategies: exploration, which aims at experimenting with new alternatives or acting on the environment to generate new stimuli [32], and exploitation, which is the use of existing knowledge (semantic or procedural) in a given situation. At this level of description, we can draw parallels with these notions in reinforcement-based machine learning, but in this second domain exploration is mostly random, whereas in problem solving as studied in learning sciences, exploration is seen as a strategy for generating new stimuli or new ideas

or behaviors by recombining prior knowledge or behaviors, or by manipulating the environment. These are therefore two quite distinct notions: one corresponding to an internal process of generating new internal representations, and the other to a behavior that is only defined with regard to a direct interaction with the environment.

**Convergent vs Divergent Thinking:** In relation to the previous distinction between two strategies — exploitation and exploration —, we can also introduce an alternation between two modes of thinking — convergent and divergent. Divergent thinking is directed towards idea generation (fostering originality and novelty), while convergent thinking focuses on selecting an effective candidate solution to the problem (ensuring adequation). These two processes, consistent with the previous definition of creativity, have been defined to be central to creative cognition [3] and are characterized by measurable indicators [55]. This brings another perspective to the goal definition that was addressed earlier: during the divergent phase, goals and their sub-goals thus unfold into tree structures that take shape through stimuli and experiences, and are reunited during the convergent phase so as to achieve the final goals.

## E. Concept Selection and Instantiation Methodology for the Ontology Implementation

In order to implement the Learner model of the ontology, we selected several concepts in relation to the previous theoretical frameworks:

**Perceptions** are internal representations of stimuli formed by the learner. Traces of these perceptions are processed in working memory and may be stored in the long term in
episodic memory. **Conceptual knowledge** as defined in III-C is for the most part stored in semantic memory and can be retrieved in working memory when it is relevant. Behavioral knowledge as defined in III-C consists of procedures and inference rules that are not convenient to represent in the OWL formalism because they are not declarative (e.g., procedural). However, we can represent the result of such inferences, which are drawn by the learner by contextualizing pieces of conceptual knowledge with regard to their perceptions. We have called these inferences **hypotheses**: they are temporary representations formed by the learner to make sense of the environment. Hypotheses are only relevant for a short time interval within the task (e.g., the execution of a plan associated with a goal) and we can assume they are only temporarily stored and used by the working memory. When they have been verified (thus believed by the learner as true), they can be maintained in the working memory to pursue the task but they are not questioned anymore (unless a new stimulus raises a contradiction). We can assume that hypotheses that were useful to complete the task may have a more durable print and be stored in episodic memory. The learner's **goals** are broken down into the four queries described in III-D (namely, *What, Why, Where, How*) and towards mastery vs performance goal orientation.

These concepts have been instantiated step by step, based on the design of the task itself and preliminary video observations of the first participants [56]. Further details were collected during a post-activity survey: the participants were asked several questions about what they did and what they learned about the activity material. Some elements were also noted throughout the task when the learners were speaking out loud, to themselves or to the experimenter (there was no directive to the participants about whether they should speak during the task or not; the experimenters were, on their side, required not to answer the participants' questions).

(a) Observe all of the possible actions and break them down into elementary actions (e.g., assembling three cubes decomposes into assembling two cubes and connecting a third one to the resulting structure).

(b) Identify the contexts in which each action was executed to determine which are the preconditions (necessary or optional) and in which state the system is after the action has been performed (post-conditions).

(c) Identify, from the material changes, what stimuli are raised and how they are perceived (e.g., when the power switch is on, a diode lights up).

(d) Specify the hypotheses and knowledge:

* Conceptual knowledge relevant to the task was mostly identified prior to the experiment, as part of the activity design (e.g., what is considered as a vehicle, what are its main elements).

* Hypothesis instances are issued from both preliminary observations by the experimenters of the resolution by different subjects and evaluation interviews after the activity, as discussed for instance in [40], [53].

(e) Finally, identify how these hypotheses and knowledge are pre-requisite for others and might drive the participant's goals.

(f) Associate the actions listed in (b) with the learner goals, issued from the activity observation, while also considering goal orientation and broken down into elementary goals corresponding to the four cognitive queries.

These are clearly preliminary prior assumptions to bootstrap a first version of the ontology, with the perspective to adapt them from further observation by learning sciences experts and logical validation by ontological reasoners.

## IV. ONTOLOGY IMPLEMENTATION

We implemented the ontology<fnref n="5" /> using the software Protégé⁶. First, we formalized the classes corresponding to the material environment of the task, and the classes referring to the learner's cognition and behavior. Then, we have defined specific classes and properties to include the observables of the task, which are situated in time, such as events and situations (in the sense of DOLCE+DnS Ultralite). Finally, we have instantiated these classes, listing their individuals based on the task design itself and the observations collected during the experiments, according to the methodology described in III-E.

### A. Modeling the Activity Process

The #CreaCube activity itself is a dul:Process; that is to say, considered in its evolution. Each scene of interest annotated in the videos is a dul:Situation, which we call an ActivityState.

Activity states are transitional and situated in time. To account for this transitionality, we use the Description & Situation ontology pattern⁷ of DOLCE+DnS Ultralite (we now use the prefix dul: for pre-defined entities).

A dul:Situation is, as per the entity specification, a relational context that is created by an observer on the basis of a dul:Description. Consequently, a dul:Description is itself a conceptualization that provides an interpretation for a set of entities in a given situation.

We consider here two aspects of the activity situation: the learner's mental representation (their representation of the scene) and the material states, from the point of view of an expert of the activity (cf Fig2). We therefore assign, for each ActivityState, two kinds of dul:Description: a MaterialStateDescription and a LearnerMentalRepresentation.

### B. Modeling the Material States

The task involves several tools (in the sense of the activity theory), which can consist in dul:PhysicalObjects (cubes, waypoints) or dul:SocialObjects (guideline recording). Physical objects have features that can themselves be considered as physical objects constitutive of the cubes (e.g., switch, wheels, magnets etc.). The cubes can be assembled into multiple configurations (encoded by a series of letters and numbers as per Fig. 1) characterizing the material states. As opposed to transitional material states, physical objects have a proper

<footnote n="5">⁵This ontology is publicly available here: https://line.gitlabpages.inria.fr/aide-group/creaonto/
⁶https://github.com/protegeproject
⁷http://ontologydesignpatterns.org/wiki/Submissions: DescriptionAndSituation</footnote>
<figure><img src="image_5.png" /><figcaption>**Fig. 3:** A partial diagram exposing the main classes defined in our ontology. The tools mediating the activity are highlighted in red boxes. In green are the other classes related to the observables. Those in blue characterize the learner's behavior and mental representations. Some classes extend the ones defined in the Cognitive Atlas (prefixed by ca:) and/or in the DOLCE+DnS Ultralite ontology (prefixed by dul:). The symbol "<" in labels stands for "isA" when the node is not expanded for concision purposes.</figcaption></figure>

space region and mass. Their state is represented by the values taken by certain properties called *dul:Quality* (e.g., color) in a given qualitative *dul:Region* (e.g., red): for example, a switch has the quality of being either ON or OFF.

Material states are characterized by a specific configurations of those physical objects, their quality values, and correspond to the observables shown on Fig 3. We have also listed the possible permutations of the cubes within each configuration shape, so that each configuration encoding (e.g., “F000-BSIW-T”) is characterized by its connections. These connections are themselves specified by the cubes they connect along what spatial axis. This allows us to give a better description of the material states and relates them to their JSON encodings.

These material states represent the problem space of the task in the sense of [6]. Indeed, the goal states of the task are those where the cubes are in a configuration that satisfies the guideline (which is itself a *dul:Description* describing the requirements for a state to be a solution state). The participants may interact with the material and change its state through their actions: this is formalized as follows in the sequel.

### C. Modeling the Learner's Behavior and Mental States

**Actions and Affordances:** Some actions are only available to the learner in some particular states of the material environment. These states define the preconditions of the action. Similarly, post-conditions define changes of state that an action results in. These pre- and post-conditions also constitute knowledge to acquire. They are represented in the ontology by

properties linking a *dul:Task* (referring to the execution of an action) to an *ActivityState* (i.e. the *dul:Situation* before and after the action has been performed). Actions are suggested by the physical objects—or their features—that the learner relates to their knowledge, which we introduced earlier as *Affordances*. Affordances allow a *dul:Task* to be executed. When appropriate, we related those affordances to the ones defined in AfNet [57], a semantic network aiming at categorizing structural affordances based on physical properties of objects.

**Stimuli and Perceptions:** Throughout the activity, the learners receive various stimuli from the material environment. Stimuli are sometimes modeled as events in DOLCE-based ontologies, as is the case in the Stimulus-Sensor-Observation ontology [58]. Indeed, in the #CreaCube activity, stimuli are most often the result of events occurring when participants proactively interact with the environment. For instance, when they flip the switch on the blue cube, a diode lights up as a response to their action. However, the learner may not always perceive stimuli as soon as a triggering event occurs, and stimuli might remain in the environment for some time before the learner focuses their attention on it (e.g., the diode that lights up is very small and some participants only notice it later). Therefore, we prefer to view them as sensory information input provided to the learner, rather than events themselves; hence, extending the *InformationObject* class. When they are filtered by the attentional focus of the learner, traces of these stimuli are kept active during the
task before being eventually consolidated as knowledge. These traces constitute *Perceptions*: they are a conceptualization of the perceived stimuli, thus a kind of *dul:Description*.

*Knowledge Retrieval, Hypotheses and Goals:* To complete the task, the learner needs to retrieve knowledge that has been acquired in the course of past experiences and learning. Most of the time, this knowledge is activated through the discovery of affordances. For instance, after finding of the wheels, the learner might retrieve the fact that wheels on a vehicle must be in contact with the ground to allow it to move. It can also be activated by the concepts evoked in the guideline.

As we explained in section III-C, we previously distinguished between conceptual knowledge and behavioral knowledge before deciding to represent only *ConceptualKnowledge* in the ontology because behavioral knowledge consists of procedures and inference rules that are, in our case, not declarative. However, we accounted for the result of inferences with the class *Hypothesis*, representing hypotheses drawn by the learner by contextualizing pieces of conceptual knowledge with regard to their perceptions.

*ConceptualKnowledge, Perceptions and Hypotheses* are all mental representations that the learner forms to make sense of the environment. These items extend the class *dul:Description* and are interdependent (i.e. some of them can be inferred or induced from others). These dependencies are represented by ontological properties, such as *requires* (logical necessity), *induces* (inductive reasoning) and *isEvidenceFor/hasEvidenceIn* (e.g., when a perception tends to confirm a hypothesis).

Instead of directly identifying discrepancies at the behavioral knowledge level (which were called *bugs* in [41] and on Fig 2), we identified *misperceptions* and *misinterpretations*. Misperceptions happen when internal representations of stimuli formed by the learner are erroneous; for example, if the learner misheard the guideline, did not see a visible feature on the cubes, or mistook the dark blue cube for another black cube. Misinterpretations refer to erroneous hypotheses that can be inferred, either from misperceptions or from some conceptual knowledge not transferable to the situation at hand. For example, the fact that wheels automatically start to turn after the user has pulled the vehicle backwards is only applicable in certain situations—most often, this mechanism is found in children car toys. However, in this task, trying to press down on the white cube by manually rolling it backwards will not cause it to move forward autonomously. Such misinterpretations can be corrected after verification showing a lack of evidence in favor of the hypothesis. Nonetheless, if it comes from a misconception that the underlying conceptual principle is a general truth that is valid in any situation, then it might hinder the completion of the task: as long as the learner keeps on trying to make the wheels work this way, they might not look for other clues.

This makes it clear that the learner needs to understand which pieces of knowledge are effectively applicable in the context of the activity and relevant to complete it. Testing hypotheses thus defines sub-goals that will help to solve the problem. As described in section III-D, goals are a way for the participant to organize their understanding of the task and plan their actions hierarchically. Global goals (essentially re-

sponding to *What* and *Why*, in relation to high-level executive functions [46]) may be instructed by an external direction—in this case, the guideline—or a metacognitive reflection, inducing (*Where, How*) sensori-motor mechanisms [46] thus yielding sub-goals at a more local level. This goal hierarchy is represented in our ontology through the property *hasSubGoal*.

# V. RESULTS AND PERSPECTIVES OF APPLICATION

## A. Modeling as an Interdisciplinary Tool

This work allowed us to confirm the multiple interest of working with ontological modeling when we bring together three disciplines (educational, cognitive and computer sciences) to tackle a problem, as follows:

* Terminological interest to begin with: beyond periphrases and phenomenological descriptions in each discipline, constraining us to define things through lexical choices and fully specified properties and relations between these key words, forcing us to clearly posit what we are talking about.

* Interest in formalization: this approach allows, without even using algorithmic reasoning skills at this stage, to take stock of what can be defined rigorously to formalize completely.

Defining a well-formulated ontology is therefore already by itself a structuring exercise, an epistemological tool in a way, before even using it. As with any modeling, it gives an exhaustive and explicit view of what is to retain compared to what is neglected. Once the specification has been set, the use of available reasoners, such as Pellet or Hermit, allows to check its consistency and query the data on the consequences that may or may not be drawn from it.

This being done, we also have an immediate check of the model properties, in particular its logical coherence and soundness. This not only allows us to verify the absence of contradictory statements but we also experimented that if our definitions are not sufficiently specified, then we can easily deduce spurious consequences (e.g., mixing categories of concepts). We thus have an operational tool to verify the completeness of our formalization.

An illustrative example of how a JSON video annotation can be translated into this formalism is given here.

## B. Reasoning About the Model

A query language, such as SPARQL in the case of RDF data, combined with a reasoner (in our case, Pellet<sup>8</sup>), allows us to retrieve specific information—asserted or inferred—from our model. This raises some perspectives of application both at the material level and at the learner level. Let us illustrate each aspect in the following examples, the former being already operational and the latter being still at an exploratory stage.

*Link the Material Configuration Encodings to the Material States:* At the material level, this ontology can be used to retrieve physical information from configuration encodings.

<footnote>8https://www.w3.org/2001/sw/wiki/Pellet</footnote>
<figure><img src="image_6.png" /><figcaption>**Fig. 4:** Examples of a SPARQL query <fnref n="9" /> on the material states: Retrieving connections and cubes for a given configuration.</figcaption></figure>

In the JSON annotations of the videos, configurations tested by the subject are encoded as strings of characters and letters (e.g., "F000-BSIW-T") that are not intelligible at a first glance, and hard to relate to real-world connections between the cubes. The ontology does this strenuous work for us: by querying a configuration by its encoding, we can retrieve the associated connections between the cubes. Corresponding ontology assertions are based on simple geometric considerations and the design of the cubes provided by the manufacturer. Conversely, we can also query the knowledge base on configurations which have, for instance, a connection between the blue cube and the red cube. Integrating an ontology-based module to the annotation interface might help to avoid encoding mistakes. A step further, we could also add some transformation rules between those configurations to validate the pre- and post-conditions of the learner's actions to verify the consistency of the annotations on a sequential scale. For example, if the learner has disconnected two cubes from each other, then we should see that there is one less connection in the new configuration.

### Inferring an Internal Representation of the Learner from an Observable:

At the learner behavior level, the main perspective of this work would be to interpret observables and infer, for example, the participants' goals and associated plans or motivations. Although more data and further investigation is needed to validate the relevance of the ontology inferences on this aspect, we propose here the general idea of such an application.

Storing domain knowledge about behavioral processes in an ontology allows to consistently interpret new data. For example, we may observe a new action which changes the state of the switch, and wonder what motivated the learner to execute this action. As illustrated by Fig. 5, a SPARQL query on the model helps us retrieve that the action may have been either a simple reaction to the stimulus of seeing the switch, in the case of a stimulus-driven behavior, or, in a

goal-driven mode, the result of a more elaborate reasoning, such as an attempt to understand the function of the blue cube (thus testing the hypothesis that it is indeed a switch) or, this hypothesis being verified, to activate the movement of the vehicle. Characterizing whether the learner is rather in a goal-driven or stimulus-driven mode, as precisely defined in [18], [56], for a given sequence of observables might be inferred from the timestamps recorded in the data. This helps us decide which statements of the ontology are more appropriate to describe the observed situation. For example, we could assume that once the learner has identified a goal, and have the means and resources at their disposal to fulfill it, then the flow of their actions might be smoother; whereas, in an exploration situation, there might be more pauses corresponding to the delay between new stimuli. However, this link between timestamps and switch of strategy is not yet well understood and needs further examination. Another difficulty is the high computational complexity of assertions regarding the learner behavior, which does not scale well with a large knowledge base and may lead to poor reasoning performances.

### Predicting the Learner's Behavior:

Instead of querying the model using backward chaining as in the previous example, we could also use forward chaining to simulate the learner's deduction, hence predicting their behavior. For example, we might wonder how the learner would react to the visual stimulus of seeing the switch. In a stimulus-driven mode, they would likely touch it or try to flip it (assuming that the knowledge relative to switches being flipped has become a procedural knowledge, that is to say automated into an almost reflex behavior). In a goal-driven mode, we may infer that the learner will elaborate the hypothesis that the switch can be used to activate the vehicle, which will result in planning goals (flip the switch, connect the cube to the structure) and sub-goals (they have to grasp the cube before connecting it to the others).

This ontology thus provides a subset of plausible behaviors. However, we still need a way to decide which of these behaviors to choose to simulate the learner. Mechanisms to explain what the learner's attention is more focused on in a given situation are still to be developed. Moreover, we need to take into account inter-individual differences: some participants might be more interested in achieving a performance goal, thus appealing for a predominant exploitation strategy, whereas others might pursue a mastery goal and try to understand as much as possible about the environment of the task, displaying more exploratory behaviors.

## VI. DISCUSSION AND CONCLUSION

This study introduces the operationalization of a creative problem-solving task. Within this type of task, we need to consider both exploration and exploitation strategies directed toward different concurrent goals (e.g., performance versus mastery). These goals refer to an objective (*What*) but we also need to consider the *Why*; that is, extrinsic motivation (e.g., task performance) and intrinsic motivation (e.g., increase knowledge). They result in concrete plans to achieve them (*Where* and *How*). The execution of such plans is the observable behavior of the learner that we attempt to capture

<footnote n="9">⁹We used the Protégé Snap-SPARQL plugin which allows us to perform logical inference and answer queries not only based on asserted data but also inferred data: https://github.com/protegeproject/snap-sparql-query</footnote>
<figure><img src="image_7.png" /><figcaption>(a) An example SPARQL query</figcaption></figure>

<table><thead><tr><th>action</th><th>stimulus</th><th>goal</th></tr></thead><tbody><tr><td>FlipSwitch</td><td>The blue cube has a switch</td><td>TFindFunctionCubeBe</td></tr><tr><td>FlipSwitch</td><td>The blue cube has a switch</td><td>TMakeConfigMoveAutonomously</td></tr><tr><td>FlipSwitch</td><td>The blue cube has a switch</td><td>TMakeConfigMoveAutonomously</td></tr><tr><td>FlipSwitch</td><td>The blue cube has a switch</td><td>TMakeConfigMoveAutonomously</td></tr></tbody></table>

Switches must be 'on' to activate an electronic device
The blue cube has a switch (assuming the color is perceived) /
Switches are used to activate an electronic device
Switches must be 'on' to activate an electronic device

(b) SPARQL results returned for the query (a): We enquired about which actions change the state of the blue switch, what stimuli may have provoked them, or what goal they might be aiming at, and if that goal might be the sub-goal of another one, as well as what knowledge they are using and what hypothesis might be tested.

<caption>Fig. 5: Examples of SPARQL queries on the learner's behavior.</caption>

and formalize in our ontology. Through this process, we take into account the stimuli received during the interactions of the learner with the modular robotic cubes, and the perception and interpretation of the different technological affordances that will support the problem-solving activity through concrete actions to be executed. The learner's prior knowledge is mobilised to understand current perceptions and transform knowledge into a way that makes sense within the context of activity [59], formulating and testing in-task hypotheses.

This formalization was achieved step by step with the aim of encapsulating domain knowledge used by learning science researchers when they analyze such an experiment, in order to make it more robust and reproducible. The purpose of this description is not to remain phenomenological; that is, driven by a theoretical framework expressed in human language to guide the analysis of the data. We hereby reformulate these notions at a computational level to give us the means to process them in a more systematic way using an algorithmic approach, as discussed here, formalizing the concepts invoked in a precise language.

As stated in the introduction, we model the learning behavior engaged in a problem-solving activity that is ill-defined at the participant level, while the activity state space is entirely formalized at the observer level. As such, our approach is voluntarily specific, while the proposed methodology is easy to generalize, as pointed out in [16] where three other computational thinking learning tasks have been also considered. From the paradigm choice to rely on an ontology modeling to the general knowledge about cognition when engaged in a creative problem solving task, an important part of this formalization translates to another learning task. Relying on a robust foundational ontology such as DOLCE also ensures compatibility and facilitates such knowledge transfer. Besides, our paradigm is based on frameworks that are already widely used in learning sciences such as the activity theory and the ITS learner/observer representation, combined with a cognitive basis. A step further, the query-based specification of goals and the hypothetico-deductive process might also be reconcilable with, e.g, the Belief-Desire-Intention (BDI) model

commonly used in AI and inspired by psychology to formalize rational agents [60], where knowledge and hypotheses would constitute beliefs with various degrees of confidence, desires may correspond to the *What/Why* of the learner's goals, and intentions to the procedures considered to reach these goals, i.e. *Where/How*.

For obvious ethical reasons, the learner is aware of being engaged in a task with learning analytic evaluation. To what extends this could perturb the outcomes has been addressed in [40], in fact only marginally. Besides, this could be turned into a significant advantage, since it is a way to engage the participants in understanding their own learning process, opening opportunities to meta-learning.

This is only a first step but it already shows that formalizing such a problem-solving activity is a complex but attainable work. To complete this formal description, one of the next steps would be to better formalize the relationship between affordances and knowledge (e.g., we could draw inspiration from [61] who proposed a design pattern involving the use of frames). We also consider to address the emotional aspect, such as how to identify observable emotions (e.g., the ones listed in Fig. 1) and link inferred cognitive states to emotional states. Finally, we need to make use of reasoning mechanisms to better infer the subject internal motivation and behaviors from the experiment observables (e.g., learner state observation, post-activity evaluation) and conduct statistical analyses on the results to assess the validity of such an application.

With respect to the state of the art, briefly reviewed previously, our approach applies to tangible activities, modeling both the task and the learner in the task, and targeting a better understanding of the learning process, more than increasing the learning performance. This approach is rather disruptive with respect to learning science because, to the best of our knowledge, there has not been such modeling of an ill-defined task using an ontology, including a triple-disciplinary contribution from learning science, cognitive neuroscience and computer science, from which could emerge computational learning science [62].
ACKNOWLEDGMENTS

Frédéric Alexandre, Margarida Romero and Thierry Viéville are gratefully acknowledged for powerful ideas at the origin of this study and their contributions to the first version of this work presented in [18], with special thanks to the latter for his punctual help regarding this paper. We also wish to thank Lisa Roux and Lola Denet for their preliminary work on the ontology, especially on the task model and the learner model, respectively.

REFERENCES

[1] M. Romero, A. Lepage, and B. Lille, "Computational thinking development through creative programming in higher education," *International Journal of Educational Technology in Higher Education*, vol. 14, no. 1, pp. 1-15, Dec. 2017, number: 1 Publisher: SpringerOpen. [Online]. Available: https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-017-0080-z

[2] F. Alexandre, J. Becker, M.-H. Comte, A. Lagarrigue, R. Liblau, M. Romero, and T. Viéville, "Open Educational Resources and MOOC for Citizen Understanding of Artificial Intelligence," Jul. 2020. [Online]. Available: https://hal.inria.fr/hal-03024034

[3] J. Guilford, *The nature of human intelligence*, ser. The nature of human intelligence. New York, NY, US: McGraw-Hill, 1967.

[4] F. Alexandre, "Creativity explained by Computational Cognitive Neuroscience," in *ICCC'20 - International Conference on Computational Creativity*, Sep. 2020, p. 4. [Online]. Available: https://hal.inria.fr/hal-02891491

[5] A. Dietrich, "The cognitive neuroscience of creativity," *Psychonomic Bulletin & Review*, vol. 11, no. 6, pp. 1011-1026, Dec. 2004. [Online]. Available: http://link.springer.com/10.3758/BF03196731

[6] A. Newell and H. A. Simon, *Human problem solving*. Englewood Cliffs, N.J.: Prentice-Hall, 1972, oCLC: 622041645.

[7] J. M. Wing, "Computational thinking," *Communications of the ACM*, vol. 49, no. 3, pp. 33-35, Mar. 2006. [Online]. Available: https://doi.org/10.1145/1118178.1118215

[8] M. Lodi, "Introducing Computational Thinking in K-12 Education: Historical, Epistemological, Pedagogical, Cognitive, and Affective Aspects," phdthesis, Dipartimento di Informatica - Scienza e Ingegneria, Alma Mater Studiorum - Università di Bologna, Apr. 2020. [Online]. Available: https://hal.inria.fr/tel-02981951

[9] L. Heiser, M. Romero, and V. Prokofieva Nelson, "Analyse des activités technocréatives au collège en contexte d'éducation FabLab," Université Côte d'Azur, Tech. Rep., Jan. 2022.

[10] W. S. De Grave, H. P. A. Boshuizen, and H. G. Schmidt, "Problem based learning: Cognitive and metacognitive processes during problem analysis," *Instructional Science*, vol. 24, no. 5, pp. 321-341, Sep. 1996. [Online]. Available: https://doi.org/10.1007/BF00118111

[11] T. Viéville, "An improved biologically plausible trajectory generator," *INRIA*, report, Sep. 2002. [Online]. Available: https://hal.inria.fr/inria-00072049

[12] M. Romero, "CreaCube, analyse de la résolution créative de problèmes par le biais d'une tâche de robotique modulaire," in *Journées Nationales de la Recherche en Robotique, JNRR 2017*. Femto-st, 2017, p. 1.

[13] W. Huang and C.-K. Looi, "A critical review of literature on "unplugged" pedagogies in K-12 computer science and computational thinking education," *Computer Science Education*, vol. 0, no. 0, pp. 1-29, Jul. 2020, publisher: Routledge _eprint: https://doi.org/10.1080/08993408.2020.1789411. [Online]. Available: https://doi.org/10.1080/08993408.2020.1789411

[14] M. Romero, D. David, and B. Lille, "CreaCube, a Playful Activity with Modular Robotics," in *Games and Learning Alliance*, M. Gentile, M. Allegra, and H. Söbke, Eds. Cham: Springer International Publishing, 2019, vol. 11385, pp. 397-405, series Title: Lecture Notes in Computer Science. [Online]. Available: http://link.springer.com/10.1007/978-3-030-11548-7_37

[15] S. Bull and J. Kay, "Open Learner Models," in *Advances in Intelligent Tutoring Systems*, ser. Studies in Computational Intelligence, R. Nkambou, J. Bourdeau, and R. Mizoguchi, Eds. Berlin, Heidelberg: Springer, 2010, pp. 301-322. [Online]. Available: https://doi.org/10.1007/978-3-642-14363-2_15

[16] S. Barnabé, L. Denet, M. Manrique, D. Menon, E. Pascual, M. Romero, and T. Viéville, "A low-cost tabletop game to collect learning analytics during computational thinking using unplugged or tangible activities," Inria, report, Dec. 2020. [Online]. Available: https://hal.inria.fr/hal-03040909

[17] Y. Engestrom, "Activity theory as a framework for analyzing and redesigning work," *Ergonomics*, vol. 43, no. 7, pp. 960-974, Jul. 2000. [Online]. Available: https://doi.org/10.1080/001401300409143

[18] C. Mercier, L. Roux, M. Romero, F. Alexandre, and T. Viéville, "Formalizing Problem Solving in Computational Thinking: an Ontology approach," in *2021 IEEE International Conference on Development and Learning (ICDL)*, Beijing, China, Aug. 2021, pp. 1-8.

[19] K. Stancin, P. Poscic, and D. Jaksic, "Ontologies in education – state of the art," *Education and Information Technologies*, vol. 25, no. 6, pp. 5301-5320, Nov. 2020. [Online]. Available: https://doi.org/10.1007/s10639-020-10226-z

[20] G. George and A. M. Lal, "Review of ontology-based recommender systems in e-learning," *Computers & Education*, vol. 142, p. 103642, Dec. 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0360131519301952

[21] A. Nouira, L. Cheniti-Belcadhi, and R. Braham, "An ontology-based framework of assessment analytics for massive learning," *Computer Applications in Engineering Education*, vol. 27, no. 6, pp. 1343-1360, 2019, _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/cae.22155. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/cae.22155

[22] J. Perisic, M. Milovanovic, and Z. Kazi, "A semantic approach to enhance moodle with personalization," *Comput. Appl. Eng. Educ.*, 2018.

[23] J. Torrente, P. Moreno-Ger, I. Martínez-Ortiz, and B. Fernandez-Manjon, "Integration and Deployment of Educational Games in e-Learning Environments: The Learning Object Model Meets Educational Gaming," *Journal of Educational Technology & Society*, vol. 12, no. 4, pp. 359-371, 2009, publisher: International Forum of Educational Technology & Society. [Online]. Available: https://www.jstor.org/stable/jeductechsoci.12.4.359

[24] K. Fuchs and P. A. Henning, "Cognitive Space Time: A Model for Human-Centered Adaptivity in E-Learning," in *2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC)*. Stuttgart: IEEE, Jun. 2018, pp. 1-9. [Online]. Available: https://ieeexplore.ieee.org/document/8436283

[25] S. Tang and M. Hanneghan, "Game Content Model: An Ontology for Documenting Serious Game Design," in *2011 Developments in E-systems Engineering*, Dec. 2011, pp. 431-436.

[26] R. Poldrack, A. Kittur, D. Kalar, C. Seppa, Y. Gil, D. Parker, F. Sabb, and R. Bilder, "The Cognitive Atlas: Toward a Knowledge Foundation for Cognitive Neuroscience," *Frontiers in neuroinformatics*, vol. 5, p. 17, Sep. 2011.

[27] J. A. Turner and A. R. Laird, "The cognitive paradigm ontology: design and application," *Neuroinformatics*, vol. 10, no. 1, pp. 57-66, Jan. 2012.

[28] A. Lieto, F. Perrone, G. L. Pozzato, and E. Chiodino, "Beyond subgoaling: A dynamic knowledge generation framework for creative problem solving in cognitive architectures," *Cognitive Systems Research*, vol. 58, pp. 305-316, Dec. 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1389041719304632

[29] A. Lieto and G. L. Pozzato, "A description logic framework for commonsense conceptual combination integrating typicality, probabilities and cognitive heuristics," *Journal of Experimental & Theoretical Artificial Intelligence*, vol. 32, no. 5, pp. 769-804, Sep. 2020, publisher: Taylor & Francis _eprint: https://doi.org/10.1080/0952813X.2019.1672799. [Online]. Available: https://doi.org/10.1080/0952813X.2019.1672799

[30] C. Masolo, S. Borgo, A. Gangemi, N. Guarino, and A. Oltramari, *Wonderweb deliverable d18: Ontology library*. Technical report, ISTC-CNR, 2003.

[31] G. Schraw, M. E. Dunkle, and L. D. Bendixen, "Cognitive processes in well-defined and ill-defined problem solving," *Applied Cognitive Psychology*, vol. 9, no. 6, pp. 523-538, 1995. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.2350090605

[32] A. Kaplan and H. Flum, "Achievement goal orientations and identity formation styles," *Educational Research Review*, vol. 5, no. 1, pp. 50-67, Jan. 2010. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1747938X09000347

[33] J. J. Gibson, "The theory of affordances," in *Perceiving, acting, and knowing: toward an ecological psychology*, J. B. Robert E Shaw, Ed. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1977, pp. 67-82. [Online]. Available: https://hal.archives-ouvertes.fr/hal-00692033
[34] M. Romero, T. Vieille, and L. Heiser, "Analyse d'activités d'apprentissage médiatisées en robotique pédagogique," in *Traité de méthodologie de la recherche en Sciences de l'Éducation et de la Formation*. B. Alberto and J.Thievenaz, 2021. [Online]. Available: https://hal.inria.fr/hal-02957270

[35] B. Albero and J. Guérin, "L'intérêt pour l'activité en sciences de l'éducation. Vers une épistémologie fédératrice ?" *TransFormations : Recherches en éducation et formation des adultes*, vol. 11, pp. 11-45, 2014, publisher: Institut CUEEP, Lille 1. [Online]. Available: https://hal.archives-ouvertes.fr/hal-01712411

[36] D. Allemang, J. Hendler, and F. Gandon, *Semantic Web for the Working Ontologist: Effective Modeling for Linked Data, RDFS, and OWL*, 3rd ed. New York, NY, USA: Association for Computing Machinery, 2020. [Online]. Available: https://hal.inria.fr/hal-02939606

[37] T. Denœux, D. Dubois, and H. Prade, "Representations of Uncertainty in AI: Beyond Probability and Possibility," in *A Guided Tour of Artificial Intelligence Research: Volume 1: Knowledge Representation, Reasoning and Learning*, P. Marquis, O. Papini, and H. Prade, Eds. Cham: Springer International Publishing, 2020, pp. 119-150. [Online]. Available: https://doi.org/10.1007/978-3-030-06164-7_4

[38] C. Mercier, H. Chateau-Laurent, F. Alexandre, and T. Viéville, "Ontology as neuronal-space manifold: towards symbolic and numerical artificial embedding," in *KRHCAI-21@KR2021*, Mar. 2021.

[39] N. Noy, "Ontology Development 101: A Guide to Creating Your First Ontology," *undefined*, 2001. [Online]. Available: /paper/Ontology-Development-101%3A-A-Guide-to-Creating-Your-Noy/c15cf32df98969af5eaf85ae3098df6d2180b637

[40] M. Romero, "Analyzing Cognitive Flexibility in Older Adults Through Playing with Robotic Cubes," in *Human Aspects of IT for the Aged Population. Social Media, Games and Assistive Environments*, ser. Lecture Notes in Computer Science, J. Zhou and G. Salvendy, Eds. Cham: Springer International Publishing, 2019, pp. 545-553.

[41] P. Dillenbourg and J. Self, "A Framework for Learner Modelling," *Interactive Learning Environments*, vol. 2, no. 2, pp. 111-137, Jun. 1992. [Online]. Available: http://www.tandfonline.com/doi/abs/10.1080/1049482920020202

[42] F. Alexandre, "Les relations difficiles entre l'intelligence artificielle et les Neurosciences," *Interstices*, Aug. 2020. [Online]. Available: https://hal.inria.fr/hal-02925517

[43] N. Cowan, "What are the differences between long-term, short-term, and working memory?" *Progress in brain research*, vol. 169, pp. 323-338, 2008. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2657600/

[44] ——, "Sensory and Immediate Memory," *Encyclopedia of Consciousness*, Jan. 2010.

[45] H. Eichenbaum, "Memory: Organization and Control," *Annual Review of Psychology*, vol. 68, no. 1, pp. 19-45, Jan. 2017. [Online]. Available: http://www.annualreviews.org/doi/10.1146/annurev-psych-010416-044131

[46] F. Alexandre, "A global framework for a systemic view of brain modeling," *Brain Informatics*, vol. 8, no. 1, p. 3, Dec. 2021. [Online]. Available: https://braininformatics.springeropen.com/articles/10.1186/s40708-021-00126-4

[47] P. R. Pintrich, "Multiple goals, multiple pathways: The role of goal orientation in learning and achievement," *Journal of Educational Psychology*, vol. 92, no. 3, pp. 544-555, 2000, place: US Publisher: American Psychological Association.

[48] P. M. Poortvliet, "Mastery Goals," *Encyclopedia of Personality and Individual Differences*, pp. 1-4, 2016, publisher: Springer.

[49] G. H. Seijs and G. P. Latham, "Learning versus performance goals: When should each be used?" *Academy of Management Perspectives*, vol. 19, no. 1, pp. 124-131, 2005, publisher: Academy of Management Briarcliff Manor, NY 10510.

[50] D. Ness and S. J. Farenga, *Knowledge under construction: The importance of play in developing children's spatial and geometric thinking*, ser. Knowledge under construction: The importance of play in developing children's spatial and geometric thinking. Lanham, MD, US: Rowman & Littlefield, 2007, pages: xxiii, 257.

[51] G. Kalmpourtzis and M. Romero, "Artifactual Affordances in Playful Robotics," in *International Conference on Games and Learning Alliance*, ser. Lecture Notes in Computer Science, I. Marfisi-Schottman, F. Belotti, L. Hamon, and R. Klemke, Eds. Cham: Springer International Publishing, 2020, pp. 316-325.

[52] T. S. McNerney, "From turtles to Tangible Programming Bricks: explorations in physical language design," *Personal and Ubiquitous Computing*, vol. 8, no. 5, pp. 326-337, Sep. 2004. [Online]. Available: https://doi.org/10.1007/s00779-004-0295-6

[53] A. Leroy, M. Romero, and L. Cassone, "Interactivity and materiality matter in creativity: educational robotics for the assessment of divergent thinking," *Interactive Learning Environments*, vol. 0, no. 0, pp. 1-12, Jan. 2021, publisher: Routledge _eprint: https://doi.org/10.1080/10494820.2021.1875005. [Online]. Available: https://doi.org/10.1080/10494820.2021.1875005

[54] L. Jamone, E. Ugur, A. Cangelosi, L. Fadiga, A. Bernardino, J. Piater, and J. Santos-Victor, "Affordances in Psychology, Neuroscience, and Robotics: A Survey," *IEEE Transactions on Cognitive and Developmental Systems*, vol. 10, no. 1, pp. 4-25, 2016. [Online]. Available: http://ieeexplore.ieee.org/document/7523298/

[55] R. Jung, B. Mead, J. Carrasco, and R. Flores, "The structure of creative cognition in the human brain," *Frontiers in Human Neuroscience*, vol. 7, p. 330, 2013. [Online]. Available: https://www.frontiersin.org/article/10.3389/fnhum.2013.00330

[56] L. Roux, M. Romero, F. Alexandre, T. Viéville, and C. Mercier, "Développement d'une ontologie pour l'analyse d'observables de l'apprenant dans le contexte d'une tâche avec des robots modulaires," Inria, report, Nov. 2020. [Online]. Available: https://hal.inria.fr/hal-03013685

[57] K. M. Varadarajan and M. Vincze, "AfNet: The Affordance Network," in *Computer Vision – ACCV 2012*, ser. Lecture Notes in Computer Science, K. M. Lee, Y. Matsushita, J. M. Rehg, and Z. Hu, Eds. Berlin, Heidelberg: Springer, 2013, pp. 512-523.

[58] K. Janowicz and M. Compton, "The Stimulus-Sensor-Observation Ontology Design Pattern and its Integration into the Semantic Sensor Network Ontology," in *Proceedings of the 3rd International Conference on Semantic Sensor Networks - Volume 668*, ser. SSN'10. Aachen, DEU: CEUR-WS.org, Nov. 2010, pp. 64-78.

[59] M. Musial and A. Tricot, *Précis d'ingénierie pédagogique*, 1st ed., ser. PEDAGOGIES EN DEVELOPPEMENT. De Boeck Supérieur, Feb. 2020.

[60] F. Toyoshima, A. Barton, and O. Grenier, "Foundations for an Ontology of Belief, Desire and Intention," *Formal Ontology in Information Systems*, pp. 140-154, 2020, publisher: IOS Press. [Online]. Available: https://ebooks.iospress.nl/doi/10.3233/FAIA200667

[61] L. Asprino, A. G. Nuzzolese, A. Russo, A. Gangemi, V. Presutti, and S. Nolfi, "An Ontology Design Pattern for supporting behaviour arbitration in cognitive agents," in *Advances in Ontology Design and Patterns*, ser. Studies on the Semantic Web, 2017, vol. 32, no. Chapter 8, p. 10.

[62] M. Romero, F. Alexandre, T. Viéville, and G. Giraudon, "LINE-Mnémosyne: Des neurosciences computationnelles aux sciences de l'éducation computationnelles pour la modélisation du cerveau de l'apprenant et du contexte de l'activité d'apprentissage," *Bulletin de l'Association Française pour l'Intelligence Artificielle*, no. 108, Apr. 2020. [Online]. Available: https://hal.inria.fr/hal-02541099