<figure><img src="image_1.png" /></figure>

HAL
open science

# Effects of a Tabletop Interface on the Co-construction of Concept Maps

Stefan Oppl, Chris Stary

► To cite this version:

Stefan Oppl, Chris Stary. Effects of a Tabletop Interface on the Co-construction of Concept Maps. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.443-460, 10.1007/978-3-642-23765-2_31 . hal-01591836

HAL Id: hal-01591836

https://inria.hal.science/hal-01591836

Submitted on 22 Sep 2017

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Effects of a Tabletop Interface on the Co-Construction of Concept Maps

Stefan Oppl¹, Chris Stary¹

¹ Department of Business Information Systems – Communications Engineering
Kepler University of Linz
Freistädterstrasse 315, 4040 Linz, Austria
{stefan.oppl, christian.stary}@jku.at

**Abstract.** Concept Mapping is a method for externalizing and reflecting knowledge about real world phenomena. In cooperative settings, concept maps can also be used to aid cooperative learning activities and the development of a common understanding about the mapping subject. This process imposes requirements on tool support that have hardly been addressed in existing concept mapping tools. We present a tabletop interface designed to meet these requirements. In an empirical study, the positive effects on the cooperative mapping process facilitated by the proposed system have been shown in comparison to a traditional, screen-based system.

**Keywords:** Concept mapping, tangible interface, tabletop, cooperation, alignment of meaning, sense making

## 1 Introduction

Concept maps are established means to organize and represent information. They can be used to support the process of eliciting, structuring, and sharing knowledge. According to their objective, to enable meaningful learning (cf. [1]), most of the applications of concept maps can be found in education. Concept maps, besides other means to represent information, use concepts as entity to structure items of interest. A concept can be a central term, an expressions or a metaphor. It represents a coherent unit of information for the person using it. In the course of mapping, concepts are put into mutual context by specifying concrete relationship, leading to a network of concepts, i.e. the concept map.

Based on the findings related to MiMs (Montessori-inspired Manipulatives) grasping information using tangible user interfaces could leverage the way to develop intelligence (cf. [2], [3]). In our research we target the social and cognitive dimensions of building capacities. We aim to support cooperative concept mapping using a tangible tabletop interface. To do so, we have to examine the fundamentals of concept mapping and its usefulness in different stages of learning (section 2). When reviewing existing tools, lack of support for the specifics of cooperative concept mapping can be identified (section 3). Our tabletop concept mapping system has been
designed and evaluated to overcome this deficiency (section 4 to 6). The contribution of this paper can be found in the detailed empirical examination of the effects the setting using the tangible tabletop system has on both, the individual's activities in cooperative mapping process and the outcome of the mapping process for the involved individuals.

## 2 Concept Mapping and its Effects on Learning

Persons express their items of interest and the relationships by means of language constructs. As language allows expressing human thoughts, cognitive structures and the network of concepts explicitly representing semantics seem to correspond in one way or another [3] [4].

The strengths of concepts mapping lie in the focused, but still meaningful representation of domains of interests. According to the (subsumption) theory of Ausubel [1] it is the type of structure (i.e. a network of nodes and relationships) allowing learners to capture large amounts of relevant information from textual and verbal inputs, assuming acquisition of new knowledge is dependent on what is already known.

### 2.1 Acquisition

As the prior knowledge of learners needs to be addressed effectively before entering learning processes, concept maps can serve as a means reflecting already acquired knowledge. According to Chang et al. [5] concept mapping as a graphic organizer can assist text learning through enhanced text comprehension and summarization abilities. When BouJadue et al. [6] have used concept maps as chemistry homework tools positive impact could be found with respect to organizing information, fostering metacognition, and engaging learners in building individual knowledge structures.

### 2.2 Co-Construction

Sutherland et al. [7] have suggested concept mapping as a methodological catalyst for organizational learning. Concept mapping offers a way to simultaneously understand complex systems in terms of both intra- and interpersonal relationships. In their evaluation study, concept mapping has been used by two stakeholder groups along a process of structured conceptualization. In that case concept mapping has facilitated the development of a jointly authored conceptual framework to be used in future program planning, development, and evaluation. The revealed knowledge conversion has also been argued by Fischer et al. [8] in the context of collaborative learning, as concept maps allow to deal with homogeneity/heterogeneity of information items, such as learning prerequisites, in a constructive way.
Okada et al. [9] could show the benefits of using concept maps as sense-making representations of conversations. Hereby, concept maps have been used as visual thinking technologies to create knowledge structures about and for conversations mediated through videoconferencing. In the course of negotiating collaborative concept mapping has been supplemented with oral narratives. In this way, each participant could represent the collective sense-making process. The participants could negotiate individual meanings of a concept in a straightforward way.

3 Existing Support for Concept Mapping

There exists a variety of tools for concept mapping, among them the CMapTools [10], Inspiration, (www.inspiration.com), SemNet [11], Smart Ideas (www.smarttech.com), Visimap (www.visimap.com), and Thinkmap (www.thinkmap.com).

Focusing on the cooperative application of concept mapping for the co-
construction of meaning, tool support has to provide both, equal access to the
mapping surface, and means to explicitly support the cooperation-. Existing tools
hardly address these issues. The CMapTools support spatially distributed
asynchronous and synchronous co-construction of concept maps by providing each
participant with an instance of the toolset running on dedicated computers, which can
be connected via a network [10]. However, negotiation processes occurring in the
course of co-construction processes are more effective when carried out in a face-to-
face setting [12]. In such a setting, not only the content of the concept map (and an
accompanying chat or forum, respectively) can be used to convey information, but
also facilities like gesture and facial expression support the process of negotiating a
common understanding of the learning object.

Traditional user interfaces based on screen, keyboard and mouse provide exclusive access to the input channels of the concept mapping support system. They hinder the co-construction-process by lacking equal access for all participants. They rather require for all inputs to pass the “filter” of the user currently using mouse and keyboard.

In mapping-environments not supported by computers, equal access is typically
enabled by a mapping surface accessible to all, such as a table [13]. The use of a table
provides immediacy of manipulation, which is attained by the physical creation of the
model. Participants immediately refer to a physical representation rather than abstract
items. They create and alter the model in a dialogue-based way until reaching
consensus about what is represented.

Mental models of individuals are externalized, questioned and can be modified at
the same time. However, due to the potential complexity of concept maps (especially
for more complex mapping subjects), computer-support is considered crucial [14]. As
an example, in the case of spontaneous idea elaborations relationships are not traced
automatically when concepts are moved in traditional settings. In computer-supported
settings this task can be handled by the system, thus allowing for spontaneous
changes without affecting prior results. Rolling back follows recorded modifications,
which ensures consistency.
Both, the requirements of equal access to the mapping surface and the need for computer-support can be met by tabletop interfaces. In this work, we present our approach for a tabletop interface to support cooperative concept mapping. Baraldi et al. [15], Tanenbaum & Antle [16], Do-Lenh et al. [17] and Maldonado et al. [18] have proposed similar approaches.

Baraldi et al. [15] have used gesture recognition on a multitouch table in order to support concept mapping. They however did not explicitly focus cooperative use cases. Tanenbaum & Antle [16] have also focused on the individual creation of concept maps using a tabletop system. While the concept map itself has been projected onto the table-surface, manipulation has been performed using tangible tokens and constraints, in particular for self-checks in knowledge-structuring and learning tasks. Maldonado et al. [18] present a tabletop interface for cooperative concept mapping. They use a multi-touch interface and a purely digital visualization of the concept map. Their evaluation, however, focused on a qualitative study of usability aspects and did not focus on the effects of the tabletop for the construction of the concept map. Do-Lenh et al. [17] have physically represented the concepts of the map for collaborative usage – users are able to interact with the system by directly manipulating the concept representations. Additional digital tools also accessible through the table surface are used to further augment the concept map (i.e. naming concepts using a virtual keyboard). Do-Lenh et al. have explicitly addressed and examined the tabletop’s effects on collaboration. Their findings are put in the context of our work below (cf. section ‘Related Studies’).

## 4 Tangible Tabletop Concept Mapping

The system we present here is based on a tabletop interface. We use tangible tokens for content representation and interaction with the user (see Fig. 1). Physical tokens have been chosen for implementation in favor of a multi-touch system, because the physical effects of graspable representations facilitate knowledge externalization and communication (e.g. [12], [13]).

Users place tokens and associate them accordingly to form a concept map. The tokens act as carriers for concepts. All interactions between the users and the system occur on the surface to enable simultaneous manipulation of the model. The table surface and a secondary, traditional display are used for visual information output. The system has been built upon the ReactIVision-Framework [19] for element tracking on the tabletop system.

For interaction the system provides a set of tokens available in different shapes and colors. Each shape represents an arbitrary concept type and has to be assigned by its users. The different token types allow for semantic structuring of the map by assigning a specific meaning to each type. To create associations between concepts, the representing tokens have to be shortly moved in close spatial vicinity (see Fig. 2). The participants of a mapping session name connections and concepts by using a wireless keyboard.
<figure><img src="image_3.png" /><figcaption>**Fig. 1:** System Overview, showing a simple procedural concept map on both, the table surface and the secondary display (computer screen)</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 2:** Labeling & Associating</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 3:** Using Tokens as Containers</figcaption></figure>

To overcome the restrictions of the limited size of the table surface and to allow for additional structuring, all tokens have been designed and constructed as containers (see Fig. 3). A (sub-)map can be saved to a embeddable token by placing the token in a designated area of the surface ("tray"). The (sub-)map then can be attached to a concept by putting the embeddable token briefly next to an open container token and simply placing it inside afterwards. Placing tokens inside a container significantly adds weight and also produces audible feedback when grasping the container. In this way, awareness about additional, currently hidden, information becomes available.

Support for exploration of knowledge domains is explicitly provided by history and reconstruction support. The system automatically tracks the creation history of the concept map and allows for navigation through the time line (similar to the concept of design history [20]). The reconstruction of former states of the map, as required in case of (experimental) changes, is supported by the system by displaying step-by-step directions on which concepts to remove, move or add to reestablish the former concept map. While undoing changes this way still cause more effort than in purely digital tools, our system is the first explicitly addressing the requirement of undoing changes in physical modeling environments, and providing an implementation to that respect.

The system has been designed to be controllable by several people simultaneously. People group around the table, which is about 110 cm in height. In order to foster
interaction, activity and changing roles among the users, they need to stand upright
rather than being seated. In the course of collaboration they position themselves
where their current focus of interaction lies.

The active table surface is 100 x 80 cm in size and allows for placement of about
15 to 20 concept tokens at a time. The software subsystem is able to track and
visualize an arbitrary number of interactions in real time. Even when in non-mapping-
states (like history navigation or concept naming), the system keeps track of changes
to the map on the table surface and updates its internal representation of the map
accordingly. Changes can then be propagated to the output channels (table surface,
secondary displays or attached remote displays) as soon as the live mapping
visualization is active again.

5 Evaluation

We first detail the finding of hypotheses and the study design before reporting the
results and outcome.

**5.1 Hypotheses**

To be able to outline the evaluation design, the metrics of “effectiveness” have to be
specified. According to Okada et al. [9], a concept map should serve as a mediator for
conversations. Sutherland et al. [7] argue to consider concept maps as a means to
develop a common understanding of the represented problem and to integrate
different viewpoints in a constructive way. As basic requirement the system must be
open with respect to the concept mapping process itself, i.e. has to allow the creation
of arbitrarily complex concept maps representing arbitrary content. If these three
requirements can be met when using the presented system, it can be considered to
support cooperative concept mapping effectively:

* H1: The tabletop system allows for the creation of arbitrarily complex concept maps.

* H2: The tabletop system enables cooperation among the participants in the mapping process.

* H2a: Using the tabletop system leads to a higher degree of cooperation during map construction compared to using a screen-based tool.

* H3: The tabletop system enables the development of a common understanding of the mapping topic by supporting the alignment of meaning.

* H3a: Development of a common understanding is facilitated when using the tabletop system in comparison to a screen-based system.

Hypothesis 1 addresses the fundamental requirement of being able to use the
system for concept mapping. This hypothesis is also used to test, whether the feature
to create hierarchical maps (using the container metaphor) is an adequate replacement
of an unlimited modeling surface (as available on a screen-based mapping tool).
Hypotheses 2 und 3 capture the specific requirements for cooperative learning and
reflection processes. The latter are supplemented with an additional sub-hypothesis, in
order to contrast our system with traditional, screen-based concept mapping tools. The goal of the sub-hypotheses is to test, whether tabletop systems allow for better, i.e. more effective, support for cooperative concept mapping than screen-based systems.

## 5.2 Related Studies

Do-Lenh et al. [17] have evaluated the usage of a tabletop concept mapping tool in cooperative settings. In an additional study [21], the same authors have also evaluated the learning outcome of collaborative tabletop concept mapping in comparison to the CMapTools as a representative of traditional, screen-based concept mapping tools.

Regarding the usage process, the authors have found out, that the tabletop system fosters a continuous flow of cooperation among the participants during the whole process of the creation of the concept map. Cooperative creation of a concept map using the CMapTools is characterized by alternating phases of largely individual content creation (i.e. having on participant taking the mapping initiative) and cooperative phases of reflection of the concept map.

Regarding the learning outcome, the authors were not able to justify the positive impact of a tabletop system on the learning outcome. On both, individual and group level, the participants using the screen-based system reached higher learning performance than the participants using the tabletop system. They however found a ‘qualitatively better collaboration’ in cases where the tabletop system had been used. They also argue that the physical setting could have influenced the intensity of collaboration, as the physical degrees of freedom might affect group dynamics. In this respect, their setting differs from ours, as people were sitting around the tabletop surface in their setup, whereas participants are standing upright around the tabletop in our case.

## 5.3 Study Design

**Context.** Two studies were carried out in the course of a lab on business process modeling. Students attending this lab learn to represent knowledge about business processes using different three process modeling languages. The goal of the lab is to recognize the fact that different scenarios of use for process models require different forms of representations. During the lab, students are introduced to three graphical modeling languages and have to apply each language by modeling a common business scenario. In their final task, the students have to collaboratively reflect on their experiences and contrast the modeling languages regarding their strengths and weaknesses for different scenarios of use. They document their findings using a concept map, which they co-create in the course of their reflection and discussion process (for a sample, see Fig. 4). This task was subject to both studies.

The scope of the first study was to examine the effects of the tabletop interface on the collaborative concept mapping process (contributing to the evaluation of H1, H2 and H3). 54 students of business information systems in the first part of their undergraduate studies participated in the study (46 male, 8 female). None of the participants had any prior knowledge in process modeling or concept mapping. To
complete the concept mapping task, they were arbitrarily grouped into 18 teams of 2 or 3 persons, who arranged around the tabletop system.

In the second study, the tabletop concept mapping scenario was compared to a collaborative mapping setting using a traditional, screen-based system (mainly focusing to the evaluation of H2a and H3a but also contributing to H1, H2 and H3). 49 students in the same phase of their studies as described above participated in the study (40 male, 9 female). To complete the concept mapping task, they were arbitrarily grouped into 23 teams of 2 or 3 persons. 11 groups created the concept map using the tabletop system again arranging themselves around the table (standing, no seats), while 12 groups used the CMapTools toolset collectively sitting in front of a computer screen.

<figure><img src="image_6.png" /><figcaption>**Fig. 4:** Sample Concept Map (screen-based viewer of the tabletop system, concepts labeled in German, association labels have not been used by the modelers)</figcaption></figure>

**Measures.** Data was collected during and after the mapping sessions from three sources: (1) Video recordings of the mapping process from two spatial perspectives – one focusing on the participants, the other covering the mapping surface in detail, (2) recordings of the evolution of the created concept maps (by the system), (3) post-mapping questionnaires on usability, usefulness and perceived outcome.

In total 7 quantitative and 5 qualitative measures were extracted from the collected data. In the following, we describe the measures and how they have been used to test the hypotheses:

*Quantitative measures:*

* Hypothesis 1 – Creation of Concept Maps:

* Size of concept map (S) – measured by number of concepts used. S can be used as a measure for the complexity of the map.

* Speed of map creation (SC) – measured in seconds per concept used. SC can be used as a measure for the system's ease of use.
* *Number of concept types used (CT)* – calculated by counting the number of different semantic types of concepts used. CT can be used as a measure for amount of content structuring.

* *Connectedness (C)* – measured in connections per concept used. Can be used as a measure for the complexity of the map.

* **Hypothesis 2 – Cooperation:**

* *Distribution of activity time among participants (ATD)* – extracted from videos; considering not only physical initiative but activity in general (including discussion, negotiation, giving directions for mapping). Cooperative activity times are distributed equally among the involved participants. ATD serves as a measure for involvement of the participants.

* *Turn Taking of physical initiative (TPI)* – extracted from videos; focusing on physical initiative and counting how often participants switched tool control. Simultaneous physical activity (SPA) on the tabletop was recorded separately (not possible in the case of CMapTools). TPI can be used as a measure for the amount of interaction occurring among the participants.

* **Hypothesis 3 – Development of common understanding:**

* *Fraction of mapping time used for discussion (FDT)* – extracted from videos. Complete mapping time was distributed among time used for map creation, time used for technical and administrative discussion and time used for discussion about the content of the map. The latter is used to calculate this measure. FTD is a measure of the amount of time dedicated to activities for the alignment of individual views.

*Qualitative measures (extracted from questionnaires and videos):*

* **Hypothesis 1 – Creation of Concept Maps:**

* *Perceived system restrictions during the mapping process (pR)* – users were asked which aspects of the system they perceived to restrict the mapping process, i.e. not letting them express what they intended to externalize. pR not only is a measure of perceived restrictedness but also gives hints on how interaction with the system can be enhanced. For the redesign of interaction, the subjective findings were cross-checked with the video recordings showing the interaction with the system in order to identify the actual pitfalls and misunderstandings causing interaction to break.

* **Hypothesis 2 – Co-creation:**

* *Perceived cooperation during the mapping process (pCP)* – users here described the amount and specifics of cooperation during the mapping process. This included explicit questions on implicit or explicit role taking and switching during the process as well as open questions on how well cooperation was perceived in general.

* *Observed cooperation during the mapping process (oCP)* – extracted from videos. Cooperation was qualitatively assessed based upon the way individual contributions where integrated in the
common concept map. It includes omitted contributions, misunderstandings or cooperative effort to create or alter content.

o *Perceived ability to contribute the common concept map (pCO)* – this measure targets perceptions of the individual contribution to the mapping result. Users had to explicitly describe in which aspect of the concept map (e.g. content, structuring, ...) they felt to have contributed to most. The results of this measure have been cross-checked with the result of pCP, as many users described the roles and contributions of their fellow group members there.

* Hypothesis 3 – Development of common understanding:

o *Perceived impact of mapping process and result on individual understanding (piU)* – users were asked, whether and to which extent the mapping process and the mapping result contributed to their individual understanding of the topic the concept map was created on. The questions were formulated in a way that encouraged users to describe, in which aspects or areas their understanding changed. pIU thus requires introspection, reflecting on and externalizing one's own cognitive structures.

o *Perceived impact of mapping process and result on development of a common understanding (pCU)* – in addition to their perception of changes in their individual understanding, users were asked whether they found the mapping process and/or result helpful for the development of a common understanding about the mapping topic across all participants. Again, users were asked to describe, where they perceived changes and based on which perceptions they identified these changes.

Along with pIU, data for pCU was gathered not only directly after the mapping session but also for a second time about 8 weeks after the session to examine long-term impact of the concept mapping session.

Measures based on *perceived understanding* might appear to be inferior when compared with “objective” measures e.g., based on the expert interpretation of mapping results. In the following, we describe why our approach to measure mental model alignment via individual perception nevertheless is valid. Changes in individual cognitive structures (which are a consequence of cooperatively creating a concept map), however, cannot be measured directly, but only via additional externalized structures (such as, again, concept maps) [22]. Diagnosing changes requires repeatedly letting people externalize their views about the same phenomena. This creates methodical problems such as better results in later sessions due to learning effects. These problems can only be addressed with certain quantitative statistical approaches (such as “Hierarchical Linear Modeling” [22]). Measuring “alignment” would, however, require being able to not only quantify changes but also gain qualitative data about how the cognitive structures have changed. Currently available approaches, that are methodologically sound and correctly deal with the challenges of change diagnosis, do not provide qualitative data.

Although considered suboptimal when examining changes in cognitive structures [22], subjective measures still appear to be the most suitable instrument when judging
cognitive structure alignment, as they at least provide the required qualitative data. The use of subjective measures, however, is a limitation of this study, as reliability of the collected data cannot be judged for these measures.

**Testing.** Hypothesis 1 is tested using the measures *Size of concept map (S)*, *Speed of map creation (SC)*, *Number of concept types used (CT)*, *Connectedness (C)* and *Perceived system restrictions during the mapping process (pR)*. S, CT and C are calculated for maps created with the tabletop and the CMapTools. As the CMapTools have already turned out to support Concept Mapping (cf. Canas et al., 2004), similar results for identical mapping tasks indicate appropriate mapping support by the tabletop. pR provides additional hints on potential restrictions on concept mapping imposed by the tabletop system.

Hypotheses 2 and 2a are tested using the measures *Distribution of activity time among participants (ATD)*, *Turn Taking of physical initiative (TPI)*, *Perceived cooperation during the mapping process (pCP)*, *Observed cooperation during the mapping process (oCP)* and *Perceived ability to contribute the common concept map (pCO)*. All measures are calculated for mapping sessions using the tabletop system and the CMapTools, respectively. Both ATD and TPI are measures for the involvement of the participants. Similar individual ATD values indicate a similar amount of participation in the mapping processes. TPI and its sub measure SPA directly indicate the strength of cooperation during the mapping process. pCP, oCP and pCO give hints on cooperation support based on subjective perceptions of the participants.

Hypotheses 3 and 3a are tested using Fraction of mapping time used for discussion (FDT), Perceived impact of mapping result on individual understanding (pIU) and Perceived impact of mapping result on development of a common understanding (pCU). FDT shows the fraction of time dedicated to discussing the content of the concept map. Discussion of mapping content is an integral part of the development of a common understanding. High values for FDT thus indicate potential for the development of a common understanding. The actual impact of system usage is only measured qualitatively using pIU and pCU. Both measures are again based on the subjective perceptions of the participants.

# 6 Results

**Size (S):** The average size of a tabletop concept map considering both studies is 14.68 concepts (SD = 7.9, n = 29). In the second study, the tabletop concept maps (M = 12.18, SD = 8.84, n = 11) were significantly smaller than those created with CMapTools (M = 25.92, SD = 7.04, n = 12) for the same mapping task (one-sided Wilcoxon-test for unpaired samples: W = 8, p < 0.001).

**Speed of Creation (SC):** The average time used to place or alter a specific concept was calculated by dividing the overall modeling time (total time minus time for discussion and technical or administrative issues) through the total number of placed concepts. On the tabletop (considering both studies) the average time per concept was
40.3 s (SD = 14.2, n = 29). In the second study, the speed of creation for tabletop concept maps (M = 32.5 s, SD = 11.26 s, n = 11) was significantly faster than for those create with CMapTools (M = 48.12 s, SD = 17.91 s, n = 12) for the same mapping task (one-sided Wilcoxon-test for unpaired samples: W = 83, p = 0.021).

**Concept Types (CT):** The tabletop basically only provides 3 different types of tokens. However, using more than 3 concept types was possible, once sub models with different semantics were embedded. Considering both studies, the average number of concept types used on the tabletop was 3.14 (SD = 1.04, n = 29). In the second study, the number of concept types of tabletop concept maps (M = 3.64, SD = 1.50, n = 11) did not significantly differ from that of maps create with CMapTools (M = 3.50, SD = 2.11, n = 12) for the same mapping task (two-sided Wilcoxon-test for unpaired samples: W = 75, p = 0.587).

**Connectedness (C):** Connectedness is calculated by dividing the overall number of connections through the overall number of concepts. On the tabletop (considering both studies) the average connectedness was 0.96 (SD = 0.41, n = 29). In the second study, the connectedness of tabletop concept maps (M = 1.09, SD = 0.36, n = 11) did not significantly differ from the connectedness of maps create with CMapTools (M = 1.23, SD = 0.20, n = 12) for the same mapping task (two-sided Wilcoxon-test for unpaired samples: W = 64.5, p = 0.476).

**Distribution of activity time (ATD):** On the tabletop (considering both studies), distribution of activity time for groups of three people (n = 19) was 49.3 % for participant A, 33.1 % for participant B and 17.6 % for participant C (participants were sorted in decreasing order from A to C). For groups of 2 persons (n = 10) was 60.4 % (SD = 12.2 %) for participant A and 39.6 % (SD = 12.2 %) for participant B.

In the second study, only groups of two persons have been analyzed for reasons of sample size (CMapTools: 11 samples for 2 persons, 1 sample for 3 persons, Tabletop: 9 samples for 2 persons, 2 samples for 3 persons). Distribution of activity time (percentual values for participant A and participant B, where A is the person with higher activity) on the tabletop system (M: 57.5 % - 42.5 %, SD: 11.3 % - 10.4 %, n = 9) did not significantly differ from the distribution of activity time for groups using CMapTools (M = 56.6 % - 43.4 %, SD = 7.4 % - 9.6 %, n = 11) for the same mapping task (two-sided Wilcoxon-test for unpaired samples calculated on the percentual differences of the activity time of both participants: W = 51, p = 0.86).

**Physical Turn Taking (TPI):** In the 29 sessions carried out on the tabletop, 17 cases have been identified (SPA), in which no explicit physical turn taking was carried out, as at least two participants were working simultaneously on the tabletop. In an average mapping time of 16 minutes and 40 seconds (SD = 10m 1s), 2 turn taking activities have been identified in 6 of the remaining sessions, another 6 sessions did not show any physical turn taking activities. In the second study, 7 cases of simultaneous physical activity have been identified on the tabletop (n = 11), while no such case was present for CMapTools (n = 12). Exclusive physical access to the mapping controls by one person (i.e. no turn taking at all) was identified in 2 cases on the tabletop (n = 11) and in 8 cases for CMapTools (n = 12). On the tabletop, the
remaining 2 cases both showed 2 turn taking activities (in an average mapping time of 7m 7s, turntaking on after 3m 33s). For the CMapTools, 2 turn taking activities were identified in two cases, 4 and 8 turn taking activities were identified one time each (in an average mapping time of 19m 22s, turn taking on average happens after 4m 42s).

**Discussion Time (FDT):** On the tabletop (considering both studies) the average fraction of discussion time per concept was 73.13 % (SD = 6.43 %, n = 29). In the second study, the fraction of discussion time of tabletop concept maps (M = 76.08 %, SD = 8.84 %, n = 11) was significantly higher than that of maps create with CMapTools (M = 57.15 %, SD = 7.49 %, n = 12) for the same mapping task (one-sided Wilcoxon-test for unpaired samples: W = 120, p < 0.001).

In the following we discuss the results of the **qualitative measurement** with respect to the hypotheses. For further results the readers are referred to [23].

## 6.1 Discussion

The results of hypotheses testing described in the section above require some further discussion and interpretation, in particular when putting evaluation results in mutual context.

**H1 – The tabletop does not allow for the creation of arbitrarily complex concept maps:** Hypothesis 1 had to be rejected – the tabletop system does not allow the creation of arbitrary concept maps. Compared to maps created with the CMapTools, the size of the maps was significantly smaller (Measure S). Measure pR also showed that the two major restrictions of the tabletop system were perceived to be the too small size of the mapping surface and the too big size of the mapping elements, respectively. The amount of concept types used did not significantly differ from maps created with the CMapTools (Measure CT). The amount of concept types limited to three (due to the current hardware implementation) was perceived as a problem by some of the users. For both, model size and semantics, the option to embed sub models does not seem to be an appropriate alternative to arbitrarily sized maps created on a single layer.

From a usability point of view, however, the tabletop does not slow down the creation of a concept map. The speed of creation actually was significantly faster for tabletop maps (Measure SC). Moreover, values for Connectedness did not show any significant difference (Measure C). Results of the measure pR also confirm that the system is considered usable from a technical point of view (except certain features of the original version, which caused confusion due to misleading UI metaphors. These features underwent a redesign based on the initial findings and were not perceived to be incomprehensible after redesign – for a more detailed elaboration on this aspect, cf. [24]).

Overall, H1 had to be rejected because of the design decision to use tangible elements for content representation. If the requirement of providing a concept mapping environment with capabilities equivalent to screen-based tools is a central one (which in the case presented here it was not), choosing a multi-touch surface with
a digital representation of the concept map might be a more suitable option. It
however remains unclear which effect a multi-touch-based system would have on
cooperation and the alignment of individual understanding – from a conceptual point
of view, physical representation should provide added value for these aspects (e.g.
[12], [13]).

**H2 – The tabletop enables cooperation among participants:** Hypothesis 2 was verified – the tabletop system does support cooperative concept mapping. While the amount of activity time is not equally distributed (Measure ATD), tests show that all participants were significantly involved. Further studies using scenarios with up to 6 participants show corresponding time distributions – involvement does not decrease for even more participants. Simultaneous mapping activities (Measure SPA) have been identified in the majority of sessions using the tabletop system. In contrast to the sessions using CMapTools, exclusive physical access to the mapping controls could hardly be identified on the tabletop. Qualitative measures (Measures pCP, oCP and pCO) consistently showed the tabletop system to be perceived supportive with respect to cooperation. Explicitly, the physical setup (standing around a table) and the ability to physically reference and modify the map were mentioned and observed as positive factors several times. oCP on the tabletop system also showed that a restricting factor for simultaneous interaction – the editable but still exclusive access to the keyboard used for labeling – in fact seems to foster cooperation, as labels are discussed intensively before being entered. Using CMapTools, oCP does not show this tendency – in most of the cases the participant in control of the keyboard decides on the label autonomously. oCP provides hints to a possible explanation for this difference: usage patterns show, that on the tabletop, placement and association of elements and labeling them is a distributed activity shared among at least two participants, while for CMapTools the exclusive access to both, keyboard and mouse, leads to temporary full control of the mapping process by a specific participant (see also description of measure TPI above).

**H2a – The tabletop leads to more cooperative activities than the screen-based system:** As for Hypothesis 2a, TPI shows an average duration for one turn which is significantly lower for the tabletop than for the CMapTools. Simultaneous physical activity (Measure SPA) can be identified in more than two thirds of the tabletop sessions and is not present in the CMapTool-sessions. In turn, exclusive access to the input channels can be identified in 75% of the CMapTool-sessions while it has only be observed in 18% of the tabletop-sessions. Video analysis (oCP) confirms the effects of exclusive access to input channels (like mouse and keyboard) by a single user and shows, that the static nature of sitting around the system (as opposed by standing in the case of the tabletop) seems to cause the differences for the measures TPI and SPA. Activity time distribution, however, did not differ significantly. However, in 20% of the screen-based-sessions, participants noted that they were not able to contribute to the concept map at all. This was not the case in any of the tabletop sessions. Accordingly, the process of cooperation was perceived suboptimal in some cases for the screen-based system.
The results for hypotheses 2 and 2a are in line with the study of Rogers et al. [25] and Lucchi et al. [26], in which the effects of different interface types (screen-based, multitouch, tangible tabletop) have been examined in more detail than in the work presented here.

**H3 – The tabletop supports the development of a common understanding:** The fraction of time dedicated to discussion (Measure TDT) as a quantitative measure used to test Hypothesis 3 show, that on average two thirds of the overall mapping time are used for activities contributing to negotiation of meaning. This amount is significantly lower in case of the screen-based system. Qualitative feedback (Measures pIU and pCU) consistently shows, that the system was perceived to have positive impacts on the development of a common understanding (with over 75% stating that it was easy to find consensus about the content to be represented in the map). Statements on pIU and pCU did not show any fundamental changes over time. People described similar perceptions when questioned directly after the mapping session and 8 weeks afterwards, where the latter were more elaborate due to perceptions of the impact in daily work practices.

**H3a – Development of a common understanding was not facilitated when using the tabletop in comparison to the screen-based system:** The perceived effects of on the development of a common understanding tested in Hypothesis 3a (Measures pIU and pCU) did not show any significant difference between the two examined tools. In both cases the majority of participants reported on the perception that it was “easy to come to a consensus” or to “find a common understanding” (80% for the CMapTools, 78% for the tabletop system).

The result for the hypotheses 3 and 3a however must not be overrated, as the concept maps created before the cooperative mapping session did not show fundamental differences in the understanding of the conceptual relationships within the mapping domain. Thus the development of a common understanding was likely to be perceived in a straightforward way. Future evaluations will require focusing on a more controversial task in order to be able to examine the effects of the systems in more detail.

## 6.2 Interpretation of Overall Results

Putting the results for each hypothesis in mutual context, leads to additional findings: The far less detailed concept maps created with the tabletop (as compared to the screen-based-system) lead to a significantly higher amount of discussion during the concept mapping process. When using the CMapTools participants have focused more on the process of representation than on the alignment of different individual understandings. Results of the examination of cooperation during the mapping process additionally show that groups using the CMapTools mostly have a dedicated “system operator” who actually creates the map. Regardless of the amount of interaction occurring during mapping, video analysis of the mapping sessions has revealed for all sessions using CMapTools that operating participants act as “filters”,
in particular, omitting content input by others which they consider not appropriate. The risk of representing the perception of information of a single participant in the map and leaving it unquestioned thus seems to be higher when using screen-based-tools compared to the tabletop.

Consequently, the restrictions of the tabletop system seem to contribute to the well perceived results on cooperative alignment of meaning – higher effort for map creation and the restricted modeling space seem to lead to more focused – however, not comprehensive – maps, which serve as an anchor point for further discussion. In the context of the design of tabletop-based interaction in groups in general, these findings direct research to a detailed study of the objectives of tabletop-supported applications. Whenever the goal is to serve as a facilitator and mediator for inter-personal communication and direct information exchange, tangible representations of the relevant data might be appropriate. In case an application mainly targets towards comprehensive representation of information, a non-tangible way of representing data (i.e. using touch-based interaction on a digitally enhanced surface) might be a better choice. Tangible representations still pose restrictions on the complexity of data to be represented. They cannot be overcome by developing workarounds, as shown for the container metaphor in this paper.

# 7 Conclusions

Concept Mapping is a method for externalizing and reflecting knowledge about real world phenomena. In cooperative settings concept maps can also be used to aid cooperative learning activities and the development of a common understanding about the mapping subject. This process imposes requirements on tool support that have hardly been addressed in existing concept mapping tools. In this paper, we have presented a tabletop interface designed to meet these requirements. In the empirical study the positive effects of the proposed system have been shown in comparison to a traditional, screen-based system.

The major limitation of this study turned out to be the selected mapping task, which has caused less controversy as expected. Due to the already existing congruence of the individual understandings of the mapping subject across the set of participants before the concept-mapping task, effects of the system on cooperative alignment of meaning could hardly be identified. Future work will have to build upon mapping topics and participants that lead to higher controversy and more effort of find a common understanding. Methodologically, one way to evaluate the development of a common understanding should be found that goes beyond analyzing individual perceptions. Candidates for such methods are dialogue-consensus-methods using structure-elaboration-techniques [13], and methods for diagnosing learning-related changes in mental models [22].

Overall, the proposed system shows positive effects on the cooperative construction of concept maps. While the physical restrictions of the tangible approach prevent the system from facilitating concept mapping as originally described in literature, the same restrictions foster discussion and steps towards cooperative alignment of meaning. For the selected sample topic the study shows that the tabletop
approach and a screen-based approach support concept mapping on different levels
while leading to similar outcomes regarding the development of a common viewpoint.
The different qualities of support will have to be explored in future studies.

References

1. Ausubel, D.P.: Educational Psychology, A Cognitive View. New York: Holt, Rinehart and Winston, Inc. (1969)

2. Antle, A. N.: The CTI framework: informing the design of tangible systems for children. In *Proceedings of the 1st international Conference on Tangible and Embedded interaction* (Baton Rouge, Louisiana, February 15 - 17, 2007). TEI '07. ACM, New York, NY, 195-202. (2007)

3. Feltham, F. G.: Do the blocks rock: a tangible interface for play and exploration. In *Proceedings of the 20th Australasian Conference on Computer-Human interaction: Designing For Habitus and Habitat* (Cairns, Australia, December 08 - 12, 2008). OZCHI '08, vol. 287. ACM, New York, NY, 188-194. (2008)

4. Quillan, R. M.: Semantic memory. In M. Minsky (Ed.), *Semantic Information Processing*, pp. 216-270. Cambridge, MA: MIT Press. (1968)

5. Chang, K.E., Sung, Y.T., Chen, I.D.: The effect of concept mapping to enhance text comprehension and summarization. The Journal of Experimental Education. 71(1):5-23. (2002)

6. Boujadue, S., Atthie, M.: The effect of using concept maps as study tools on achievement in chemistry. *Eurasia Journal of Mathematics, Science & Technology Education*, 4(3): 233-246. (2008)

7. Sutherland, St., Katz, St.: Concept mapping methodology: A catalyst for organizational learning. *Evaluation and Program Planning*, 28, 257-269. (2005)

8. Fischer, F., Mandl, H.: Knowledge convergence in computer-supported collaborative learning: The role of external representation tools. *Journal of the Learning Sciences*, 14(3), 405-441. (2005)

9. Okada, A., Tomadaki, E., Buckinham Shum, S., Scott, P.: Fostering open sensemaking communities by combining knowledge maps and videoconferencing. *Upgrade, IX(3)*, 27-36. (2008)

10. Cañas, A., Hill, G., Carff, R., Suri, N., Lott, J., Eskridge, T., Gómez, G., Arroyo, M., and Carvajal, R.: Cmaptools: A knowledge modeling and sharing environment. In *Concept Maps: Theory, Methodology, Technology, Proceedings of the 1st International Conference on Concept Mapping*. Pamplona, Spain: Universidad Pública de Navarra. (2004)

11. Fisher, K. M., Faletti, J., Patterson, H. A., Thornton, R., Lipson, J., Spring, C.: Computer-based concept mapping: SemNet software - a tool for describing knowledge networks. *Journal of College Science Teaching*, 19(6):347-352. (1990)

12. Hornecker, E.: Understanding the benefits of graspable interfaces for cooperative use. In *Proceedings of Coop*, pp. 4-7. (2002)

13. Dann, H.-D.: Variation von Lege-Strukturen zur Wissensrepräsentation. In *Struktur-Lege-Verfahren als Dialog-Konsens-Methodik*. Ein Zwischenfazit zur Forschungsentwicklung
bei der rekonstruktiven Erhebung subjektiver Theorien, B. Scheele, Ed., vol. 25 of Arbeiten zur sozialwissenschaftlichen Psychologie. Aschendorff, pp. 2–41. (1992)

14. Novak, J., and Cañas, A. J.: The theory underlying concept maps and how to construct them. Technical Report IHMC CmapTools 2006-01, Florida IHMC (2006)

15. Baraldi, S., Del Bimbo, A., Valli.: A. Interactive concept mapping through gesture recognition on a tabletop. In Proceedings of CMC2006, San José, Costa Rica. (2006)

16. Tanenbaum, K., and Antle, A.: Using physical constraints to augment concept mapping on a tangible tabletop. In Proceedings of the International Conference on Education and Information Technology, pp. 539–547. (2008)

17. Do-Lenh, S., Kaplan, F., Sharma, A., and Dillenbourg, P.: Multi-finger interactions with papers on augmented tabletops. In TEI ’09: Proceedings of the 3rd International Conference on Tangible and Embedded Interaction. ACM, pp. 267–274. (2009)

18. Maldonado, R.M., Kay, J., Yacef, K.: Collaborative concept mapping at the tabletop. In: ACM International Conference on Interactive Tabletops and Surfaces. pp. 207–210. ITS ’10, ACM, New York, NY, USA (2010)

19. Kaltenbunner, M., and Bencina, R.: reacTIVision: a computer-vision framework for table-based tangible interaction. In TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, ACM Press, pp. 69–74. (2007)

20. Klemmer, S., Thomsen, M., Phelps-Goodman, E., Lee, R., and Landay, J.: Where do web sites come from? capturing and interacting with design history. Human Factors in Computing Systems, CHI Letters, 4(1). (2002)

21. Do-Lenh, S., Kaplan, F., and Dillenbourg, P.: Paper-based concept map: the effects of tabletop on an expressive collaborative learning task. In Proceedings of the 2009 British Computer Society Conference on Human-Computer Interaction, British Computer Society, pp. 149–158. (2009)

22. Ifenthaler, D.: Diagnose lernabhängiger Veränderung mentaler Modelle - Entwicklung der SMD-Technologie als methodologisches Verfahren zur relationalen, strukturellen und semantischen Analyse individueller Modellkonstruktionen. PhD thesis, University of Freiburg. (2006)

23. Oppl, S.: Unterstützung expliziter Articulation Work. PhD-Thesis, Technical University of Vienna. (2010)

24. Oppl, S.: Konsistente Verwendung von Metaphern als Erfolgskriterium für komplexe Tangible User Interfaces. In Workshop-Proceedings der 9. fachübergreifenden Konferenz Mensch und Computer 2009. (2009)

25. Rogers, Y., Lim, Y., Hazelwood, W., and Marshall, P.: Equal Opportunities: Do Shareable Interfaces Promote More Group Participation Than Single User Displays? Human-Computer Interaction, 24(1-2):79–116. (2009)

26. Lucchi, A., Jermann, P., Zufferey, G., Dillenbourg, P.: An empirical evaluation of touch and tangible interfaces for tabletop displays. In: Proceedings of TEI 2010. ACM Press (2010)