# Towards Linking Tool Functionalities to Processes of Collaborative Learning

Sebastian Simon, Iza Marfisi-Schottman, Sébastien George

► To cite this version:

Sebastian Simon, Iza Marfisi-Schottman, Sébastien George. Towards Linking Tool Functionalities to Processes of Collaborative Learning. 16th International Conference on Computer-Supported Collaborative Learning (CSCL), International Society of the Learning Sciences (ISLS), Jun 2023, Montréal, Canada. pp.205-208. [https://hal-04176094](https://hal-04176094)

HAL Id: hal-04176094

https://univ-lemans.hal.science/hal-04176094

Submitted on 2 Aug 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

# Towards Linking Tool Functionalities to Processes of Collaborative Learning

Sebastian Simon, Iza Marfisi-Schottman, Sébastien George

sebastian.simon@univ-lemans.fr, iza.marfisi@univ-lemans.fr, sebastien.george@univ-lemans.fr

Université Le Mans, France

**Abstract:** A large variety of tools for collaborative learning has been developed in the last 20 years. While the impact of these tools on collaboration is acknowledged by the research community, a fine-grained understanding of links between the various functionalities offered by these tools and collaborative processes does not exist yet. In this paper, a meta-analysis was carried out on 49 collaboration studies to examine the relationship between individual functionalities on collaborative processes.

## Introduction

Education is in a deep process of transformation. In a context of pandemics, fragmentation and social disruptions, societies have to adapt their way of teaching future generations. The Internet allows for remote classrooms, but the Covid19 pandemic has shown how isolation degrades mental health (Cullen et al., 2020). Semi-/autonomous, technology-assisted learning in small groups might be the compromise needed between full class configurations and fully remote learning. The main beneficial component of working in groups for learning is **collaboration** (Laal & Ghodsi, 2012). However, this complex mechanism is not fully understood yet (Cress et al., 2021). This challenge could be met by further studies with digital tools for collaborative learning. However, designing collaborative software remains challenging, precisely because a fine-grained understanding of how tool functionalities impact collaboration is still missing.

As we will show in the first part of this paper, previous studies often measure the effect of entire tools, consisting of several functionalities, to promote collaboration (i.e., voting systems, participation visualizations, shared workspaces...). We present, in the second part, the method and the results of our meta-analysis of 49 studies. The objective is to compare the results of previous studies in order to identify the impact of tool functionalities along three collaborative dimensions. Finally, we put the results of our work into perspective with remaining and future work.

## Challenges and previous work

Among the variety of tools that have been developed to support collaboration, large horizontal interactive displays have been found to positively impact collaboration (Mateescu et al., 2019) when embedded in collaborative activity design (Stahl, 2006). Since the advent of technology allowing for large touchscreens, the latter have been the subject of extensive research for small collaborative group learning in lab and classroom settings (Martinez-Maldonado et al., 2014). While many of those experimentations have proven some positive impact on collaboration, it is not clear which software functionality fosters which aspect(s) of collaboration. This is due to the fact that the tested tools often offer a large variety of functionalities to effectively support and enforce collaboration (Sugimoto et al., 2004). In addition, definitions for collaboration and frameworks have changed substantially and vary across disciplines, leading to **conceptual uncertainty** (i.e. different theoretical concepts for defining and measuring collaboration) among available studies on collocated collaboration.

Therefore, linking collaborative functionalities to collaborative learning processes currently presents a great challenge, due to a conceptual uncertainty, tools consisting of multiple functionalities and a limited understanding of the interplay between functionalities within tools.

In this paper, we propose a novel meta-analysis to provide insights on which functionality may impact which dimensions of collaboration. In order to reduce conceptual uncertainty, we use a simplified version of the comprehensive collaboration framework by Simon et al. (2022), that provides three categories of collaborative processes: Participation (i.e. the level of involvement by participants), Awareness (behavioural, cognitive and social) and Coordination flow (i.e. collective coordination strategies). This framework is the result of an analysis of existing frameworks across different domains.

This work attempts to set a basis for a more fine-grained vision of the interplay between tool functionality and collaborative processes than previous studies. As such, the results can be used to evaluate tools and will simultaneously serve the authors for the development of new modular, collaborative multi-user software frameworks enforcing teaching scenarios and collecting user data for an understanding of tool-mediated collocated collaboration on the process level.
<figure><img src="image_3.png" /></figure>

# Analysis

## Method

As depicted in Figure 1, we initially explored various studies on collaboration around interactive tabletops, from sources such as the international handbook of international learning sciences (Cress et al., 2021) and other journal articles. We identified **20 recurrent tool functionalities**. In order to obtain a qualitative corpus of studies, we examined **30 articles** that Mateescu et al. (2019) filtered in their recent systematic review (12/2019) of more than 1400 articles on collaborative tools. They selected papers related to collaboration around interactive tabletops that meet eligibility criteria such as “sound experimental methodology” (i.e. authors must provide information about the method used, sample size and statistical details) and screening criteria (i.e. article with 10 pages or longer, English language etc.). We then included papers from our initial exploratory study that met Mateescu (2019)’s eligibility criteria but had been filtered due to the screening criteria “10 pages or longer”. We include those studies because work on isolated tool functionalities often fits 8 pages. The 20 functionalities identified in our initial exploratory study, reappeared throughout the works identified by Mateescu et al. (2019).

Consequently, we identified which of the framework’s three **collaborative process categories** *Participation*, *Awareness* and *Coordination flow (PAC)* were impacted by each tool. Each paper was screened for terminology that aligned with the PAC categories. As a result, we were able to summarize our findings in a table listing all studies, functionalities and their impact by category (see Table 1).

After analyzing the different studies, we summed up occurrences of each functionality across studies and their correlation to PAC. The results were then used to construct a PAC vector. For example, a functionality that was present in one study impacting Awareness and Coordination and, in another study, impacting Awareness and Participation would generate a vector of <1,2,1> in the PAC space. In order to account for functionality exclusive studies, we corrected those vectors for the number of functionalities present in each study. A functionality that was exclusively tested as a tool would create a weight of 1, whereas functionalities that *shared* a tool with other functionalities would result in weights inferior to 1 (since effects on distinct collaboration categories are less evident). This weight was calculated by PAC category and functionality so that a functionality correlating with two or three categories and being tested individually on one category (arguably a more interesting scenario) would then weight heavier on the individual category. For example, a functionality impacting Participation in 3 studies and Awareness in 2, but being tested on Awareness individually and Participation with two others would result in a vector of <3/3, 2/1, 0> = <1, 2, 0>. Each component of a functionality vector consists of three values in the PAC space. Each number of occurrences in a category is weighted by the mean of appearances of the functionality among other functionalities within a tool by category. Finally, we used the calculated vectors to represent each functionality in a schema relating collaborative categories and tool functionalities (see Figure 2).

<figure><img src="image_4.png" /><figcaption>**Figure 1:**
Methodology</figcaption></figure>

## Results

In total, we identified **20 collaborative functionalities** that reappeared (in different frequencies) across **49 studies of tools**, meeting eligibility and partial screening criteria of Mateescu et al. (2019). We found reported impact of these tools on the PAC collaborative process categories: *Participation* in 26 studies, *Coordination* in 14 and *Awareness* in 23 studies.

On average, a tool was composed of 2,8 functionalities (in a range from 1 to 6). Parallel input appeared 36 times across the 49 studies of the corpus (being, by now, a distinct feature of interactive tabletops). Functionalities impacting only one category are rare – the more studies include functionalities, the more categories are impacted. This is due to both potentially more functionalities in different tools and the complex interplay between functionalities. The functionality that has been studied the most exclusively (1,5 functionalities per tool) was *tool bridging*. Functionalities were most often associated to the *Participation* category (on average 3.5 times/functionality), then *Awareness* (2.9 times/functionality) and *Coordination* (2.5 times/functionality).

Furthermore, we were able to identify **groups of functionalities** in Table 1: Functionalities n°1-5 provide *artefact manipulation* (visual sorting, filtering etc.), n° 6-8 concern *input methods*, n° 9-11 are what Mateescu et al. (2019) coined *Mirroring tools* providing users (or educators) with feedback on their actions, n° 12-16 provide functionality for *metaprocess manipulation* (planning, structuring the activity or enforcing activity design), n°17-19 support territorialities and finally, n°20 encourages participation through playfulness.
<figure><img src="image_5.png" /><figcaption>Table 1</figcaption></figure>

<table><caption>Occurrences and impact on PAC space by tool functionality</caption><thead><tr><th rowspan="2">N°</th><th rowspan="2">Tool functionality</th><th colspan="3">Aggregated occurrences in studies</th><th colspan="3">Average number of functionalities in studies</th><th colspan="3">PAC vector</th><th rowspan="2">Total number of occurrences in studies</th></tr><tr><th>N Awareness</th><th>N Coordination</th><th>N Participation</th><th>A Awareness</th><th>A Coordination</th><th>A Participation</th><th>V Awareness</th><th>V Coordination</th><th>V Participation</th><th>N</th></tr></thead><tbody><tr><td>1</td><td>Sorting virtual artefacts</td><td>1</td><td>1</td><td>1</td><td>4</td><td>3</td><td>3</td><td>0,3</td><td>0,3</td><td>0,3</td><td>2</td></tr><tr><td>2</td><td>Resizing virtual artefacts</td><td>2</td><td>3</td><td>2</td><td>3,5</td><td>2,7</td><td>2,5</td><td>0,6</td><td>1,1</td><td>0,8</td><td>6</td></tr><tr><td>3</td><td>Grouping virtual artefacts</td><td>3</td><td>0</td><td>4</td><td>2,7</td><td>0</td><td>2,8</td><td>1,1</td><td>0</td><td>1,5</td><td>6</td></tr><tr><td>4</td><td>Problem decomposition</td><td>2</td><td>1</td><td>0</td><td>3</td><td>3</td><td>0</td><td>0,7</td><td>0,3</td><td>0</td><td>2</td></tr><tr><td>5</td><td>Tool bridging</td><td>0</td><td>2</td><td>2</td><td>0</td><td>2,5</td><td>3</td><td>0</td><td>0,8</td><td>0,7</td><td>4</td></tr><tr><td>6</td><td>User Embodiment</td><td>1</td><td>1</td><td>0</td><td>3</td><td>3</td><td>0</td><td>0,3</td><td>0,3</td><td>0</td><td>1</td></tr><tr><td>7</td><td>Parallel input</td><td>17</td><td>11</td><td>19</td><td>2,5</td><td>2,5</td><td>2,5</td><td>6,7</td><td>4,3</td><td>7,5</td><td>36</td></tr><tr><td>8</td><td>Tangible tokens</td><td>4</td><td>2</td><td>3</td><td>2,3</td><td>3,5</td><td>2,7</td><td>1,8</td><td>0,6</td><td>1,1</td><td>6</td></tr><tr><td>9</td><td>Educator awareness tools</td><td>1</td><td>0</td><td>0</td><td>3</td><td>0</td><td>0</td><td>0,3</td><td>0</td><td>0</td><td>1</td></tr><tr><td>10</td><td>Participation feedback</td><td>1</td><td>0</td><td>3</td><td>1</td><td>0</td><td>0</td><td>3</td><td>0</td><td>4,5</td><td>3</td></tr><tr><td>11</td><td>Movement traces</td><td>2</td><td>0</td><td>1</td><td>3,5</td><td>0</td><td>3</td><td>0,6</td><td>0</td><td>0,3</td><td>2</td></tr><tr><td>12</td><td>Orchestration tools</td><td>2</td><td>2</td><td>1</td><td>3</td><td>3</td><td>4</td><td>0,7</td><td>0,7</td><td>0,3</td><td>5</td></tr><tr><td>13</td><td>Process building</td><td>0</td><td>2</td><td>1</td><td>0</td><td>1,5</td><td>1</td><td>0</td><td>1,3</td><td>1</td><td>2</td></tr><tr><td>14</td><td>Voting mechanisms</td><td>1</td><td>1</td><td>0</td><td>3</td><td>3</td><td>0</td><td>0,3</td><td>0,3</td><td>0</td><td>1</td></tr><tr><td>15</td><td>Activity enforcement</td><td>2</td><td>1</td><td>4</td><td>2,5</td><td>2</td><td>3</td><td>0,8</td><td>0,5</td><td>1,3</td><td>6</td></tr><tr><td>16</td><td>Scripting</td><td>1</td><td>0</td><td>0</td><td>2</td><td>0</td><td>0</td><td>0,5</td><td>0</td><td>0</td><td>1</td></tr><tr><td>17</td><td>Dynamic territoriality</td><td>2</td><td>0</td><td>3</td><td>3</td><td>0</td><td>4</td><td>0,7</td><td>0</td><td>0,8</td><td>4</td></tr><tr><td>18</td><td>Static territoriality</td><td>4</td><td>3</td><td>5</td><td>3,8</td><td>3,3</td><td>3,2</td><td>1,1</td><td>0,9</td><td>1,6</td><td>8</td></tr><tr><td>19</td><td>Shared and individual devices</td><td>4</td><td>0</td><td>3</td><td>3,5</td><td>0</td><td>3</td><td>1,1</td><td>0</td><td>1</td><td>5</td></tr><tr><td>20</td><td>Physics engine</td><td>1</td><td>1</td><td>0</td><td>3</td><td>3</td><td>0</td><td>0,3</td><td>0,3</td><td>0</td><td>1</td></tr></tbody></table>

Figure 2 provides a distilled view of the data. Since many functionalities impacted more than one category, we opted for a Venn-diagram with three overlapping areas representing the three collaborative process categories PAC (green, blue and red). Collaborative conditions (black), such as participants' skills and teaching scenario, influence collaborative processes and vice versa.

<figure><img src="image_6.png" /><figcaption>Figure 2:</figcaption></figure>
<figure><img src="image_7.png" /></figure>

Figure 2 provides in fact another view on the PAC framework. The position of each functionality is calculated and mapped onto the space in figure 2. The three categories can be considered “attraction points” to each functionality so that a functionality with a vector <1, 0, 0> (e.g., “Physic engines”) will be attracted by Participation only (and thus be placed close to the Participation label. Functionalities that have been found to impact all three collaboration categories are present in the middle of the diagram. The legend is composed of the functionality name, the number of occurrences in our study (n) and a short description.

This vision provides multiple insights: Firstly, there seems to be a drag towards *Awareness* and *Participation*. Indeed, there is no functionality exclusively to *Coordination flow*. The *Awareness-Coordination* cluster accounts for three functionalities while *Participation-Awareness* contains five. Functionalities with influence on all three PAC categories (or whose impact cannot be isolated) are distributed among an area closer to Participation and Awareness then Coordination flow.

## Limitations

Identifying links between tool functionalities and collaborative processes, is as logical, in the course of collaborative research, as it is complex: the tool as a collaborative condition does not exist in isolation: activity design, participants’ collaborative skills and the environment are all major factors that were not taken into account in this first review on functionalities and collaboration. In the reviewed studies, they vary considerably. Further research is required to include most notably the type of activity. While certain functionalities may improve collaboration independently from activity type (e.g. mirroring tools), some functionalities may particularly enhance an activity type (e.g. dynamic territoriality for brainstorming tasks). Another issue is that new, specialized functionalities, by their novel nature, do not occur frequently across literature, tying their results to their individual study context. This and the low number of identified functionalities supporting single categories calls for both a more detailed conceptual framework and a common software framework to test individual functionalities and establish more detailed links between collaborative processes, tool functionalities and importantly, activity design.

## Conclusion

In this work on functionalities and collaborative processes, we investigated and illustrated how functionalities impact different aspects of collaboration, based on a meta-analysis of 49 studies. As a result, we identified 20 tool functionalities and detailed their potential impact on collaboration. The resulting mapping (see Figure 2) is a first, although incomplete, step towards a better understanding of the interplay between tools and collaboration.

## References

* Cress, U., Rosé, C., Wise, A. F., & Oshima, J. (Eds.). (2021). *International handbook of computer-supported collaborative learning*. Springer.

* Cullen, W., Gulati, G., & Kelly, B. D. (2020). Mental health in the COVID-19 pandemic. *QJM: An International Journal of Medicine, 113*(5), 311–312.

* Laal, M., & Ghodsi, S. M. (2012). Benefits of collaborative learning. *Procedia - Social and Behavioral Sciences, 31*, 486–490.

* Martinez-Maldonado, R., Clayphan, A., Ackad, C., & Kay, J. (2014). Multi-touch technology in a higher-education classroom: Lessons in-the-wild. *Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design*, 220–229.

* Mateescu, M., Pimmer, C., Zahn, C., Klinkhammer, D., & Reiterer, H. (2019). Collaboration on large interactive displays: A systematic review. *Human-Computer Interaction, 36*, 1–35.

* Simon, S., Marfisi-Schottman, I., & George, S. (2022). A Conceptual Framework for Creating Mobile Collaboration Tools. In I. Hilliger, P. J. Muñoz-Merino, T. De Laet, A. Ortega-Arranz, & T. Farrell (Eds.), *Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption* (pp. 601–607). Springer International Publishing.

* Stahl, G. (2006). *Group Cognition: Computer Support for Building Collaborative Knowledge* a book by Gerry Stahl.

* Sugimoto, M., Hosoi, K., & Hashizume, H. (2004). Caretta: A System for Supporting Face-to-Face Collaboration by Integrating Personal and Shared Spaces. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 41–48.