DOI: 10.5281/zenodo.15640622
title: From Human Teams to Hybrid Intelligence Teams: Identifying, Characterizing, and Evaluating Foundational Quality Attributes - Supplementary Material

description: # GENERAL INFORMATION

## Title

From Human Teams to Hybrid Intelligence Teams: Identifying, Characterizing, and Evaluating Foundational Quality Attributes - Supplementary Material

### Dataset DOI

[https://zenodo.org/doi/10.5281/zenodo.15640622](https://zenodo.org/doi/10.5281/zenodo.15640622)

## Authors: 

- **Dr. Davide Dell'Anna** ([ORCID](https://orcid.org/0000-0002-1162-8341), d.dellanna@uu.nl, Utrecht University, The Netherlands, [Personal webpage](https://www.davidedellanna.com) - **Dr. Pradeep K. Murukannaiah** ([ORCID](https://orcid.org/0000-0002-1261-6908), p.k.murukannaiah@tudelft.nl, Delft University of Technology, The Netherlands- **Dr. Mireia Yurrita** ([ORCID](https://orcid.org/0000-0002-9685-4873), m.yurritasemperena@uu.nl, Utrecht University, The Netherlands- **Dr. Bernd Dudzik** ([ORCID](https://orcid.org/0000-0002-4522-074X), b.j.w.dudzik@tudelft.nl, Delft University of Technology, The Netherlands- **Prof. Davide Grossi** ([ORCID](https://orcid.org/0000-0002-9709-030X), d.grossi@rug.nl, University of Groningen, The Netherlands, and University of Amsterdam, The Netherlands- **Prof. Catholijn M. Jonker** ([ORCID](https://orcid.org/0000-0003-4780-7461), c.m.jonker@tudelft.nl, Delft University of Technology, The Netherlands, and Leiden University, The Netherlands- **Dr. Catharine Oertel** ([ORCID](https://orcid.org/0000-0002-8273-0132), c.r.m.m.oertel@tudelft.nl, Delft University of Technology, The Netherlands- **Prof. Pınar Yolum** ([ORCID](https://orcid.org/0000-0001-7848-1834), p.yolum@uu.nl, Utrecht University, The Netherlands 

# DESCRIPTION

This repository contains the supplementary material for the paper "From Human Teams to Hybrid Intelligence Teams: Identifying, Characterizing, and Evaluating Foundational Quality Attributes".

The repository contains details of the three HI Comp competitions (themed "fundamental qualities of HI teams") run at AAMAS 2024, HHAI 2024, and at the 13th Dutch Hybrid Intelligence Centre Consortium Meeting in 2024. The repository also contains the resulting dataset of 48 usage scenarios formulated by participants during the competitions, and the supplementary material containing the qualitative and quantitative analysis performed on such dataset for the paper mentioned above.

The material in this repository was generated and collected in the context of the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research, [https://www.hybridintelligence-centre.nl/](https://www.hybridintelligence-centre.nl/), under Grant No. (024.004.022). 

The dataset was generated via competitions that involved teams of human participants describing Hybrid Intelligence Usage Scenarios. The competitions were conducted to evaluate the understanding and relevance of the proposed quality model for hybrid intelligence teams, and to support the development of high-quality HI (Human-AI) teams, by exploring the possible benefits, risks, and consequences of collaboration between humans and AI systems. The dataset of scenarios constitutes the first repository of scenarios for researchers and practitioners to guide the development and evaluation of HI teams. All the details and results from the competitions are also available online at [https://github.com/Hybrid-Intelligence-Competition](https://github.com/Hybrid-Intelligence-Competition).

The dataset supplements the article "From Human Teams to Hybrid Intelligence Teams: Identifying, Characterizing, and Evaluating Foundational Quality Attributes", where detailed information about the quality model, and the related methodologies can be found. 

It is  important to mention that these competitions were carried out in compliance with the General Data Protection Regulation (EU GDPR) 2016/679, and the Utrecht University guidance for working with personal data from human participants. The project was approved by the Ethics Assessment Committee of Utrecht University. In that sense, all participants were informed since the beginning -via an informed consent form- that the scenarios that they described would be made available upon the completion of the project.

## Keywords

Hybrid Intelligence; Quality model; Human-agent teamwork; Sociotechnical systems; Team Diagnostic Survey; Competitions; Scenarios; Human-AI Teams;

## Date of data collection

2024-04-16 to 2024-06-14  

## Date of dataset publication

2025-06-11 

## Funding

This research was partially supported by Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research, https://www.hybridintelligence-centre.nl/, under Grant No. (024.004.022) and by the BOLD Cities initiative.

# FILE OVERVIEW

Below is reported the structure of this repository.

```JAAMAS_A_Quality_Model_for_Hybrid_Intelligence_Teams_Supplementary_Material│└───Competitions Details│    └───HI_Comp_AAMAS_2024                          # Folder containing all the details about the competition run at AAMAS 2024,│    │                                                # including a .csv file with all the scenarios submitted by the participants.│    └───HI_Comp_HHAI_2024                           # Folder containing all the details about the competition run at HHAI 2024,│    │                                                # including a .csv file with all the scenarios submitted by the participants.│    └───HHI_Comp_HICentreCM_2024                    # Folder containing all the details about the competition run at the 13th Dutch│                                                     # HI Centre Consorium meeting in 2024, including a .csv file with all the │                                                    # scenarios submitted by the participants.    └───Quantitative_Analysis                           # Folder containing all the replication material for the quantitive analysis │                                                    # of the data collected during the competitions│        Raw Data and Quantitative Analysis.xlsx     # File containing the Raw data collected during the competition and the results │                                                    # of the quantitative analysis│        Correlation Analysis.xlsx                     # File containing the results of the correlation analysis│        Correlation Analysis - Script.ipynb            # Script (Jupyter notebook) to run the correlation analysis.│└───Qualitative_Analysis                            # Folder containing all the replication material for the qualitive analysis │                                                    # of the data collected during the competitions.│        Report_qualitative_analysis-clean.pdf        # Report of the statements extracted through qualitative analysis.│        Raw Data Qualitative Analysis.xlsx            # File containing the Raw data collected during the competition and the results │                                                    # (the same Raw data in the 'Raw Data and Quantitative Analysis.xlsx' file).│   README.md                                       # This file│   HI_Teams_Quality_Model.jpg                      # A figure that overviews the Quality Model for Hybrid Intelligence Teams```

 

Last updated: 2025-06-11