<figure><img src="image_1.png" /></figure>

HAL
open science

# Integrating AI in Human-Human Collaborative Ideation

Joongi Shin, Janin Koch, Andrés Lucero, Peter Dalsgaard, Wendy E. Mackay

<figure><img src="image_2.png" /></figure>

## To cite this version:

Joongi Shin, Janin Koch, Andrés Lucero, Peter Dalsgaard, Wendy E. Mackay. Integrating AI in Human-Human Collaborative Ideation. Extended Abstracts of the CHI'23 SIGCHI Conference on Human Factors in Computing Systems, Apr 2023, Hamburg, Germany. pp.1-5, 10.1145/3544549.3573802. hal-04023507v2

HAL Id: hal-04023507

https://inria.hal.science/hal-04023507v2

Submitted on 12 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Integrating AI in Human-Human Collaborative Ideation

Joongi Shin

Aalto University
ESPOO, Finland
joongi.shin@aalto.fi

Janin Koch

LISN, Université Paris-Saclay, CNRS,
Inria
Orsay, France
janin.koch@inria.fr

Andrés Lucero

Aalto University
ESPOO, Finland
lucero@acm.org

Peter Dalsgaard

Aarhus University
Aarhus, Denmark
dalsgaard@cavi.au.dk

Wendy E. Mackay

LISN, Université Paris-Saclay, CNRS,
Inria
Orsay, France
mackay@lri.fr

## ABSTRACT

People can generate more innovative ideas when they collaborate with one another, collectively exploring ideas and exchanging viewpoints. Advancements in artificial intelligence have opened up new opportunities in people's creative activities where individual users ideate with diverse forms of AI. For instance, AI agents and intelligent tools have been designed as ideation partners that provide inspiration, suggest ideation methods, or generate alternative ideas. However, what AI can bring to collaborative ideation among a group of users has not been fully understood. Compared to ideating with individuals, ideating with multiple users would require understanding users' social interaction, transforming individual efforts into a group effort, and—in the end—making users satisfied that they collaborated with other group members. This workshop aims to bring together a community of researchers and practitioners to explore the integration of AI in human-human collaborative ideation. The exploration will center around identifying the potential roles of AI as well as the process and form of collaborative ideation, considering what users want to do with AI or humans.

## CCS CONCEPTS

* **Human-centered computing → Collaborative content creation.**

## KEYWORDS

Collaborative ideation, AI agent, Human-AI collaboration, Human-Human collaboration, Ideation partner, Facilitator

## ACM Reference Format:

Joongi Shin, Janin Koch, Andrés Lucero, Peter Dalsgaard, and Wendy E. Mackay. 2023. Integrating AI in Human-Human Collaborative Ideation. In *Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23)*, April 23–28, 2023, Hamburg, Germany. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

## 1 BACKGROUND

Ideation techniques for generating, evaluating, and refining ideas are the foundation for people's creative endeavors ranging from art to design to problem-solving. Research has shown that the quality of ideas often increases when people collaborate with each other [19, 21]. This is due to the fact that group members can express diverse viewpoints that individuals may not consider otherwise and collaborate on ideation efforts by building on each other's contributions [1, 2, 10]. This *human-human collaborative ideation* can be observed in diverse contexts such as when individuals with shared interests seek better ideas or when groups with conflicting needs seek to formulate satisfying solutions for everyone. Extensive research has been performed to understand and support collaborative ideation. For instance, various ideation techniques such as brainstorming and ideation cards are practiced in the design field of design, and guidelines have been proposed that any group of users can follow [3, 7, 16, 17]. Interactive technologies that enable a digital form of ideation have been shown to be useful for idea exchange or consensus-building within the HCI community [5, 12, 23]. This form of ideating 'through' machines is now evolving to the next phase where humans ideate 'with' artificial intelligence systems.

In recent years, research in human-AI collaborative ideation has actively investigated multiple potential forms of AI as an ideation partner (Figure 1). For instance, intelligent design tools have been developed to proactively suggest inspirational materials or alternative designs related to users' own ideas and have been shown to broaden users' perspectives during idea generation [6, 9]. Similarly, conversational agents such as chatbots have been developed to ask critical questions to users and help them articulate their own ideas [4, 22]. This shows that AI-embedded systems or AI agents can improve users' ideation processes and outcomes that satisfy individual users.

Compared to individual ideation, however, collaborative ideation involves group members' social interaction, such as building consensus or empathizing with other group members' viewpoints. How AI systems can bring to collaborative ideation among a group of users has not been fully explored yet. A few studies have looked into the potential of AI as a facilitator of humans' collaborative ideation, encouraging group members to build on each other's contributions [11, 20], moderating equal turn-taking [15], summarizing their key ideas [8], or guiding consensus-building [18]. While
<figure><img src="image_4.png" /><figcaption>**Figure 1:** Examples of human-AI collaborative ideation systems. An intelligent design tool helping users design a mood board (left) [9] and a chatbot assisting consensus-building (right) [18].</figcaption></figure>

studies often show AI's capabilities to contribute to ideation, open questions remain how intelligent systems can support collaborative ideation among people, e.g.: *What would users want from AI systems and what would users want from humans despite AI's capability in doing the same?*

This workshop aims to explore the future of collaborative ideation between humans and intelligent systems with researchers and practitioners from diverse disciplines. This workshop will encourage participants from art, design, social science, computer science, and HCI to integrate AI in a variety of collaborative ideation processes. Going beyond identifying the potential roles of AI, participants will explore roles that AI or humans could or should play and construct collaborative ideation scenarios that provide concrete directions for future research.

## 2 WORKSHOP GOALS

This workshop provides a forum for HCI researchers from diverse backgrounds to discuss the future of human-human collaborative ideation supported using AI systems (Figure 2). We aim to exchange the understanding and expectation of human actors and AI in multiple processes of collaborative ideation (e.g., inquiry, idea generation, and consensus-building) and the forms of collaboration (e.g., multiple users with multiple AI, (a)synchronous, and (non)collocated). We expect to address conflicting viewpoints on the subjects, uncover the potential and shortcomings of AI, and establish a common ground that sets the direction of integrating AI into humans' collaborative ideation.

Accordingly, the goals of this workshop are:

* Identify human actors' roles that AI can perform better or worse.

* Explore new roles, processes, or forms of collaborative ideation that AI could enable.

* Examine what humans would prefer to be performed by humans or AI.

* Develop potential scenarios of integrating AI in human-human collaborative ideation.

* Investigate the implications of AI that can enable such scenarios.

## 3 ORGANIZERS

The workshop organizers are all active researchers in the area of interactive systems that support users' creative and collaborative activities. The contact person will be Joongi Shin (joongi.shin@aalto.fi).

**Joongi Shin** is a Postdoctoral Researcher in the User Interfaces Research Group at Aalto University. His work focuses on conversational agents that support collaborative ideation among a large number of users. His research explores how artificial facilitators can guide group members to effectively build on each other's ideas and reach a consensus in asynchronous and non-collocated settings.

**Janin Koch** is a permanent researcher at Inria Paris-Saclay. Her research interests include collaborative artificial intelligence for exploratory creative tasks. Her work aims to define, study and evaluate human-machine interaction to construct ideas and concepts together with intelligent machines.

**Andrés Lucero** is an Associate Professor of Interaction Design at Aalto University. His work focuses on the design and evaluation of novel interaction techniques for mobile devices and other interactive surfaces.

**Peter Dalsgaard** is a Full Professor of Interaction Design at Aarhus University and director of the Centre for Digital Creativity. He studies real-life use of digital systems in creative processes, develops and experiments with prototypes of new systems and services, and develops theories to understand the role and nature of digital media in creative processes.

**Wendy E. Mackay** is a Research Director at Inria and head of the ExSitu research group focused on the design of human-computer
## Potential roles in human-human collaborative ideation

<figure><img src="image_5.png" /><figcaption>**Figure 2:** The discussion space of integrating AI in human-human collaborative ideation. The collaborative ideation between human participants involves complex social interaction, which AI might be able to simulate in the future. However, it is unknown what humans would want to perform with other humans despite AI's capability.</figcaption></figure>

partnerships, with an emphasis on interactive tools that support early-stage creative design.

# 4 PRE-WORKSHOP PLANS

## 4.1 Website

We will create a website to introduce the subject of our workshop, including the call for participants, workshop schedule, and contact information (https://www.joongishin.com/co-ideation-ai). During the workshop, this website will be used to update the progress and share materials for asynchronous participants. After the workshop, we will use the website as a platform for fostering further discussion about the subject, presenting selected position papers and a brief summary of the workshop.

## 4.2 Recruitment

We aim to recruit researchers and practitioners from diverse fields such as art, design, social science, computer science, and HCI. The participants will be valued for their unique knowledge and viewpoints that can bring into the discussion: Experts in (collaborative) ideation with less understanding of AI or vice versa will be all welcomed. We will recruit participants via diverse channels such as social media (e.g., Twitter and Facebook) as well as mailing lists of HCI, Computational Design, and Design research communities. We aim for 18-24 participants. The participants will be asked to submit 2 to 4 pages of a position paper. We will select participants by the relevance and quality of their position paper and by considering the diversity of participants.

## 4.3 Mode of Workshop

We intend to conduct our workshop in-person only. Based on participants' position papers and backgrounds, we will predefine groups of 4-5 participants that can include multiple viewpoints in each group. To be prepared for participants who may fail to join the workshop in person on the day, we will hold a synchronous online meeting (e.g., Zoom). Such participants will be able to listen to discussions from other attendees and join one of the participant groups to participate in workshop activities. We will post the progress and materials on our website for them to download.

## 4.4 Required facilities and specifications

The workshop will require a VGA projector for the organizers' and participants' presentations. Tables and chairs will be needed for 4-5 participants to perform workshop activities as a group as well as note-taking materials such as pens, notes, and papers.

# 5 WORKSHOP STRUCTURE

We aim to bring together mixed viewpoints of researchers and practitioners from diverse fields. We intend to uncover conflicting understandings and expectations of human actors, AI, and social interaction in human-human collaborative ideation. This will help to establish a concrete common ground for fruitful discussion during the workshop. As a result of this workshop, a set of roles to be performed by humans or AI; the process and form of collaborative ideation enabled by AI; and representative scenarios that might provide clear directions for future research ideas in the field of HCI will be identified. With consent, we will record the entire workshop sessions and document the artifacts of workshop activities.

**Morning session: Co-design the roles of AI in human-human collaborative ideation**

9:00 - 9:10 Introduction and scene setting from the organizers.

9:10 - 9:30 **Icebreaker activity:** Each participant will spend a couple of minutes writing down their expectations on what AI agents can bring to collaborative ideation between humans on a note card. Then, they will be asked to exchange their cards while introducing themselves to the other attendees.

9:30 - 10:20 **Lightning talk:** Participants will present their submitted position papers in front of the other attendees in the form of a lightning talk (3-min talk with 2-min Q&A).

10:20 - 10:30 **Break**

10:30 - 12:00 **Co-design session:** Participants will co-design the roles of AI from multiple viewpoints, identified from their position papers. They will be guided to perform an ideation method called Dialogue-Lab [13]. This method is developed by one of the co-authors (Andrés Lucero), which designates multiple areas (stations) in a room for
<figure><img src="image_6.png" /><figcaption>**Figure 3:** Ideation techniques in this workshop. Examples of Dialogue-Lab activities (left) [13] and video prototyping (right) [14] from the authors' research.</figcaption></figure>

<figure><img src="image_7.png" /></figure>

unique topics (Figure 3 left). Pairs of participants will cycle around the stations for a discussion, focusing on one topic at a time.

12:00 - 13:30 *Lunch*

**Afternoon session: Co-design the process and form of human-human collaborative ideation**

13:30 - 15:30 *Co-design session:* Based on the designed roles of AI, participants will co-design the scenario of integrating AI in human-human collaborative ideation. For diversity, each group of participants will be assigned different roles of AI. They will be asked to perform role-playing techniques to sketch detailed interactions between AI and humans.

15:30 - 15:40 *Break*

15:40 - 16:30 *Video prototyping:* Participants will be asked to create a video prototype of their scenarios [14] (Figure 3 right), where one of the group members acts as AI.

16:30 - 17:30 *Presentation of results:* Each group of participants will present their scenarios to the group (10-min presentation with 5-min Q&A).

17:30 - 18:00 *Closing discussion:* As a closing remark, organizers will lead the last discussion with all participants to share their thoughts on the impact of AI agents in human-human collaborative ideation, potential barriers to investigating the subjects, and potential formats of publishing the workshop result.

**6 POST-WORKSHOP PLANS**

The output from the workshop will be summarised and published on our workshop website. This will describe the roles of AI in human-human collaborative ideation. The scenario and video prototype will be added to visualize the potential process and form of collaboration related to the potential benefits and shortcomings of AI. To expose our insights to a wider audience, we will post them on active social platforms for design and AI communities. We expect the result of the workshop can make a special issue in both design

and HCI journals (e.g. ToCHI), considering the combined perspectives from humans' interpersonal interaction and interaction with AI. This will nurture further research in the community and potential collaboration among attendees. We will discuss this at the end of the workshop with our participants. We will also welcome additional researchers who would be inspired by our workshop output.

**7 CALL FOR PARTICIPATION**

People can generate better ideas when they collaborate with one another. AI has been studied to support individual users' ideation process. In contrast, what AI would bring to collaborative ideation among a group of users is unknown. Compared to individual ideation, collaborative ideation requires transforming individuals' ideation effort to group effort considering complex social interaction among them. Despite AI's capability in simulating interpersonal interaction between humans, what humans want AI to do and what humans want other humans to do is an open-ended question. Accordingly, new roles, processes, and forms of human-human collaborative ideation could emerge supported by AI. This CHI 2023 workshop welcomes researchers and practitioners from diverse fields such as art, design, social science, computer science, and HCI to discuss the integration of AI agents into human-human collaborative ideation. The workshop will focus on:

* Identify human actors' roles that AI can perform better or worse.

* Explore new roles, processes, or forms of collaborative ideation that AI could enable.

* Examine what humans would prefer to be performed by humans or AI.

* Develop potential scenarios of integrating AI in human-human collaborative ideation.

* Investigate the implications of AI that can enable such scenarios.

Participants should submit a 2- to 4-page position paper (including references) in the CHI Extended Abstracts Format. The proposal should be emailed to joongi.shin@aalto.fi. The position papers need to describe participants' interests and/or previous work related to
the workshop topic. We will select papers based on their relevance, quality, and diversity. We will limit the size of the workshop to 24 people. At least one author of each accepted submission must attend the workshop and all participants must register for both the workshop and for at least one day of the conference. Please contact joongi.shin@aalto.fi for any questions.

## ACKNOWLEDGMENTS

The research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2022R1A6A3A03056886), ERC Advanced Grant CREATIV (321135), Independent Research Fund Denmark (0132-00111B) Managing Ideas in Creative Work, the Aalto University Department of Communications and Networking, the Finnish Center for Artificial Intelligence (FCAI), Academy of Finland (grants 'Human Automata' and 'BAD') and by The Humane-AI-Net excellence network funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No 952026.

## REFERENCES

[1] Vincent R. Brown and Paul B. Paulus. 2002. Making Group Brainstorming More Effective: Recommendations From an Associative Memory Perspective. Current Directions in Psychological Science 11, 6 (2002), 208–212. https://doi.org/10.1111/1467-8721.00202 arXiv:https://doi.org/10.1111/1467-8721.00202

[2] Hernan Casakin and Georgi V. Georgiev. 2021. Design creativity and the semantic analysis of conversations in the design studio. International Journal of Design Creativity and Innovation 9, 1 (2021), 61–77. https://doi.org/10.1080/21650349.2020.1838331 arXiv:https://doi.org/10.1080/21650349.2020.1838331

[3] Haakon Faste, Nir Rachmel, Russell Essary, and Evan Sheehan. 2013. Brainstorm, Chainstorm, Cheatstorm, Tweetstorm: New Ideation Strategies for Distributed HCI Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI '13). Association for Computing Machinery, New York, NY, USA, 1343–1352. https://doi.org/10.1145/2470654.2466177

[4] Yoshiko Goda, Masanori Yamada, Hideya Matsukawa, Kojiro Hata, and Seisuke Yasunami. 2014. Conversation with a Chatbot before an Online EFL Group Discussion and the Effects on Critical Thinking. The Journal of Information and Systems in Education 13, 1 (2014), 1–7. https://doi.org/10.12937/ejsise.13.1

[5] P. Grünbacher and R.O. Briggs. 2001. Surfacing tacit knowledge in requirements negotiation: experiences using EasyWinWin. In Proceedings of the 34th Annual Hawaii International Conference on System Sciences. 8 pp.–. https://doi.org/10.1109/HICSS.2001.926243

[6] Youngseung Jeon, Seungwan Jin, Patrick C. Shih, and Kyungsik Han. 2021. FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 576, 18 pages. https://doi.org/10.1145/3411764.3445093

[7] Jay F. Nunamaker Jr., Lynda M. Applegate, and Benn R. Konsynski. 1987. Facilitating Group Creativity: Experience with a Group Decision Support System. Journal of Management Information Systems 3, 4 (1987), 5–19. https://doi.org/10.1080/07421222.1987.11517775 arXiv:https://doi.org/10.1080/07421222.1987.11517775

[8] Soomin Kim, Jinsu Eun, Joseph Seering, and Joonhwan Lee. 2021. Moderator Chatbot for Deliberative Discussion: Effects of Discussion Structure and Discussant Facilitation. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 87 (apr 2021), 26 pages. https://doi.org/10.1145/3449161

[9] Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI? Design Ideation with Cooperative Contextual Bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300863

[10] Nicholas W. Kohn, Paul B. Paulus, and YunHee Choi. 2011. Building on the ideas of others: An examination of the idea combination process. Journal of Experimental Social Psychology 47, 3 (2011), 554–561. https://doi.org/10.1016/j.jesp.2011.01.004

[11] Sung-Chul Lee, Jaeyoon Song, Eun-Young Ko, Seongho Park, Jihee Kim, and Juho Kim. 2020. SolutionChat: Real-Time Moderator Support for Chat-Based Structured Discussion. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376609

[12] Weichen Liu, Sijia Xiao, Jacob T. Browne, Ming Yang, and Steven P. Dow. 2018. ConsensUs: Supporting Multi-Criteria Group Decisions by Visualizing Points

of Disagreement. Trans. Soc. Comput. 1, 1, Article 4 (jan 2018), 26 pages. https://doi.org/10.1145/3159649

[13] Andrés Lucero, Kir sikka Vaajakallio, and Peter Dalsgaard. 2012. The dialogue-labs method: process, space and materials as structuring elements to spark dialogue in co-design events. CoDesign 8, 1 (2012), 1–23. https://doi.org/10.1080/15710882.2011.609888 arXiv:https://doi.org/10.1080/15710882.2011.609888

[14] Wendy E Mackay. 1988. Video Prototyping: a technique for developing hypermedia systems. In CHI '88 Conference Companion Human Factors in Computing Systems, Vol. 5. Citeseer, 1–3.

[15] Torsten Maier, Nicolas F. Soria Zurita, Elizabeth Starkey, Daniel Spillane, Christopher McComb, and Jessica Menold. 2022. Comparing human and cognitive assistant facilitated brainstorming sessions. Journal of Engineering Design 33, 4 (2022), 259–283. https://doi.org/10.1080/09544828.2022.2032623 arXiv:https://doi.org/10.1080/09544828.2022.2032623

[16] Paul B Paulus, Jonali Baruah, and Jared B Kenworthy. 2018. Enhancing collaborative ideation in organizations. Frontiers in psychology 9 (2018), 2024.

[17] Dorian Peters, Lian Loke, and Naseem Ahmadpour. 2021. Toolkits, cards and games – a review of analogue tools for collaborative ideation. CoDesign 17, 4 (2021), 410–434. https://doi.org/10.1080/15710882.2020.1715444 arXiv:https://doi.org/10.1080/15710882.2020.1715444

[18] Joongi Shin, Michael A. Hedderich, Andrés Lucero, and Antti Oulasvirta. 2022. Chatbots Facilitating Consensus-Building in Asynchronous Co-Design. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Oregon, USA) (UIST '22). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3526113.3545671

[19] Pao Siangliulue, Kenneth C. Arnold, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Toward Collaborative Ideation at Scale: Leveraging Ideas from Others to Generate More Creative and Diverse Ideas. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work; Social Computing (Vancouver, BC, Canada) (CSCW '15). Association for Computing Machinery, New York, NY, USA, 937–945. https://doi.org/10.1145/2675133.2675239

[20] Pao Siangliulue, Joel Chan, Steven P. Dow, and Krzysztof Z. Gajos. 2016. IdeaHound: Improving Large-Scale Collaborative Ideation with Crowd-Powered Real-Time Semantic Modeling. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST '16). Association for Computing Machinery, New York, NY, USA, 609–624. https://doi.org/10.1145/2984511.2984578

[21] James Surowiecki. 2004. The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business. Economies, Societies and Nations 296, 5 (2004).

[22] Thiemo Wambsganss, Tobias Kueng, Matthias Soellner, and Jan Marco Leimeister. 2021. ArgueTutor: An Adaptive Dialog-Based Learning System for Argumentation Skills. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 683, 13 pages. https://doi.org/10.1145/3411764.3445781

[23] Roshanak Zilouchian Moghaddam, Brian P. Bailey, and Christina Poon. 2011. IdeaTracker: An Interactive Visualization Supporting Collaboration and Consensus Building in Online Interface Design Discussions. In Human-Computer Interaction – INTERACT 2011, Pedro Campos, Nicholas Graham, Joaquim Jorge, Nuno Nunes, Philippe Palanque, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 259–276.