# Systematic Integration of Solution Elements: How Does Digital Creativity Support Change Group Dynamics?

Florian Perteneder, Susann Hahnwald, Michael Haller, Kurt Gaubinger

► To cite this version:

Florian Perteneder, Susann Hahnwald, Michael Haller, Kurt Gaubinger. Systematic Integration of Solution Elements: How Does Digital Creativity Support Change Group Dynamics?. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.547-565, 10.1007/978-3-642-40483-2_39. hal-01497461

HAL Id: hal-01497461

https://inria.hal.science/hal-01497461

Submitted on 28 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Systematic Integration of Solution Elements: How Does Digital Creativity Support Change Group Dynamics?

Florian Perteneder<fnref n="1" />, Susann Hahnwald<fnref n="2" />, Michael Haller<fnref n="1" /> and Kurt Gaubinger<fnref n="2" />

**Abstract.** In practice, most creativity techniques are still performed with traditional tools, such as pen and paper, whiteboards, and flipcharts. When transforming these techniques into a digital environment, the reduction of organizational overhead is the main goal to foster accessibility. Still, we do not know if overhead reduction fosters creativity or if it eliminates an important part of the creative process. To get a deeper understanding of these effects, we compare the performance of the creativity technique SIS (Systematic Integration of Solution Elements) in a traditional setting with a setup based on multiple interactive surfaces. By using a mix of diverse evaluation methods, we show how the use of a digital interactive creativity room can really foster creativity and produce better results.

**Keywords:** Creativity, Design, Creativity Techniques, Interactive Environment, Systematic Integration of Solution Elements, Collaboration

## 1 Introduction

Creativity and innovation are predictors of success in our knowledge-based society. With the increasing availability of digital whiteboards, the development of tools to support creativity has become an emerging field [13, 15, 16, 18].

Since the '70s, psychologists and practitioners have put a lot of effort in developing numerous methods for supporting creativity more effectively [13, 26, 27]. Although there are a lot of creativity techniques for versatile needs and tasks, only a small set of traditional tools and media, such as pen and paper, sticky notes, flipcharts, whiteboards, and pinboards, are used. Due to the limited possibilities to edit and copy content, especially complex creativity techniques often require a huge amount of organizational overhead (e.g. copying content, placing flipchart papers on pinboards). Digital implementations usually aim to limit this overhead and increase the ease of use [4, 20]. However, it is not clear if the efforts of digitizing the creative process lead to the desired effect of decreased task execution time or if they even have negative side effects on creativity and inspiration. New media in creative processes change the balance of power, involvement and satisfaction of participants, as well as

<footnote n="1">¹ University of Applied Sciences Upper Austria, Media Interaction Lab, Hagenberg, Austria
mi-lab@fh-hagenberg.at</footnote>

<footnote n="2">² University of Applied Sciences Upper Austria, School of Engineering/Enviromental/Sciences, Wels, Austria
{susann.hahnwald, kurt.gaubinger}@fh-wels.at</footnote>
the general dynamics of collaborative sessions. To provide a theoretical understanding for developing adequate and practical interaction processes and applications, it is necessary to carefully study the effects of digital solutions and their impact on group creativity.

In this paper, we analyze the impacts of a digital environment on group dynamics and creative outcome when performing a complex creativity method. Therefore, we performed an experiment to compare the traditional paper-based way [12, 34] of performing the creativity technique SIS (Systematic Integration of Solution Elements) [36] to its implementation in a digital, interactive environment (cf. Fig. 1). We use a combination of different evaluation approaches to get a deeper understanding of the emerging side effects that come with overhead reduction. Finally, we present the results of our analysis and discuss their implications on environments that are supposed to support collaborative creativity, and on creative tasks in general.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Performing the SIS method in an interactive environment. Participants can discuss and present their ideas using digital paper and a large interactive whiteboard.</figcaption></figure>

# 2 Related Work

## 2.1 Creativity Techniques

While Osborn’s idea of verbal brainstorming [23] is widely known and used, many studies showed that this collaborative way of idea generation is not the best choice regarding quantity of ideas [24] due to negative social effects (production blocking, evaluation apprehension, and free riding) [8, 37]. Individual idea generation, also known as nominal brainstorming, is a way to overcome these issues and to increase the number of ideas produced [2, 32]. Nevertheless, verbal brainstorming is still preferred in many practical situations as it yields diverse perspectives when team members provide complementary skills and expertise [30].

Soon after the introduction of brainstorming, psychologists as well as practitioners developed new and improved creativity techniques [12, 25, 26, 35], focusing on different categories of problems. Most of these techniques include aspects of verbal
brainstorming (collaborative), nominal brainstorming (individual), or both. The com-
bination is considered as the best solution, especially when dealing with complex
problems.

## 2.2 Supporting Creativity Techniques in Interactive Environments

A considerable amount of recent work has explored digital support for creativity, focusing on brainstorming or discussion support. Most systems or tools, however, focus on the implementation of one specific creativity technique. As we identified both individual and collaborative work as crucial parts of creative work, we pay special attention to this aspect when presenting existing work. In addition, we were interested in flexible and open approaches that support a variety of different creativity techniques.

**Individual vs. Collaborative Work.** There are multiple concepts that support both individual as well as collaborative aspects of the creative process [4, 14, 16, 18]. While some concepts provide shielded, private space for undisturbed, individual work [14, 16], other implementations provide private, non-shielded space for individual content creation [4, 18]. In general it seems as if the need for shielded space rises when tasks become more complex and require e.g. extensive sketching instead of simple input of single words.

Post Brainstorm [15] forgoes any dedicated features to support individual phases
during the session. Since the system was designed as a substitution for traditional
media used within the brainstorming sessions at IDEO, it offers a lot of flexibility
regarding the import and arrangement of content, which is important for their creativi-
ty method.

**Flexibility.** Although there is some research about systems supporting very specific creative methods [20] and about providing certain improvements to the ideation process [1, 35], many systems try to provide at least some flexibility to the user. Warr and O’Neill [38] recommend providing users with flexible tools, such as free-hand drawing tools. TEAM STORM [16] aims not to impose structures by providing flexibility regarding private or public work. However, flexibility regarding a change of the underlying creativity technique is not intended.

Concepts based on the use of virtual sticky notes [14, 18] or paper strips [4] can
provide a fast way of interaction but have issues with supporting concepts that require
more extensive sketching.

## 3 Systematic Integration of Solution Elements

The creativity method *Systematic Integration of Solution Elements (SIS)* or *Successive Integration of Problem Elements (Successive Element Integration – SEI)* – was designed by Schlicksupp for extracting synergies from interdisciplinary teams [36].
SIS is a creativity technique to be used on problems that require rather complex solutions and offer only a restricted number of possible solutions, such as industrial or product design tasks. The basic idea of this method is to merge the benefits of individual (cf. nominal brainstorming) solutions to an integrated solution in collaborative work (cf. verbal brainstorming). Due to the integration of all individual solutions and the positive way of analyzing only their benefits (and not their drawbacks), SIS is a technique that leads to high identification of each participant with the final result.

## 3.1 The SIS Process

The SIS creativity technique is designed for 4-8 participants. The attendees develop individual solutions during the first working phase and integrate these ideas collaboratively during the moderated later phases. Usually the individual solutions as well as the integrated solutions consist of sketches with additional explanations.

After the initial problem framing, SIS defines a specific process that consists of three phases (cf. Fig. 2).

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Schematic representation of the different phases of the SIS process when used within a group of 4 participants. For our experiment we divided the entire process into three (artificial) stages (at the very right) to perform process-based measurements. Between the stages, we asked participants to complete questionnaires to measure their psychological state.</figcaption></figure>
*Phase 1 – Individual Solution Finding* is characterized by an individual idea gener-
ation phase. Each participant comes up with a potential solution for the predefined
problem on a sheet of paper. The goal is to get a wide variety of possible solutions by
overcoming negative group dynamic effects.

*Phase 2 – Solution Analysis* consists of two parts. First the solution generated in
phase 1 is presented by its creator. Second, beneficial ideas and advantages from the
particular solution are extracted collaboratively and written down (on a flipchart or
whiteboard) by the moderator.

*Phase 3 – Solution Integration* describes the process of combining the benefits of different solutions in one integrated solution. As this integration is performed collabo-ratively, the final solution does not only benefit from diverse ideas during generation, but also from different perspectives and from the expertise of a multidisciplinary team. This helps to consider the pros and cons of certain features in order to find the best solution for the final result.

As depicted in Fig. 2, phase 2 and 3 alternate according to the number of partici-
pants.

4 SIS Implementation for a Digital Environment

In a traditional setup, when performing the SIS method, all individual sketches are drawn by using pen and paper [12, 34]. Due to limited editing possibilities, sketches have to be redone and feature lists have to be copied manually when new solutions are integrated. This causes enormous organizational overhead. Our main goal is to pro-vide a digital environment that minimizes this overhead.

The SIS method involves multiple types of interaction, which also appear in other
creativity techniques. Similar to traditional, non-digital tools, the interactive environ-
ment as well as the applications were not designed to specifically support SIS but to
provide a flexible solution for a variety of creative activities.

**4.1 Environment**

It is essential to adequately support individual and private content creation (phase 1) as well as collaborative, usually moderated work in a group (phase 2 and 3). For this reason, we decided on a twofold solution: We use digital paper (using Anoto pens<fnref n="1" />) to facilitate private content creation in combination with a large-scale interactive whiteboard that offers public space for discussion or collaborative work.

**Digital Paper (phase 1).** We chose digital paper for individual work for a number of reasons. Primarily, it captures handwriting on paper and enables displaying the content to the public. On the whiteboard, created content can be edited, copied and moved. Second, other than with digital solutions, such as tablets, people are very familiar with paper, so it requires almost no learning to use this technique. Third, paper

<footnote n="1">¹ www.anoto.com</footnote>
provides adequate privacy during content generation. In addition, the possibility to
take it to another place can be indirectly used for shielding during content creation.
Finally, the approach is easily scalable for larger groups.

**Interactive Whiteboard (phase 2 and 3).** To support collaborative work, we use a large-scale multi-user interactive whiteboard based on the system presented in [17]. Due to its flexibility, it serves multiple purposes: It can be used for public note taking and to display prepared content, such as presentation slides. In addition, it can be used to display ideas, sketched on the above mentioned digital paper. As it is a digital device, it also offers benefits in terms of editing and space management capabilities compared to traditional media.

**4.2 Applications**

Digital paper is a suitable solution for phase 1 as it enables free-form sketching as well as handwritten content. In contrast, the requirements for the whiteboard software are diverse. First, participants have to be able to present their individual solutions to others. Second, there should be a way to emphasize and mark the benefits of a certain solution. In addition, the creation of integrated solution directly on the whiteboard should be supported. To meet these requirements two different applications were provided.

**Paper Application.** The first application allows users to display the content written on paper to the entire group (cf. Fig. 3, left).

It is very important to provide smooth and simple transitions of content between different devices and media. The bare possibility of providing ways to move and copy content in a digital system is no guarantee of actual overhead reduction compared to a traditional setting. For this reason, all pages that contain content are displayed per default in a small preview on the whiteboard without any additional interaction necessary. To ensure privacy during the individual content creation, the creator is able to trigger the public visibility by using a printed button directly on the paper sheet.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Content written on digital paper is displayed on the whiteboard for presentation (**left**). Content generated on paper is clipped into the Sketching Application for further usage (**right**).</figcaption></figure>
Individual and also groups of pages can be maximized for presentation or comparison (cf. Fig. 3, *left*). In addition, the application provides the possibility to select and copy content into a free-form sketching application for further editing (cf. Fig. 3, *right*).

**Sketching Application.** This application provides the possibility to create and edit content simultaneously for multiple users. Among others, editing includes erasing, selection, transformation and duplication of content. Moreover, the canvas is divided into pages. This way, the application provides effortless spatial navigation. During the SIS sessions, the page navigation was typically used to shift previous solutions and to generate new space (cf. Fig. 4).

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Typical configuration of the digital environment. One third of the whiteboard is used for the paper application (blue) to display the previously created individual solutions; two thirds are used for the sketching application (green) to create combined solutions. In the course of the session the combined solutions are shifted to the right (symbolized by the arrow) to gain new space.</figcaption></figure>

# 5 Evaluation Methodologies

There are two types of methods that can be used to evaluate the effectiveness and efficiency of creativity techniques. While outcome-based approaches focuses on the final results produced during the process [29], process-based approaches focuses on the ideation process itself.

## 5.1 Outcome-based Approach

Outcome-based approaches focus on evaluating the final result of the ideation process. For assessing ideation quality, researchers usually are guided by the following four steps: First, unique ideas from an ideation session are identified; second, a quality score is assigned to each individual idea (usually done by domain experts, who understand and interpret the ideas [21]); third, by using one of the four approaches
discussed below, a metric value is computed, which is used, forth, to make statistical
comparisons between treatments of every session threshold [25].

There is a huge variety of criteria to give a score to each individual idea: The crea-
tivity of an idea is usually assessed through novelty - how unusual or unexpected an
idea is compared to the other ideas [29] - and quality (feasibility or the readiness for
implementation and the detail of description [8, 9]) [21]. Other criteria are utility [21],
effectiveness [32], the value ideas could create [3], the importance of an idea within a
specific context [33], and the magnitude of impact an idea might have [7]. Usually
semantic differentials, such as Likert scales, or rubrics that evaluate one or more di-
mensions are used to measure those criteria [25].

## 5.2 Process-based Approach

To fully evaluate the effectiveness and efficiency of a creativity technique and to
understand the reasons for specific results, the overall process needs to be analyzed.
By using process-based approaches, such as the concept of flow, occurrences of cog-
nitive processes inherent to creative thought are evaluated. Other common evaluation
methods are, e.g. video analysis and the ‘think aloud’ technique [28].

**Concept of Flow.** How does the user feel ideating? How does the user feel at the beginning, the middle and the end of the task? To answer these questions about how participants experience ideation, Csikszentmihalyi's concept of flow [10] can be used. It describes a complex psychological state that is characterized as a situation of perceiving an optimal and enjoyable experience by engaging in an activity with total involvement, concentration, and enjoyment. This results in intrinsic motivation and a sense of time distortion. Being in the flow contains: Having clear goals, focusing attention, losing self-consciousness, having an altered sense of time, enjoying the sense of control, and a merge of action and awareness. In summary, it describes the perfect balance between one's skills and the situations challenges, leading to an autotelic experience [6]. Therefore, one way to measure the psychological state of participants is to measure the balance between the skills and challenges, which is divided into eight possible dimensions: *apathy*, *worry*, *anxiety*, *arousal*, *flow*, *control*, *boredom*, and *relaxation* [22]. These parameters can be used as a barometer for reflecting the success of the ideation [10]. The Flow Wheel (cf. Fig. 5) asks participants to relate their amount of skills to the challenge during the recent activity by putting a single dot into one of the eight dimensions [11].
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** The Flow Wheel adapted by Dorta [11]. If adequate skills (x-axis) match adequate challenges (y-axis), the user comes into the flow state that is considered most productive.</figcaption></figure>

**Open Coding Video Analysis.** Video analysis is a popular approach to gather qualitative data of complex processes and to analyze them quantitatively.

The concrete nature of the findings is often uncertain before the analysis. In this case, open coding video analysis is a promising way to get a deeper understanding of the observed process [31, 19]. In this method, events in the videos are categorized by using codes that are defined during the coding process. This way, the approach is very flexible but also time consuming, as passages of the video have to be recoded as new codes emerge. Consolvo et al. [5] refer to a similar technique called LSA (Lag Sequential Analysis), as a valuable technique to generate quantitative and statistical data to observe ubicomp environments.

## 5.3 Our Evaluation Approach

Both categories, process-based as well as outcome-based evaluation methods, were used for evaluation. To get a broad insight into the ideation process, we chose to use a variety of process-based evaluation techniques. We used the Flow Wheel to find out about the participants' psychological state during the different stages of SIS. In addition, we used open coding video analyses to investigate overhead reduction and capture effects on group dynamics that might be a result of the changed environment. The evaluations were completed by an expert evaluation. Experts with different backgrounds (design, technic, and marketing) were asked to rate different aspects of the final results. Based on the above mentioned literature about outcome-based methodology we decided to use five criteria: maturity, usability, consumer benefit, level of detail, and novelty.
# 6 Experiment

To explore the digital implementation of the SIS method in use and to compare it with a traditional implementation, we conducted an experiment with students from the department of Innovation and Product Management. The main objective of this study was to get insight on how the use of an interactive, digital environment alters creative processes and group dynamics compared to a non-digital solution. Moreover, we wanted to find out how a digital environment fosters creativity and if it helps to get better ideas.

## 6.1 Reasons for Observing SIS

There are multiple reasons why we chose to use the creativity method SIS for our experiment. Most importantly it is a rather complex technique and involves a lot of different components such as individual and moderator guided collaborative work that also occur in other creativity techniques. In addition, the tools used in the traditional setting (flipcharts, pencil and paper) are rather generic and not tightly bound to this creativity method. Moreover, SIS does not limit the input to a specific type, like e.g. Method 635 [26], as the users can use sketches or handwriting to phrase ideas. Therefore, we believe that it is an appropriate technique to be studied as it is possible to draw conclusions for a variety of other creativity techniques as well.

## 6.2 Participants

32 of first-year students from the local university participated in the study. The participants were divided into eight groups of four, as four is a commonly used number to study various effects on group dynamics in small groups [2, 8]. There were seven females and 25 males between the age of 18 and 38 (M = 24.2, SD = 5.1). Participants in a group were either familiar or very familiar with each other. All participants made use of computers on a regular basis and had prior experience with pen-based interactive whiteboards. However, they were neither familiar with the used applications nor with the SIS technique and its digital implementation.

## 6.3 Moderators

As the moderator is a crucial factor in creativity methods alike SIS we decided to use different moderators in the experiment. This way we tried to balance the effects of single persons on the overall results. Overall five moderators (two female, three male) conducted one or two sessions each. The moderators stayed with the group when switching between the classical and the digital condition. They were not selected from the participant pool. The chosen moderators were more advanced students, who had considerable experience in presentation and workshop moderation. All moderators had prior training with the interactive environment and were familiar with SIS.
## 6.4 Apparatus

In the classic condition two flipcharts were used for public note taking and sketching (cf. Fig. 6. left). In addition, sheets of DIN-A3 paper were used by the participants during the Individual Solution Finding phase. Finally, the room was equipped with two pinboards to provide additional space to mount produced content.

The digital condition was conducted on a large (5.2 m × 1.17m) interactive whiteboard, driven by three Hitachi ultra-short-throw projectors with a total resolution of 3,072 × 768 (cf. Fig. 6. right). The whiteboard system was capable of handling simultaneous multi-user input through multiple Anoto pens (ADP-301). In addition, sheets of paper (DIN-A4) were provided to capture handwriting. For this purpose, multiple Anoto pens (ADP-201) equipped with a ball-pen refill were provided.

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** The classical condition (two flipcharts, two pinboards, paper DIN-A3) (**left**). The digital condition (one large interactive whiteboard, digital paper DIN-A4) (**right**).</figcaption></figure>

## 6.5 Tasks

Due to the use of within-subject design, we had to define two different tasks of a similar degree of complexity, the participants were equally familiar with. Therefore, we asked the participants to design an easy-to-handle drip coffee maker in one and a novel vacuum cleaner in the other condition. We chose these tasks due to their medium complexity that enables participants to come up with different solutions but also with the realistic prospect to complete the tasks within one hour.

## 6.6 Procedure

Every group of four completed two tasks (~60 min each) in total, one using the classical and one using the digital environment. This resulted in a two hour session per group with a short break between the two conditions. Each group was assigned to a moderator that were told to keep track of the overall duration. The tasks and also the order of the conditions were counterbalanced. The SIS process and also the ways of analysis were the same for each method. Both tasks were unveiled to the participants one week before the study, so that they had some time to think about possible solutions.

To evaluate the ideation process using the Flow Wheel, the SIS process was divided into three stages (cf. Fig. 2). At the end of each stage, the participants were asked to complete a questionnaire to measure their mental and psychological state. At the
end of both sessions, a questionnaire was handed out to the participants and moderation team to compare the experiences during the sessions. It included several questions focusing on problem description and understanding, visualization, further processing of generated ideas, documentation, and interaction. All sessions got videotaped for later analysis.

# 7 Results

In the final questionnaire, the digital condition was preferred by far to the classical setting, especially regarding overview, further processing, and fast visualization of the collected concepts. Also the expert evaluation confirms the advantage of the digital condition. To find out more about the causes of these positive results, we carefully analyzed the process using the above mentioned methodologies.

## 7.1 Outcome-based Results Based on Expert Evaluation

All 16 integrated solutions were visually standardized after all sessions had been completed, to avoid mapping of solutions to a specific condition. In addition, a short explanation was added by the moderator. A heterogeneous group of experts (2 senior designers, 2 senior technicians, 2 marketing experts) rated these solutions independently on a 10-point scale by using the five above mentioned criteria (maturity, usability, consumer benefit, level of detail, and novelty). We can make several interim conclusions from the scores in Fig. 7.

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Outcome-based expert evaluation of the final, visually standardized results. Each colored section represents one particular value on a 10-point scale (1=very bad, 10=outstanding).</figcaption></figure>
A Wilcoxon Signed-Rank Test comparing both techniques showed no significant difference in the maturity (z = -0.771, p = .441). The aspect of novelty was rated better in the digital condition (X̄<sub>digital</sub> = 6.5) than in the classical condition (X̄<sub>classic</sub> = 4), z = -2.524, p = .012. The outcome of the digital condition was also considered superior by the experts in terms of usability (X̄<sub>classic</sub> = 5 vs. X̄<sub>digital</sub> = 6, z = -2.38, p = .017), consumer benefit (X̄<sub>classic</sub> = 5 vs. X̄<sub>digital</sub> = 6, z = -2.521, p = .012), and level of detail (X̄<sub>classic</sub> = 5 vs. X̄<sub>digital</sub> = 6, z = -1.68, p = .039). In the following section we are going to examine the reasons for these results.

## 7.2 Process-based Results Based on the Flow Wheel

The diagrams in Fig. 8 show that the overall distribution of the psychological states is not entirely different for the two conditions. In both, we can see that most participants went from *flow* at the beginning (stage 1) towards the *control* segment at the end (stage 3). As the conditions did not really differ up to this point, this concord is not a surprise. However, it is interesting to see that, when using the classical SIS technique, already in the middle of the session (stage 2) quite a notable number of participants were in a state of boredom. This trend continued in stage 3, as the challenge level continued dropping. In contrast, in the digital condition, participants experienced a better balance between skills and challenges. Therefore they were kept longer and more consistently in the flow and control dimensions.

Fig. 9 shows that participants were immediately able to start their task (stage 1) and to get into the state of flow. Stage 2, in contrast, was new to them. In the digital

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** Psychological state of the participants during the different stages of the session.</figcaption></figure>
condition, the majority stayed in flow (15) and control (9) and only one continued to be anxious. Most of the participants stayed in flow (14) or control (8) also in stage 3, only eight participants felt either bored or relaxed. In contrast Fig. 9 shows a much higher drop-off of participants in the flow for the stages 2 and 3 in the classical condition.

<figure><img src="image_10.png" /><figcaption>Fig. 9. Number of the participants in the flow state during the different stages.</figcaption></figure>

## 7.3 Process-based Results Based on Video analysis

The collected video data was subdivided into fragments of five seconds and analyzed by using open coding [31, 19]. For each fragment the corresponding phase of the SIS technique was determined. Moreover, the physical activity (e.g. sitting, standing, environment adaption) as well as the core activity (e.g. listening, talking, private / public writing, idle) was identified for each participant and the moderator.

**Overhead Reduction.** Regarding the reduction of overhead, the video analysis gave us the expected clear confirmation. The average amount of overhead decreased from 8:21 min (SD = 1:02 min) in the classical condition to 4:44 min (SD = 0:44 min) in the digital condition (cf. Fig. 10). The measured time includes activities, such as moving flipcharts and pinboards as well as pining sheets of paper and copying content in the classical condition. In the digital condition, navigation and moving, duplicating, and transforming content were activities considered as overhead.

<figure><img src="image_11.png" /><figcaption>Fig. 10. Overhead Comparison: Idle refers to the time required for organizational tasks that block the creative process. Time needed to complete in-process questionnaires as well as phase 2 was also often used for organizational tasks.</figcaption></figure>
**Effects on the Overall Process.** While the reduction of overall organizational overhead is quite high (43%) in the digital condition, the reduction of the *Idle* time - the timespan where the creative process has to be stopped to perform tasks considered as overhead - is much smaller (26%). In both conditions, the time to complete questionnaires was used for organization. In addition, in the classical condition phase 2 was heavily used to copy or duplicate content while the participants were busy with presenting. On multiple occasions in the classic conditions, the moderator asked the participants to continue with the discussion, while he/she was still performing organizational tasks.

Fig. 11 shows the temporal sequences observed from two sample groups (1, 2) under the digital (D) and classical (C) condition. Although the overall sequence of the phases was more or less the same, certain differences can be observed.

<figure><img src="image_12.png" /><figcaption>**Fig. 11.** Four samples of temporal sequences. 1D and 2D represent digital, 2C and 1C classical conditions.</figcaption></figure>

1C and 2D were the first conditions conducted in the selected group. This is why the introduction was a lot longer. It is also noticeable that the illustrated digital conditions (1D, 2D) had a longer overall duration. Although this finding corresponds with the overall, average durations (classic: 54:06 min; digital: 56:47 min), there are too many different influences to identify a clear trend. For instance, the comparatively long problem framing phase in 2D was due to multiple questions of the participants. Furthermore the integration of the third solution in 1C and the fourth solution in 2C were extremely short due to large benefit overlaps between individual and combined solutions.

Comparing the idle phases between the classical and digital conditions shows the average number of idle phases per session is exactly the same (M = 18.25; SD<sub>digital</sub> = 2.6; SD<sub>classical</sub> = 0.5). The phases, however, last 7.1% longer in the classical condition. This difference seems to be the result of a small number of extraordinary long idle phases that occur when the moderator is no longer able to handle the overhead simultaneously with running the session. These long idle phases appear in nearly all classical conditions (cf. Fig. 11). In contrast, the idle phases' durations are more equal in the digital conditions.
**Effects on Participants' Performance.** Since there are a lot of variables (e.g. personality, involvement) that affect participants' performance, the results in this section have to be handled with care. By all means, the observed sessions in both conditions were very dynamic. Especially when the moderator was busy with writing or organizing, subgroups that actively discussed issues or did something else, unrelated to the actual problem of the session, emerged very fast.

Fig. 12 unveils that the communication of the moderators in phase 2 (Benefit Deduction) and phase 3 increases in the digital condition. Most likely this effect occurred due to the reduced amount of overhead performed in these phases. Moreover, increased attention during phase 2 (Presentation) might also affected this result as the moderators had a better understanding of the solutions. Surprisingly the number of the statements of the participants in phase 2 (Benefit Deduction) rose simultaneously. We suppose this is at least partly due to the increased activity of the moderator.

<figure><img src="image_13.png" /><figcaption>**Fig. 12.** Percentage of speech in the different phases and conditions. Due to the possibility of simultaneous comments, the combined time for moderators and participants may exceed 100%.</figcaption></figure>

# 8 Discussion

In general, the analyses confirm that digital environments are not only capable of substituting the traditional paper-based environment, but also provide additional values reducing the overhead and can lead to better results. Especially the results of the expert evaluation are very convincing.

The results of the flow analysis (cf. Fig. 8 and Fig. 9) shows that participants stayed considerably longer in a state of flow during the digital condition. Excitement about the novelty of the digital approach might be one possible explanation. However, as all participants were already familiar with the interactive environment, the factor of novelty was eased to a great extent. Therefore, we assume that the ability to stay in the flow for a longer period was primarily caused by other reasons, such as changed interaction patterns between the group and the moderator.

In the classical condition, the moderators attempted to keep the creative process alive by doing organizational work during the actual creative phases. Investigating the
amount of speech (cf. Fig. 12), we can see that this additional work leads to a de-
creased participation of the moderator especially in the second part of phase 2 and 3.
Without a moderator leading the discussion, the results of most discussions between
the participants ended quickly or the participants didn't stay focused. The experiment
confirmed that a strong moderator is a very important part in the creativity technique.
With decreasing his/her workload, which results in a higher participation and availa-
bility, the group productivity and process quality can be increased significantly.

In the light of this finding it is even more important to improve both the digital en-
vironment and the interaction concepts to decrease the required overhead even fur-
ther. However, as currently most of the organizational tasks are accomplished by the
moderator, also the adaption of the creativity techniques themselves should be con-
sidered. Digital environments enable smooth transitions between different private and
public media and thus support more dynamic workflows. This could help the group to
reduce their dependency from the moderator or to distribute the organizational over-
head among all participants. This way, it would be possible to use the observed dy-
namic creation of subgroups and discussions more productively and sustainably.

9 Conclusion and Future Work

In this paper, we compared the performance of the creativity technique SIS in a tradi-
tional, paper-based setting to a digital environment with focusing on reducing the
organizational overhead. Using a set of versatile methodologies, we did not only
prove the advantage of a digital environment, but also investigate the negative effects
of overhead when performing a creativity technique. Our findings show that the re-
duction of overhead does not necessarily result in a decrease of the task execution
time. It rather leads to a higher participation of the moderator, higher likelihood for the
participants to stay in a state of flow, and finally better and more advanced results.

In future work, we think it is important to intensify the research on interface and
interaction techniques that support smooth transitions and effortless content manage-
ment in digital environments to further reduce unnecessary overhead. Moreover, it
will be interesting to develop new concepts of creativity techniques that enrich the
dynamic possibilities of digital, interactive environments.

**Acknowledgements.** The presented research has received funding from the European Union's Seventh Framework Programme FP7/2007-2011 under grant agreement n° 318552. The experiment was realized in the “sprint>lab” that is supported by the Austrian Research Promotion Agency (FFG) within the research program COIN. The authors would like to thank the participants as well as the moderators, who helped us to realize the extensive study. We would also like to thank our colleagues from the Media Interaction Lab for their support with the implementation and their useful comments.
References

1. Bao, P., Gerber, E., Gergle, D., Hoffman, D.: Momentum: Getting and Staying on Topic during a Brainstorm. CHI 2010. pp. 1233–1236 ACM Press, New York, NY, US (2010)

2. Baruah, J., Paulus, P.B., Effects of training on idea generation in groups. Small Group Research 2008 39(5) pp. 523–41 (2008).

3. Boehm, B., Gruenbacher, P., Briggs, R.: Developing groupware for requirements negotiation: Lessons learned. IEEE Computer 18. 3 pp. 46–55 IEEE Press (2001)

4. Clayphan, A., Collins, A., Ackad, C., Kummerfeld, B. Kay, J.: Firestorm: a brainstorming application for collaborative group work at tabletops, Technical Report 678, University of Sydney (2011)

5. Consolvo, S., Arnstein, L., Franza, B.R.: User Study Techniques in the Design and Evaluation of a Ubicomp Environment. Ubicomp 2002. pp. 73–90 Springer (2002)

6. Csikszentmihalyi, M.: Artistic problems and their solution: an exploration of creativity in the arts. PhD Thesis. University of Chicago, Chicago (1965)

7. Dennis, A.R., Valacich, J.S., Nunamaker, J.F. Jr.: An experimental investigation of the effects of group size in an electronic meeting environment. IEEE Transactions on Systems, Man, and Cybernetics 20. 5 pp. 1049–1057 IEEE Press (1990)

8. Diehl, M. Stroebe, W.: Productivity Loss In Brainstorming Groups: Toward the Solution of the Riddle. Journal of Personality and Social Psychology 53. 3 pp. 497–509 (1987).

9. Diehl, M. Stroebe, W.: Productivity Loss In idea-generation groups: Tracking down the blocking effect. Journal of Personality and Social Psychology 61. 3 pp. 392–403 (1991)

10. Dorta T., Lesage A., Pérez E., Bastien J.M.C.: Signs of Collaborative Ideation and the Hybrid Ideation Space. ICDC 2010. pp. 199–206 Springer, Kobe, Japan. (2010).

11. Dorta, T., Pérez, E. Lesage, A.: The Ideation Gap: Hybrid tools, Design flow and Practice, In Design Studies. 29. 2 pp. 121–141 (2007)

12. Eversheim, W.: Innovation Management for Technical Products: Systematic and Integrated Product Development and Production Planning. pp. 384 Springer (2010)

13. Geschka, H., Schaude, G.R., Schlicksupp, H.: Modern Techniques for Solving Problems. Chemical Engineering. pp. 91–7 (1973)

14. Geyer, F., Pfeil, U., Höchtl, A., Budzinski, J., Reiterer, H.: Designing Reality-Based Interfaces for Creative Group Work. C&C 2011. pp. 165–174 ACM Press, New York (2011)

15. Guimbretière, F., Stone, M. Winograd, T.: Fluid interaction with high-resolution wall-size displays. UIST 2001. pp. 21–30 ACM Press, New York (2001)

16. Hailpern, J., Hinterbichler, E., Leppert, C., Cook, D. Bailey, B.P.: TEAM STORM: Demonstrating an Interaction Model for Working with Multiple Ideas during Creative Group Work. C&C 2007. pp. 193–202 ACM Press, New York (2007)

17. Haller, M., Leitner, J., Seifried, T., Wallace, J., Scott, S., Richter, C., Brandl, P., Gokceza-de, A., Hunter, S.: The NiCE Discussion Room: Integrating Paper and Digital Media to Support Co-Located Group Meetings. CHI 2010. pp. 609–618 ACM Press, New York (2010)

18. Hilliges, O., Terrenghi, L., Boring, S., Kim, D., Richter, H. Butz, A.: Designing for collaborative creative problem solving. C&C 2007. pp. 137–146 ACM Press, New York (2007)

19. Isenberg, P., Tang, A., Carpendale, S.: An exploratory study of visual information analysis. CHI 2008. pp. 1217–1226 ACM Press, New York (2008)

20. Liikkanen, L.A., Kuikkaniemi, K., Lievonen P., Ojala P.: Next Step in Electronic Brainstorming: Collaborative Creativity with the Web. CHI 2011 EA. pp. 2029–2034 ACM Press, New York (2011)
21. Liikkanen, L.A., Hämäläinen, M.M., Häggman, A. Björglund, T., Koskinen, M.P.: Quantitative Evaluation of the Effectivness of Idea Generation in the Wild. HCD 2011. pp. 120–129 (2011).

22. Massimini, F. Carli, M.: La selezione psicologica umana tra biologia e cultura. In Massimini, F. and Inghilleri, P., eds., L’esperienza quotidiana. Franco Angeli, Milan (1986)

23. Osborn, A.F. Applied Imagination: principles and procedures of creative thinking. Scribner, New York (1953)

24. Paulus, P.B., Brown, V.R.: Toward More Creative and Innovative Group Idea Generation: A Cognitive-Social-Motivational Perspective of Brainstorming. Social and Personality Psychology Compass 1. 1, pp. 248-265 (2007)

25. Reinig, B., Briggs, R. Nunamaker, J.F. Jr.: On the Measurement of Ideation Quality. In Journal of Management Information Systems 23(4). pp. 143-161 (2007)

26. Rohrbach, B.: Kreativ nach Regeln – Methode 635, eine neue Technik zum Lösen von Problemen. Absatzwirtschaft 12. pp. 73–75 (1969)

27. Schlicksup, H.: Innovation, Kreativität und Ideenfindung. pp. 124-126 Vogel Business Media, Würzburg (2004)

28. Shah, J.: Experimental Investigation of Collaborative Techniques for Progressive Idea Generation Techniques in Engineering Design. ASME Design Theory and Methodology Conference (1998)

29. Shah, J.J., Smith, S.M., Vargas-Hermandes, N.: Metrics for measuring ideation effectiveness. Design Studies 24(2). pp. 111–134 (2002)

30. Sutton, R.I. and Hargadon, A.: Brainstorming Groups in Context: Effectiveness in a Product Design Firm. Administrative Science Quarterly 41. pp. 685-718 (1996)

31. Tang, A., Tory, M., Po, B., Neumann, P., Carpendale, S.: Collaborative Coupling over Tabletop Displays. CHI 2006. pp. 1181–1290 ACM Press, New York (2006)

32. Taylor, D.W., Berry P.C., Block, C.H.: Does Group Participation When Using Brainstorming Facilitate or Inhibit Creative Thinking? Administrative Science Quarterly 3. June pp. 23-47 (1958)

33. Valacich, J.S., Dennis, A.R., Connolly, T.: Idea generation in computer-based groups: A new ending to an old story. Organizational Behavior and Human Decision Processes 57. 3 March pp. 448–467 (1994)

34. VanGundy, A.B. Techniques of Structured Problem Solving. 2nd ed. Van Nostrand Reinhold pp. 168-169 New York (1988)

35. Wang, H.-C., Cosley, D., Fussel, S. R.: Idea Expander: Supporting Group Brainstorming with Conversationally Triggered Visual Thinking Stimuli. CSCW 2010. pp. 103–106 ACM Press, New York (2010).

36. Warfield, J.N., Geschka, H., Hamilton, R.: Methods of Idea Management. Academy for Contemporary Problems. Columbus (1975).

37. Warr, A., O'Neill, E.: Understanding Design as a Social Creative Process. C&C 2005. pp. 118–127 ACM Press, New York (2005)

38. Warr, A., O'Neill, E.: Tool Support for Creativity using Externalizations. C&C 2007. pp. 127–136 ACM Press, New York (2007)