# On the Reliability and Factorial Validity of the Assessment Scale for Creative Collaboration

Aekaterini Mavri, Andri Ioannou, Fernando Loizides

► To cite this version:

Aekaterini Mavri, Andri Ioannou, Fernando Loizides. On the Reliability and Factorial Validity of the Assessment Scale for Creative Collaboration. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.783-792, 10.1007/978-3-030-29381-9_46 hal-02544565

HAL Id: hal-02544565

https://inria.hal.science/hal-02544565

Submitted on 16 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On the reliability and factorial validity of the Assessment Scale for Creative Collaboration

Aekaterini Mavri¹[0000-0002-5076-5891], Andri Ioannou²[0000-0002-3570-6578] and Fernando Loizides³[0000-0003-0531-6760]

¹ Cyprus Interaction Lab, Department of Multimedia and Graphic Arts, Cyprus University of Technology, 30 Archbishop Kyprianou Street, 3036 Limassol, Cyprus,

² Research Centre on Interactive Media, Smart Systems and Emerging Technologies (RISE), Julia House, 3 Themistokli Dervi Street, Nicosia 1066, Cyprus

³ School of Computer Sciences & Informatics, Cardiff University, Cardiff, CF24 3AA, UK

**Abstract.** Creativity, a primary objective across academic disciplines, has received considerable attention over the past few decades. While much focus has been put on the measurement of individual creativity, a notable research gap remains regarding social collaborative creativity that occurs in blended learning settings. This work offers an initial validation of the psychometric properties of a self-reported instrument, the Assessment Scale for Creative Collaboration (ASCC) that can measure learner perceptions of creative collaboration in a team within a computer-supported collaborative learning (CSCL) context. In this study, 236 undergraduate and graduate students rated the key variables of creative collaboration. Exploratory factor analysis resulted in a three-factor scale (21 items) measuring ‘Synergistic Social Collaboration’, ‘Distributed Creativity’ and ‘Learning Regulation and Achievement’. Cronbach's alphas indicated good internal consistency for the subscales. An instrument with psychometric properties for the assessment of creative collaboration is much-needed for the growing community of researchers and practitioners looking into creativity in education. It is also critical in advanced technical subjects, such as Design, HCI and Engineering, where collaboration is essential in developing innovative products.

**Keywords:** Creative Collaboration, Blended Learning, Psychometric Measure

## 1 Introduction and theoretical background

Recent decades have seen the rise of creativity as a critical element in higher education (HE). Creativity can provide a competitive advantage for today's young graduates and enhance their employment prospects as they transition into innovation-oriented digital industries [33]. Yet, the field appears significantly under-researched [5]. The bulk of present research has largely focused on organizational settings, while creativity in education, particularly in the areas of Design, HCI, and Engineering, has not been the focus of targeted investigation. Furthermore, collective, versus individual, creativity
has yet to be robustly investigated, despite the expanding practice of sociocultural
learning approaches in HE [29].

A multi-dimensional construct, creativity has always been challenging, especially in investigations seeking to identify the elements required for its effective practice and evaluation, as well as its technology-supported configurations. Research has provided a number of frameworks and models that attempt to theorize about creativity. These primarily focus on personality characteristics [41] and outcomes [15]. Various methodologies, such as the observatory [20], self-reported [27], evaluative [2, 15] and neurobiological [5] aim to capture different perspectives of the construct. Some of the assessment strategies for creativity include protocol analysis [12], purpose-specific coding for content analysis, behavior and activity-based testing [32], interaction analysis [29] and external evaluation of creative products [2, 15]. Lastly, the majority of psychometric measures, such as the ‘Torrance Test of Creative Thinking’ (TTCT) [30] and the ‘Kaufman Domains of Creativity Scale’ (K-Docs) [12], focus on individual dimensions of creativity. For collaborative endeavors, the assessment of creativity remains largely under-explored. A few studies have qualitatively observed brainstorming teams in an attempt to document collaborative creativity [23]. Others reported on distributed creativity through a computer-mediated discourse analysis approach [30]. What remains missing from the literature is an instrument aimed at the assessment of social creativity in education.

This work focuses on the creative collaborative processes of students in blended HE
settings – especially those in highly technical fields, such as Design, HCI and
Engineering, which require high levels of collaboration to develop innovative products.

As such, this study seeks to derive a psychometrically valid measure for the
evaluation of participant perceptions of such group processes using an existing
instrument, the Assessment Scale for Creative Collaboration (ASCC), as the main
deliverable of the European-funded CoCreat Lifelong Learning Project [36]. The
reliability measure for the 25 items of the instrument was reported at an earlier stage of
the project; its psychometric properties had yet to be assessed.

The scale measures the principal variables of creative collaboration as perceived by
team members in blended learning settings, based on underlying CSCL and creativity
theories [7, 9, 10, 17]. The term refers to the collaboration processes between people
across creative and other disciplines. The initial 25 items of the scale measure the
creative processes that stem from ill-defined problems, which initiate cycles of
imagination, divergent thinking and problem-solving that are driven by learners'
interest and engagement in a task. Learners draw from prior subject-level knowledge
and withstand time pressures to develop novel and appropriate outcomes.

The purpose of this study is to extend these findings by:

1. Determining the factors of ASCC and presenting its subscales' reliability.

2. Interpreting and analyzing the conceptual relationships of subscale variables, guided by background work.

In short, an instrument with psychometric properties for the assessment of creative
collaboration—so as to be able to research creativity in HE—is both needed and not yet
in place. The process for its development is described in the following sections.
# 2 Methodology

## 2.1 Instrumentation

The ASCC questionnaire uses a 7-point Likert scale and aims to elicit information about key concepts of creative collaboration in CSCL settings. It employs the term ‘creative collaboration’ based on multiple CSCL theories. The 25 questionnaire items prompt for divergent thinking, domain-level knowledge, critical thinking, response to real-life problems, social aspects of co-present and distant collaboration, conceptual factors of interest and engagement in collaboration, individual and joint time-management, learning regulation and time pressure.

## 2.2 Participants

The ASCC was completed by a total of 236 international undergraduate and graduate students who had prior experience of collaborative projects. The sample falls within the fair-to-good range of roughly 10 observations per item [24]. Students were asked to complete the questionnaire based on their most recent collaborative project experience.

## 2.3 Parallel Analysis (PA)

To define the statistically significant factors (eigenvalues) to be extracted, we first conducted Parallel Analysis (PA). Both PA and scree plot suggested a three-factor structure for the 25 items of the ASCC, which also matched the eigenvalue of >1 criterion.

## 2.4 Exploratory Factor Analysis (EFA)

Descriptive statistics presented an average range of item means of 4,26 – 5,92 (M=5,29) and adequate diversity in opinions (SD=1,46). EFA was conducted, using the Principal Axis Factor (PAF) extraction method as well as an Oblimin Oblique rotation method (delta=0) on the ASCC’s variables, which were expected to be correlated - a typical phenomenon in social studies. The Kaiser-Meyer-Olkin measure of sampling adequacy was found to be of an optimal value of .913. The Bartlett’s test of sphericity, reporting on the homogeneity of the correlation matrix [30], was found to be significant (χ² (300) = 3117,52 p < .001). The three factors obtained accounted for 47,28% of the total variance in the ASCC variables. Extracted factor eigenvalues and respective total variance percentages were as follows: Factor 1=9,084 and 36,33%, Factor 2=1,672 and 6,68%, Factor 3=1,065 and 4,26%.

A within variables approach indicated that the variables have a moderate to high level of common variance based on the extracted communality values: >.5 accounted for the 48%, >.4 accounted for the 40% and the rest for values of <4. The rotated pattern matrix (pattern coefficients) results indicated an initial set of eleven variables for Factor 1, seven variables for Factor 2, and seven variables for Factor 3. We retained variables with the following criteria: a) a pattern coefficient of 0,4 and above and b) significant differences in cross-loading values (approximately ≥ 0,20) [21].
<table><caption>Table 1. Scale dimensions, descriptions and individual items</caption><thead><tr><th>Dimension 1</th><th>Synergistic Social Collaboration</th><th>Theoretical Origin</th></tr></thead><tbody><tr><td colspan="3">A 9-item subscale that assesses social collaborative learning and the conceptual variables of interest and emotional factors such as belonging, mutuality and trust</td></tr><tr><td>Group interest in the task</td><td>1. Everyone in our group was interested in the task.</td><td>Interest</td></tr><tr><td>Trust between participants</td><td>2. Classmates/colleagues in my group trust each other.</td><td>Social Collaborative Learning</td></tr><tr><td>Orientation towards the task</td><td>3. Everyone in my group wanted to make a successful product.</td><td>Interest</td></tr><tr><td>Safe atmosphere</td><td>4. We had a feeling of belonging together.</td><td>Social Collaborative Learning</td></tr><tr><td>Communication</td><td>5. We were all able to express our ideas, even controversial ones, freely.</td><td>Creativity</td></tr><tr><td>Discussion of ideas</td><td>6. We were able to share and discuss our ideas with each other.</td><td>Creative Collaboration</td></tr><tr><td>Level of collaboration</td><td>7. We understood each other's viewpoints at the start of the project.</td><td>Social Collaborative Learning</td></tr><tr><td>Adequate knowledge base</td><td>8. Our group had the necessary knowledge to be able to complete our task.</td><td>Social Collaborative Learning</td></tr><tr><td>Shared knowledge and goals</td><td>9. I had a good idea of what the others in my group knew that is relevant to this activity.</td><td>Interest</td></tr><tr><th>Dimension 2</th><th>Distributed Creativity</th><th>Theoretical Origin</th></tr><tr><td colspan="3">A 7-item subscale that assesses collective divergent thinking and externalization, the degree of tension and perceived co-presence in distant teams</td></tr><tr><td>Problem boundaries stretched or broken</td><td>10. We weren't always certain about how to carry out the task which led us to explore different possibilities.</td><td>Creativity</td></tr><tr><td>A degree of disagreement or tension</td><td>11. We sometimes disagreed, but we discussed our different points of view.</td><td>Creativity</td></tr><tr><td>Group-based time pressure</td><td>12. My group was pressured to complete in time.</td><td>Time Pressure</td></tr><tr><td>Degree of co-presence (formally - text based)</td><td>13. We were able to share information between group members e.g. via a wiki or shared document.</td><td>Interest</td></tr><tr><td>Possibilities for externalizing representations</td><td>14. We could see or find out what other people knew or were thinking about. For example, we could draw, write or build things on the computer that the other group members could see and/or read</td><td>Creativity</td></tr><tr><td>Degree of co-presence (informally - SN)</td><td>15. We were able to chat informally with the other group members via text or social networking.</td><td>Interest</td></tr></tbody></table>
<table><tr><td>Level of divergent thinking</td><td>16. My group generated diverse and novel ideas in response to the task.</td><td>Creativity</td></tr><tr><td><strong>Dimension 3</strong></td><td><strong>Time Regulation and Achievement</strong></td><td><strong>Theoretical Origin</strong></td></tr><tr><td colspan="3"><strong>A 5-item subscale assesses the degree of individual and collective time-management as components of learning regulation and achievement</strong></td></tr><tr><td>Stretching boundaries</td><td>17. We went beyond the set task.</td><td>Creativity</td></tr><tr><td>Group-level time management</td><td>18. Our group organized our time for learning well.</td><td>Time Management</td></tr><tr><td>Individual time management</td><td>19. I organized my time for learning well</td><td>Time Management</td></tr><tr><td>Emotional expression</td><td>20. The set task/activity enabled us to express our emotions.</td><td>Social Collaborative Achievement</td></tr><tr><td>Level of imagination</td><td>21. Between us we used a lot of imagination</td><td>Creativity</td></tr></table>

Qualitative judgements about the retention of variables were made during post-PAF-processing. With the exception of items 2, 4 and 7 in Factor 1, the rest cross-loaded on other factors, but were maintained due to their compliance with retention criterion (b). Factor 2 loaded with a total of seven items. Item 16 failed the retention criteria, but was retained due to its critical conceptual significance related to divergent thinking. Factor 3 loaded with a total of seven items, out of which two did not match retention criteria and were thus dropped from the instrument. Factor 3 resulted in a total of five variables (see Table 1).

## 2.5 Reliability Analysis

Following FA, we proceeded to investigate the three subscales' internal consistency reliability and expected the following: a) a Cronbach's alpha coefficient minimum of α = 0,70 for the subscales [6] (the minimum value of 0,7 is acceptable for newly developed scales [16]), b) Inter-item correlations ranges of 0,3 and 0,7 to indicate homogeneity but no redundancy [25], c) small inter-item correlations standard deviation, preferably ≤ .1 [25] and finally, d) a minimum value of 0,4-0,75 for corrected item-to-totals as indicated in the item-total statistics results [21]. These are presented in Table 2, while individual reliability results for each sub-scale are outlined in the following three sections.

<table><caption>**Table 2.** Initial Reliability Statistics for the ASCC Subscales (N = 236)</caption><thead><tr><th></th><th>Cronbach's alpha</th><th>Mean inter-item correlations</th><th>SD of inter-item correlations</th><th>No. of items</th></tr></thead><tbody><tr><td>Factor 1</td><td>,924</td><td>,695</td><td>0,01</td><td>11</td></tr><tr><td>Updated*</td><td>,893*</td><td>,654*</td><td>0,00*</td><td>9*</td></tr><tr><td>Factor 2</td><td>,778</td><td>,505</td><td>0,01</td><td>7</td></tr><tr><td>Factor 3</td><td>,758</td><td>,529</td><td>0,01</td><td>5</td></tr></tbody></table>
**Subscale 1.** This subscale presented an optimal level of internal consistency at α = ,92 [6]. Most items fell within the inter-item-correlation ranges, apart from three items, which were above the value of 0,7. A closer examination in conjunction with the item-to-total correlation results, indicated that two out of three were far higher than the recommended upper limit and were therefore deleted. Item 1 was retained as a key conceptual variable of ‘interest’ within the subscale. A second reliability analysis, resulted in a lower (updated*), but still high, Cronbach’s value of α = ,89 (see Table 2).

**Subscale 2.** Reliability analysis of its seven items concluded an acceptable value of Cronbach’s α = ,77 (see Table 2). This subscale presented an item (12) that failed to meet the minimum criteria, in a few of the inter-item-correlation ranges. Based on the fact that it measures ‘time pressure’, a key conceptual element inherently linked to creativity and collaboration, the variable was retained.

**Subscale 3.** Reliability analysis of the subscale’s five items concluded an acceptable Cronbach’s value of α = ,76 (see Table 2). Item 20 scored just below the minimum value of 0,3 in the inter-item-correlation matrix (0,29). It was nevertheless retained in the subscale due to its critical theoretical significance (see Table 1). As all subscale coefficients resulted high alpha values (α ≥ ,70), the scale presents high internal consistency.

**3 Discussion**

This work undertook an initial validation of ASCC in response to an increasing need for instruments to assess collaborative creativity in HE team-work settings. EFA resulted in a three-factor scale, with a total of 21 items measuring ‘Synergistic Social Collaboration’, ‘Distributed Creativity’ and ‘Time Regulation and Achievement’.

**3.1 Subscale 1: Synergistic Social Collaboration**

The choice of term for this subscale relies on the role of synergy amongst collaborative team members in the production of greater results than the sum of separate individual parts. It comprises concepts related to both co-present computer-supported, as well as distant collaborative learning (CSCL) [11, 22]. The first subscale includes 11 items and addresses all co-present, physical, computer supported and distant collaborative learning (CSCL). It includes a number of affective variables, such as the sense of belonging, mutuality and trust between participants, as well as cognitive variables such as the ability to develop a shared understanding of individual viewpoints within a group (see Table 1).

The persistent recurrence of interest as an intrinsic motivational variable is anticipated, as it appears strongly intertwined with literature on collaborative learning and creativity. With both affective as well as cognitive traits, the construct of interest and engagement is linked to conceptualizations about one’s self as well as the social,
physical, and conceptual environment (i.e. 'Shared knowledge and goals') [34]. It is an intrinsic component of task-value derived from the Expectancy-Value theory. It also acts as a motivation for, and expectation of, success in performing a task i.e. 'Orientation towards task success' [35]. This is supported by the high correlation value between the two variables, 'Group engagement' and 'Task Success' (r = ,664).

Further theoretical associations confirm the structure of this subscale. For example, as interest and engagement grow, learners and collaborators in a field become naturally more inquisitive and explorative ('Discussion of early ideas') leading to in a field and further generation and analysis of ideas occurs. The 'Discussion of early ideas' is evidently clearly a significant stage in both collaborative and creative learning processes, that and it is also highly correlated to 'Group Engagement' (r = ,558) in the subscale. The ASCC report posits that this variable, typically related to brainstorming activities, is explicitly linked to Collaborative creativity literature [19]. Similarly, 'Adequate knowledge base' is regularly encountered across theoretical domains. Sufficient level of domain knowledge is projected by Amabile [2] in her componential theory of creativity and is also a primary variable in social constructivism as a precursor to higher-level cognitive functions in collaborative learning. Prior knowledge is also strongly connected to interest and engagement in this subscale (r = ,550) and across the literature [18].

## 3.2 Subscale 2: Distributed Creativity

Drawing from Sawyer's and DeZutter's [29] definition, this seven-item subscale is labeled 'Distributed Creativity', as the majority of its variables relate to this concept. Creativity is presented in the form of original ideas or products of the team-driven 'Level of divergent thinking', which are deemed suitable for a purpose (i.e. 'My group generated different and novel ideas in response to the task'). This type of collective creativity is heightened in response to ill-defined problems that lack explicit directions ('Problem boundaries stretched or broken'). Furthermore, a moderate 'Degree of disagreement and tension' within a respectful and trusting context is a positive precursor to collective novelty in ideas. This is supported in the correlation value between these two variables (r = ,452). A 'Degree of disagreement or tension' in the form of argumentative exchange can also enforce reflective reasoning during a collective creative task [36]. Tension in itself denotes evidence of engagement and interest, which is also found in the form of 'Degree of co-presence' (formal/informal and offline/online) in the subscale.

Another point of interest is the positive relationship between time pressure, and creativity, which is evident in the subscale. This relationship appears to work in opposite ways in the literature. Studies have shown that working under pressure impedes creativity by leading participants to choose safer options, rather than the more exploratory or time-consuming [1]. That said, working with mild-to-moderate time pressure, as a "challenge stressor" [28] can be beneficial, reportedly triggering creative effort and motivation. Apart from the high inter-item correlations in 'Degree of co-presence' (r = ,544), these variables correlate with 'Externalizing representations' (r = ,473), the latter of which is also highly correlated with 'Level of divergent thinking' (r
= ,476), presenting the second-highest correlation in the sub-scale. The link between creativity and externalization in social collaboration is key, particularly in the domains of Design and Engineering.

The process of using physical or digital artefacts such as paper sketches, texts or 3D-prototypes to portray thoughts on to tangible objects is used for communicative, coordinative, explorative and reflective creative activity [38]. These require a high degree of co-presence amongst team members, which is made evident through the subscale's high inter-item correlation structure between the two variables.

Finally, ‘Group-based time pressure’ and ‘Stretching problem boundaries’ present high inter-item correlations (r = ,463). The latter correlates well with the ‘Degree of disagreement or tension’ (r = ,443). We note that ‘Stretching problem boundaries’ refers to the exploration of different possibilities, as opposed to ‘Stretching boundaries’ in subscale 3, which suggests going beyond or improving upon the expected quality of the assigned deliverable.

3.3 Subscale 3: Time Regulation and Achievement

This factor’s name draws from the inherent interaction between learning regulation (encompassing time regulation) and achievement, based on relevant literature [26]. It consists of five items. ‘Individual’ and ‘Group-level time management’ carry—as expected—high inter-item correlation (r = ,636) in the subscale. The literature indicates that time-management is a primary component of learning regulation [26, 31] and reports on its three components, namely, self-regulation, co-regulation (pairs) and “socially shared regulation” [13] (teams). Shared learning regulation, in terms of time and effort, concerns tactics that are implemented according to a plan, so as to increase learning gains (self) or attain a collective target.

Additionally, the relationship between self/collective regulation ('Group time-management'), efficacy and creative achievement ('Stretching boundaries') is well documented in behavioral studies [26]. Bandura [4] states that an understanding of one's abilities, as well as "being purposive" towards an end goal is what leads to drafting and following a systematic learning action plan to the point of completing or going beyond the end goal ('We went beyond the task'). These two variables bear positive inter-correlations in the subscale (r=,463). Imagination, a factor synonymous with divergent thinking, is also highly correlated with 'Stretching boundaries' and 'Group time management' (r = ,435). The literature denotes that creative individuals consciously seek to regulate their practices to produce novel outcomes [37].

'Emotional expression' also relates to conceptions of regulated learning as a means of commitment and orientation towards an end goal. The term has a positive correlation with achievement ('Boundaries stretched') in the subscale (r = ,334). It is also supported by the literature [28], which widely agrees that collaboration built on socio-emotional spaces that foster inter-connectedness amongst participants enhances creativity. Conversely, negative emotional load stemming from a distrustful or restrictive atmosphere hinders expression and generates poor creative outcomes.
## 4 Conclusion

An instrument with psychometric properties that can assess creative collaboration is much-needed in the growing community of researchers focusing on creativity in higher education. This is especially important in the educational areas of Design, HCI and Engineering, where collaboration is key to produce innovative outcomes. The ASCC, a self-reported questionnaire, is designed to measure the perceived creative collaboration amongst teams working in blended learning settings. The objective of this study was to examine the ASCC's psychometric properties by (i) determining its subscales and presenting their reliability value and (ii) interpreting and analyzing the relationships of the subscale variables, guided by conceptual groundings from earlier work. Factor analysis resulted in a three-factor structure (21 items), namely, ‘Synergistic Social Collaboration’ (9 items), ‘Distributed Creativity’ (7 items), and ‘Time Regulation and Achievement’ (5 items), all bearing good reliability values. Future improvements could focus on using a Confirmatory Factor Analysis to provide additional validity for the ASCC instrument.

### References

1. Amabile, T.M. et al.: Creativity under the gun. Harvard business review. 80, 52–63 (2002).

2. Amabile, T.M.: Social psychology of creativity: A consensual assessment technique. Journal of personality and social psychology. 43, 5, 997 (1982).

3. Amabile, T.M., Pillemer, J.: Perspectives on the social psychology of creativity. The Journal of Creative Behavior. 46, 1, 3–15 (2012).

4. Bandura, A.: Social cognitive theory of self-regulation. Organizational behavior and human decision processes. 50, 2, 248–287 (1991).

5. Batey, M.: The measurement of creativity: From definitional consensus to the introduction of a new heuristic framework. Creativity Research Journal. 24, 1, 55–65 (2012).

6. Cronbach, L.J.: Coefficient alpha and the internal structure of tests. Psychometrika. 16, 3, 297–334 (1951).

7. Dillenbourg, P. et al.: The evolution of research on computer-supported collaborative learning. In: Technology-enhanced learning. pp. 3–19 Springer (2009).

8. Eccles, J.: Expectancies, values and academic behaviors: Achievement and achievement motives: Psychological and sociological approaches. Presented at the , San Francisco (1983).

9. Ferrari, A. et al.: 23. ICT as a driver for creative learning and innovative teaching. Measuring Creativity. 345 (2009).

10. Fischer, G., Shipman, F.: Collaborative design rationale and social creativity in cultures of participation. In: Creativity and Rationale. pp. 423–447 Springer (2013).
11. Gaggioli, A. et al.: The creative link: Investigating the relationship between social network indices, creative performance and flow in blended teams. Computers in Human Behavior. 42, 157–166 (2015).

12. Gero, J., Kan, J.: Empirical results from measuring design creativity: use of an augmented coding scheme in protocol analysis. In: DS86: Proceedings of The Fourth International Conference on Design Creativity, Georgia Institute of Technology, Atlanta, GA, USA. (2016).

13. Hadwin, D.H. et al.: Handbook of self-regulation of learning and performance. Taylor & Francis (2011).

14. Hirst, G. et al.: A cross-level perspective on employee creativity: Goal orientation, team learning behavior, and individual creativity. Academy of management journal. 52, 2, 280–293 (2009).

15. Horn, D., Salvendy, G.: Product creativity: conceptual model, measurement and characteristics. Theoretical Issues in Ergonomics Science. 7, 4, 395–412 (2006).

16. Lance, C.E. et al.: The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? Organizational Research Methods. 9, 2, 202–220 (2006). https://doi.org/10.1177/1094428105284919.

17. Lew, K.H. et al.: The relationship among creativity thinking ability, creative personality and creative product. Advanced Science and Technology Letters. 36, 58–62 (2013).

18. Linnenbrink-Garcia, L. et al.: Developing conceptual understanding of natural selection: The role of interest, efficacy, and basic prior knowledge. The Journal of Experimental Education. 80, 1, 45–68 (2012).

19. Mamykina, L. et al.: Collaborative creativity. Communications of the ACM. 45, 10, 96–99 (2002).

20. Meneely, J., Portillo, M.: The adaptable mind in design: Relating personality, cognitive style, and creative performance. Creativity Research Journal. 17, 2–3, 155–166 (2005).

21. Netemeyer, R.G. et al.: Scaling procedures: Issues and applications. Sage Publications (2003).

22. Paulus, P.B. et al.: Collaborative creativity—Group creativity and team innovation. In: Handbook of organizational creativity. pp. 327–357 Elsevier (2012).

23. Paulus, P.B., Nijstad, B.A.: Group creativity: Innovation through collaboration. Oxford University Press (2003).

24. Pearson, R.H., Mundform, D.J.: Recommended Sample Size for Conducting Exploratory Factor Analysis on Dichotomous Data. Journal of Modern Applied Statistical Methods. 9, 2, 359–368 (2010). https://doi.org/10.22237/jmasm/1288584240.

25. Pett, M.A. et al.: Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Sage (2003).

26. Pintrich, P.R.: A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students. Educational Psychology Review. 16, 4, 385–407 (2004). https://doi.org/10.1007/s10648-004-0006-x.
27. Plucker, J.A., Renzulli, J.S.: Psychometric approaches to the study of human creativity. Handbook of creativity. 35–61 (1999).

28. Prem, R. et al.: Thriving on challenge stressors? Exploring time pressure and learning demands as antecedents of thriving at work. Journal of Organizational Behavior. 38, 1, 108–123 (2017). https://doi.org/10.1002/job.2115.

29. Sawyer, R.K., DeZutter, S.: Distributed creativity: How collective creations emerge from collaboration. Psychology of aesthetics, creativity, and the arts. 3, 2, 81 (2009).

30. Scott, T.J.: Distributed Affect As a Framework for Understanding Creative Collaboration. In: Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. pp. 335–336 ACM, New York, NY, USA (2015). https://doi.org/10.1145/2757226.2764768.

31. Stoeger, H., Ziegler, A.: Evaluation of a classroom based training to improve self-regulation in time management tasks during homework activities with fourth graders. Metacognition and Learning. 3, 3, 207–230 (2008). https://doi.org/10.1007/s11409-008-9027-z.

32. Torrance, E.P.: Torrance Tests of Creative Thinking: Normstechnical Manual : Research Ed. : Verbal Tests, Forms A and B : Figural Tests, Forms A and B. Flere materialer. Personell Press (1966).

33. Turbot, Sebastien: Is higher education equipping young people for the jobs market?, https://www.weforum.org/agenda/2015/06/is-higher-education-equipping-young-people-for-the-jobs-market/.

34. Wentzel, K.R., Miele, D.B.: Handbook of motivation at school. Routledge (2009).

35. Wigfield, A., Eccles, J.S.: Expectancy–value theory of achievement motivation. Contemporary educational psychology. 25, 1, 68–81 (2000).

36. Wishart, J. et al.: CoCreat Research Plan. Internal report of the CoCreat project. (2011).

37. Zampetakis, L.A. et al.: On the relationship between individual creativity and time management. Thinking Skills and Creativity. 5, 1, 23–32 (2010). https://doi.org/10.1016/j.tsc.2009.12.001.

38. Zurita, G. et al.: Introducing a Collaborative Tool Supporting a Learning Activity Involving Creativity with Rotation of Group Members. J. UCS. 22, 10, 1360–1379 (2016).