Bidirectional Analysis of Creative Processes: A Tool for Researchers

Young peoples’ creative production combines such activities as analysis of existing forms and genres; participation in processes including drafting, journaling, and modeling; sustained work on a particular idea; and mentor critique. In our work with a variety of settings, we have found that young artists’ progress depends on their engagement in all of these tasks, and that analysis of this work depends on looking across and through these varied practices. While typical descriptive analyses of such processes move forward, we have found that moving bidirectionally—from final product backward and from initial idea forward—better helps us to understand participants’ learning. We use examples from two creative production contexts to describe a technological tool developed for bidirectional artifact analysis by outlining its features, affordances, and constraints. Why Analyze Creative Process? In recent years, new literacies research has begun to re-focus the meaning of “literacy” and “multi-literacies” (e.g. Cope & Kalantzis, 2009; Moje, Overby, Tysvaer, & Morris, 2008). Beyond the comprehension and production of texts, “being literate” in a particular content area has come to describe fluency in specific “constellations” (Steinkuehler, 2007) of practices, modes, and ways of knowing. By producing content, learners use medium-appropriate tools to create artifacts and develop metarepresentational competence, an understanding of what tools are useful for expressing which ideas (Halverson, 2013). These understandings of literacy have particular purchase in digital environments and creative arts education, where interactions are often shaped by the production and critique of artifacts for a responsive audience. Reframing literacy and learning in terms of how students use tools to communicate ideas requires an emphasis on measuring the process and the products of learning (Halverson, 2012). As researchers in the learning sciences, we seek to understand and design learning environments that support these complex practices. In order to do so, we must construct methods that help researchers describe the processes of creative production and the creation of successful literacy artifacts. We need analytical lenses that take the complexities of creative works themselves—from online video to school essay to multimedia collage—as units of analysis and trace these works through creation, drafting, and revision. Thus, we need tools that enable such analyses. Because these social, artifactual, and often multimodal processes extend over varying periods of time, it is difficult to capture them using existing methods and tools for qualitative analysis. While our prior work has focused on the development of tools for analyzing youth-made films (Halverson, Bass, & Woods, 2012), these tools do not generalize to creative processes more broadly, nor do they capture the process of creation. Rather, they afford final product analysis. Altogether, creative production combines such activities as engagement with existing genres, problems, or frameworks; participation in interlaced processes (drafting, journaling, modeling, etc); sustained work on a particular idea; and audience-based critique (Barrett, 2000; Soep, 2006). In our work with young artists, media, and settings, we have found that their progress depends on their participation in all of these tasks—as well as ongoing engagement with their audience(s) (Magnifico, 2010, 2012; Halverson, 2012). In other words, the analysis of creative process depends on not focusing on particular types or genres of work, as in discourse analysis or artifact analysis, but looking across and through varied practices. We first developed methods for understanding creative production processes (Halverson & Magnifico, 2013). We now have developed a tool to look across and through full trajectories of art development. Using Javascript, HTML 5, SVG, and CSS technologies, we present an alpha version and examine its affordances for data analysis and sense-making. We need such a tool to document creative practices, examine the origins of particular ideas and critiques, and trace how artists take up these concepts (in future drafts or more broadly). This year’s ICLS theme, “Learning and Becoming in Practice” highlights learning trajectories over time: ways in which learners adopt certain kinds of practices and become certain kinds of people (e.g. Duncan & HmeloSilver, 2009). Our tool helps streamline the analysis, visualization, and representation of the learning involved in creative production. As a field, the learning sciences value theory-building and technological design around such questions, and we are eager to participate in such ongoing conversations and inquiry. Here, we use examples from two creative production contexts—creative writing and mobile game design—to describe our tool for bidirectional artifact analysis. We outline its features and offer suggestions for using the tool in other research contexts that value the role of process in understanding learning. ICLS 2014 Proceedings 839 © ISLS Audience as Fundamental to Learning Through Production Much of the original literature that developed and documented rhetorical (e.g. Ong, 1975) and cognitive (e.g. Bereiter & Scardamalia, 1987; Flower & Hayes, 1981) viewpoints on audience implicitly assumes that when writers write with an audience in mind, these imagined readers have a measurable effect on the writing. This assumption is rooted in ethnographic and expert-novice research on how writers think and how writers write, and has been documented repeatedly with different kinds of writers. Understandings of audience and collaboration have broadened, however, as new media have pushed writers and artists into spaces where they may create, read, and view “among the audience” (Lunsford & Ede, 2009). This shift can transform writers into speakers, readers into critics, and both into collaborators. Within online and face-to-face artistic communities, it is increasingly plausible to think of readers and viewers in terms of Ong’s (1975) “collectivity, acting. on one another and on the speaker [or writer, or artist] as members of an audience do” (p. 11). Considering how feedback guides artistic process suggests that audiences for creative work may play both a cognitive role in artistic production, in that sharing work forces creators to consider readers, and a social one, in that creators define themselves as community members by the work that they share (Magnifico, 2010). If we extend the metaphor of literacy from producing text to creating artifacts using various multimodal tools for meaning making, we can apply the same insight about audience to creative processes in general. Audience is central to the learning of creative production, not an afterthought. As a result, analyses of creative production processes must include an understanding of audience feedback and interaction as a core part of tracing learning. The Role of Representation in Creative Processes diSessa (2004) argues that representation design is a creative endeavor, “a venue in which creative and artistic skills are at a higher than normal premium” (p. 300). Thus, observing creative production helps us understand the role of external representation in learning (e.g. Eisner, 2002). “Getting smart” in the context of producing art “means coming to know the potential of the materials in relation to the aims of a project or problem” (Eisner, 2002, p. 72). The capacity to build representations from different materials for multiple situations is a marker of creative expertise (Hayes, 1989), and has been called “essential in investigations of creativity” (p. 262) (Hasirci & Demirkan, 2007). Since creative production requires creative representation, young artists must learn to construct and evaluate multiple representations. We have called this process “the representational trajectory” and its outcome the development of “metarepresentational competence” (Halverson, 2013). The representational trajectory describes how producers focus on a central idea or concept, use this idea to motivate the development of skills and tools to represent their concept, and design a product that reflects this relationship between concept and tools. To study and describe this learning process requires multiple forms of evidence at multiple points in time, often with the engagement of multiple peers and mentors (Halverson, 2013). Creative production settings often encourage novices to use several media to produce their art. While representations may shift and ideas may transform through drafting, feedback, and revision, understanding how these artifacts contribute to the final “text” is necessary to understand the processes that producers employ. Bidirectional Artifact Analysis To address this methodological need, we developed bidirectional artifact analysis (BAA), an analytic method for understanding creative production that employs ethnographic observations of participants, artifacts they create, and interviews over time (Halverson & Magnifico, 2013). A key component of BAA is this concept of time. Most education research moves forward in time because learning continually builds up from prior knowledge. Our concept of time is grounded in Enyedy’s (2005) description of bidirectionality: “go[ing] ‘backwards’ in time in an attempt to trace the origins of this intervention and ‘forwards’ in time to examine what subsequent impact it had on the way other students reasoned” (p. 437). Moving bidirectionally helps us understand the role of social, collaborative feedback and audiences in young peoples’ learning. By turning the analytical lens, we see how learners’ reflections on past representations become shared prior knowledge. BAA involves three steps: 1) identifying learner-created artifacts; 2) documenting relevant data around artifacts; 3) constructing threads across the data types to trace the final product’s core ideas and too