Tool-building for amateur creativity in virtual reality

This thesis studies amateur creativity in the context of building virtual reality experiences. Access to creative self-expression is an important part of flourishing as a human, and new technologies often dramatically affect how we live our lives. So, it is the underlying mission of this thesis to ensure that after its period of rapid commercialization, virtual reality can still be used creatively by everyday people. I introduce the term "folk design" to mean design pursued by amateurs for their own enjoyment and for the connection to their community that the process brings. Supporting folk design in virtual reality naturally leads to more specific research questions: how can virtual reality experiences enrich someone's life? How can we create tools that are approachable and enjoyable to use from within virtual reality? Can we better understand the needs of amateur creators within their social context? The goal of this thesis is to develop potential values and strategies that might influence future experiences and tools for virtual reality. In order to do so, I use a Research through Design methodology to develop design theory. Design theory is not universal; it is specific to the context of people, places, and things where it was developed. It is often said that the goal of design research is to produce guidance that sometimes brings about effects that are aligned with its underlying values. It is left to future work to see what other situations the design theory developed here can be useful in. Nevertheless, it is strongly shown that it is useful in the context of amateur creation of virtual reality environments. The research process of this thesis began with a consideration of musical creativity in virtual reality. Armed with values on living well and the benefits of making music, I first analyzed existing commercial experiences focused on music in virtual reality. Next, I explored new musical interactions and experiences that would encourage musical creativity in ways not addressed by the commercial experiences. After building a feature-length experience, I next considered how to build tools that would enable amateurs to create similar experiences, but from within virtual reality and without specialized technical knowledge. After experimenting with a sculptural block-based programming language that enabled users to control their environment with sound, I broadened my focus to every aspect of the virtual environment. I built tools powered by interactive machine learning for creating landscapes, music, and virtual creatures (requiring both sound and animation), all by providing examples of the desired appearance or behavior. Finally, I developed insights on how to support amateur creators in social environments. I first looked at a context that succeeded in this area, observing public making workshops in museums. When translating those insights into VR, I discovered that a major area of difference between physical and virtual reality was creators' comfort in communication. So, I developed communication techniques for indirect, whimsical communication that enabled creators to express their ideas, show their personality, and develop a shared language. Ultimately, the tools I developed enabled real-world amateur creators to folk design in virtual reality. With this research, I contributed three kinds of knowledge. From analyzing and building experiences in virtual reality, I developed theoretical perspectives on how to infuse these experiences with the values of human flourishing. Of particular importance is doing vs. being, the balance of focuses on action, agency, reflection, and calm. From building tools for the creation of virtual environments, I proposed reusable strategies for using interactive machine learning in creative tools. One such strategy, chained mappings, connects several machine learning models together so that they can develop complex relationships from simple inputs, with creators able to provide semantically meaningful examples at multiple levels. And, from my observations of amateur creativity in museums, I articulated insights on supporting social creativity. Overarching these insights is ad hoc social connection, the short-lived yet strong bonds between unfamiliar creators working near each other. My tools for indirect, whimsical communication in virtual reality then led me to articulate seven areas of consideration that should be covered by any such set of tools, such as influence, personalization, and expression. Taken together, these contributions form a design framework for supporting amateur creativity in virtual reality. Each contribution stands alone and may be useful to guide any project incorporating a focus on human flourishing through creativity. As a whole, the framework empowers everyday people to express themselves, to play, and to build community. It also helps knit together a growing body of academic research on supporting creativity not for the end result, but for the joy of the process itself

Keywords: Music.
Advisor: Wang Ge
Chafe Chris
Fiebrink Rebecca Anne