Final report for the Tangible Handimation project

Computer animations have in recent years become very common in feature movies, commercials, and computer games. The goal of the Tangible Handimation project was to demonstrate a possibility to improve existing practice for production of animated content for visual media and entertainment. An additional goal was to demonstrate the possibilities for collaborative work among animators by allowing several animators to animate one or many objects at the same time in real-time. The core novelty of Tangible Handimation is that it allows individual recording of animation part through intuitive controllers, which in combination with replay and rewind functionality drastically decreases turnaround time. This report describes the background of the project, the UI and software development conducted during the project, and feedback received by professional animators. The planned adjustments of Tangible Handimation and current dissemination of results concludes the report. 1. Background Computer animations have in recent years become very common in feature movies, commercials, and computer games. After being popularized by companies such as Pixar with the movie Toy Story, the rapid development of graphical processing power has allowed a proliferation of computer animated production possibilities, or phrased in other words “Although there was a time when elaborate computer animation was restricted to blockbuster movies, much of the three-dimensional work being generated today occurs in small to mid-sized companies” (Bureau of Labor Statistics, 2004). The profession group that is responsible for turning mathematical descriptions of models into believable characters, multimedia artists and animators, was the fastest growing group in 2004 within the motion picture and video industries (Bureau of Labor Statistics, 2004), and that is excluding the video and computer game industry with at the same time had world sales reaching $25.4 billion (ESA, 2006). In parallel with the growth of computer animation in games the costs for games have grown from an average of $40,000 in the early 1990s to an average of $10 million in 2004 (ESA, 2006). One of primary reasons for this increase is three dimensional graphics which currently “require ‘millions of polygons’ and ‘teams of artists,’ thereby significantly increasing the cost of making a video game” (Crandall & Sidak, 2006). Although computer animation began with lengthy waiting times for rendering on dedicated computers, the advances in computer games have changed this and leading game engine programmers such as Carmack of Id Software believes that “[eventually] real-time rendering will be so dynamic that animators will be able to produce films using game engines” (Kushner, 2003). Thus, the bottleneck in computer-animated characters lies not within the costs of the hardware used for rendering. Instead, current techniques for animation either limit acting through letting people set specifications and then letting computer perform them (keyframe animation and path animation) or require expensive sensing technologies (motion capture). None of these approaches let several animators collaborate with ease in real-time. Handimation (Svensson, 2007) is a novel approach to computer animation that solves part of this problem by allowing animation to be done in real-time rendering systems. The system allows an animator to record and playback animations of individual model axis in real-time using a sequencer-based interface, using input from various input devices, e.g. the keyboard or mouse. Figure 1: The Handimation system used to create a scene with two characters. Handimation was conceptualized by a professional animator as a tool to speed-up the process of making animation; an explicit goal was to be able to produce a two-minute short film in one day, assuming the script was written, dialogue was recorded, and sets created in advance. Another goal was to let animators work in real-time so that they in practice performed when they produced the movements the puppets should have, and the idea of having an interface based upon a sequencer metaphor had been identified. The view of animators as performer gave a requirement of being able to take input from various devices since different device, e.g. physical marionettes with sensors, might better suit different types of performances. An initial attempt to implement this vision was carried out as a Master Thesis project which resulted in Handimation. Handimation is a general connection interface for connecting input from mouse, keyboard, and joystick to different virtual 3D objects. The system is able to record, rerecord, replay and stream an arbitrary number of data tracks concurrently. The prototype is built on Virtools, an authoring tool for creating interactive 3D content. It runs as a compilation in Virtools Dev 3.0 and loads 3D models in VirTools NMO-format. The available 3D objects to be animated with Handimation are loaded at startup and imminently visible to the user. A sequencer-based interface is overlaid providing recording and replaying functionality through data tracks, where each real-time movement to be recorded requires a data track. The interface can be hidden to be able to use the whole screen for visual feedback when animating. Figure 2: details of a data track in Handimation. An arbitrary number of data tracks can be created and deleted as needed. Each data track resembles the I/O interface of most audio/MIDI sequencers with input and output options but handling data routing slightly differently. Input options consist of what input device to use, what axis of that device to use, and a multiplicator to adjust the signal strength. Output options consist of a selection of position-, directionand up-vector components for all objects in the scene. The choices for data routing consists of play (P) to activate replay of already recorded data, record (R) to make the track collect data from an input when recording, and through (T) forwards streamed data from the input device to the output object. 2. Initial Plan The main objective of the Tangible Handimation project was to introduce support for a broader range of interaction device to enable animators to make use of skill sets from traditional crafts such as puppeteering. Specifically, the plan was to extend Handimation to support for collaboration for multiple users and input from cheap input devices like the Nintendo Wiimote. This new version was called Tangible Handimation since it provides tangible interface support. The goal of the project was that this then could be used to explore more expressive interaction, with the hopes of making use of tacit knowledge animators have that do not easily map to current computer interfaces. A secondary objective was to provide Tangible Handimation with collaborative support, allowing skills from different animators to be used simultaneously in real-time. This to allow multiple animators to animate the same character simultaneously where each can take control different parts e.g. one controls the head and face while another the walk and body pose. While this can allow quicker animations of individual characters another possibility was possibly to let each animator control an individual character in real-time and thereby let them act against each other through their characters. To reach the goals of the project researchers were involved that were knowledgeable in computer animation, tangible interfaces, and collaborative environments since these had been identified as primary research areas. The project consortium thus consisted of three core partners: the Game studio at Interactive Institute II AB, the ICE (Interactive Collaborative Environments) lab at SICS AB, and Tussilago AB. II was mainly responsible for software development and usability issues in the finished prototype while SICS provided expertise regarding hardware, tangible interaction and collaboration. Tussilago, from which the original Handimation idea had originated, acted as multimedia producer and as end-user for internal testing. Further, as the project was seen as part of an iterative development process in the vein of established best practices within interaction design, external representatives of potential future users were contacted. Focusing on professional animators this meant that key people at SVT and Film i Väst, the main regional employers of animators, were involved to enable access to feedback from professionals. The Center of Visualization in Göteborg (www.center-of-visualization.org) was also involved in the project since this provided a platform for disseminating the results of the project to regional companies within the visualization sector. The initial plan for the project was to divide the work into three phases. First, some already identified technical issues with the Handimation prototype needed to be addressed. Second, a workshop was planned to collect opinions from professionals to create a basis for a new design. Third, implementation of the new design would be done with continuous feedback from the internal tester. A final feedback session with external participants would be conducted if resources allowed. 3. Results The project started out with improving the already identified technical issues. These included changing the animation recording and playback timing to be independent of computer performance. Furthermore during the first phase initial support for the Nintendo Wiimote to provide with tangible input. Also in the first phase planning and preparation for the workshop was conducted. Recruiting participants though turned out to take more time than initially planned mainly due to difficulties to find a suitable date. Therefore it was decided to take the software development further and different filtering techniques were investigated as well as SenseBoard support was implemented. SenseBoard is another input device that might overcome some problems encou