reating realistic animated models of people is a central task in digital content production. Traditionally, highly skilled artists and animators construct shape and appearance models for a digital character. They then define the character's motion at each time frame or specific key-frames in a motion sequence to create a digital performance. Increasingly, producers are using motion capture technology to record animations from an actor's performance. This technology reduces animation production time and captures natural movements to create a more believable production. However, motion capture requires the use of specialist suits and markers and only records skeletal motion. It lacks the detailed secondary surface dynamics of cloth and hair that provide the visual realism of a live performance. Over the last decade, we've investigated studio capture technology with the objective of creating models of real people that accurately reflect the time-varying shape and appearance of the whole body with clothing. 1 We've developed a system for surface motion capture that unifies the acquisition of shape, appearance, and motion of the human body from multiple-view video (see also the "Related Work" sidebar on page 22). It captures a shape and appearance model for the performer in full wardrobe as well as the model animation and surface dynamics to create highly realistic digital content from the performance, unencumbered by a specialist suit. Our system solves two key problems in performance capture: scene capture from a limited number of camera views and efficient scene representation for visualization. The multiple-view video and scene reconstruction in this work is available at http://www.ee. surrey.ac.uk/cvssp/vmrg/surfcap. We record an actor's performance in a dedicated multiple-camera studio with controlled lighting and a chroma-key background. Because of cost, rather than use professional cameras such systems typically use machine vision cameras, which don't give the color quality required for production work in broadcast or film. Our work aims to demonstrate our surface capture technology's potential for high-quality entertainment content. We therefore developed a studio system using a limited number of professional film-quality highdefinition (HD) cameras. In our studio, we spaced eight HD cameras equally around a circle of 8 meters in diameter about 2 meters above the studio floor (see Figure 1 ). This gives a performance volume of 4 ´4 ´2 meters with a wide-baseline 45-degree angle between adjacent camera views. We capture performances using Thomson Viper cameras in High-Definition Serial Digital Interface (HD-SDI) 20-bit 4:2:2 format with 1,920 ´1,080 resolution at 25 Hz progressive scan. Eight dedicated PC capture boxes using Digital Video Systems HD capture cards record synchronized video from all eight cameras uncompressed direct to disk. This studio setup enables highquality performance capture from wide-baseline camera positions. Recording a performance from multiple video cameras shows the actor from a fixed set of viewpoints. Providing a complete 3D model for interactive visualization requires geometric surface reconstruction, as Figure 2 shows. To recover geometric shapes from camera images, we must calibrate the camera parameters. To calibrate a camera, we image a known object and derive the camera transformation that reproduces the object in a set of captured images. Techniques typically use a planar grid of known geometry that's visible from all cameras. Several public domain tools are available to achieve this (for example, http://www.vision.caltech.edu/bouguetj/ calib_doc). This approach is suitable for camera systems in which the images have an overlapping field of view, but is impractical in studios where cameras surround the capture volume. For studio production where cameras might be reconfigured many times in one day with location and zoom changes, a simple and quick method of calibration is of key practical importance. For rapid and flexible calibration of a multiple camera studio, we developed a wand-based calibration technique. Wand-based calibration uses two spherical Surface capture is a fully automated system for capturing a human's shape and appearance as well as motion from multiple video cameras to create highly realistic animated content from an actor's performance in full wardrobe.
One of the first pioneering work in surface capture for performance-based animation is proposed in REF with a fully automated system to capture shape and appearance based on visual hull extraction, feature matching and dense reconstruction.
Surface Capture for Performance-Based Animation
IEEE Computer Graphics and Applications
IEEE Computer Graphics and Applications
Computer Science Medicine