REPRESENTATION IN SPACE AND TIME CONTROL SCORE AND RESPONSIVE INTERFACE TO A VIRTUAL PERFORMANCE
Year: 2007

This lecture is about the application of the simulative methodology brought over by the digital culture to the design, portability and fruition of a performance. The simulative methodology is a computer based approach that organizes and delivers the aural and visual elements of a performance according to the temporal sequence defined by a control score, to provide a realistic experience through a responsive interface to a virtual reality environment (Brooks 1999) The result is a virtual performance. The simulative methodology provides a novel approach to the representation and control of a performance, bringing together the descriptive power of storyboard drawings (that here become more or less realistic 3D objects and scenes) the algorithmic and interaction capabilities of the computing machine, and the attractive mediation of a virtual reality environment (though adequate interface design is necessary to realize a satisfactory simulation) With more and more sophistication in computer graphics representations (with high mimetic quality) and the simulation power provided by the programming environments, the virtual performances become a powerful tool to design and pre visualize a complex event. Both static (representation) and dynamic (algorithmic) aspects must obey the lexicon and the rules of a formal language that become the symbols manipulated by the control system. Contemporary shows are complex events with contributions from several sources (each requiring specific competencies) and the existence of a common paradigm for representation and control is a viable possibility for providing a connection over operational attitudes that hardly communicate (consider, e.g. video and programming practices) and that conversely require neat and unambiguous responses. The virtual performance is useful to designers to provide a first idea of what the final performance will be to commissioners and collaborators (like storyboard and animatic for animated cartoons) to producers to understand the ideas conceived by an author and decide about the funding of a project (like pilot shows for TV series) to audience to experience the performance in a different modality (like TV representations of theatrical performances) to artists (different than the creator) to propose a new performance of the same show (like score in music) Each category of possible users requires an appropriate interface for the best experiencing results: we go from immersive stereoscopic view through helmets and binaural audio through headphones to standard computer monitors and stereophonic loudspeakers; also the interface control depends on the user, from position tracking to joysticks to mouse and keyboard. The acceptance of such simulative techniques will depend on several elements that include the expressivity/complexity of the descriptive language, the effectiveness of computer graphics and aural representational items, the viability of the user interface. In this lecture we present the Enthusiasm software, an open source project (on http:/enthusiasm.sourceforge.net/ that incorporates such issues in a comprehensive environment. Enthusiasm has been used to design and visualize live performances, multimedia shows, game environments. For example, Figure 1 reports six screenshots from an underdevelopment multimedia show, the concert performance "Orchestra Meccanica Marinetti" by the artist Motor (see the website http:/www.toshare.it/ACTION_SHARING/omm_eng.htm for more information) The performance features drummer robots playing steel bins on a stage guided by a conductor/performer (the inspiration source are the "Tambours du Bronx" with videos played over two screens above the stage and sounds processed in real time and delivered by loudspeakers all around the audience.