Programming models for the development of interactive audio applications

There are several systems that use external hardware, stand-alone applications, or special purpose languages to facilitate experimentation with audio processes ([22, 35, 44] and [29, section 3.2]). These packages take advantage of the extra efficiency gained by hardware support and by focusing a complete application on a particular problem. However, as computation becomes cheaper, increasingly more complex synthesis and analysis algorithms can be performed in real time on high end workstations without the help of specialized hardware. Environments that are geared specifically towards manipulating signals might solve that particular problem extremely well, but cannot be used at the level of notes or user interaction; these problems must then be solved independently. Adding support for audio to an already existing programming language would eliminate the need to create and support specialized, ad-hoc syntaxes. A general purpose language would ease the transition from the signal level to the note level, because all of the code would exist in the same framework. An extensible language would allow applications to combine low level audio routines with packages from just about any domain, including networking, artificial intelligence, and scientific computation. Finally, an interpreted language would give the user immediate feedback, and would allow applications to be flexible and configurable. We extended one portable and available language that meets these requirements, Tcl/Tk [30], with the framework for a traditional patch-based audio system. The resulting package, "swish," lets the user create applications that implement, control, and interact with audio processes at a high level. Since Tk provides access to a widget set, the interaction can be either textual or graphical. In order to incorporate a unit generator patcher into Tcl/Tk, it was necessary to reconcile the programming model presented by the audio system with that presented by the language. Using ideas found in [3], we replaced the traditional graphical dataflow machine with a more general "task" model. In the task model, the temporal relationships are specified independently of the functional relationships. We show that this model is still amenable to both graphical and textual representations, fits naturally into the Tcl/Tk framework, and is more general and flexible than several other existing systems. Finally, after examining the way that a typical patch is executed, we discuss the need for a language that is capable of performing incremental compilations. A typing system, lexical scope, and a functional programming style are other language features that might be useful, but are not found in Tcl/Tk. We use these observations to examine several languages that may become available in the near future. In the conclusion, we hypothesize about the role that audio will play in the near future in the developing networked multimedia environments, and argue that the applications that are going to have the most influence on the way that we use and view computers are going to be the ones that make portability a central concern. Thesis Supervisor: Tod Machover Title: Associate Professor