Tracking Musical Beats in Real Time

Identifying the temporal location of downbeats is a fundamental musical skill. After briefly discussing the perceptual information available in beat tracking, we survey previous attempts to automate this process in both real time and non-real time. Our attempt to add flexibility to Mont-Reynaud’s model [4] by parameterizing the confidence and history mechanisms failed to yield a satisfactory beat tracker. Observing that this and previous models are constrained to hold a single current notion of beat timing and placement, we find that they will fail to predict beats and not recover beyond the point at which a mistake is first made. We propose a new model that uses beam search [1] to simultaneously consider multiple interpretations of the performance. At any time, predictions of beat timing and placement can be made according to the most credible of many interpretations under consideration. Credibility is determined by a heuristic evaluation function. Introduction Identifying the temporal location of downbeats is a fundamental musical skill that comes naturally to most people. Even people without musical training can tap their foot with the beat when they hear a musical performance and can generally perform this task with ease and precision even in the face of unusual rhythms, musical expressiveness, and imprecise performances. (An alternate name for this task, foot tapping, arises from this characteristic human activity.) Musical performances contain a great deal of information that presumably facilitates the ease with which humans do beat tracking. Examples of such information are temporal information like note onsets and offsets, pitches and their interval relationships, and the relationships of multiple voices in polyphony. A fully general, automatic beat tracker would be of great value in many applications such as human-computer improvisation, music transcription, music editing and synchronization, and (ethno-) musicological studies. It would open up new possibilities for computers in music by allowing the computer to synchronize with tempos external to it without the use of explicit synchronization information. Musicians generally would agree that it is much harder to play in an ensemble and follow a collective tempo than it is to set their own tempo and require other musicians to follow. Our goal is to transform the computer from a tempo setter to a tempo follower. A beat tracker would solve this problem and allow an outside agent (e.g. a human performer) instead of the computer to control the tempo. Implicitly, we have been discussing beat tracking in terms of realtime reactions. Humans tap their foot in realtime and we would want a computer to improvise with a human performer in realtime. Though we could imagine that a transcription task, for example, not be performed in realtime since there are generally no time constraints implicit in that task. However, we feel that performing beat tracking in realtime is important since a non-realtime beat tracker would find far fewer uses in the computer music world. Thus, except for review of some previous beat tracking work, we consider only realtime beat tracking for the remainder of this paper. Perceptual Information Available in Beat Tracking Beat tracking is differentiated from the score following of Dannenberg [3] and Vercoe [12] primarily by the absence of a score as part of the input to the process. The minimum input needed by a beat tracker is temporal 1 information consisting of the note onset times of a single instrument voice from which inter-onset intervals may be derived. Additional input may include pitch information, and an initial tempo so that bootstrapping is avoided. A polyphonic beat tracker would use similar information derived from multiple voices. The output of a beat tracker is a prediction of where future beats will be in a musical performance. To facilitate discussion, we extend the musical concept of beat to include two aspects, period and phase. In a musical performance, we will say that the period or 1Defined as the time from one note onset to the next note onset.