Abstract:
An apparatus and method are provided for processing an audio signal conveying a musical passage so as to reveal the sequence of musical chords contained within that passage. The signal is amplified, filtered, and converted to digital data, which are then processed using digital filters to determine in real time the amplitude of every note within a predetermined note range. The most prominent notes are compared to chord patterns to determine which, if any, chord is implied, and the chord name is then displayed to the user. Further provided is a means for detecting and correcting for any deviation of the pitches of the notes in the passage from their standard frequencies.

Description:
DESCRIPTION 
     1. Field of the Invention 
     The present invention relates generally to the fields of microprocessor-based systems for analyzing analog signals, and in particular to microprocessor-based systems for analysis of music. 
     2. Background Art 
     It is desirable for a number of reasons for musicians to be able to analyze the progression of chords in a given musical passage. Traditionally, musicians have relied on their musical ear, developed through training and practice, to derive chords. However, ear training is inherently limited by the musician&#39;s innate ability. 
     The prior art provides devices and methods for analyzing acoustic signals. These include amplification and filtering of acoustic signals, both analog and digital, analog-to-digital conversion of acoustic signals, and microprocessor-based signal processing. 
     DISCLOSURE OF INVENTION 
     The present invention provides an electronic system for analyzing the chords present in a given musical passage. In a preferred embodiment, the system includes input means for receiving an analog signal, analog-to-digital conversion means for converting the analog signal into a digital signal, single-note filter means for determining the presence and relative volumes of individual notes within the digital signal, and computing means for determining what chords are characterized by the detected individual notes. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a block drawing of the components of a preferred embodiment of the present invention; 
     FIGS. 2A-E are schematic drawings of an external device according to a preferred embodiment of the present invention; 
     FIG. 3 is a schematic drawing of a high-pass filter used in a preferred embodiment of the present invention; 
     FIG. 4 is a schematic of a low-pass filter used in a preferred embodiment of the present invention; 
     FIG. 5 is a flow chart of a digital filter algorithm used in a preferred embodiment of the present invention; 
     FIG. 6 is a flow chart showing algebraically the processes performed in the flow chart shown in FIG. 5; 
     FIG. 7 is a flow chart of a filtering routine used in a preferred embodiment of the present invention; 
     FIG. 8 is a flow chart of a preferred embodiment of a user interface loop. 
     FIG. 9 is a diagram demonstrating the ratios used in a preferred embodiment of the present invention to determine the sharpness or flatness of an incoming acoustic signal 
     FIGS. 10A-E show the graphical output on a computer monitor screen in a preferred embodiment of the present invention. 
    
    
     DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 
     The present invention provides an electronic system for performing tonal analysis on passages of recorded music. Its primary function and purpose is to determine the sequence of chords that comprise a given musical passage. 
     In a preferred embodiment, the present invention includes (1) a software package that runs on a personal computer, such as an IBM-PC or compatible with a clock rate of at least 16 MHz and with a VGA card and monitor and (2) an external device that is connected to the personal computer through the computer&#39;s serial port. The external device receives as input a standard audio signal from a tape deck, compact disc player, or microphone. The output of the system is a graphic display of a timeline of chords on the PC&#39;s monitor. This chord timeline informs the user of the name, time, and duration of every chord that occurs in the musical sequence. 
     The present system provides especially accurate chord-sequence transcriptions on music that makes use of mid-range instruments and embodies a clear progression of chords. For example, songs with consistent guitar strums or strong vocal harmonies are excellent candidates for transcription. Music which is lacking harmonic content, or whose harmonic elements are exclusively high-range (above the pitch C5), do not lend themselves for ready transcription employing the present system. 
     The present system employs a chain of processes that converts a particular audio signal into a set of note volumes, which are then translated into chords and mapped onto a timeline. A note volume is the amplitude of a particular frequency component of the acoustic signal corresponding to a note on a 12-tone diatonic scale. A chord embodied in an acoustic signal will display a characteristic set of note volumes. In a preferred embodiment of the present invention, a range of four octaves, i.e., 48 notes, is used. However, it would of course be possible to expand the range Of note volumes, as desired. 
     FIG. 1 is a block diagram of the steps in the chain. An amplifier 10 takes as its input 12 a pair of audio signals. It provides as an output the sum of the two signals multiplied by a gain that is controllable by the user. 
     In the present preferred embodiment, four octave filters 14a-d divide the amplified signal into four separate signals, each of which carries the frequencies belonging to a particular octave range. However, it is possible to implement an alternative embodiment of the present invention in which the signal is not divided into separate components, but digitized as a whole. In such a system, however, it is desirable, using means known in the art, for a low-pass filter to be provided in order to filter out aliasing frequencies. Further, in alternative embodiments, it would be possible to increase the number of filters to increase the range of the system. 
     An analog-to-digital converter 16 converts the four analog signals into four digital signals, which are multiplexed into a single digital signal for serial transmission. An external serial port 18 transmits the digital signal in a standard serial format to the PC. The PC serial port 20 receives the serial input and reconstructs the digital signal. A single-note filter 22 de-multiplexes the digital signal back into its four component signals. It then digitally filters each of those signals twelve times, each time isolating the frequency-band of a particular note. The note volumes 24 are then determined from the results of this filtering. The set of note volumes is then compared against a library of characteristic sets of note volumes to provide the final interpretation into chords which are then mapped against a timeline. 
     A preferred embodiment of the present invention includes an external device 26 and a personal computer 28 and is embodied partially in hardware and partially in software. In particular, the single-note filtering, final chord interpretation, and final output generation are performed using software. However, as would be obvious to a practitioner of ordinary skill in the art, it would be possible to alter this configuration is a number of ways without departing from the essence of the invention. 
     FIGS. 2A-E provide a detailed schematic of a preferred embodiment of the external device 26 shown in FIG. 1. 
     The amplifier 10 of FIG. 1 is provided in FIG. 2C by three LM 324 op-amps U13A, U13B, and U13C. Its inputs are two connectors X1 and X2 for receiving the stereo audio signal. The amplifier performs three functions between its inputs and its output: (1) it receives and stabilizes the two incoming audio signals; (2) it combines the two signals into one; and (3) it amplifies the combined signal by a gain controllable by the user. 
     The two audio input signals are stabilized by the resistor/capacitor combinations R8/C15 and R9/C16 attached to each microphone lead X1 and X2. The stabilized signals are added together using a voltage adder circuit U13A, U13B, R10, and R11. The combined signal is then amplified with an inverting amplifier U13C, R12, R13, and S2. The inverting amplifier incorporates a switch S2 and a potentiometer R13, both of which allow the gain of the circuit to be controlled by the user. 
     The four octave filters constitute the remainder of the circuitry shown in FIG. 2C. As is apparent from FIG. 2C, the octave filters receive as their input the stabilized, added, and amplified audio input signal, and their outputs are the four signals labelled &#34;FI[1 . . . 4]&#34;. 
     The function of the octave filters is to separate the frequencies of the audio signal into four bands, each carrying frequencies that fall within a particular octave range. The first octave filter 14a isolates all frequencies between the pitches C1 and B1; the second octave filter 14b, the frequencies between C2 and B2; the third octave filter 14c, the frequencies between C3 and B3; and the fourth octave filter 14d, the frequencies between C4 and B4. These four octaves comprise the range over which the present preferred embodiment is capable of detecting notes. 
     This separation of frequencies is necessary and desirable for two reasons. The first reason is that lower frequencies generally tend to have much greater amplitudes than higher frequencies, and thus tend to dominate the signal. If the signal were to be digitized as a whole, the dominant lower frequencies would drown out the higher frequencies. By digitizing each octave range separately, the system allows both strong and weak frequencies to be digitized with equal resolution. 
     The second reason for the separation of frequencies is that the minimum sampling rate of higher frequencies is greater than that of lower frequencies. For example, in order for a signal carrying the pitch A4 to be accurately digitized, it must be sampled at a rate of at least 880 Hz. By contrast, a signal carrying the pitch A1 need only be sampled at a rate of 110 Hz. By isolating each of the octave ranges and sampling and processing it separately, the system can devote the most amount of processor time where it is most needed, i.e., on the higher octave ranges. 
     Each of the four octave filters is actually a pair of filters, a high-pass filter followed by a low-pass filter. The general circuit diagram for the high-pass filter used in the present preferred embodiment is shown in FIG. 3. The formulae used to calculate appropriate values for R1 and R2 are as follows: ##EQU1## 
     The circuit values for the high-pass component of the first octave filter 14a are determined in the following way. We choose a sharpness factor μ of 0.2 and a cutoff frequency f of 29.14 Hz. Note that this latter value is slightly below the C1 frequency of 32.70 Hz; this discrepancy makes an allowance for deviations in component values. Selecting a capacitor value C of 0.1 μF, we are able to determine the values of resistors R1 and R2. Solving the equations, we obtain R1=10.93K and R2=273.1K. Rounding these values to the nearest standard resistances, we assign R1=11.0K and R2=274K. These are the values used in the actual octave filter circuit 14a, where R14 and R15 correspond to resistors R1 and R2 in the general schematic. 
     For the remaining octave filters 14b-d, appropriate values for f are 58.27 Hz, 116.54 Hz, and 233.08 Hz, respectively, while μ and C are assigned the same values as for the first octave filter. The resistor values are then determined using the circuit equations as shown above. 
     The general circuit diagram for the low-pass filter used in the present preferred embodiment is shown in FIG. 4. The general formulae used to calculate appropriate values for R1, R2, and R3 are as follows: ##EQU2## 
     The circuit values for the low-pass component of the first octave filter 14a is determined in the following way. A sharpness factor μ of 0.2 has been found to be desirable. A cutoff frequency f of 69.30 Hz is used. Note that the value of f is slightly higher than the B1 value of 61.74 Hz, again to allow for deviations in component values. Selecting a C value of 0.1 μF and an R4 value of 1K, we are able to solve the equations for R2 and R3, and obtain R2=57.42K and R3=527.5K. These values approximate to R2=57.6K and R3=523K, which are the resistances used in the actual octave filter circuit R17 and R18. R1 controls the gain and is determined experimentally; a value of 300K works well and thus is used for R16 in the actual circuit. 
     In the remaining three octave filters 14b-d, the parameter f is assigned the frequencies 138.59 Hz, 277.18 Hz, and 554.37 Hz, respectively, while the parameters μ, C, and R4 have the same values as before. The resistor values are then determined using the circuit equations as shown above. 
     FIG. 2D shows a schematic drawing of the analog-to-digital converter 16 used in the present preferred embodiment. Its inputs are the four signals labelled &#34;FI[1 . . . 4]&#34; as well as control signals AD[0 . . . 2], START --  AD, ADC&gt;UART, and CLK from the microprogrammed control unit (MCU) depicted in FIG. 2B. The output of the FIG. 2D analog-to-digital converter is the eight-bit word D[0 . . . 7]. 
     The overall function of the analog-to-digital converter is to convert the four analog signals received as an input from the four octave filters into a single multiplexed digital signal. The converter achieves this end by sampling each of the signals at regular intervals and converting each sample to an eight-bit number. The sampling interval is different for each octave; as mentioned before, the sampling rate of higher frequencies must be greater than that of lower frequencies. Octave range 1, corresponding to the output from the first octave filter, is sampled at a rate of 300 Hz; octave range 2, at a rate of 600 Hz; octave range 3 at a rate of 1200 Hz; and octave range 4 at a rate of 2400 Hz. The analog-to-digital converter performs a conversion every 4800 Hz, selecting a different octave range each time. It selects the octave ranges in the following pattern, which repeats continuously: 
     
         4, 3, 4, 2, 4, 3, 4, 1, 4, 3, 4, 2, 4, 3, 4, X 
    
     The above pattern realizes the aforementioned sampling rates for each octave range. The &#34;X&#34; indicates that no octave range is selected on the 16th step; the analog-to-digital converter is idle during that time. On the 16th step, the byte 00000000 is sent to the serial port by the MCU. This zero-byte is a message to the PC that the octave range selection pattern is about to repeat again. By sending this end-of-pattern message, the external device is able to remain synchronized with the PC. 
     Each of the &#34;FI[1 . . . 4]&#34; signals is centered about the zero-volt axis as it enters the analog-to-digital converter. Because the converter expects its input to be between zero and five volts, the signals must be re-centered about the 2.5-volt axis and limited to the zero-to-five-volt range. The re-centering of each signal is accomplished with four inverting amplifier circuits (U18A-D). The voltage limitation is achieved with a pair of diodes (D3-D10) following each inverting amplifier. Note that the lower voltage limit is actually 0.1 volts rather than 0 volts; this provision exists so that the signals are never digitized to zero; zero is reserved for the end-of-pattern message mentioned above. The four signals are then digitized in the aforementioned pattern by an ADC0809 analog-to-digital converter integrated circuit (U19). 
     FIG. 2E is a schematic of the external serial port 18 used in a preferred embodiment of the present invention. Its inputs are the eight-bit word labelled &#34;D[0 . . . 7]&#34; as well as MCU control signals AD[0 . . . 2], *UART --  CS, ADC&gt;UART, *MCU&gt;UAR, RESET, CLK. The external serial port has as its output the signal TD. 
     The function of the external serial port is to transmit in standard RS-232 format the digital signal D[0 . . . 7] it receives from the analog-to-digital converter 16. The conversion to this format is a two-step process. First, the digital signal is serialized by an 8250 UART integrated circuit U20, the output of which is in the zero-to-five-volt range. Second, this serial signal is converted to the RS-232 range by an amplifier circuit U21. The signal is then ready to be sent to the PC through a nine-pin connector U22. 
     The PC serial port receives the RS-232 signal sent from the external device. Its output is the digital signal that it reconstructs from this input. The PC serial port is identical to the one on the external device; both provide an 8250 UART with an RS-232 converter, and both are configured to the same data format and baud rate. The PC serial port is initialized in software using means known in the art (using the &#34;init --  comport()&#34; procedure shown in the source code listing in the Appendix). Once initialized, the serial port is activated automatically whenever a data byte is sent; no additional code is required for its operation. 
     The single-note filter is implemented entirely in software as an interrupt service routine. This routine reads the digital signal that is generated by the PC serial port and de-multiplexes this signal back into its four component signals. It then digitally filters each of those signals twelve times, each time isolating the frequency-band of a particular note. The note volumes are then determined from the results of this filtering. 
     FIG. 5 is a diagram showing a preferred embodiment of a digital filter algorithm that used to isolate each note&#39;s frequency-band. In the present preferred embodiment, the digital filter algorithm is implemented in software. 
     In FIG. 5, each square box R0, R1, and R2 represents a register that remembers the value of its last input. Each circle Σ represents an accumulator whose output equals the sum of its inputs. Each triangle A, B, and C represents a multiplier whose output equals its input multiplied by some fixed gain. The flow-chart shown in FIG. 6 conveys the same information algebraically. 
     The characteristics of this filter, i.e., its center frequency F and its sharpness factor Q, are determined entirely by the gain factors A, B, and C, in accordance with the following relationships: ##EQU3## Depending on the values we choose for these gain factors, the filter can display any desired center frequency and sharpness. Hence, a single filter structure with changeable gain factors can be used to implement all twelve of the filters needed for each octave range. 
     The actual filtering routine (called &#34;serial --  handler()&#34; in the source code listing in the Appendix) is invoked whenever a data byte is received by the serial port. Its implementation is outlined in the flow-chart shown in FIG. 7. 
     This routine utilizes the filtering algorithm of FIGS. 5 and 6 described earlier, but executes it twelve times (using different sets of registers and gain factors each time) in order to filter all twelve notes of the octave to which the current input corresponds. The routine also stores the current maximum value of each filter output (using the &#34;peak[]&#34; array in the source code listing). Doing so allows the other software routines to determine what the current volume of any particular note is by reading the &#34;peak[]&#34; array. 
     FIG. 8 is a flow chart of a preferred embodiment of a user interface loop. The user interface loop is a continuously repeating set of software routines which provide an interface between the user and the program. (These routines are all listed in the source code listing in the Appendix, where the top-level interface routine is the &#34;main --  loop()&#34; procedure.) 
     The first step 31 in the interface loop gets and processes any keystroke that has been made by the user. If no keystroke has occurred, this step is bypassed; otherwise, the program reads the keystroke from the keyboard buffer and performs the appropriate function. Keyboard commands generally involve editing an on-screen item, such as altering a parameter value, or changing the system state in some way, such as beginning a transcription. (All of these functions are handled in the source code listing within the &#34;get --  command()&#34; procedure.) 
     The second step 32 in the interface loop checks a timer to see if a certain time interval has lapsed. If not, the program returns to the first step 31. Otherwise, the timer is reset and the program continues. The purpose of this step 32 is to ensure that the remaining steps in the loop are executed at regular intervals. Note that the timer is incremented every time the interrupt service routine (&#34;serial --  handler()&#34;) is invoked. 
     The third step 33 determines which octave range of notes should currently be displayed by the interface loop. This octave range is chosen using the same repeating selection pattern that is used by the note detection chain described above. It should be noted, however, that the octave range that is being processed by the note detection chain and the octave range that is being displayed by the user interface loop are not necessarily the same octave range. If the current position in the octave range selection pattern is any but the idle (16th) position, the program next executes steps 34a and 35a; otherwise, it executes steps 34b and 35b. 
     Step 34a updates the signal strength indicator for the current octave range. The signal strength indicator is a screen item which indicates the peak amplitude and degree of clipping of each of the four digital signals entering the PC. (In the source code listing in the Appendix, the interrupt service routine &#34;serial --  handler()&#34; keeps track of the peak amplitudes in the &#34;signal --  amplitude[]&#34; array. The same routine keeps track of the number of times each signal reaches its cutoff point in the &#34;signal --  cutoff[]&#34; array.) Step 34a of the interface loop makes use of these arrays to convey to the user the peak amplitude and degree of clipping of each signal. 
     Step 35a updates the note volume indicator for each note in the current octave range. The note volume indicator is a screen item which displays the pitch and volume of each note as it is played. The program computes each note&#39;s volume by subtracting its filter response (found in the &#34;peak[]&#34; array) from that of the larger of its two neighbors. The program also labels each note which attains a volume above a certain threshold. 
     Step 34b updates the intonation indicator. The intonation indicator is a screen item which indicates the extent to which the detected notes are sharp or flat. The program is able to calculate this information by comparing the filter response of the strongest note with those of its two neighbors. If the lower-pitched neighbor has a greater amplitude than the higher-pitched neighbor, then the notes are flat; the reverse situation implies the notes are sharp. The degree of flatness or sharpness is determined by the ratio of the center note&#39;s amplitude with that of the larger of its two neighbors; a large ratio implies good intonation, whereas near equality implies poor intonation. This is shown graphically in FIG. 9. 
     Step 35b updates the chord timeline. The chord timeline is a screen item which tells the user the name of the chord currently being played. The program determines this chord name by summing the note volumes for each family of notes (e.g., the volumes of all the C#&#39;s are added together). The program picks from these sums the three most predominant note families, and then looks in a reference table to determine which chord, if any, corresponds to the three note families selected. If there is such a chord, the program conveys its name to the user. (This process is implemented in the source code listing in the Appendix within the &#34;transcribe --  chord()&#34; procedure.) 
     FIGS. 10A-E show the graphical output of the present preferred embodiment. FIG. 10A shows a signal strength indicator, which in the present preferred embodiment appears as the left-most window at the bottom of the monitor screen. It indicates to the user if the signal coming from the tape deck, CD player, or other audio signal source is too strong or too weak. 
     The signal strength indicator displays four vertical bars, the heights of which are constantly fluctuating. The heights of these bars correspond to the strength of the incoming audio signal. If the tops of the bars turn magenta, it indicates that the signal is being clipped. The signal control knob on the external device should be adjusted so that the tallest bar is roughly half the window height. This allows the signal strength to be maximized but clipping to be minimized. 
     FIG. 10B shows an intonation indicator that, in a preferred embodiment of the present invention, appears as the second window from the left at the bottom of the monitor screen. The intonation indicator tells the user to what extent the notes of the musical passage are sharp or flat. When the bar inside the window is magenta, it indicates that the notes are sharp; when red, it indicates that the notes are flat. The size of the bar indicates to what extent the notes are sharp or flat. It is desirable, therefore, that this bar be as short as possible. 
     FIG. 10C shows a pitch adjuster that, in a preferred embodiment of the present invention, appears as the third window from the left at the bottom of the monitor screen. The pitch adjuster allows the user to change the intonation of the computer so that it matches that of the music. Thus, the intonation indicator and the pitch adjuster are used in conjunction with one another. 
     There is an arrow on the pitch adjuster which points to a number. That number is the adjusted frequency of the pitch A4, whose standard frequency is 440.0 Hz. In the present preferred embodiment, the value is adjusted by pressing CTRL-F on the keyboard. The arrow becomes highlighted, and may be moved up or down using the arrow keys. If the intonation indicator is mostly magenta, the arrow should be moved up several places. If the intonation indicator is mostly red, the arrow is moved down several places. If the intonation indicator is both red and magenta and very short, the arrow should not be moved at all. The &lt;ENTER&gt; key is then pressed. The arrow de-highlights, and the computer adjusts its intonation to the new setting just selected. The user looks at the intonation indicator again. If it is still mostly red or mostly magenta, or if the bar is very tall in both directions, the process is repeated until the proper intonation is achieved. 
     FIG. 10D shows a note volume indicator that, in a preferred embodiment, appears inside the rightmost window at the bottom of the monitor screen. The note volume indicator displays the pitch and volume of every note as it is played. When a note is played, a bar appears, the position and height of which correspond to the pitch and volume of the note. If the note&#39;s volume is sufficient strong, the bar becomes highlighted and the note&#39;s name appears underneath it. The user may adjust the volume at which the notes become highlighted and labelled by pressing CTRL-N. Doing so causes a horizontal line, called the noise threshold, to appear in yellow. The noise threshold may then be moved up or down with the arrow keys, raising or lowering the volume at which notes are recognized. The user must press the &lt;ENTER&gt; key when done adjusting the noise threshold. 
     FIG. 10E shows a chord timeline that, in a preferred embodiment, occupies the remainder of the screen. It informs the user what chords have been played over the course of the musical passage. When the user presses CTRL-B, a column of chord names appears running down the left edge of the screen, as well as a page number and the message &#34;Transcribing&#34; at the top of the screen. A number of horizontal lines should also appear stretching to the right as time passes. These lines indicate which, if any, musical chord is currently predominant. If a line is thick, it indicates a major triad; if it is thin, it indicates a minor triad. This information, in conjunction with the chord name to the left of the line, tells the user the complete name of the chord being played. 
     The user may add comments above the chord timeline in order to make it easier to remember where the chords belong within the music. If the music has lyrics, the user may wish to fill in the words as they are sung. If the music is mostly instrumental, the user may find it useful to make a comment at the downbeat of every measure. To make a comment, the user simply types the comment while the transcription is taking place, and the comment appears at the current place in the transcription. Comments are separated by pressing either the space bar or the &lt;ENTER&gt; key. The transcription is termination by pressing CTRL-E. The user may then page through the transcription using the PgUp and PgDn keys. 
     In the present preferred embodiment, the user may press CTRL-W to write the transcription to disk. When the program asks for the name of the file, the user enters a name of not more than eight characters. The transcription will be written to a file bearing that name in the C: SCRIPT FILES directory. After a number of transcriptions have been written, the user may read one back by pressing CTRL-R. The user then enters the name of the file, and the transcription is loaded into memory for the user to examine. 
     In the present preferred embodiment, the program is exited by pressing CTRL-X. ##SPC1##