Abstract:
Methods and apparatus are provided that help users learn to play one or more instruments by employing simplified note recognition techniques, improved feedback, and a simplified keyboard tablature. In one aspect, users are encouraged to play specified notes within a specified time frame. The pitch and duration can be represented in any suitable manner, but are advantageously displayed together as a single icon ( 232 A- 232 F). Preferably, the icon has an elongated shape in which the horizontal length correlates with duration of the note, and its vertical position on a display correlates with a pitch of the note. Notes can thus be represented by elongated lines, bars, ellipses, or even missiles or arrows. In another aspect, an improved musical tablature effectively clusters juxtaposed black keys ( 412 A,  412 C,  412 E) and white keys ( 412 B,  412 D) on a display for easier visualization.

Description:
FIELD OF THE INVENTION  
         [0001]    The field of the invention is music teaching.  
         BACKGROUND OF THE INVENTION  
         [0002]    Learning to play an instrument has proved problematic for generations of aspiring musicians, and the use of traditional musical notation has often made such learning even more difficult. Even in advanced countries, only 2-3% of musicians are classified as musically “literate”.  
           [0003]    A threshold problem is learning to produce desired notes on an instrument. Some individuals seem to pick this up naturally, but others have a great deal of difficulty remembering which keys, strings, or other elements produce which notes. Electronics have been employed for many years to solve this problem.  
           [0004]    One class of solutions provides substantially immediate feedback to the user (musician) as to which note is being played. For example, U.S. Pat. No. 3,577,824 to Lavan (May 1971), teaches a device that produces small lighted areas on a display in response to depressions on the keys of a piano. The lights are associated with traditional stave notation, so that the user learns which keys correspond to which notes. More recently microprocessors have greatly extended this solution to the world of desktop, laptop, and even hand-held devices. An example is U.S. Pat. No. 4,827,826 to Hiroyuki et al. (May 1989), which provides a visual display that indicates which notes are being played. A particularly sophisticated embodiment was marketed as a computer game called Miracle Piano™ by Software Toolworks™.  
           [0005]    U.S. Pat. No. 3,886,838 to Scherrer (June 1975) teaches a similar approach, in which lights are randomly displayed on a computer screen, with the user trying to identify the notes and play them on the instrument. Here again, substantially immediate (real-time) feedback is provided to the user by the computer.  
           [0006]    Problems remain, however, in that these note-to-screen or screen-to-note systems fail to address quality of play. An early mechanical system described in U.S. Pat. No. 3,760,680 to Moore (September 1973) purported to teach playing style, but the Moore system as well as other known systems fail to adequately teach musical expression, changes in tempo, loudness, and so forth. For example, the known systems do nothing at all to teach legato playing, where successive tones are produced in a closely connected, smoothly gliding manner. The importance of legato playing is expressed in the following quote from How to Teach Piano Successfully, by James Bastien:  
           [0007]    “The execution of the legato principle requires the student to play a key, hold it, and release it when the next key is played . . . . The process requires intricate finger coordination which may take some time to develop. However, it is vitally important for the beginner to understand this process and to be able to do it easily.” 
           [0008]    See also “pianoFORTE: A System for Piano Education Beyond Notation Literacy,” by Stephen W. Smoliar, Institute of Systems Science, National University of Singapore; John A. Waterworth, Department of Informatics, Umea University, Sweden; and Peter R. Kellock, Institute of Systems Science, National University of Singapore, published in ACM Multimedia 95—Electronic Proceedings, Nov. 5-9, 1995, San Francisco, Calif.  
           [0009]    Computer based systems have been developed that judge a performance or practice session, and provide feedback on the quality of play. Examples are U.S. Pat. No. 4,098,165 to Akiyama (July 1978) and U.S. Pat. No. 6,073,113 to Tohgi et al. (June 3000). Unfortunately, those systems are extremely complicated, and require the user to understand traditional musical notation.  
           [0010]    Traditional musical notation is not straightforward, and the existing systems have not satisfactorily met the challenge of either obviating or conveniently teaching such notation. This failure is in part due to the high level of complexity involved. Traditional musical notation combines two distinct symbol codings, one for pitch, and the other for duration. Correct interpretation of the pitch notation requires constant reference to the key signature, while correct interpretation of duration symbols requires constant reference to the time signature. Other notations code for tempo, playing style, and expression. All of this can be extremely difficult for beginners to master, and indeed even for non-beginners as well.  
           [0011]    These difficulties have been addressed over the years through non-traditional musical notations known as tablatures. For example, in the field of stringed instruments, the finger positions are often shown as black dots on a schematic of the frets and/or neck of an instrument. Tablatures are also known for keyboard type instruments. A continuing problem, however, is that the tablatures themselves are too complicated. In piano, for example, tablatures have been invented that provide a line for each black or white key. But there are so many keys that such an approach employs too many lines to be particularly useful. Another problem is that piano keys are usually tuned such that the change in frequency from one key to the next is not at all consistent. Still additional complexity results from the apparently inconsistent pattern in which the black keys are interspersed between the white keys. Yet another problem results from applying traditional nomenclature to a piano type keyboard, because the very same key on the keyboard can be considered a sharp in one key signature, and not sharp in another key signature. What is needed is a tablature that accommodates all these factors, while visually simplifying the keyboard.  
           [0012]    None of the known systems have been particularly effective in resolving these problems, and none have been commercially successful. Thus, there is still a need for systems and methods that help users learn to play musical instruments, and to either learn traditional musical notation, or at least learn a tablature that supplements or replaces traditional notation.  
         SUMMARY OF THE INVENTION  
         [0013]    Methods and apparatus are provided that help users learn the playing of one or more instruments by employing simplified note recognition techniques, improved feedback, and a simplified keyboard tablature. Other methods and apparatus are provided that help users learn to sing.  
           [0014]    In one aspect users are encouraged to play specified notes within a specified time frame. A preferred system comprises visual representation of a pitch and a time frame, and a visual feedback that identifies whether the pitch is played within the time frame. The pitch and the duration can be represented in any suitable manner, but are advantageously displayed together as a single icon. Preferably the icon has an elongated shape in which the horizontal length is correlated with duration of the note, and its vertical position on a display is correlated with a pitch of the note. Notes can thus be represented by elongated lines, bars, ellipses, or even missiles or arrows.  
           [0015]    The time frame can be represented in any suitable manner, including a “window” having leading and trailing edges. To encourage legato playing, and alter the tempo of play, one or both of the icon and the window can appear to move relative to one another on the display. For example, enlarging the window in the proper direction would increase the flexibility of “correct” play, while narrowing the window would require less flexible playing.  
           [0016]    The visual feedback can comprise any sort of signal, from a modification in color, intensity, or other aspect of an icon, to a signal that is considerably distanced from the icon. Exemplary feedback strategies include modifications to the color or extent of the icons, time frame, or other elements.  
           [0017]    The visual representations of the pitch, the duration, and time frame are preferably electronically generated, and displayed concurrently on a single display, such as a computer screen. Numerous game aspects are contemplated, including, for example, where the icons comprise projectiles or other recognizable shapes. Also contemplated are methods of using this technology to teach musical notation, and playing of musical instruments, especially on a keyboard type instrument, and most especially on a piano style keyboard. To that end statistics can be maintained with respect to a user&#39;s playing, and displayed either during or after play. A computer rendition of playing can also be stored electronically during play, and then run through the system at a later date so that the user can observe his playing with respect to one or more parameters.  
           [0018]    An improved musical tablature is preferably employed for piano style keyboards, in which representations ofjuxtaposed black and white keys are clustered on a display for easier visualization. Here, a horizontal gray zone having relative thickness of 5 may represent a clustering of 3 black and 2 white interposed keys, ajuxtaposed white zone having relative thickness of 2 may represent a clustering of two white keys, and a juxtaposed gray zone having relative thickness of 3 may represent a clustering of 2 black and 1 white interposed keys. Individual keys can be displayed in their relative positions within the zones as they appear on the keyboard. This simplification enables the tablature to encompass the same 43 key range as the 10 lines of the grand staff with seven gray zones.  
           [0019]    Various objects, features, aspects, and advantages of the present invention will become more apparent from the following detailed description of preferred embodiments of the invention, along with the accompanying drawings in which like numerals represent like components. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0020]    [0020]FIG. 1 is a perspective view of a prior art music box.  
         [0021]    [0021]FIG. 2 is a schematic of a system according to the inventive subject matter that includes a computer running a computer game, and a screen display providing visual feedback to a user.  
         [0022]    [0022]FIG. 3 is a schematic of a screen display depicting a tablature as part of a computer game.  
         [0023]    [0023]FIG. 4 is a schematic of a correspondence between a keyboard and a musical tablature.  
         [0024]    [0024]FIG. 5 is a schematic of a correspondence between a traditional musical notation and a corresponding tablature. 
     
    
     DETAILED DESCRIPTION  
       [0025]    [0025]FIG. 1 is a prior art system of a music box  100 . The music box  100  comprises a disk  110  having a plurality of spaced tines  112 , and a row of metal pins  120  having various lengths correlated with various notes. As the disk  110  rotates about a spindle  114  in the direction of arrow  116 , individual pins  130  contact individual tines  112  in a particular sequence and timing, causing the pins  130  to be “plucked”. Vibration of the plucked pins reproduces a musical score. A player piano works in substantially the same way.  
         [0026]    Music box technology is very old, and there are doubtless numerous variations known in the field. Significantly, however, all music boxes are based on the limitation that the order and spacing of the notes is entirely fixed. Other than variations in how fast the disk or cylinder rotates, which consequently affects the tempo of the music produced, there is no flexibility at all that would give rise to “expression” of the player. In other words, individual notes are not played within an allowable time frame (playing window), but are always played according to a predictable timing. If one were to interpret music box playing in terms of time frame, the time frame would be zero. This is not what is contemplated herein, where time frames are always considered to be non-zero.  
         [0027]    [0027]FIG. 2 generally depicts a system  200  having a processor  210  operatively coupled (in a manner not shown) to a memory  212 , a display screen  220 , and keyboards  230  and  240 . The display generally depicts the playing of a game in which a plurality of missiles  232 A- 232 F move leftward towards a window  234  as shown by arrow  236 , and a target  238 .  
         [0028]    System  200  is depicted here as a modem laptop computer system. Such depiction is intended to be illustrative rather than limiting. It is contemplated, for example, that in the future the processor  210  and memory  212  (depicted here as partially exploded from the remainder of the computer) could be much smaller, or located outside of the computer screen housing. The computer and memory portion of the system could, for example, be located at a distal hosting computer many miles away. The computer and memory portion of the system could also be included as part of a single board computer that fits in the keyboard, or on the user&#39;s person. Similarly, the “screen” could be a stand alone CRT, liquid crystal or other display, a projection on a wall or onto a pair of glasses, or even an electrical connection directly into a user&#39;s brain distal. By way of further illustration, keyboard  230  is a standard QWERTY keyboard, and keyboard  240  is a standard, or non-standard sized musical type keyboard. Either or both of those keyboards may be split into two or more physically separate pieces, and either or both are optional. In that regard virtual keyboards are contemplated where electronics are used to detect movements of the user&#39;s fingers in the air, or on an ordinary desktop, or anywhere else.  
         [0029]    It is currently preferred that the processor  210  is housed in a laptop computer electronically connected to a stand-alone CRT or liquid crystal display screen. With current programming technology, suitable laptop computers advantageously include a processor clock speed of at least 500 MHz, a memory of at least 2 GB, and at least one I/O card (not shown) capable of a MIDI connection to a musical instrument or keyboard. Keyboard  240  is preferably laid out in the manner of a normal electronic piano keyboard, with both black and white keys.  
         [0030]    In accordance with traditional terminology, the term “key” is used herein with reference to both physical keys and musical keys. Physical keys can be any collection of physical or virtual spacing of operational elements used to produce sounds of varying pitches. The usual reference is to the keys of a keyboard instrument such as mechanical or electronic piano, although keys on a French horn, oboe, or other instrument are also contemplated. Musical keys are collections of pitches in traditional staff notation to which a musical piece may be limited. Exemplary keys are G, F#, and C.  
         [0031]    The terms “black” and “white” with reference to keys refers to the black and white keys of a piano or piano style keyboard. The white keys (e.g., numbers  242  in FIG. 2) are generally used to produce one type of note, such as non-sharp and non-flat notes, while the black keys (e.g., numbers  244  in FIG. 2) refer to another type of note, such as sharps or flats, at least on a particular key. Of course, it should be understood that the “black” and “white” keys referred to herein need not actually be black and white in color. They could instead be green and gold, or black and ivory, white and white, or any other color patterns. The “black” and “white” keys could additionally or alternatively be raised or not-raised, rough or smooth, or even laid out on separate levels as in an organ keyboard.  
         [0032]    In the particular example of FIG. 2, the display screen  220  depicts the playing of a game in which a plurality of missiles  232  move leftward towards a target  238  in the direction of arrow  236 . The missiles  232 A- 232 F are disposed spatially across the screen such that their vertical position relates to different pitches, with the higher positions relating to higher pitches, and the lower positions relating to lower pitches. In this example missile  232 A corresponds to note middle C, while missile  132 B corresponds to note B above middle C. The missiles  232 A- 232 F are also configured horizontally in an elongated fashion corresponding to lengths (i.e., durations) of notes. Here, missile  232 A corresponds to a quarter note, and is half as long as missile  132 B, which corresponds to a half note.  
         [0033]    A window  234  is interposed in the path of the missiles and the game aspect involves the player (also referred to herein interchangeably as the user) pressing a key on the musical keyboard  240  while individual ones or multiples of the missiles  232 A- 232 F traverse the window  234 . To destroy each missile and prevent it from reaching the target  238  (which may, for example, be depicted as the Earth), the user must press the correct key on the keyboard  240  that corresponds with the pitch depicted by the missile. The player quickly realizes that pressing of the keyboard keys in the sequence and timing required to destroy them results in playing of a musical score.  
         [0034]    To encourage legato playing, preferred embodiments require that the correct key on the keyboard  240  be played for a sufficient time period corresponding to the length of each missile, as the missile crosses the window  234 . In a preferred game, pressing the correct key in that fashion causes the corresponding missile to “explode” and vanish from the screen. The correct duration is typically at least 0.2 seconds, but may be at least 0.5 seconds, at least 1 second, at least 2 seconds, at least 5 seconds or even longer. Pressing a wrong key, or pressing a correct key when the corresponding missile is not crossing the window  234 , will not destroy the missile. When some number of missiles passes through the window  234  to the target  238 , the game is terminated.  
         [0035]    Many variations should be apparent to those skilled in the art. As a simple example, the speed with which the missiles move relates to tempo of the score being played, and the speed can be altered by the computer, the user, or both. As another example, the missiles  32 A- 232 F need not move towards the window  234 . Instead, the window  234  may move towards the missiles  32 A- 232 F. As yet another example, the missiles  232 A- 232 F could be made to change color, or intensity, or brightness, or some other characteristic rather than exploding.  
         [0036]    It should be appreciated that the missiles  232 A- 232 F are merely icons for notes, preferably notes in a musical score. Other icons could be used as well, including other military themed icons, or non-military theined icons. By way of example, contemplated icons include elongated rectangular bars or narrow ellipses, or well known objects such as hats or automobiles. Moreover, in other embodiments the elongation could be replaced by some other signification, including, for example, a color or a brightness. In still other (much less preferred) embodiments, it is contemplated to separate out the icons into their pitch and duration parameters, so that two representations are employed for each note.  
         [0037]    [0037]FIG. 3 generally depicts a display  300  having a window  310  with a left side  311  and a right side  312 , a tablature  320  comprising icons  322 A- 322 D and shaded zones  340 , a feedback representation  324 , and statistics  330 .  
         [0038]    The window  310  is a visual depiction of a time frame. Here, the window  310  is a shaded zone on the display that defines a time frame within which the user attempts to play a pitch for a corresponding duration. The term “shaded” is used here in its broadest possible context. “Shading” may be represented by any or all of a darkening, a color change, an apparent texture change, and so forth.  
         [0039]    While it is preferred to have only one window, multiple windows are contemplated. For example, one could have one window for the playing of one hand, and a second window for the playing of another hand. It is also possible to have multiple windows for multiple players in a band, orchestra, or other cluster. The multiple windows would then preferably be spread out across multiple screens.  
         [0040]    The time frame is defined by the amount of time it takes for an icon (or at least some specified portion of the icon) to traverse the window. Certainly, the icon could move toward a stationary window, the window could more toward a stationary icon, or both window and icon could move toward each other. In a preferred class of embodiments, the icon moves and the window is stationary. In this class, a substantially correct response involves the user playing the note while the leading edge of the icon is visually located between the right side  312  of the window  310  and the left side  311  of the window  310 . Additionally, in order to achieve a substantially correct response, the user may be required to hold the note for a duration corresponding to the length of the icon.  
         [0041]    The time frame within which the user is to play the note is affected by the rate at which the window or icon moves, and the width of the window. Thus, the time frame decreases as the rate of movement goes up and the size of the window goes down. The user or the computer may advantageously control some or all of these factors. For example, a beginner may not be able to play the note within the time frame if the rate of movement is relatively fast. For learning purposes, the computer may start out with a relatively wide window, and relatively slow icon movement. As the player progresses, the window may be narrowed somewhat, and the movement of the icon may be increased.  
         [0042]    The tablature  320  comprises icons  322 A- 322 D, set against a background of juxtaposed grayed and white zones. The position of the icon within a zone preferably corresponds to the position of the corresponding physical key on a keyboard. As described above, the icons are preferably displayed using an elongated shape in which the length correlates with the duration. A more detailed description of the tablature  320  is discussed with FIG. 4.  
         [0043]    Embodiments of the inventive subject matter may optionally include some sort of visually recognizable feedback  324  as to whether the player played the pitch for the duration within the window. Feedback  324  may be represented as a visually recognizable alteration in the icon such as a change in color, form, or some other visually recognizable change. A change in color may include changes in intensity, shading, brightness, hue, saturation, reflective properties, and so on. Changes in form may include a representation of the icon “blowing up” being destroyed, or being visually eliminated. Feedback  324  may be represented by a change in an extent of the icon including size.  
         [0044]    Feedback  324  in the form of a visually recognizable alteration may be correlated with a user failing to properly play the pitch for the duration within the time frame or the user playing the pitch for longer than the duration. For example, the alteration may be visually displayed as a green light, yellow light, or red light depending on relative correctness. It is also contemplated that the feedback  324  may be represented by something other than an alteration in the icon such as a visual indication next to the icon, a sound, a graph, or merely a change in score.  
         [0045]    Feedback  324  is preferably displayed in a prompt manner so that the player can quickly alter his/her playing. Preferred feedback occurs in less than 2 seconds, more preferably in less than 1 second, and still more preferably displayed in less than 0.5 second. In current computer argot, such feedback may be said to occur in “real-time”. That term, however, is difficult to define precisely because display of the feedback  324  will likely vary substantially depending on variables such as CPU speed, contention for resources, and communication line speed.  
         [0046]    Statistics  330  may comprise any useful summarization of play, including such values as elapsed time, level of play, temp, number of notes played correctly and incorrectly, and so forth. These statistics may be presented as numbers, graphs, icons, and so forth. For example, quality of play may be presented as a position on a bar, or a color from green to yellow to red. Some or all of the statistics may be stored and recalled, and perhaps averaged over multiple sessions. Such information may be transmitted to a remote sight, and thereby, users may enter contests, communicate with teachers remotely, and compete with one another.  
         [0047]    [0047]FIG. 4 generally depicts a schematic of a correspondence between a keyboard  410  and a tablature  420 . The keyboard  410  comprises clusters of keys  412 - 416 , respectively, which correspond to shaded zones  422 ,  424 ,  426  and unshaded zones  423  and  425  of the tablature  420 . Those skilled in the art will immediately recognize that each of the shaded zones  422 ,  424 ,  426  represents at least 2 black keys and at least one intervening key. Thus, shaded zone  412  represents three black keys  412 A,  412 C,  412 E and two intervening white keys  412 B and  412 D. The non-shaded or white zones  423 ,  425  occur whenever there are two juxtaposed white keys on the keyboard  410 .  
         [0048]    Icons on the tablature are represented as described above, positioned in the proper shaded or unshaded zone, and vertically within the zone as a function of the respective position of the corresponding key within the cluster. Thus, icons  425 A,  425 B correspond to physical keys  415 A,  415 B, respectively. Additional embodiments include a tablature that comprises at least some Braille, and in this respect, it is further contemplated that a tablature may be specially printed in Braille.  
         [0049]    The specific clustering here is thought to be particularly advantageous, but not absolutely necessary. It would be possible, for example, to cluster together 2 black keys, an intervening white key, and the two outside juxtaposed white keys. Furthermore, the terms “shaded” and “non-shaded” or “white” are to be construed euphemistically, to refer to any visually distinctive renderings. Thus, “shaded” keys may actually be dark green while “non-shaded” or “white” keys are light green. Similarly, “shaded” keys may actually be striped while “non-shaded” or “white” keys are stippled. In any case, by clustering keys and associating the clusters with shaded zones on the tablature, an improved tablature is provided.  
         [0050]    [0050]FIG. 5 is a depiction of a correspondence between a traditional staff type musical notation  510 , and a corresponding tablature  520  of a few bars of the song “Frere Jacques”. Those skilled in the art will appreciate that FIG. 5 is entirely analogous to FIG. 4. Here, quarter notes  512 A- 512 D in the traditional notation  510  correspond to quarter-note length bars  522 A- 522 D in the tablature  520 . The vertical lines such as  524  in the latter indicate that a given succeeding note should be played immediately after the preceding note.  
         [0051]    The notes, which would be played by the left hand are not represented in the tablature  520 . Such notes could, however, be represented by appropriate symbols.  
         [0052]    Thus, specific embodiments and applications of music teaching devices and methods have been disclosed. It should be apparent, however, to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except in the spirit of the appended claims. Moreover, in interpreting both the specification and the claims, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced.