Apparatus and method for embedding content within a MIDI data stream

A method implemented on a data processing device is described comprising: generating supplemental data defining one or more characteristics of one or more light-emitting diodes (“LEDs”) on the data processing device; embedding the supplemental data within a musical instrument digital interface (“MIDI”) stream; decoding the supplemental data concurrently with decoding the MIDI stream; and modifying the one or more characteristics of one or more of the LEDs responsive to decoding the supplemental data.

BACKGROUND

1. Field of the Invention

This invention relates generally to the field of data processing. More particularly, the invention relates to an apparatus and method for encoding and decoding multimedia content and other data within a MIDI stream.

2. Description of the Related Art

Musical Instrument Digital Interface (“MIDI”) is a protocol used for interchanging musical information between musical instruments, synthesizers and computers. It defines the codes for a musical event, including, for example, the start of a note, the note's pitch, length, volume and various other musical attributes (e.g., instrument, vibrato level, . . . etc). It also defines codes for various button, dial and pedal adjustments used on most synthesizers. Since the advent of “General MIDI,” which defines a standard set of MIDI instruments, MIDI has become widely used for musical backgrounds in multimedia applications.

Instead of digitizing and recording the actual sound waves (e.g., as in a tape recorder), a computer with a MIDI interface stores the music as keystroke and control codes. As such, a MIDI recording typically consumes significantly less space than an actual digitized audio recording. Once a MIDI recording is stored on a hard drive or other mass storage device, the recording can then be edited in an entirely different manner than with conventional recording. For example, the rhythm can be changed by editing the timing codes in the MIDI messages and the key of the original recording can easily be transposed (e.g., from B major to D major). Moreover, unwanted notes or groups of notes can easily be removed and/or replaced.

SUMMARY

A method implemented on a data processing device is described comprising: generating supplemental data defining one or more characteristics of one or more light-emitting diodes (“LEDs”) on the data processing device; embedding the supplemental data within a musical instrument digital interface (“MIDI”) stream; decoding the supplemental data concurrently with decoding the MIDI stream; and modifying the one or more characteristics of one or more of the LEDs responsive to decoding the supplemental data.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Embodiments of the invention may be implemented on a data processing device such as that described in co-pending application entitled ADJUSTABLEDATAPROCESSINGDISPLAY, Ser. No. 09/714,320, Filed Nov. 15, 2000, which is assigned to the assignee of the present application and which is incorporated herein by reference. Certain embodiments of the data processing device will now be described followed by a detailed description of an apparatus and method for embedding content within a MIDI data stream. As an initial matter, however, it should be noted that the specific data processing device described below is not required for implementing the underlying principles of the invention. Rather, the invention may be implemented on virtually any type of data processing device including standard personal computers, personal digital assistants and wireless telephones.

FIGS. 1a–dillustrate a data processing device100with an adjustable display103according to one embodiment of the invention. In one embodiment, the data processing device100is comprised of a keyboard101, a control knob/wheel102(e.g., for scrolling between menu items and/or data), and a set of control buttons105(e.g., for selecting menu items and/or data).

In one embodiment, the display103is pivotally coupled to the data processing device100. More specifically, the display103pivots around a pivot point109, located within a pivot area104, from a “closed” position illustrated inFIG. 1ato an “open” position illustrated inFIGS. 1b–c. When in a closed position, the display103covers the keyboard101thereby decreasing the size of the device100and protecting the keyboard101. Even when the display is in a closed position, however, the control knob102and control buttons105are exposed and therefore accessible by the user. The motion of the display103from a closed position to an open position is indicated by motion arrow106illustrated inFIGS. 1b–c. As illustrated, when in an open position, the keyboard101is fully exposed. Accordingly, it will be appreciated that the display is viewable, and data is accessible by the user in both an open and a closed position (although access to the keyboard is only provided in an open position).

In one embodiment, a switch within the device100(not shown) is triggered when the display103is moved from one position to the next. Hardware/software within the device may be configured to read the position of the switch and invert images rendered on the display based on the switch position. Accordingly, images are rendered on the display103right-side-up, regardless of whether the display103is in an open or a closed position. In addition, in one embodiment, a different user interface (or other operating systems functions) may be triggered by the switch. For example, when the display is moved into a closed position, a user interface may be displayed which is more easily navigable with only the control buttons105and control knob102(i.e., without the use of the keyboard101). Various other interface functions may be triggered by the switch consistent with the underlying principles of the invention. Moreover, various different types of switches may be employed on the device100including standard mechanical switches, electrical switches (e.g., capacitive/magnetic switches), or any combination thereof.

If standard electrical wiring is used to electrically couple the data processing device100and the display103, the pivot area104should be wide enough to accommodate the wiring. However, various different types of electrical connections may be employed between the data processing device100and the display103while still complying with the underlying principles of the invention. For example, in one embodiment, the display103may be communicatively coupled to the processing device100via a wireless connection (e.g., using the Bluetooth standard, IEEE 802.11b, a capacitive coupling, . . . etc). If configured with a wireless connection, the display103may be detachable from the processing device100.

Moreover, various types of physical connections may be used to rotatably mount the display103to the processing device100. For example, in one embodiment, the device100is cooperatively mated to the display103with a set of circular guide rails or tracks (not shown).

The control knob102and control buttons105may be programmed to perform various functions within applications executed on the processing device100. For example, if an email client application is executed on the device100, the control knob102may be configured to scroll through the list of email messages within the user's inbox (e.g., with the current email message highlighted on the display103). One of the control buttons105may be configured to select a particular email message within the list. A second control button may be configured as a “back” button, allowing the user to back out of selected email messages and/or to move up through the menu/folder hierarchy. A third control button may be configured to bring the user to a desired location within the email application (e.g., to the top of the menu/folder hierarchy) or within the operating system executed on the processing device100.

In one embodiment, the functions to be executed by the buttons105and control knob102may be programmed by the end-user. In addition, various different control elements may be employed on the processing device100while still complying with the underlying principles of the invention.

In one embodiment, a cursor control element107is provided within the keyboard101. The cursor control element107acts like a typical set of control keys, providing for movement of a cursor in any direction specified by the user (i.e., up, down, left and right).

In one embodiment, the data processing device100is also provided with audio telephony (e.g., cellular) capabilities. To support audio telephony functions, the embodiment illustrated inFIGS. 1a–dincludes a speaker120for listening and a microphone121for speaking during a telephone conversation. Notably, the speaker120and microphone121are positioned at opposite ends of the data processing device100and are accessible when the screen103is in a closed position and an open position.

As illustrated inFIG. 1d, one embodiment of the data processing device100also includes a detachable camera115for capturing images. InFIG. 1d, the lens116of the camera115is facing out of the plane of the figure. The camera may be inserted into an input port110such as that shown inFIG. 1c, and may be rotatable around an axis defined by the input orientation of the input port. In one embodiment, the input port110is the same port as that used to communicatively couple a telephone headset (now shown) to the data processing device100.

In one embodiment, one or more light emitting diodes (“LEDs”) or similar light-producing elements are embedded within or beneath the control knob102. Accordingly, in this embodiment the control knob is comprised of a translucent material so that the LED colors are viewable by the end user. In one embodiment, a red, a blue and a green LED are provided. By manipulating the values of red, green and blue (e.g., via an LED device driver), virtually any color within the visible spectrum may be generated. The LED colors may be manipulated in a variety of different circumstances, several of which are described below.

In one embodiment, the data processing device illustrated inFIGS. 1a–dincludes a MIDI controller capable of processing MIDI data to render MIDI audio content. In addition, in one embodiment, the data processing device includes a supplemental MIDI data decoder capable of extracting and processing supplemental data (e.g., multimedia content) embedded within the MIDI data stream.

FIG. 2generally illustrates one embodiment of a MIDI encoder module202used to encode the supplemental data within the MIDI stream. The encoder module202embeds two types of supplemental data within the standard MIDI data stream201: LED data203defining LED colors to be synchronized with playback of the MIDI audio; and vibrate data204indicating points in time during the playback of the MIDI audio at which the data processing device should vibrate. It will be appreciated that various additional types of supplemental data may be encoded within the MIDI stream while still complying with the underlying principles of the invention.

In one embodiment, the MIDI encoder module202encodes the supplemental data within MIDI fields designated for “General Purpose Controller” data (e.g., Status Byte=176; Second Byte=16–19). Alternatively, or in addition, one of the various MIDI fields designated as “Undefined” may be used (e.g., Status Byte=176; Second Byte=20–31).

FIG. 4illustrates an exemplary MIDI data stream205generated by the MIDI encoder module202which contains LED and/or vibrate data203and204, respectively, embedded at specific points (e.g., t1–t2) so as to be synchronized with the audio generated by the standard MIDI data. For example, LED data defining a bright red LED color may be embedded at particularly loud and/or fast points within a musical composition. Conversely, a dark blue or purple color may be selected for softer and/or slower points within the composition. Of course, the underlying principles of the invention are not limited to any particular color and/or synchronization scheme.

In the example shown inFIG. 4, the LED data is comprised of a red component401, a green component402and a blue component403. By manipulating the levels of each component, the MIDI encoder module202may generate LED data defining any color and any brightness level within the visible spectrum. Various alternate color encoding schemes may also be used including, by way of example but not limitation, Hue Saturation Value (“HSV”), Hue Saturation Brightness (“HSB”), and luminance/chrominance components (“YUV”).

A the vibration data204defines points in time at which the device should vibrate, a vibration level and/or a vibration period. In one embodiment, the encoder module202embeds the vibration data204at points at which the bass of the musical composition rises above a specified threshold value, thereby simulating heavy bass on a relatively small device. The specified threshold value may be based on both volume and pitch. For example, the encoder module202may specify that only notes below a low C (“C3”) should vibrate, and only if the volume level is above a specified threshold.

FIG. 3illustrates a MIDI controller300according to one embodiment of the invention. The MIDI controller300is comprised of both a standard MIDI decoder340for decoding the standard MIDI audio data and a supplemental data decoder301for decoding the LED data203and vibration data204(or other data embedded within the MIDI stream). In one embodiment, the supplemental data decoder301is comprised of a set of registers for storing each of the extracted red, green and blue values and/or the vibration values. Alternatively, the supplemental decoder module301may store each of these values in single, contiguous memory space (e.g., with regions of the memory space dedicated to storing each value).

In the particular embodiment illustrated inFIG. 3, the supplemental data decoder301controls a red strobe module310, a green strobe module311, and a blue strobe module312based on the red, green and blue values401,402, and403, respectively, extracted from the MIDI stream. More specifically, based on the control signals provided by the supplemental data decoder301, the strobe modules control the rate at which the red, green and blue LEDs320are strobed, and thereby change the color and brightness generated by the LEDs320(modifying LED brightness by strobing is well known in the art).

In one embodiment the “control signals” provided to the strobe modules310–312by the supplemental data decoder301may simply be the red, green and blue values extracted from the MIDI data stream and temporarily stored in memory. The strobe modules310–312will then translate these values to strobe rate values and independently adjust each of their strobe levels accordingly. Alternatively, the supplemental data decoder301may itself translate the underlying red, green and blue values into a strobe rate value, which it will then provide to the red green and blue strobe modules,310,311, and312, respectively. The underlying principles of the invention remain the same regardless of which portion of the system converts the red, green and blue values to a strobe rate value.

As mentioned above, in addition to LED values, the supplemental data decoder301also extracts vibration values embedded within the MIDI stream. The supplemental data decoder301controls a vibrator module330configured on the data processing device based on the extracted vibration values. As mentioned above, the vibration values may indicate the level of vibration and/or the period of vibration.

In one embodiment, the MIDI controller300processes MIDI streams upon receipt of an incoming call to the data processing device (i.e., and thereby generates an audible, visible and/or physical indication of the incoming call). In the embodiment illustrated inFIG. 3, an indication of an incoming call is provided to the MIDI controller300by a caller identification module360. In one embodiment, the caller identification module360initially attempts to identify the number of the incoming call370using various caller identification techniques such as, for example, automatic number identification (“ANI”). If the caller identification module360identifies the number, it then performs a lookup in a caller database380stored on the data processing device (e.g., in Flash memory). The user may associate certain MIDI data streams with certain callers within the caller database380. Accordingly, if the caller identification module360locates a particular caller within the caller database380, it extracts the identity of the MIDI data stream205associated with the caller (e.g., the MIDI file name) and provides the identity of the MIDI data stream to the MIDI controller300. The MIDI controller then renders the identified MIDI data stream, including the embedded LED and vibration data203and204, respectively, as described above. If the caller identification module380is unable to locate the caller within the caller database380, it may identify a default MIDI data stream to the MIDI controller300. Alternatively, it may not provide any indication to the MIDI controller (i.e., the data processing device will “ring” in a standard manner, without using MIDI).

In one embodiment, the MIDI controller300processes the MIDI streams as described above upon receipt of any type of incoming electronic message including, by way of example but not limitation, incoming e-mail messages and instant messages.

In one embodiment, the user may create his/her own MIDI data streams, store the data streams on the data processing device, and (as mentioned above) associate the data streams with potential callers. One example of a particular MIDI data stream used to indicate an incoming call will now be described with respect toFIG. 5. The first graph500illustrated inFIG. 5indicates how the volume of the MIDI audio (or standard telephone ringer) will change over time in response to an incoming call. In one embodiment, the volume is manipulated using the standard MIDI protocol. The second graph501indicates how the LED brightness and vibration level or duration will change over time in response to an incoming call.

As illustrated, when an incoming call is initially received, the MIDI audio or standard ringer volume is set to zero. The volume remains at zero for some predetermined period of time t1. During the same period of time, however, the vibrate level/duration and/or the LED brightness is at the highest level. Following the initial time period, the MIDI audio or ringer volume will consistently increase up to its maximum at t2. During the same period of time, the LED brightness and/or vibration level/duration will continually decrease until it reaches zero at t2.

Thus, the user will initially be notified of a call using inaudible notification techniques (a useful feature, for example, if the user is in a meeting). However, if the user does not answer the call for a specified period of time, the data processing device/wireless telephone will begin generating an audible notification—at a low volume at first, gradually increasing to its maximum value.

It should be noted that the specific audio, LED, and vibration parameters illustrated inFIG. 5are for the purpose of illustration and should not be read to limit the scope of the present invention. A virtually unlimited number of audio, LED and vibration combinations may be configured on the data processing device while still complying with the underlying principles of the invention.

In one embodiment, the user may select from several predetermined incoming call settings such as those illustrated inFIG. 5. In addition, in one embodiment, the user may specify her/her own incoming call notification settings. For example, the user may create a MIDI stream/file with audio characteristics such as those illustrated in graph500. In addition, the user may embed supplemental LED/vibrate data within the MIDI stream/file having the characteristics shown in graph501.

Embodiments of the invention may include various steps as set forth above. The steps may be embodied in machine-executable instructions which cause a general-purpose or special-purpose processor to perform certain steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

Throughout the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. For example, although LED brightness is controlled via strobe units in the embodiments described above, other brightness control mechanisms may be employed while still complying with the underlying principles of the invention. Moreover, other visual effects may be controlled by embedding supplemental data within the MIDI data stream. For example, the characteristics of the data processing device's LCD screen103may be manipulated in addition to the LED embedded within the control knob102. For example, the backlight for LCD screen may be turned on or off and the contrast of the LCD screen may be modified. Accordingly, the scope and spirit of the invention should be judged in terms of the claims which follow.