Browser-based music rendering apparatus method and system

Atomic music segments are visually and sonically rendered within a browser window as directed by a set of interface controls thus providing the ability to directly control various performance parameters while also communicating the intentions of the composer and arranger in a manner similar to traditional sheet music. In certain embodiments, individual voices may be selectively displayed, muted, or attenuated in order to focus a practice session or performance on particular parts. In one embodiment, atomic music segments and their associated lyrics are sequentially highlighted as the music progresses, providing a convenient means for reviewing or practicing music. Each atomic music segment may include one or more notes that have a substantially common onset time, thus providing an essentially indivisible unit of music convenient for user interaction and control.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to systems and methods for distributing and viewing sheet music and more particularly relates to apparatus methods and systems for browser-based visual and sonic rendering of sheet music.

2. Description of the Related Art

FIG. 1is an illustration of one example of a prior art published musical selection100. As depicted, the published musical selection100includes a variety of elements and markings that communicate the intended expression of the music printed thereon. The published musical selection100enables individuals and groups such as musicians, singers, hobbyist, and churchgoers to practice and perform music composed and arranged by others.

A title110identifies the name of the selection being performed. A tempo indicator112indicates the intended tempo or speed of performance. A key signature114specifies the key in which the music is written. A time signature118denotes the unit of counting and the number of counts or beats in each measure120. The depicted measures120are separated by bar lines122.

A system130typically contains a system indicator131and one or more staffs132composed of staff lines134that provide a frame of reference for reading notes136. The notes136positioned on the staff lines134indicate the intended pitch and timing associated with a voice or part140.

The published musical selection100may include lyrics150consisting of verses160. Within each verse160, words162and syllables164are preferably aligned with the notes136in order to suggest the phonetic articulations that are to be sung with each note136.

The elements associated with the published musical selection100are the result of hundreds of years of refinement and provide means for composers and arrangers to communicate their intentions for performing the musical selection. However, the process of formatting published music is typically a very tedious and time consuming process that requires a great deal of precision. Furthermore, adding or changing an instrument or transposing the selection to a new key requires the musical selection to be completely reformatted. Additionally, to be effective the published musical selection100typically requires either an accompanist who can play the music, or performers who can sight read the music. In many circumstances, such individuals are in limited supply.

In contrast to the published musical selection100, a media player200provides an alternate means of distributing music. As depicted, the media player200includes a play button210, a stop button220, a pause button230, a next track button240, and a previous track button250. The media player200provides a variety of elements that provide a user with direct control over a musical performance without requiring musical literacy or skill. However, the level of control provided by the media player200is quite limited and is typically not useful for practicing and performing music.

What is needed are systems, apparatus, and methods that provide users additional control over a musical performance while also communicating the intentions of the composer and arranger of the music. Preferably, such methods and systems would work within a standard browser and facilitate musical practice and performance for individuals and groups with a wide range of musical skill and literacy.

SUMMARY OF THE INVENTION

The present invention provides control over performance parameters such as dynamic voice selection and volume control within a standard browser window. The present invention overcomes the performance limitations typically associated with rendering music within a standard browser window through various techniques including formatting music data into units convenient for visual and sonic rendering. Referred to herein as atomic music segments, each note within an atomic music segment has a substantially common onset time enabling multiple notes to be processed as a single functional unit.

The use of atomic music segments, and formatting and rendering techniques associated therewith, enables the present invention to efficiently update a visual representation of sheet music within a standard browser in response to various changes such as transposing a key, disabling a voice, changing an instrument, hiding lyrics, or other user requested preferences or rendering options.

In one aspect of the present invention, a method for rendering music within a browser window includes displaying a song as a sequence of user-selectable atomic music segments, each atomic music segment comprising at least one note, and playing the song in response to a user-initiated event. Additionally, the method may also include sequentially highlighting the atomic music segments as each segment is sonically rendered within the browser window.

In certain embodiments, the internal representation of an atomic music segment has one or more notes with a substantially common onset time and includes a duration indicator that indicates the duration until the next segment (i.e. note onset) within the song. Thus, each atomic music segment is essentially an indivisible unit of music convenient for user interaction and control. In one embodiment, each duration indicator is quantized to a shortest inter-note interval of the song thus reducing the amount of data required to represent a song. Each note may also include a voice indicator that indicates which voice or part the note corresponds to. In one embodiment, the pitch of each note is indicated via an octave indicator and semitone indicator.

The structure used by the present invention to represent atomic music segments facilitates efficient and coordinated visual and sonic rendering of digital sheet music. The atomic music segments may be interspersed with other data elements that facilitate an accurate visual rendering of the sheet music such as system indicators, measure indicators, and annotations. A user is provided with direct control over various performance aspects while the intentions of the composer and arranger are communicated in manner that is consistent with traditional sheet music.

In certain embodiments, a visual rendering of the sheet music is accomplished by rendering the song as a sequence of music systems comprising one or more staffs. In one embodiment, notes are placed on the staffs in a visually appealing manner by computing a default width for each atomic music segment within the system, adjusting the segment width of selected segments in order to encompass their associated lyrics, and proportionally decreasing unadjusted segments to fit the atomic music segments within the available system width.

In another aspect of the present invention, an apparatus and system for rendering music includes, in one embodiment, a visual rendering module configured to display a song as a sequence of user-selectable atomic music segments, each atomic music segment comprising at least one note, and a sonic rendering module configured to play the song in response to a user-initiated event. The visual rendering module may be further configured to highlight a selected atomic music segment within the song, such as the segment currently being played by the sonic rendering module, in response to a change in the playback position.

In one embodiment, the visual rendering module includes a system builder that builds a music system, a segment builder that builds each atomic music segment, a spacing adjuster that adjusts the spacing of segments and staffs to prevent collisions with lyrics, a note renderer that renders basic note shapes, and a detail render that renders slur, ties, annotations, markings, and the like.

The sonic rendering module may be configured with a song loader that receives and loads a song for playback and a sound font loader that receives and loads a note palette or sound font to facilitate dynamic synthesis of notes and chords. Furthermore, the sonic rendering module may also include a playback module that facilitates coordinated visual and sonic rendering of the acoustic music segments that comprise the song, and a transpose module that facilitates transposing a song to a different key.

In addition to the visual and sonic rendering modules, the apparatus and system for rendering music within a browser window may also include as set of interface controls and associated event handlers that enable a user to control the rendering process. In one embodiment, the interface controls include controls that enable a user to control the playback tempo, mute or unmute specific voices, change the volume of each voice, specify a particular instrument, activate or inactivate autoscrolling of the sheet music during playback, include or omit the lyrics of a song, and search the lyrics, titles, and topics of a particular song or library of songs.

The aforementioned elements and features may be combined into a system for rendering music within a browser window. In one embodiment, the system includes a server configured to provide digitally encoded music, a browser-equipped client configured to execute a script, and a browser script configured to display a song as a sequence of user-selectable atomic music segments, and play the song in response to a user-initiated event. In certain embodiments, the browser script is further configured to sequentially highlight the atomic music segments in response to a change in a playback position.

In another aspect of the present invention, a method for rendering music within a browser window includes receiving a note palette, the note palette comprising a plurality of sampled sounds corresponding to a plurality of notes referenced in a song, receiving a plurality of atomic music segments, each atomic music segment comprising one or more notes, and mixing the digital samples that correspond to each note within an atomic music segment to provide a digital audio segment. The described method facilitates real-time dynamic control of the rendering process by a user and facilitates providing options such as changing the tempo of a song and dynamically muting or attenuating a selected voice.

In another aspect of the present invention, a method for rendering music within a browser window includes displaying a song within a browser window, the song comprising at least one music staff, at least one verse, and a plurality of voices, determining a set of selected voices and/or their desired volumes from at least one interface control, and playing the selected voices within the song adjusted to the desired volumes. The method may also include dynamically changing the selected voices and/or their desired volumes during playback. The described method facilitates real-time control of the music rendering process by a user within a standard browser.

In another aspect of the present invention, a method for rendering music within a browser window includes displaying a song within a browser window, the song comprising at least one music system and at least one voice, and automatically scrolling the at least one music system in response to completing playback of a current system. The described method enables a user to view an entire song during playback in an automated manner.

In another aspect of the present invention, a method for rendering music within a browser window includes storing a song as a sequence of atomic music segments and providing the song to a browser-equipped client. In one embodiment, each atomic music segment contains one or more notes, each note within a segment has a substantially common onset time. The described method facilitates efficient distribution and perusal of sheet music.

In another aspect of the present invention, a method for rendering music within a browser window includes receiving a song from a server, the song comprising a plurality of voices, displaying the song within a browser window, reformatting the song in response to a user inactivating a selected voice of the plurality of voices. The described method facilitates loading a song with a large number of voices such as an orchestral score and viewing only those voices that are of interest such as voices corresponding to a specific instrument.

These features and advantages of the present invention will become more fully apparent from the following description and appended claims, or maybe learned by the practice of the invention as set forth hereinafter.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides a browser-based apparatus method and system for visual and sonic rendering of sheet music that provides functionality beyond the capabilities of the prior art sheet music and prior art digital media players described in the background section. Specifically, the present invention segments song data into atomic music segments and uses each atomic music segments as a fundamental unit for rendering music. Preferably, each note within an atomic music segment has a substantially common onset time, thus forming an essentially indivisible unit of music convenient for user interaction and control.

The present invention overcomes the performance limitations typically associated with rendering music within a standard browser window. Specifically, the use of atomic music segments enables the present invention to provide real-time control over performance parameters such as voice selection and volume control while operating within a standard browser window.

Furthermore, the use of atomic music segments and formatting techniques associated therewith enables the present invention to efficiently update a visual representation of sheet music in response to various changes such as transposing a key, disabling a voice, changing an instrument, hiding lyrics, or other user requested preferences or rendering options.

FIG. 3is a schematic block diagram depicting one embodiment of a music publishing system300of the present invention. As depicted, the music publishing system300includes one or more atomic music servers310, one or more atomic music clients320, and an internet330. The music publishing system300facilitates distribution and perusal of electronic sheet music to users of the internet330via a conventional browser.

The atomic music servers310provide digitally encoded songs312to the atomic music clients320. The digitally encoded songs312may be encoded as a sequence of atomic music segments each segment thereof having one or more notes with a substantially common onset time. Providing digitally encoded songs312encoded in the aforementioned manner facilitates page-oriented streaming of song data and reduces the latency associated with reviewing music. Furthermore, the sequence of atomic music segments provides convenient units for visual rendering, sonic rendering, and user interaction using a standard browser.

In addition to the digitally encoded songs312, the atomic music servers310may provide one or more atomic music rendering modules (not shown) to the browser-equipped clients320. In one embodiment, the atomic music rendering modules are provided as a securely encoded Macromedia Flash™ script (i.e. a .swf file).

FIG. 4is a block diagram depicting one embodiment of a music publishing apparatus410of the present invention. As depicted, the music publishing apparatus410includes a set of interface controls410, one or more interface event handler(s)420, a visual rendering module430, a sonic rendering module440, and a search module450. In one embodiment, the music publishing apparatus410is achieved via one or more scripts provided by a server and executed by a browser.

The interface controls410enable a user to control rendering options, and the like, associated with the apparatus400. In one embodiment, the interface controls410enable a user to control volume, tempo, muting of voices, and other audio-related options. The interface controls410may also provide control over the visual display of a song. For example, in one embodiment the interface controls410enable a user to display music with or without lyrics, autoscroll to a next line of music, and print a song.

In the depicted embodiment, the interface event handlers420respond to changes in the interface controls410in order to effect the requested changes. For example, if a user mutes a particular voice an interface event handler420may inform the sonic rendering module440that the particular voice has been muted. An interface event handler420may also change one or more variables corresponding to the requested changes or invoke specific procedures to effect the change. For example, in response to a user disabling lyrics via an interface control, an interface event handler may change a lyric display variable and invoke a page redraw function that accesses the lyric display variable.

The visual rendering module430displays a song within a browser window. In the depicted embodiment, specific elements of the song are rendered by the various sub-modules which include a system builder432, a segment builder434, a spacing adjuster436, a note renderer438, and a detail renderer439. The song may be rendered within the same window as the interface controls410or with a separate window.

The system builder432builds a system comprising one or more staffs. In one embodiment, the system builder432computes an initial estimate of the space needed by the system and allocates a display region within the browser window for building the system. The system builder may draw the staffs within the allocated display region upon which notes corresponding to one or more voices will be rendered. In addition, the system builder may draw staff markings and allocate space for measure indicators and the like.

The segment builder434builds individual music segments within a system. The segments may be atomic segments having one or more notes with a substantially common onset time and one or more lyric segments that correspond to the notes. Under such an arrangement, the onset of all the notes of the segment may be within a single quantization interval and treated as an atomic unit for both visual and sonic rendering. In one embodiment, the segment builder434computes a default width for each segment based on the duration of the segment and number of segments within the system.

The spacing adjuster436may adjust the spacing provided by the system builder432and the segment builder434. For example, the width of particular segments may be increased by the spacer adjuster436in order to encompass lyrics that exceed the width of that segment, and the width of other segments may be decreased to accommodate those segments whose widths are increased. In addition to adjusting the (horizontal) width of segments, the spacing adjuster436may also adjust the vertical space between staffs to prevent collisions between notes and lyrics.

The note renderer438renders the basic notes of each segment within the system being rendered. The detail renderer439renders additional details such as slurs, ties, and annotations that result in a highly polished visual rendering of each system in the song.

The sonic rendering module440plays the visually rendered song in response to a user-initiated event such as depressing a play control (not shown). In the depicted embodiment, playing the visually rendered song is accomplished via a number of sub-modules including a song loader442, an optional sound font loader444, a playback module446, and a transpose module448. The various modules of the sonic rendering module440facilitate coordinated visual and sonic rendering of music such as sequentially highlighting music segments synchronous to playback (i.e. sonic rendering) of the segments.

The song loader442loads a song within a locally accessible location. In one embodiment, the song loader442retrieves a digitally encoded song312from an atomic music server310as described in the description ofFIG. 3. In certain embodiments, the song loader442may convert a track-based song encoding to a segment-based song encoding preferable for use with the present invention.

The optional sound font loader444may load a sound font associated with a song or a sound font selected by a user. In certain embodiments, the sound font is a set of digital audio segments that correspond to notes. In one embodiment, the sound font is restricted to those notes that are referenced in the song.

The playback module446plays the loaded song in response to a user-initiated event or the like. Playback is preferably synchronized with visual rendering such as highlighting each music segment as it is played. Synchronized playback may be accomplished via a callback function invoked by a segment-oriented player. For example, a segment-oriented player may activate the notes within a music segment and invoke a highlight function within the visual rendering module to de-highlight the previously highlighted segment and highlight the current music segment.

The transpose module448transposes the notes within a song in response to a user request or the like. In certain embodiments, the transpose module448shifts each note within each music segment up or down a selected number of half-steps and invokes a redraw function to update the visual rendering of the song. Updating the visual rendering of the song may include adjusting the spacing between staffs to account for the vertical shifting of notes. Updating may also include respacing the atomic music segments for various factors such as a change in the available system space due to a key signature change.

The search module450enables a user to search one or more songs for specific words or topics. In one embodiment, a search may be conducted on the lyrics of the currently loaded song, or the titles, topics, or lyrics of songs within a library of songs stored on a selected server.

FIG. 5is a flow chart diagram depicting one embodiment of a music rendering method500of the present invention. As depicted, the music rendering method500includes a receive segments step510, a receive palette step520, a display segments step530, a mix notes step540, a highlight selected segment step550, a play segment step560, an advance segment step570, and a respond to requests step580. The music rendering method500may be conducted in conjunction with, or independent of, the music publishing apparatus400and provides visual and sonic rendering of sheet music in an efficient coordinated manner. While depicted in a certain order, the steps of the depicted method may be rearranged in an order most suitable for the environment in which it is deployed.

The receive segments step510receives one or more music segments to be visually and sonically rendered within a browser or the like. In one embodiment, the music segments are provided as a digitally encoded song such as the digitally encoded song312. The receive palette step520receives a sound palette, or the like, for use with the music segments received in step510. In one embodiment, the sound palette is a set of audio segments corresponding to notes of a particular instrument. The receive palette step is an optional step that may not be needed in certain embodiments.

The display segments step530displays the received segments in a browser window or the like. The display segments step530may be conducted in the manner described previously in the description of the visual rendering module430ofFIG. 4or subsequently in the system formatting method ofFIG. 10.

The mix notes step540mixes the notes of the next segment to be played. In one embodiment, the mix notes step540involves invoking a play function for each active note by referencing a corresponding digital audio segment from a sound palette and specifying an envelope for the digital audio segment that corresponds to the selected volume for the voice and the specified note duration. Invoking a play function in such a manner for each note reduces the required size of the sound palette, provides for efficient processing, and provides for dynamic voice selection and volume control. In another embodiment, the mix notes step540sums digital audio segments from a sound font or sound palette into a next note. Preferably, only notes corresponding to active voices are mixed at volume levels prescribed by one or more interface controls.

The highlight selected segment step550highlights the currently selected segment. In one embodiment, the currently selected segment is automatically advanced as the music progresses from segment to segment and corresponds to the next note mixed in step540. Subsequently or concurrently to step550, the play segment step560plays the next segment in the song. In one embodiment, the next segment is the selected segment that is highlighted in step550.

The respond to requests step570, responds to user requests such as volume changes or the like. One embodiment of step570is the interface service method1100depicted inFIG. 11.

The end test580ascertains whether playback should end. In one embodiment, playback should end if a user activates a stop control or the song has ended. If playback should end, the method ends585. If playback should continue, the method loops to the advance selected segment step590. The advance selected segment step590automatically advances the selected segment to the next segment to be played. Subsequently, the depicted method continues by looping to the mix notes step540.

FIG. 6is a text-based diagram depicting one embodiment of an atomic segment data structure600of the present invention. The depicted atomic segment data structure600includes a segment duration605, one or more notes610with voice, octave, semitone, and duration indicators620,630,640, and650and may include one or more lyric segments660. The atomic segment data structure600facilitates coordinated visual and sonic rendering of music in an efficient manner.

The segment duration605indicates the duration of the music segment. In one embodiment, the duration is a quantized value representing the number of fundamental time units until the next music segment. The notes610indicate the notes that are to be activated within the music segment. The voice indicator620indicates which voice a particular note is associated with.

The octave and semitone indicators630and640indicate the octave and semitone to be played. The duration indicator650indicates the duration of the note. In one embodiment, each note begins at approximately the same time. However, the notes may have a duration650that is different that the segment duration605and may exceed the segment duration605.

The lyric segments660contain the lyrics670associated with the music segment. In certain embodiments, the lyric segments660also include a language indicator680.

FIG. 7is a screen shot depicting one embodiment of an upper portion of a music rendering interface700of the present invention. The depicted music rendering interface700includes a number of interface controls such as play controls710, an autoscroll control720, lyric controls730, one or more print controls740, and search controls750. Additionally, the music rendering interface700includes a search results pane760and a sheet music pane770with a visual rendering of the currently selected song. The music rendering interface700provides a user with an interactive environment for reviewing, practicing, and performing music.

The depicted play controls710enable a user to start, stop, and pause a sonic rendering of the current selection. The autoscroll control720enables a user to activate an autoscroll feature which facilitates automated viewing of the system currently being played. The depicted lyric controls730enable a user to selectively view the lyrics. In another undepicted embodiment, a language selector enables a user to specify a language for the displayed lyrics.

The print controls740enable a user to generate a printed copy of the music. The depicted search controls750facilitate a user to conduct a search of a song library. In another embodiment, the search controls facilitate finding specific words in the current selection. The search pane760displays results of a user requested search.

The depicted sheet music pane770is organized as a set of user-selectable atomic music segments including a highlighted segment780. In one embodiment, the highlighted segment780corresponds to the current playback position of a sonic rendering of the current selection.

FIG. 8is a screen shot depicting one embodiment of a lower portion of the music rendering interface700of the present invention. In addition to the previously introduced elements, the depicted music rendering interface700includes a set of voice controls810including muting controls810aand volume controls810b, one or more tempo controls820, one or more transpose controls830, and an information pane840.

The voice controls810enable a user to selectively control the balance of the various voices or parts in a song. The depicted muting controls810aenable a user to dynamically mute or unmute each voice. In one embodiment, the visual rendering of the sheet music pane770is redrawn to hide muted voices. The depicted volume controls810benable a user to dynamically adjust the playback volume of each voice.

In another embodiment, a separate set of voice display controls (not shown) enable a user to visually hide individual parts or voices such that the music pane770is respaced and redrawn showing only the visually selected voices. Having separate voice display controls and muting controls provides a user to increase flexibility over prior art solutions.

The depicted information pane840displays information about the current selection such as the author of the lyrics, the composer of the music, a tune name, and a meter pattern. The tempo controls820facilitate adjusting the playback tempo. In one embodiment, the tempo may be dynamically adjusted during playback. The transpose controls830enable a user to transpose a song a selected number of half-steps.

FIG. 9is a flow chart diagram depicting one embodiment of a page scrolling method900of the present invention. As depicted, the page scrolling method900includes a next segment step910, a new system test920, an end of page test930, and a scroll page step940. The page scrolling method900may be conducted in conjunction with the music publishing apparatus400depicted inFIG. 4or the music rendering method500depicted inFIG. 5. While the depicted method assumes a single sheet of auto-scrolled music, one of skill in the art will recognize how the method900may be extended to other scenarios.

The next segment step910advances to the next segment in a song. Advancing to the next segment may include waiting for a timeout event that indicates completion of the current segment. In one embodiment, advancing to the next segment also involves traversing a linked list of data structures containing a description of each segment and their associated notes and lyrics. The new system test920ascertains whether the next segment is on a new system. If not, the method loops to the next segment step910. If the next segment is on a new system, the method proceeds to the end of page test930.

The end of page test930ascertains whether the end of a page of sheet music has been reached. If the end of the page has been reached, the method ends950. If the end of the page has not been reached, the method loops to the scroll page step940. The scroll page step940scrolls a page of sheet music such that the new system is in a viewable location such as near the top or middle of a browser window. Subsequent to the scroll page step940, the method loops to the next segment step910and continues processing.

FIG. 10is a flow chart diagram depicting one embodiment of a system formatting method1000of the present invention. As depicted, the system formatting method1000includes a compute default widths step1010, an adjust segment widths step1020, and a decrease unadjusted widths step1030. The system formatting method1000facilitates displaying a page of sheet music in an aesthetic yet efficient manner.

The compute default widths step1010computes a default width for each segment in a system. The default width may be expressed in units of pixels or similar convenient units such as percentage of the system width. In one embodiment, the space available on the system for rendering segments is proportionally allocated as a weighted average of the available width per segment and the available width per duration count.

The adjust segment widths step1020, adjusts the width of certain segments from their computed defaults. In one embodiment, segments having lyrics which exceed their default width are adjusted such that their widths encompass their associated lyrics. To account for the additional width allocated for encompassing lyrics, the decrease unadjusted widths step1030decreases the width of unadjusted segments to bring the total system width below the available rendering space. In one embodiment, the width of each unadjusted segment is proportionally decreased in order to match the total width of all of the segments to the space available on the system for rendering segments.

FIG. 11is a flow chart diagram depicting one embodiment of an interface service method1100of the present invention. As depicted, the interface service method1100includes a mute test1110, a mute step1120, an unmute test1130, an unmute step1140, a volume test1150, an adjust volume step1160, a segment change test1170, and a change segment step1180. The interface service method1100facilitates dynamic changes in sonic rendering options during playback of song.

The mute test1110ascertains if a mute request has occurred. If a mute request has occurred the selected voice is muted1120. Similarly, the unmute test1130ascertains if an unmute request has occurred. If an unmute request has occurred the selected voice is unmuted1140.

The volume test1150ascertains if a volume change request has occurred. If a volume change request has occurred the volume of the selected voice is adjusted1160. The segment change test1170ascertains if a user has selected a different playback position. In one embodiment, a different playback position is selected by clicking on a segment corresponding to a desired playback position. If the user has selected a different playback position the segment is changed1180to the indicated segment.

The present invention provides a browser-based apparatus, method, and system for rendering sheet music. The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.