Abstract:
This method and system is primarily implemented for multiple computers-guided performance by an ensemble of musicians utilizing the Internet/Intranet network. Different parts (“sheet music”), accompanying sounds, instructions, etc., of a larger, multi-layer composition are individually controlled and automatically displayed at appropriate times onto the respective monitors of multiple computers. These multiple parts are loaded without user input, individually timed and retrieved for each computer. Different elements of the composition are separately displayed for each of the performers, in such a way that when executed together they create a unified totality of the composition/presentation. This system has a number of possible configurations, such as controlling contemporaneous, multicontent classroom instructions and/or artistic presentations. In particular, an embodiment of this method is used for multiple computers-guided music or multimedia performance where performers do not need a conductor or to be physically proximate to one another.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]    This application claims priority benefits under 35 U.S.C. § 119(e), and incorporates by reference provisional application Nos. 60/287,782 filed May 2, 2001, and 60/295,840 filed Jun. 6, 2001. 
     
    
     
       FIELD OF THE INVENTION  
         [0002]    The invention combines existing networking technology, integrating new configurations and apparatus with new techniques of music composition to benefit specifically but not exclusively, new types of creations and events for the performing arts. The invention pertains to the field of use of a computer network in live, simultaneous multi-content presentations. Instead of the computer being used merely as a tool, the computer is an active participant in an ensemble of computers, each playing a separate part in the performance. The result is a symphony of computers with live performers dispersed throughout the venue and/or across many locations. The invention provides a new way of utilizing computer processing and networking technology in live music/multimedia concert automation, and is a specific system tailored to accommodate the intricacies of a new type of music performances.  
         BACKGROUND AND SUMMARY OF THE INVENTION  
         [0003]    Initially, it is to be noted that throughout this document the several words have specialized definitions understood by those skilled in the fields of music and computers:  
           [0004]    “Composition” (usually designed by a composer) means an artistic arrangement of music parts (score) and various other content/elements which when performed or executed together form a unified whole.  
           [0005]    “Performance” means execution or presentation of the composition by the performers.  
           [0006]    “Part” means a portion or a layer of the composition to be performed by a particular instrument, voice; role or any other element of the composition to be assigned for execution  
           [0007]    “Sheet Music” means a graphic representation of music notation to be read and executed by a performer trained in reading music.  
           [0008]    “Cue” means the place in composition where, or signaling the time in the composition when, the element or part is to be executed.  
           [0009]    “Midi” (Musical Instruments Digital Interface) is commonly used digital language for connecting electronic instruments so they communicate with each other.  
           [0010]    “Script” refers to any known Internet programming language such as Java Script or VB Script.  
           [0011]    “Session” in some scripting languages, is the time while the browser is opened in an Internet Browser environment.  
           [0012]    “Localhost” the root of the URL address when a computer operates as its own server.  
           [0013]    “Computer Program” includes: software (i.e. WordPerfect), scripts (i.e. Java Script) and codes (i.e. HTML).  
           [0014]    Computer technology enables users in different locations to communicate with each other. The network can be a platform for many shared activities, including music making. In most cases however, music web sites are designed to present music recording for listening or downloading (i.e. MP3) or to display music material for viewing or sale. Existing web sites are not designed to create a combined simultaneous performance outcome or material. This is because most Internet sites so far, merely communicate or display data “on demand”. Music composition and its performance require predetermined synchronization and timing of many single elements. An orchestra can consist of 100 musicians most of whom are playing different instruments and different parts. In an orchestra, the musician must be in close proximity of other players, e.g. on a stage, and if the ensemble is large, a conductor is needed to lead all the players, among others, to achieve certain precision in the timing of the execution.  
           [0015]    The principal benefit of a computers-guided performance is to create a totally new way of performing and listening to music. This system also enables performances in a variety of non-traditional venues. In a traditional concert hall, the ensemble is confined to the stage and the listeners are constrained in rows of seats. Such a “static” experience begs the addition of alternative ways of listening to music that transforms the relationship between the performers and the public. A particular use of the method, named by the inventor “webOrchestra”, offers an embodiment for multiple computers-guided music/multimedia performance where performers do not need to be physically close to each other on stage nor do they require a conductor.  
           [0016]    Until now, performances of this type would only be possible if a conductor, utilizing a closed circuit TV, led the performance. The “webOrchestra” method is inexpensive and practical. It provides an unlimited number of possible configurations creating an opportunity for substantial community (audience) participation. Even those who do not play an instrument will be able to trigger sounds, pre-composed musical phrases and other elements of the composition from their respective computers and thus participate in the interactive composition.  
           [0017]    The invention utilizes well-known server technology to guide a musical ensemble performance. The invention automates and times the display of the parts of the composition so no “page turning” or other action by the performer is needed in order to play the music on traditional instruments (e.g. trumpet, violin etc.). In addition, other multimedia elements can be scheduled to accompany the performance. In cases when the musicians are dispersed throughout large spaces, the composer should accommodate for the architecture of the venue, acoustics and other site-specific aspects. In some cases, an amplification system has to be used and separately designed for the performance to be effective.  
           [0018]    The invention is described below with terminology understood by persons knowledgeable of the practices of those skilled in the art of music and music notation reading, with reference to acts and operations that are performed by the system in the course of a musical event (concert) or equivalent of such an environment. The apparatus includes individual computers connected to a network with a browser e.g. Internet Explorer™, and a server storing the content files with the parts and other elements of the composition as well as performers&#39; passwords and various computer programs (i.e. scripts) controlling the server&#39;s output. A plurality of computers coupled to the network, also have a display monitor, at least one input device and at least one output device, audio and MIDI (Musical Instrument Digital Interface) output and a version of Microsoft™ PWS (Personal Web Server) or IIS (Internet Information Server) or similar computer program.  
           [0019]    In order to implement these mechanisms, the invention utilizes a JavaScript and/or VBScript for Active Server Pages (ASP), web browser and the Windows Media Player (for accompanying audio/midi). Use of other programming languages and computer systems is possible. Each of these items is well known in the art and does not require further explanation.  
           [0020]    Using well-known techniques, an HTML page is created. All of the elements/parts of the composition to be stored on the server can be created using any number of well-known tools. For example, Adobe Photoshop, Paint, music notation/audio/ midi recording programs such as Finale, Cubase, etc., can be used to create content using GIF (graphic interchange format) image files, WAV or other audio formats. Once the scripts for timing of the distribution of the elements have been created, specific individual pages to be displayed should also be created. All pages should be placed on the web server in separate files that the associated user will access. Specific techniques for completing these steps are well known and need not be specifically described.  
           [0021]    The principal embodiment of the invention is adapted for use with performers stationed a small distance from each, not necessarily within each other&#39;s view, but able to hear one another. In another embodiment, if the distances necessitate, microphones/amplifications can be employed. Displays of different musical parts, (“electronic sheet music” and instructions) are synchronized and timed by scripts running from the server and simultaneously exhibited to the respective performers, on monitors of multiple computers. Each performer or small group of performers is assigned a computer terminal to view and be guided by the appropriate pages of musical composition (notes) and instructions (such as a metronome count of tempo or text instructions for musical interpretation). This system also allows showing the music notes in color, and thus two or more versions of the music can be played from one screen. For example, one player plays only red notes, the other only black notes etc.  
           [0022]    The invention utilizes a network, such as the Internet or an Intranet to execute the scripts and automatically display the appropriate musical part to be played. The program uses a database containing passwords for each authorized user. Performers receive a password via email or other means. Each password is specific for the person and/or the part to be played.  
           [0023]    The query refers the performer to the appropriate part or other content assigned to that password.  
           [0024]    Depending on the size of a venue, computer terminals (connected to the Internet or a local network) are distributed throughout the building taking into consideration acoustical aspects, architecture and movement of the audience. A performer (or a small group) is placed in front of each of the terminals. After connecting to the site&#39;s URL, and the entry page the performer will enter a password, which is pre-assigned by the content provider (composer). This information should match the assigned part of the composition, as well as any additional information that will link the musician to the appropriate URL/site where his/her part is stored with instructions and timing script.  
           [0025]    After reading the instructions the performer clicks to begin playing and waits for the “sheet music” to appear on the monitor. Each computer terminal displays only the part that is assigned to it. From this point on the elements of the performance are coordinated between the terminals and automatically displayed at appropriate times, without any user input. The script (after checking the time) then displays the first page of the music to be played by the assigned instrument. The page is timed until the next cue and the second page is displayed and so on until the whole composition is sequentially transmitted. This routine is the same for each performer, yet the content they receive is different and specifically timed so when heard/executed or seen together it creates the totality of the composition. In addition, the part can include a lead-in visual or audio metronome or other real time instruction. As the musician continues to play, other elements could be timed and activated during the sequence. For instance, a timed MIDI signal could send information to an outboard device such as an electronic keyboard, lighting system etc. An accompanying audio could be streamed from the server or come from a radio/TV broadcast. The output then might be sent to the venue&#39;s PA system, depending on the size and the acoustics of the location. Because each computer&#39;s internal clock can be set differently, the timing is controlled by server-side scripts (e.g. JavaScript, VBScript) or similar computer program. The scripts check the time and send out the appropriate commands. This is to secure timing of varied contents on multiple computers. The scripts select the appropriate pages of music and distribute the parts including tempo and other instructions, at the proper time to the respective terminals to be played together. If a script checks the time and it is not appropriate for the next cue, it will automatically wait for the next cue to retrieve the appropriate content. The inventor has used VisualBasic Script to create Active Server Pages (ASP) to control timing, content and coordinate the display or execution of the elements of the composition, as designed by the composer. However, those of skill in the art of computer network programming could use other programming language (software) to carry out the invention.  
           [0026]    The type of network used is immaterial, except that it should be fast in response (i.e. DSL, or cable). In addition to the network (Internet/Intranet), other suitable method for executing this invention could use Microsoft™ PWS (Personal Web Server) or IIS (Internet Information Service) individually for each computer to operate as its own server. This approach is recommended only if the Internet connection is slow or unreliable. In such case, each computer&#39;s internal clock has to be set to a Remote Time Server e.g. swisstime.ethz.ch, tick.usno.navy.mil or another public Remote Time Server—a service which provides a constantly updated time of day reading (in hours, minutes, and seconds) and means of setting individual computer&#39;s clocks. Then all respective files with the elements of the composition and appropriate scripts should be downloaded to the individual computers. After this, the computers may be disconnected from the network and moved to a desired location for the duration of a performance.  
           [0027]    Although the following is not intended as the principal embodiment of this technique, this method also can be used to replace “sheet music” stands and page-turning for traditional orchestras. In such case, instead of printed sheet music, each musician, including the conductor, would have his/her individual computer screen instead of a traditional music stand.  
           [0028]    The program, process and the method described here can work on variety of types of computers and servers and the process can be creatively augmented on the principle presented by the inventor. The principle of this method can have numerous embodiments, which may be created within the scope and spirit of the method by persons skilled in the fields of the invention.  
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0029]    The invention will now be described with reference to the preferred embodiment illustrated in the appended drawings, in which:  
         [0030]    [0030]FIG. 1 is a chart illustrating the overall system and the interrelationship of the various hardware components with multiple terminals displaying a plurality of elements to be performed together.  
         [0031]    [0031]FIG. 2 is a diagram showing system architecture for display of the “sheet music” and optional inclusion of an accompanying element on a single performer computer terminal.  
         [0032]    [0032]FIG. 3 is a flow chart illustrating sign on routine.  
         [0033]    [0033]FIG. 4 is an alternative embodiment of the invention displayed on individual computers without a need for constant connection to the network by using Personal Web Server individually on each computer.  
         [0034]    [0034]FIG. 5 Diagram of the content of the composition to be created in the formats suitable for each of the specified parts and the design of the composition.  
         [0035]    [0035]FIG. 6 is an illustration of time sequence design for two instruments.  
         [0036]    [0036]FIG. 7 is a browser environment on a client computer displaying a sheet music page with animated images and colors providing additional instructions and leading the performer. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0037]    The invention will be understood from the specification and drawings by those skilled in the art of music and computers. In some instances, well known methods and procedures have not been illustrated in detail so as not to obscure aspects of the invention described in detail in the body of this document.  
         [0038]    [0038]FIG. 1 is a chart illustrating the overall system of the invention, including connections between hardware components and multiple terminals displaying a plurality of elements to be performed together in order to create the totality of the composition. The various elements of the composition are separately and individually timed and displayed on or executed by the computers of the performers  11 ,  12 ,  13 , in such a way that when played (if an instrument) or executed, (if a non-instrumental part of the performance such as lighting or MIDI controlled electronic instrument/sound) together they create a unified totality of the composition. Optional outputs  14  can be used for additional elements of the composition, such as Midi commands, streaming audio, audio files and visual projections; separately and individually executed from each computer, amplified and distributed throughout the venue. A server  15  stores all content including Scripts to run each part via Internet or Intranet  17 . An optional addition can be a prearranged radio/TV broadcast  16  so additional sounds and/or images could be used simultaneously and timed into the multimedia performance.  
         [0039]    [0039]FIG. 2 is a diagram showing system architecture for display of the “sheet music” on a single client computer terminal  31  for use for the performance of one performer  35 . The computer terminal  31  is among a plurality of computers, each with a network connection to a server  21  and requiring different content interrelated to the total design of the composition. The diagram also illustrates the integration of multiple elements for one of the computer terminals that participate in the performance.  
         [0040]    a. on demand, the server displays “webOrchestra” home page  22  linking to various pages that include:  
         [0041]    Log on page for the performers who already obtained their password  24   
         [0042]    Application with instructions how to obtain a password  33   
         [0043]    b. If the performer enters a correct password, the server displays the initial page of the part assigned to this particular terminal  25 . The page includes instructions for the performer and the server automatically pre-loads the assigned elements of the composition into the terminal&#39;s cache  26 . This technique is widely used to subsequently speed up the display of the elements on the computer when they are later recalled in the actual performance.  
         [0044]    c. If ready to begin, the performer clicks START and the server responds by linking to the page that runs the assigned script controlling the timing  27 ,  28  of the sheet music display and execution of other elements. From here on the performance parts are automatically timed, displayed  30  and executed  32 ,  34  by the server onto the respective terminal  31 , without any user input. In the sequential display of the sheet music and execution of other elements the server, using a script or other programming language, constantly verifies the timing of the display of the elements to match the pre-designed timing of the composition as planed by the composer. To better coordinate the performance each page might include additional instructions i.e. a visual metronome indicating for a particular page. In addition, if the performer is late or stops playing the page will disappear changing to the next page in sequence, at the predetermined time required by the design of the composition and programmed into the respective scripts by the composer.  
         [0045]    The routine is the same for each terminal; however, the content and timing are individually customized for each musician to perform the respective part in relation to the plurality of performers participating in the performance of the whole composition. As is familiar to those skilled in the art, the server  21  and the computer system  31  include an operating system and an Internet browser. The browser is a standard browser for displaying content, including hyperlinks. A user can customize the browser to maximize the viewing area, memory allocation etc. In operation, the following steps are preformed: after opening the “home” page at  22 , the player links to the password page at  24 . Depending on the preassigned password entered, the content will be associated with this terminal at  25  or reserved for another computer at  23 . For faster loading during the performance, the content will be pre-loaded into the computers&#39; cache at  26 . After the performers click START at their respective terminals, all individual elements (pages and files) of the composition are automatically displayed at  30 , one after another in a sequential order in time at  27 , predetermined by the composer/content provider at  28 . When a current music page is displayed, one or more files such as audio/midi, image projections, etc.  32 , relating to that music page can be included and sequentially executed by sending the output from the terminal to a midi device, projector or, in case of an accompanying electronic audio signal, directly to the loudspeakers through the amplification  34 . The music page then automatically switches to the next page, which can also include other accompanying media and so on. Different contents are individually customized for each performer, retrieved at  29  and individually displayed at  30  on each computer terminal  31  at the appropriate time  27 . Without any user input, automatically and independently for each computer, the next page is retrieved from the server; and at an appropriate time automatically displayed for a single performer. Simultaneously, another part is retrieved for another performer thus displaying multiple parts on multiple computers, which when executed, will result in the totality of the composition. If the venue or distances are large, a microphone  36  can be used to send the sound of the performer to the PA system or to the network to be heard on other computer(s) if they are located at a great distance.  
         [0046]    [0046]FIG. 3 is a flow chart illustrating a sign-on routine. It includes the timing bypass subroutine  44  and password bypass subroutine  39 .  
         [0047]    After signing in at  38 , any entry through a valid password at  39  is stored as a “session” so if the performer should stop playing for a period of time, the performer can quickly join the performance at any time without reentering the password.  
         [0048]    If a wrong password is entered a page is linked to instructions  37  on how to receive a valid password. Performers can receive individual passwords via email at  41 ,  42  or by other means. The password is specific for the person and/or for the element of the composition to be executed and is associated with the corresponding files containing the assigned elements for the terminal from which the password was received  40 . The assigned passwords are stored in a server file/database  48  and verified by the server subroutine  39 .  
         [0049]    Once the password is accepted, the server automatically associates the appropriate parts and other elements of the composition for display on the terminal from which the password was received. The server retrieves the files from its storage  49  and pre-loads them at  43 . After reading the instructions about the performance, the performer clicks START and from here on the assigned parts are automatically displayed or executed (subroutine A) with no need for any user input.  
         [0050]    The content pages are displayed and other elements are executed automatically and sequentially one after another at the appropriate, predetermined, time-controlled at  44 ,  45  by the script. If the time for the display is not right, as predetermined by the composer, the script will wait at  46  before displaying the part at  47 . (subroutine B)  
         [0051]    The process is the same for each performer yet resulting in simultaneous, time-controlled display of different parts and elements of the composition, individually and independently on the respective terminals associated to the respective parts by individual passwords.  
         [0052]    [0052]FIG. 4 is an alternative embodiment of the invention displayed on individual computers without a need for constant connection to the network by installing Microsoft™ Personal Web Server™ or similar program, individually on each computer. The invention may be alternatively or simultaneously embodied for displaying from public or private computer networks. The choice of preferred network depends on speed and reliability. The extreme of this embodiment would be the exclusive use of PWS (Personal Web Server) or equivalent program running each computer separately.  
         [0053]    1. All respective files with the parts of the composition and appropriate scripts must be downloaded to the individual computers by e-mail request  53  and a download  50 . Alternatively, the content could be provided on a CD-ROM. The download might also include instructions how to install and run Microsoft™ Personal Web Server™ or equivalent program that makes the computer operate as its own server.  
         [0054]    2. Each computer&#39;s internal clock has to be set to a Remote Time Server  51 , e.g. swisstime.ethz.ch, tick.usno.navy.mil or another public Remote Time Server—an Internet service which provides a constantly updated time (in hours, minutes, and seconds). After this, the computers may be disconnected from the network  52  and moved to a desired location for the duration of the performance. By installing and running Microsoft™ Personal Web Server™ or equivalent system at  54 ,  55  each computer terminal becomes its own server.  
         [0055]    3. Each performer logs on to “localhost” by opening the browser to the same URL address as if connected to the Internet  56 , except that the root of the address is changed to “localhost”. For example, http://localhost/weborchestra/weborch.htm instead of normal http://www.weborchestra.com/weborchestra/weborch.htm. This procedure displays the first page of the previously downloaded content, now stored on the local hard drive. The performer then proceeds the same way as described herein when using the Internet/Intranet network.  
         [0056]    The process of the performance and the final result stay the same as if using the Internet network. It allows performers distant from each other to be guided by the computers to produce the totality of the composition. The routine  57  of automatically displaying the parts becomes the same as in FIG.3. This method of utilizing Microsoft™ Personal Web Server™ or equivalent computer program basically enables the performance without constant dependence on the network connection.  
         [0057]    [0057]FIG. 5 is a diagram of the content of the composition to be created in the formats suitable for each of the specified instruments and the design of the composition. In order to implement the invention, the following steps are employed to provide content:  
         [0058]    a. Create a composition in a format suitable for this technique  60  and for display within a web page that loads different “sheet music” images or other multimedia formats, at certain times predetermined by the composer;  
         [0059]    b. Divide the material into files specifically associated with a particular performer or group of performers and assign separate passwords  61 ;  
         [0060]    c. Assign specific timing/cues for each part  62 ;  
         [0061]    d. Add timed instructions, scripts with timing and display commands  63 ;  
         [0062]    e. Assign individual passwords to the respective parts  64 ;  
         [0063]    f. Create an HTML (or equivalent) layout  65 ;  
         [0064]    g. Upload to a server and distribute passwords  66 ;  
         [0065]    [0065]FIG. 6 is an illustration of time sequence design for two instruments using multiple contents  69  from one server to display individually on each of the musicians&#39; computers including additional, optional elements  70 .  
         [0066]    [0066]FIG. 7 is a known browser environment  72  on a client computer. It represents a portion of the “sheet music” display in a browser environment with color notes  73 , which are referred to in instructions at the beginning of the performance. For instance, the instructions could direct the musician not to play red color notes or to wait until the musician hears the color marked notes etc. The use of color representation of “sheet music” in this invention enables the parts to be displayed on one of the monitors as several versions with “hidden” cues, instructions, and variations. Also, red/color numbers serving a visual metronome to show the required tempo for a page or other timed animation  71  such as words “PLAY”, “WAIT”, could be included to lead the musician playing this part.  
         [0067]    Having described and illustrated the principles of the invention, it will be recognized that the embodiment can be modified without departing from the basic principles of the invention. This method opens new ways of utilizing computer networks for creating multi-contents events and specifically the music/multimedia performances that could not have existed before.