Patent ID: 12206928

DETAILED DESCRIPTION

While the present invention is susceptible of embodiment in many different forms, there is shown in the drawings and will herein be described in detail one or more specific embodiments, with the understanding that the present disclosure is to be considered as exemplary of the principles of the invention and not intended to limit the invention to the specific embodiments shown and described. In the following description and in the several figures of the drawings, like reference numerals are used to describe the same, similar or corresponding parts in the several views of the drawings.

The system for the synchronization via one or more electronic device of one or more of media content includes a plurality of components such as one or more of electronic components, hardware components, and computer software components. A number of such components can be combined or divided in the system. An example component of the system includes a set and/or series of computer instructions written in or implemented with any of a number of programming languages, as will be appreciated by those skilled in the art.

The system for the synchronization via one or more speaker system include a plurality of components such as one or more input and output (1/0) device, analogue to digital and digital to analogue device (AD/DA), amplifier device, active and passive speaker systems, and an other electronic device.

The system in one example employs one or more computer-readable signal-bearing media. The computer-readable signal bearing media store software, firmware and/or assembly language for performing one or more portions of one or more implementations of the invention. The computer-readable signal-bearing medium for the system in one example comprises one or more of a magnetic, electrical, optical, biological, and atomic data storage medium. For example, the computer-readable signal-bearing medium comprises floppy disks, magnetic tapes, CD-ROMs, DVD-ROMs, hard disk drives, downloadable files, files executable “in the cloud,” and electronic memory.

FIG.1is a schematic block diagram of a networked environment100for real-time customization and synchronization of media content via one or more of one or more electronic device, that comprises client-side networked environment105, server-side networked environment110, and network115. Network115comprises one or more of one or more speaker system127, microphone system128, input and output (1/0) device198, Internet, private virtual network, extranet, fiber optic network, wide area network (WAN), local area network (LAN), wired network, wireless network, and an other type of network.

Client-side networked environment105comprises client-side device120and client-side playback device125that is operably connected with client-side device120. Client-side device120comprises, for example, one or more of one or more tablet120, phone120, smart device120, virtual reality headset120, augmented reality headset120, computer program120, computer browser120, media player120, game console120, virtual device120, and an other computing device120.

Client-side device120runs one or more applications. Client-side device120deploys over network115.

Client-side playback device125is configured to play media content. For example, client-side playback device125plays media content received from client-side device120. Alternatively, or additionally, client-side playback device125plays media content received directly over network115. For example, client-side playback device125comprises one or more of one or more headphone125, earphone125, earbud125, earworn wearable125, screen125, television125, monitor125, in-venue projector125, home theater125, three-dimensional digital projector125, and an other client-side playback device125. For example, client-side playback device125comprises one or more of one or more open headphone125, semi-open headphone125, closed headphone125, and an other type of headphone125.

Client-side playback device125operates in an environment with sensor system126. For example, sensor system126comprises one or more of one or more visual sensor126, thermal sensor126, vibratory sensor126, position sensor126, and an other sensor126. For example, the sensor system126comprises one or more of one or more head motion tracking sensor126, eye motion tracking sensor126, haptic sensor126, and an other position sensor126. For example, the sensor system126may comprise a spatial resolution of two degrees of freedom in all directions so as to provide one or more smooth audio transitioning and stable sound imaging.

Client-side playback device125operates in an environment with speaker system127. For example, client-side playback device125operates in an environment with one or more of one or more channel-based speaker system127, object-based speaker system127, scene-based speaker system127, 3d audio speaker system127, and an other speaker system127environment. For example, client-side playback device125operates in one or more of single-channel speaker system127and multi-channel speaker system127environment. For example, single-channel speaker system127and multi-channel speaker system127may comprise one or more location of one or more speakers in one or more designed position and zone within a physical space. For example, client-side playback device125may operate in an object-based speaker environment. For example, the object-based speaker system127may contain one or more of an object (e.g. sound sources) that contains metadata describing the intended object position and other spatial properties. For example, client-side playback device125may operate in a scene-based speaker system127environment. For example the, the scene-based speaker system127may use one or more of spherical harmonic basis function that describes how the sound pressure in media content changes as a function of time and direction. For example, client-side playback device125operates in an environment with one or more speaker127. For example, speaker system127may comprise one or more of one or more woofer, such as to cover low-frequency range, tweeter, such as to cover high-frequency range, and an other audio transducer.

Client-side playback device125operates in an environment with microphone system128. For example, microphone system128may comprise one or more of one or more microphone array, spot microphone, and an other microphone system.

Client-side device120comprises one or more client-side memory130and client-side data storage135.

Client-side memory130is defined herein as including both volatile and nonvolatile memory and data storage components. For example, client-side memory130comprises one or more client-side buffers. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon loss of power. For example, client-side memory130may comprise one or more random access memory (RAM), read-only memory (ROM), hard disk drive, solid-state drive, USB flash drive, memory card, floppy disk, optical disc such as compact disc (CD) or digital versatile disc (DVD), magnetic tape, and other memory components. For example, RAM may comprise one or more static random access memory (SRAM), dynamic random access memory (DRAM), magnetic random access memory (MRAM), and other forms of RAM. For example, ROM may comprise one or more programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and other forms of ROM.

Client-side memory130comprises one or more client-side customization application145, and client-side synchronization application147.

Client-side memory130further comprises a client-side device unique identifier. The client-side device unique identifier is a number unique to this particular device. In other words, each device in the world will have its own number that no other such device will have. A copy of the client-side device unique identifier, known as a client-side unique identifier, will be transmitted by the client-side device120in a message or packet to the server-side computing device170. Then a copy of the client-side transmitted unique identifier, known as a server-side unique identifier, will be transmitted back from the server-side computing device170to the client-side device120. The server-side unique identifier received by the client-side device120will then be compared with the client-side device unique identifier to help determine the integrity of the messages and as a security check.

Optionally, the client-side memory130further comprises an other client-side application (not pictured). The other client-side application comprises one or more additional client-side application, additional client-side service, additional client-side process, and additional client-side functionality. For example, an other client-side application runs background services. For example, an other client-side application runs boot processes. For example, an other client-side application runs other client-side applications.

Client-side data storage135comprises one or more single database, multiple database, cloud application platform, relational database, no-sequel database, flash memory, solid state memory, and an other client-side data storage device. Client-side data storage135may be located in a single installation that may be local to the server-side computing device170. Alternatively, client-side data storage135may be located in a single installation that may be local to the client-side device120. Alternatively, client-side data storage135may be distributed in a plurality of locations. Client-side data storage135may be distributed in a plurality of geographical locations. Client-side data storage135may be distributed in a plurality of geographical locations located in the same time zone. Client-side data storage135may be distributed in a plurality of geographical locations, wherein not all the geographical locations are located in the same time zone.

Client-side data storage135comprises one or more of item prices, order information, media content, and other information. For example, the media content comprises one or more of an audio track, a video track, an other media track, a motion picture, a commercial, a motion picture trailer, a demonstration (“demo”), a commentary, extra content, and an other form of additional content. The media content comprises one or more of media data, media content files, and other media content. The motion picture comprises one or more of a feature-length theatrical production, short-film production, an animated production, a broadcast television production, a pay television production, a documentary, a commercial, a trailer, and an other motion picture. The media data comprises one or more audio track, multi-channel track, commentary, and other media data. The audio track comprises one or more English language audio track, audio track in a language other than English, and customized audio track. The commentary comprises one or more commentary by one or more directors of a motion picture, a commentary by one or more actors in a motion picture, a commentary by contributors to a motion picture other than the directors and actors, and commentary by persons other than contributors to a motion picture.

Client-side customization application145is configured to store media content. For example, client-side customization application145stores playable media content in client-side data storage135. The playable media content comprises one or more segmented media content track, non-segmented media content track, and an other playable media content. Optionally, client-side customization application145performs media processing of the playable media content. For example, client-side customization application145passes the playable media content through one or more of bandpass transfer function, headphone transfer function (HpTF), compensation filters, equalization filters, and an other DSP algorithm, to regularize audio signals, and adjust one or more timber, spectral cue, localization, and an other attribute of sound. For example, HpTF may be used to one or more smooth, accentuate and cancel fluctuations in the low, mid and high frequencies of one or more headphone. For example, client-side customization application145automatically uploads a pre-determined HpTF to one or more headphone. For example, client-side customization application145may receive headphone data (e.g. model, serial number and other identifying information) and upload from client-side data storage135a customized HpTF for one or more headphone. For example, client-side customization application145may determine delay-processing time of these upload functions and adjust the client-side playback to compensate.

For example, client-side customization application145parses the playable media content into a chronological sequence that substantially matches the sequence of the motion picture. For example, client-side customization application145writes the playable media content to one or more of client-side data storage135and client-side memory130. For example, client-side customization application145writes the playable media content to a media content file located in one or more client-side data storage135and client-side memory130.

Client-side synchronization application147is configured to connect with server-side networked environment110so as to substantially synchronize between server-side networked environment110and client-side device120media content played on client-side playback device125. Client-side playback device125comprises one or more of one or more screen125, television125, monitor125, cellular phone125, laptop computer125, desktop computer125, notebook125, tablet125, headset125, channel-based playback system125, object-based playback system125, scene-based playback system125, 3d audio playback system125, and an other client-side playback device125. Client-side playback device125plays for the user one or more audio media content, video media content, and an other form of media content. For example, networked environment100may be synchronized with other sensory experiences such as, for example, one or more smoke effects, water droplets, moving chairs, and the like. For example, more than one client-side playback device125may be used simultaneously.

As explained below in greater detail, particularly inFIG.8, client-side synchronization application147is configured to perform one or more of sampling and recording a client-side running media play time (CRMPT) at which the client-side media player plays the media on the client. The CRMPT is defined as an elapsed running time for customized media content that is being played by the client-side media player on the client. If no customized media content is being played by the client-side media player, the CRMPT is defined as zero. The CRMPT recorded by client-side synchronization application147represents a real world time value based on the host system clock of the client. Then, client-side synchronization application147creates a client-side message or packet that it transmits to server-side networked environment110.

Server-side networked environment110comprises server-side data storage165, server-side computing device170that is operably connected with server-side data storage165, and server-side playback device175that is operably connected with server-side data storage165. Server-side data storage165is a second location where, as mentioned above in relation to client-side data storage135, client-side customization application145may store the playable media content.

Server-side data storage165comprises one or more of item prices, order information, media content, and other information. For example, the media content files comprise one or more of one or more audio track, video track, an other media track, motion picture, commercial, motion picture trailer, demonstration (“demo”), commentary, extra content, and an other form of additional content. The media content comprises one or more media data, media content files, and other media content. The motion picture comprises one or more of one or more feature-length theatrical production, short-film production, animated production, broadcast television production, pay television production, documentary, commercial, trailer, and an other motion picture. The media data comprises one or more of one or more audio track, multi-channel track, commentary, and other media data. The audio track comprises one or more of one or more English language audio track, audio track in a language other than English, and customized audio track. The commentary comprises one or more commentary by one or more director of a motion picture, commentary by one or more actor in a motion picture, commentary by contributors to a motion picture other than the directors and actors, and commentary by persons other than contributors to a motion picture.

Server-side data storage165comprises one or more single database, multiple database, cloud application platform, relational database, no-sequel database, flash memory, solid state memory, and an other server-side data storage device. Server-side data storage165may be located in a single installation that may be local to client-side device120. Alternatively, server-side data storage165may be located in a single installation that may be local to server-side computing device170. Alternatively, server-side data storage165may be distributed in a plurality of locations. Server-side data storage165may be distributed in a plurality of geographical locations. Server-side data storage165may be distributed in a plurality of geographical locations located in the same time zone. Server-side data storage165may be distributed in a plurality of geographical locations, wherein not all the geographical locations are located in the same time zone.

Server-side computing device170comprises one or more server, computer, cloud-computing device, and distributed computing system.

Server-side computing device170may be located in a single installation. Alternatively, server-side computing device170may be distributed in a plurality of geographical locations. For example, server-side computing device170may be distributed in a plurality of geographical locations located in the same time zone. For example, server-side computing device170may be distributed in a plurality of geographical locations wherein not all the geographical locations are located in the same time zone.

Server-side playback device175is configured to play media content. For example, server-side playback device175plays media content received from server-side computing device170. Alternatively, or additionally, server-side playback device175plays media content received directly over network115. For example, server-side playback device175comprises one or more of one or more in-venue projector175, home theater175, television175, monitor175, three-dimensional digital projector175, and an other device175.

Server-side playback device175operates in an environment with speaker system127. For example, server-side playback device175operates in an environment with one or more of one or more channel-based speaker system127, object-based speaker system127, scene-based speaker system127, 3d audio speaker system127, and an other speaker system127environment. For example, server-side playback device175operates in one or more of single-channel speaker system127and multi-channel speaker system127environment. For example, single-channel speaker system127and multi-channel speaker system127may comprise one or more location of one or more speakers in one or more designed position and zone within a physical space. For example, server-side playback device175may operate in an object-based speaker environment. For example, the object-based speaker system127may contain one or more of an object (e.g. sound sources) that contains metadata describing the intended object position and other spatial properties. For example, server-side playback device175may operate in a scene-based speaker system127environment. For example the, the scene-based speaker system127may use one or more of spherical harmonic basis function that describes how the sound pressure in media content changes as a function of time and direction. For example, server-side playback device175operates in an environment with one or more speaker127. For example, speaker system127may comprise one or more of one or more woofer, such as to cover low-frequency range, tweeter, such as to cover high-frequency range, and an other audio transducer.

Server-side playback device175is configured to communicate with server-side computing device170. For example, server-side playback device175communicates with server-side computing device170using one or more of one or more satellite, antenna, cable, network115, and an other communication method. Server-side playback device175comprises one or more of one or more digital projector175, hologram projector175, screen175, television175, monitor175, cellular phone175, laptop computer175, notebook175, tablet175, headset175, multi-channel playback system175, and an other server-side playback device175.

Server-side computing device170comprises server-side memory180. Server-side memory180is defined herein as including both volatile and nonvolatile memory and data storage components. For example, server-side memory180comprises one or more server-side buffers. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon loss of power. For example, server-side memory180may comprise one or more random access memory (RAM), read-only memory (ROM), hard disk drive, solid-state drive, USB flash drive, memory card, floppy disk, optical disc such as compact disc (CD) or digital versatile disc (DVD), magnetic tape, and other memory components. For example, RAM may comprise one or more static random access memory (SRAM), dynamic random access memory (DRAM), magnetic random access memory (MRAM), and other forms of RAM. For example, ROM may comprise one or more programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and other forms of ROM.

The server-side computing device170comprises one or more of server-side master application185, server-side customization application190, and server-side streaming application195. Server-side master application185is configured to provide synchronization timing information to one or more of client-side synchronization application147, and server-side streaming application195.

Optionally, server-side computing device170further comprises an other server-side application (not pictured). The other server-side application comprises one or more of additional server-side application, additional server-side service, additional server-side process, and additional server-side functionality.

For example, the other server-side application runs background services. For example, the other server-side application runs boot processes. For example, the other server-side application runs other server-side applications.

As explained below in greater detail, particularly inFIGS.9, server-side master application185is configured to perform one or more of sampling and recording server-side running media play time (SRMPT) at which the server-side media player plays the media on the server. The SRMPT is defined as an elapsed running time for customized media content that is being played by the server-side media player on the server. If no customized media content is being played by the server-side media player, the SRMPT is defined as zero. For example, the motion picture's SRMPT time might clock in at 6 minutes, 10 seconds, and 10 frames. The SRMPT recorded by server-side master application185represents a real world time value based on the host system clock of the server. Then, server-side master application185creates a server-side message or packet that it transmits to client-side synchronization application147.

As explained below in greater detail, particularly inFIG.10, server-side master application185is configured to store media content. For example, the server-side master application185stores playable media content in server-side data storage165. The playable media content comprises one or more segmented media content track, non-segmented media content track, audio stimulus, and an other playable media content. Optionally, server-side master application185performs media processing of the playable media content. For example, the server-side master application185passes the playable media content through a pattern match algorithm to cross-correlate audio data within one or more frame and sequence of frames of playable media content to one or more ith speaker via one or more microphone system. For example, the server-side master application185may determine delay-processing time of this algorithm and adjust the server-side playback to compensate.

Server-side customization application190is configured to store playable media content to be played by server-side playback device175. For example, server-side customization application190stores the playable media content in server-side data storage165. Optionally, server-side customization application190performs media processing of the playable media content. For example, server-side customization application190passes the playable media content through one or more bandpass transfer function, compensation filters, equalization filters, and an other DSP algorithm, to regularize audio signals, and adjust one or more timber, spectral cue, localization, and an other attribute of sound. For example, server-side customization application190parses the playable media content into a chronological sequence that substantially matches the sequence of the motion picture. For example, server-side customization application190writes the playable media content to one or more server-side data storage165and server-side memory180. For example, server-side customization application190writes the playable media content to a media content file located in one or more server-side data storage165and server-side memory180.

Server-side streaming application195segments media content for deployment via network115to client-side device120. Server-side streaming application195supports multiple alternate data streams, two or more of which can have different bit rates from each other. Server-side streaming application195also allows for client-side device120to switch streams intelligently as network bandwidth changes. Server-side streaming application195also provides for media encryption and user authentication over encrypted connections.

Speaker system127may receive audio via one or more server-side computing device170, network115, an analogue connection, and a digital connection. For example, speaker systems127may be one or more actively and passively used with amplifiers.

Microphone system128, may receive audio via speaker system127. Microphone system128may be placed on the wall, on the speaker, or within the venue environment.

Input and output (1/0) device198may receive audio via one or more directly from server-side computing device170, over the network115, an analogue connection, and a digital connection. For example, input and output (1/0) device198may perform one or more digital to analogue and analogue to digital conversions. For example, input and output (1/0) device198may route audio to one or more of one or more amplifier, speaker and an other electronic device.

FIG.2is a schematic block diagram of server-side computing device170in an alternative configuration of a networked environment for real-time synchronization of media content via one or more of one or more electronic device and speaker system.

Server-side computing device170comprises one or more server-side data storage165, server-side playback device175(not pictured), server-side memory180, server-side processor210, and server-side local interface220. Server-side local interface220is operationally connected with one or more server-side data storage165, server-side memory180, and server-side processor210. Server-side memory comprises one or more server-side master application185, server-side customization application190, and server-side streaming application195. For example, server-side processor210comprises a server-side computer. For example, server-side local interface220comprises a bus. For example, server-side local interface220comprises a bus and further comprises one or more of an accompanying address/control bus or other bus structure.

Software components stored in one or more server-side memory180and server-side data storage165are executable by server-side processor210. In this respect, the term executable means a program file that is in a form that can ultimately be run by server-side processor210. For example, a compiled program is executable if it may be translated into machine code in a format that can be loaded into a random access portion of server-side memory180and run by server-side processor210. For example, source code is executable if it may be expressed in a proper format, such as object code, that may be loaded into a random access portion of server-side memory180and run by server-side processor210. For example, source code is executable if it may be interpreted by another executable program to generate instructions in a random access portion of server-side memory180and run by server-side processor210. An executable program may be stored in one or more portions or components of server-side memory180. For example, server-side memory180comprises one or more random access memory (RAM), read-only memory (ROM), hard disk drive, solid-state drive, USB flash drive, memory card, floppy disk, optical disc such as compact disc (CD) or digital versatile disc (DVD), magnetic tape, and other memory components.

One or more data and components stored in one or more server-side memory180and server-side data storage165are executable by server-side processor210. For example, server-side processor210can execute one or more server-side master application185, server-side customization application190, and server-side streaming application195.

For example, as an alternative to the setup inFIG.1with server-side data storage165separate from server-side computing device170, server-side data storage165may be located in server-side computing device170. For example, server-side data storage165may be located in server-side memory180.

Server-side processor210comprises one or more processors. Server-side memory180comprises one or more memories. For example, server-side memory180comprises at least one memory configured to operate in a parallel processing circuit. In such a case, server-side local interface220may serve as network115. For example, server-side local interface220may facilitate communication between two processors. For example, server-side local interface220may facilitate communication between a processor and a memory. For example, server-side local interface220may facilitate communication between two memories. Server-side local interface220may comprise additional systems designed to coordinate this communication. For example, server-side local interface220may comprise a system to perform load balancing. Server-side processor210may comprise an electrical processor. Alternatively, or additionally server-side processor210may comprise a non-electrical processor.

Any logic or application described herein, including but not limited to server-side master application185, server-side customization application190, and server-side streaming application195that comprises software or code can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as, for example, server-side processor210in a computer system or other system. In this sense, the logic may comprise, for example, statements including instructions and declarations that can be fetched from the computer-readable medium and can be executed by the instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system. For example, the computer-readable medium may comprise one or more RAM, ROM, hard disk drive, solid-state drive, USB flash drive, memory card, floppy disk, optical disc such as a CD or a DVD, magnetic tape, and other memory components. For example, RAM may comprise one or more SRAM, DRAM, MRAM, and other forms of RAM. For example, ROM may comprise one or more PROM, EPROM, EEPROM, and other forms of ROM.

FIG.3is a flowchart of a method300for real-time synchronization of stock media content.FIG.3applies to the baseline method viewed from the client side.

The order of the steps in the method300is not constrained to that shown inFIG.3or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

According to this method, to which we shall refer in shorthand as the baseline method, the client-side device performs the synchronization of the client-side playback with the server-side playback.

In block305, a client-side synchronization application receives a command to begin playback from the user. In another embodiment, the command to begin playback may be triggered by a pre-determined schedule based on one or more of one or more geographical location, time zone, system clock of server-side computing device, system clock of an external electronic device and client-side device. The stock media content may be customized and played on a client-side playback device in synchronization with server-side playback of the stock media content by a server-side master application. Block305then transfers control to block330.

Next, in block330, the client-side synchronization application creates a client-side message or packet. The client-side message or packet comprises one or more of a client-side unique identifier and a client-side start host time (CSHT). The client-side unique identifier uniquely identifies the client-side device. The CSHT is defined as a host time at which transmission of a server-side message or packet is requested on the client side. For example, the CSHT may comprise the time at which client-side playback starts. For example, the client-side device comprises one or more of a tablet, a notebook computer, a laptop computer, a cellular phone, and another client-side computing device. For example, the client-side unique identifier comprises an alphanumeric string.

For example, the client-side synchronization application communicates over a network so as to obtain server-side timing information from the server-side master application. For example, to communicate with the server-side master application, the client-side synchronization application browses the network. The client-side synchronization application then resolves the server-side master application. The client-side synchronization application then connects via the network to the server-side master application. Further, one or more of a server-side clock and an external clock may be used to synchronize the server-side master application with the client-side synchronization application. Block330then transfers control to block335.

Next, in block335, the client-side synchronization application sends the client-side message or packet to the server-side master application. Optionally, to facilitate error detection, a client-side transmitted error code can be added to messages or packets sent from the client-side to the server-side. To facilitate verification of data authenticity, the client-side synchronization application saves the client-side message or packet to one or more of client-side memory and client-side data storage. Block335then transfers control to block340.

Next, in block340, the client-side synchronization application waits for a pre-determined time period for a valid response from the server-side master application. If a valid response is received within the pre-determined time period, the process continues to step360. If the client-side synchronization application does not receive a valid response from the server-side master application within a pre-determined time period, the request times out, and the process loops back to step330.

Next, in block360, the client-side synchronization application receives and processes the server-side message or packet and substantially synchronizes one or more clocks in the client-side synchronization application with one or more clocks in the server-side master application. Further details of how to perform step360are given inFIG.8. Optionally, an additional step may be inserted here of relaying to the user, by the client-side device, one or more offers to purchase customized goods prepared by the server-side computing device using the user's customization preferences. For example, the customized goods comprise one or more of customized physical goods, customized services, and customized media content. Block360then transfers control to block365.

Next, in block365, the client-side synchronization application receives and processes one or more head-related impulse response (HRIR) signal, head-related transfer function (HRTF) signal, reverberant environment binaural room impulse response (BRIR) signal, numerically simulated signal, and other binaural signal utilizing one or more of visual sensor, thermal sensor, vibratory sensor, position sensor, and an other sensor. For example, the client-side synchronization application may determine delay-processing time of the (HRIR), (HRTF), (BRIR), and numerically simulated signal and substantially synchronize one or more clocks in the client-side synchronization application with one or more clocks in the server-side master application. Block365then transfers control to block370.

Next, in block370, the client-side customization application receives and processes one or more bandpass transfer function, headphone transfer function (HpTF), compensation filters, equalization filters, and an other DSP algorithm. For example, the client-side customization application may determine delay-processing time of the (HpTF), and substantially synchronize one or more clocks in the client-side synchronization application with one or more clocks in the server-side master application. Block370then transfers control to block375.

Next, in block375, the client-side synchronization application begins playback of the customized media content in synchronization with playback of the stock media content by a server-side playback device. Block375then terminates the process.

FIG.4is a flowchart of a method for initializing server-side master application for playing stock media content in a server-side computing device.FIG.4applies to the baseline and live methods viewed from the server side.

The order of the steps in the method400is not constrained to that shown inFIG.4or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

In block405, a server-side master application establishes a network service. For example, a network service may comprise of publishing one or more of a Domain Name System (DNS), setting up a valid TCP listening socket or a UDP socket, and initializing the service with a name, type, domain and port number. Block405, then transfers control to block410.

In block410, a server-side master application receives and processes channel-based, object-based, scene-based, 3d audio, and audio layout files with audio routing settings. The audio layout file may be received at the time when the server-side master application is initiated or installed. Alternatively or additionally, the audio layout file settings may be received from the server-side data store. Further, the audio layout file may comprise of one or more of discrete channels mapped to a speaker system located in a venue, public, office, school, home and an other environment. For example, one or more of discrete channel may be assigned a unique identifier of one or more of audio tracks in a one or more of stock media content. For example, the stock media content may be one or more of a mono, stereo and multichannel tracks. For example, the unique identifier may be matched to one or more of spherical coordinates of one or more of speaker systems located in a venue, public, office, school, home, and an other environment. For example, the origin of the spherical coordinate is determined from one or more of the center of a room, venue, public, office, school, home, and an other environment. For example, the spherical coordinates may be defined by a radius, an azimuthal angle in mathematical positive orientation when measured counter-clockwise and zenith angle with 0° pointing to the equator and +90° pointing to the North Pole. Alternatively or additionally, a non-spherical coordinate system may be used. Block410then transfers control to block415.

Next, in block415, the server-side master application searches for an input and output device (1/0) identifier over the network. Block415then transfers control to block420.

Next, in block420, the server-side master application processes and stores into server-side memory the input and output device (1/0) identifier. Block420then transfers control to block425.

Next, in block425, the server-side master application opens one or more of a stock media content file from one or more of server-side data storage and server-side memory. For example, a server-side master application evaluates and determines if the stock media content file is encoded. If the stock media content file is encoded, then the server-side master application processes and decodes the encoded stock media content file using one or more encoder matrix, decoder matrix, numerical computing algorithm, spatial audio renderer, and third party plug-ins. For example, a decoder matrix may generate one or more of speaker signals for a specific venue arrangement. Block425then transfers control to block430.

Next, in block430, the server-side master application processes the stock media content and determines data types. For example, data types may comprise of one or more of audio sample rates, bits per channel, number of tracks and the like. Block430then transfers control to block435.

Next, in block435, the server-side master application, using an input and output (1/0) device identifier and the audio layout file, evaluates and maps the stock media content to discrete channels. For example, the server-side master application may map as many channels as the server-side computing device's memory, file system and CPU allows. Block435then transfers control to block440.

Next, in block440, the server-side master application initiates playback by a server-side playback device of the stock media content to one or more of the discrete channels of the speaker systems. Alternatively or additionally, the server-side master application may use one or more of a digital signal-processing algorithm to process the stock media content playback. For example, digital signal processing algorithms may comprise one or more of compression, reverb and EQ. Block440then terminates the process.

FIG.5is a flowchart of a method500for real-time synchronization of stock media content.FIG.5applies to the baseline method viewed from the server side.

The order of the steps in the method500is not constrained to that shown inFIG.5or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

The server-side master application initiates playback by a server-side playback device of the stock media content. The client-side device performs the synchronization of the client-side playback with the server-side playback.

In block505, a server-side master application receives and processes from a client-side device under a user's control a client-side message or packet comprising one or more of a client-side unique identifier and a client-side start host time (CSHT). Block505then transfers control to block545.

Next, in block545, the server-side master application creates a server-side message or packet. For example, the server-side message or packet comprises one or more of a server-side unique identifier, the CSHT, a server-side start host time (SSHT), a server-side end host time (SEHT), and a server-side running media play time (SRMPT). The server-side unique identifier comprises a copy of the client-side unique identifier.

Optionally, the server-side master application may use the SRMPT to control General Purpose Input and Output peripherals (GPIO). For example, GPIO may control one or more of motion chairs, projectors, venue lightings, security cameras, masking systems, and wind, liquid, fire and odor effects and the like. Block545then transfers control to block555.

Next, in block555, the server-side master application transmits the server-side message or packet to the client-side synchronization application, to be used by the client-side synchronization application, along with previously received stock media content selected by a user to be customized, and customized media content previously created using customization preferences obtained from a user, to substantially synchronize in real-time, using the CSHT, client-side playback of the customized media content with server-side playback of the stock media content. Block555then transfers control to block575.

Next, in block575, the server-side master application begins playback of the stock media content in synchronization with playback of the customized media content by a client-side playback device. Block575then terminates the process.

FIG.6is a flowchart of a method600for real-time synchronization of stock media content.FIG.6applies to the live method viewed from the client side.

The order of the steps in the method600is not constrained to that shown inFIG.6or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

According to this method, which covers the live method, the server-side streaming application streams the media content for direct playback using a substantially immediate streaming technology. For example, the server-side streaming application may use one or more HpTF, HRIR, HRTF, BRIR, numerically simulated signal, an other signal, and an other algorithm to process the media content. For example, the streaming technology comprises user datagram protocol (UDP). According to the live method, there is no need for synchronization of the client-side playback and the server-side playback, because the playback is substantially immediate.

In block605, the client-side synchronization application receives a play command from the server-side master application to play the customized media content. Block605then transfers control to block610.

Next, in block610, the client-side synchronization application processes the command to begin playback of the customized media content and a client-side playback device begins a playback, using the server-side streaming application, of the customized media content that is substantially synchronized with a server-side playback device's playback of the stock media content. Block610then terminates the process.

FIG.7is a flowchart of a method700for real-time synchronization of stock media content.FIG.7applies to the live method viewed from the server side.

The order of the steps in the method700is not constrained to that shown inFIG.7or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

The server-side master application initiates playback by a server-side playback device of the stock media content.

In block705, the server-side streaming application live streams the customized media content to the client-side synchronization application. For example, the server-side streaming application live streams the customized media content using user datagram protocol (UDP).

FIG.8is a flowchart of a method for receiving and processing a server-side message or packet and synchronizing one or more clocks in a client-side synchronization application with one or more clocks in a server-side master application.FIG.8provides more detail regarding sub-steps of step360inFIG.3for the baseline method.

The order of the steps in the method800is not constrained to that shown inFIG.8or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

In block805, the client-side synchronization application receives a server-side message or packet from the server-side master application. The server-side message or packet comprises one or more of a server-side unique identifier, a client-side start host time (CSHT), a server-side start host time (SSHT), a server-side end host time (SEHT), and a server-side running media play time (SRMPT). Optionally, the server-side message or packet further comprises a server-side transmitted error code. Error code algorithms can be used to perform error detection. For example, one or more of a checksum and a longitudinal redundancy check can be used to perform the error check. Optionally, one or more server-side messages or packets is encrypted. Optionally, one or more client-side messages or packets is encrypted. Block805then transfers control to block810.

Next, in block810, the client-side synchronization application reads the server-side message or packet into one or more client-side buffers. That is, the client-side synchronization application breaks the server-side message or packet into one or more pieces, with each piece corresponding to a data type. Then at least one of the one or more pieces is stored by the client-side synchronization application in a client-side buffer. For example, the server-side unique identifier may be stored in a first client-side buffer. For example, the CSHT may be stored in a second client-side buffer. For example, the SSHT may be stored in a third client-side buffer. For example, the SEHT may be stored in a fourth client-side buffer. For example, the SRMPT may be stored in a fifth client-side buffer. For example, the server-side transmitted error code may be stored in a sixth client-side buffer. For example, other components of the server-side message or packet may be stored in other client-side buffers. For example, some of the client-side buffers may be coterminous. For example, all the client-side buffers may be coterminous. Block810then transfers control to block815.

Next, in block815, the client-side synchronization application records a CEHT. The CEHT is defined as a host time at which transmission of a server-side message or packet ends on the client side. For example, the CEHT may comprise the time at which server-side playback stops. The CEHT is the world value time that the client-side synchronization application records based on the client-side synchronization application's host time.

For example, the time measurement may comprise a timestamp. For example, the real world time may comprise a seconds field representing an integral number of seconds. For example, the real world time may further comprise a microseconds field representing an additional number of microseconds elapsed in addition to the integral number of seconds. For example, the host time measurement represents the number of seconds elapsed since the start of the UNIX epoch at midnight Coordinated Universal Time (UTC) Dec. 31, 1969-Jan. 1, 1970. Block815then transfers control to block820.

Next, in block820, the client-side synchronization application evaluates the integrity of the server-side message or packet.

Preferably, but not essentially, the client-side synchronization application evaluates the integrity of the server-side message or packet by comparing the CEHT and the SEHT. If the CEHT is substantially equal to the SEHT, the client-side synchronization application verifies integrity of the server-side message or packet. If the CEHT is not substantially equal to the SEHT, the client-side synchronization application verifies a lack of integrity of the server-side message or packet.

Then the client-side synchronization application compares the client-side received error code with the server-side transmitted error code received in the server-side message or packet.

Optionally, or additionally, the client-side synchronization application evaluates the integrity of the server-side message or packet by calculating a server-side received error code for the server-side message or packet, and then the client-side synchronization application compares the server-side received error code with the server-side transmitted error code comprised in the server-side message or packet.

If the calculated server-side received error code is substantially equal to the server-side transmitted error code, the client-side synchronization application verifies integrity of the server-side message or packet. If the calculated server-side received error code is not substantially equal to the server-side transmitted error code, the client-side synchronization application verifies a lack of integrity of the server-side message or packet.

Optionally, or additionally, the client-side synchronization application evaluates the integrity of the server-side message or packet by comparing the server-side unique identifier with the client-side device unique identifier.

If the server-side unique identifier is substantially equal to the client-side device identifier, the client-side synchronization application verifies integrity of the server-side message or packet. If the server-side unique identifier is not substantially equal to the client-side device identifier, the client-side synchronization application verifies a lack of integrity of the server-side message or packet.

If the client-side synchronization application verifies a lack of integrity of the server-side message or packet, the client-side synchronization application deletes the server-side message or packet and sends a message to the client-side synchronization application to request transmission of a replacement server-side message or packet. The process loops back to step805.

If the client-side synchronization application verifies the integrity of the server-side message or packet, the client-side synchronization application accepts the server-side message or packet. Block820transfers control to block835.

Next, in block835, using one or more of the CEHT, CSHT, SEHT, and SSHT, the client-side synchronization application then calculates a half-round-trip time. The round-trip time is equal to the time it takes a signal to leave from the client-side synchronization application, arrive at the server-side master application, be received by the server-side master application, be processed by the server-side master application, be retransmitted by the server-side master application, and arrive back at the client-side synchronization application, less the time it takes for the signal to be processed by the server-side master application. For example, the round-trip time RT may be calculated according to equation (1) as:
RT=(CEHT−CSHT)−(SEHT−SSHT).  (1)

Where the server-side processor is faster than a minimal time, for example, where the server-side processor is faster than a playback synchronizing error threshold time, the difference (SEHT minus SSHT) may be negligible. In these cases, the round-trip time RT may be calculated according to equation (2) as:
RT=CEHT−CSHT.  (2)

The client-side synchronization application then computes a half-round-trip time HRT by multiplying the round-trip time RT by 0.5, according to equation (3):
HRT=0.5*RT  (3)

Block835then transfers control to block840.

Next, in block840, the client-side synchronization application reads the value of SRMPT. For example, the client-side synchronization application reads from the fifth client-side buffer the value of the SRMPT. Block940then transfers control to block845.

Next, in block845, using one or more of CEHT, CSHT, SEHT, SSHT, SRMPT, and HRT, the client-side synchronization application calculates a playback offset time Tpo. The playback offset time TPo is defined as the time increment by which media playback at the client-side should be speeded up or slowed down relative to media playback at the server-side to synchronize the server-side media and the client-side media.

The playback offset time TPo may be defined as the timing difference between client-side playback and server-side playback. The playback offset time TPo may be calculated according to equation (4):
Tpo=HRT+BD,  (4)

where BD is buffer size delays caused by inadequate sizing of one or more of the client-side buffers. Buffer size delays may be safely ignored if they are sufficiently small. Block845then transfers control to block850.

Next, in block850, using one or more of the computed playback offset time TPo and the SRMPT, the client-side synchronization application then computes a client-side running media play time (CRMPT). The CRMPT is defined as the elapsed running time for the media content that is being played by the media player on the client. For example, the client-side synchronization application calculates the CRMPT according to equation (5):
CRMPT=SRMPT+Tpo.  (5)

Alternatively, or additionally, the client-side synchronization application makes multiple requests in parallel to the server-side master application. Then, using one or more of statistical analysis and machine learning algorithms, the client-side synchronization application estimates the value of the CRMPT. Block850then transfers control to block855.

Next, in block855, using one or more of the computed CRMPT and the computed playback offset time Tpo, the client-side synchronization application synchronizes client-side playback of customized media content with server-side playback of stock media content. For example, the client-side synchronization application adjusts the playback to synchronize the client-side synchronization application to the client-side playback device. For example, the client-side synchronization application adjusts the playback to synchronize the client-side synchronization application to the client-side playback device by offsetting the playback time of the client-side media player by the playback offset time Tpo.

Alternatively, for example, a single system call can be used to calculate one or more of the CRMPT and playback offset time Tpo. Alternatively, multiple systems calls may be performed to improve accuracy. For example, accuracy may be improved via one or more of linear regression, a sum of averages, a technique for evaluating statistical significance, a machine learning algorithm, and the like.

One or more of automatic re-synchronization and manual re-synchronization may be performed as needed. For example, after a while the clock may drift, meaning that one or more of a stream and a download of the media content that starts out synchronized may gradually become de-synchronized. In such cases, resynchronization may be performed by one or more of manual execution and automatic execution of a re-synchronization of the client-side synchronization application in the client-side synchronization application with the server-side master application in the server-side master application.

The client-side synchronization application may request at random or at pre-determined time intervals the player's synchronization time on the server-side master application in the server-side computing device. If there is a difference of more than a pre-determined offset, the client-side device will vary the rate of the playback until the drift is corrected. In another embodiment, a digital signal-processing algorithm may be used to time compress or stretch the playback. Block855then terminates the process.

FIG.9is a flowchart of a method for receiving and processing a client-side message or packet and synchronizing one or more clocks in a client-side synchronization application with one or more clocks in a server-side master application.FIG.9provides more detail regarding sub-steps of step545inFIG.5for the baseline method.

The order of the steps in the method900is not constrained to that shown inFIG.9or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

In block905, the server-side master application receives a client-side message or packet from the client-side synchronization application. The client-side message or packet comprises one or more of a client-side unique identifier and a client-side start host time (CSHT). Optionally, the client-side message or packet further comprises a client-side transmitted error code. Error code algorithms can be used to perform error detection. For example, one or more of a checksum and a longitudinal redundancy check can be used to perform the error check. Block905then transfers control to block910.

Next in block910, the server-side master application reads the client-side message or packet into one or more server-side buffers. That is, the server-side master application breaks the client-side message or packet into one or more pieces, with each piece corresponding to a data type. Then at least one of the one or more pieces is stored by the server-side master application in a server-side buffer. For example, the client-side unique identifier may be stored in a first server-side buffer. For example, CSHT may be stored in a second server-side buffer. For example, the client-side transmitted error code may be stored in a third server-side buffer. For example, other components of the client-side message or packet may be stored in other server-side buffers. For example, some of the server-side buffers may be coterminous. For example, all the server-side buffers may be coterminous. Block910then transfers control to block915.

Next, in block915, the server-side master application records a server-side start host time (SSHT).

For example, the time measurement may comprise a timestamp. For example, the real world time may comprise a seconds field representing an integral number of seconds. For example, the real world time may further comprise a microseconds field representing an additional number of microseconds elapsed in addition to the integral number of seconds. For example, the host time measurement represents the number of seconds elapsed since the start of the UNIX epoch at midnight Coordinated Universal Time (UTC) Dec. 31, 1969-Jan. 1, 1970. Block915then transfers control to block920.

Next, in block920, the server-side master application evaluates the integrity of the client-side message or packet.

Preferably, but not essentially, the server-side master application evaluates the integrity of the client-side message or packet by comparing the CSHT and the server-side start host time (SSHT). If the CSHT is substantially equal to the SSHT, the server-side master application verifies integrity of the client-side message or packet. If the CSHT is not substantially equal to the SSHT, the server-side master application verifies a lack of integrity of the client-side message or packet.

Optionally, or additionally, the server-side master application evaluates the integrity of the client-side message or packet by calculating a client-side received error code for the client-side message or packet, and then the server-side master application compares the client-side received error code with the client-side transmitted error code comprised in the client-side message or packet.

If the calculated client-side received error code is substantially equal to the client-side transmitted error code, the server-side master application verifies integrity of the client-side message or packet. If the calculated client-side received error code is not substantially equal to the client-side transmitted error code, the server-side master application verifies a lack of integrity of the client-side message or packet.

If the server-side master application verifies a lack of integrity of the client-side message or packet, the server-side master application deletes the client-side message or packet and sends a message to the client-side synchronization application to request transmission of a replacement client-side message or packet. The process loops back to step905.

If the server-side master application verifies the integrity of the client-side message or packet, the server-side master application accepts the client-side message or packet. Block920transfers control to block925.

Next, in block925, the server-side master application records a server-side running media play time (SRMPT). If no customized media content is being played by the server-side media player, the SRMPT is set to zero. Block925then transfers control to block930.

Next, in block930, the server-side master application creates a server-side message or packet. The server-side message or packet comprises one or more of a server-side unique identifier, the CSHT, the SSHT, a server-side end host time (SEHT), and the SRMPT of the media being played. Optionally, to facilitate error detection, the server-side master application adds a server-side transmitted error code to one or more server-side messages it sends to the client-side synchronization application. Block930then terminates the process.

FIG.10is a flowchart of a method for receiving and processing a server-side message, packet, digital signal or analogue signal, and synchronizing one or more clocks in a server-side master application with one or more clocks in server-side playback device and speaker system.FIG.10applies to the baseline method and live method viewed from the server side.

The order of the steps in the method1000is not constrained to that shown inFIG.10or described in the following discussion. Several of the steps could occur in a different order without affecting the final result.

In block1005, the server-side master application is configured to calculate and record, by the server-side computing device, a server-side start host time (SSHT) of stock media content. Block1005then transfers control to block1010.

Next, in block1010, the server-side master application creates a server-side audio stimulus, the server-side audio stimulus comprising one or more of one or more speech, music, noise, frequency log sweep, sine sweep, and an other audio stimulus. Block1010then transfers control to block1015.

Next, in block1015, the server-side master application sends to one or more ith speaker, one or more of a server-side message, packet, digital signal, and analogue signal, comprising one or more server-side audio stimulus, to one or more ith speaker, via a communications network within or around venue, public, office, school and home environment. Block1015then transfers control to block1020.

Next, in block1020, the server-side master application reads and processes, via one or more microphone and an other receiver device, audio data within one or more of one or more frame and sequence of frames of media content, and cross-correlate, by a pattern match algorithm, the audio data. Block1020then transfers control to block1025.

Next, in block1025, the server-side master application reads and records a server-side end host time (SEHT) of media content for each ith speaker. Block1025then transfers control to block1030.

Next, in block1030, the server-side master application calculates, using one or more (SSHT) and (SEHT), a playback offset time TPo for the longest delay interval for each ith speaker. Block1030then transfers control to block1035.

Next, in block1035, the server-side master application evaluates and records the longest delay interval of each ith speaker. Block1035then transfers control to block1040.

Next, in block1040, the server-side master application calculates a playback offset time TPo for the longest delay interval of each ith speaker. Block1040then transfer control to block1045.

Next, in block1045, the server-side master application substantially synchronizes, using one or more of the SRMPT and the Tpo, the server-side playback of stock media content with and for each ith speaker.

Alternatively, the server-side master application may further adjust the playback offset time TPo by calculating system latencies introduced in each ith speaker feed, using one or more of bandpass transfer function, equalizer, and an other DSP algorithm. In addition, the pattern match algorithm may separate one or more of audience noise and an other noise from the signal captured by one or more microphone system. Block1040then terminates the process.

Advantages of the invention include the ability to individually track a potentially unlimited number of audience members using the unique identifiers comprised in their individual devices. Additionally, media player time is divided into different buffers to improve precision. For example, the minutes and seconds go into different buffers, and then into different packets.

While the above representative embodiments have been described with certain components in exemplary configurations, it will be understood those skilled in the art that other representative embodiments can be implemented using one or more of different configurations and different components. For example, it will be understood by those skilled in the art that the order of certain fabrication steps and certain components can be altered without substantially impairing the functioning of the invention.

For example, one or more of audio, video, and another entertainment format can be playing on the client-side. For example, one of more of audio, video, and another entertainment format can be played on the server-side.

For example, while this application for simplicity at times verifies that the media to be customized is a motion picture, embodiments of the invention are also applicable to silent pictures, video recordings of concerts, audio recordings of concerts, video recordings of interviews, audio recordings of interviews, and countless other media. For example, instead of being implemented by the client-side synchronization application147and the server-side master application185, the steps of the flowchart depicted inFIGS.3and5may be implemented by one or more of the server-side computing device170and the client-side device120.

For example, instead of being primarily implemented by the client-side customization application145, the server-side master application185, and the server-side customization application190, the steps of the flowchart depicted inFIGS.8and9may be implemented by one or more of the server-side computing device170and the client-side device120.

For example, instead of being implemented by the client-side customization application145, the server-side master application185, the server-side customization application190, and the server-side streaming application195, the steps of the flowchart depicted inFIGS.6and7may be implemented by one or more of the server-side computing device170and the client-side device120. For example, instead of being primarily implemented by the server-side master application185, the steps of the flowchart depicted inFIG.4may be implemented by one or more of the server-side computing device170and the client-side device120.

For example, instead of being located in the client-side memory130, one or more of the client-side customization application145, and the client-side synchronization application147may be located in a section of the client-side device120other than the client-side memory130.

For example, instead of being located in the server-side memory180, one or more of the server-side master application185, the server-side customization application190, and the server-side streaming application195may be located in one or more of the server-side data storage165and a section of the server-side computing device170other than the server-side memory180. For example, instead of being a freestanding component of the server-side networked environment110, the server-side memory180may be located in the server-side computing device170.

For example, the client-side data storage135may be separate from the client-side device120rather than being comprised in the client-side device120. For example, the server-side data storage165may be comprised in the server-side computing device170rather than being separate from the server-side computing device170.

For example, instead of being the client-side device120and the client-side playback device125being two separate entities, the client-side playback device125may be comprised in the client-side device120. For example, instead of the client-side customization application145and the client-side synchronization application147being two separate entities, the client-side synchronization application147may be comprised in the client-side customization application145. For example, instead of the client-side customization application145and the client-side synchronization application147being two separate entities, the client-side customization application145may be comprised in the client-side synchronization application147.

The representative embodiments and disclosed subject matter, which have been described in detail herein, have been presented by way of example and illustration and not by way of limitation. It will be understood by those skilled in the art that various changes may be made in the form and details of the described embodiments resulting in equivalent embodiments that remain within the scope of the invention. It is intended, therefore, that the subject matter in the above description shall be interpreted as illustrative and shall not be interpreted in a limiting sense.