Patent Description:
Many live scenes, such as concerts, television shows, ballets, theatre plays, operas, sport events, are broadcasted for a distant audience to enjoy both watching and listening to the show.

As light and sound do not travel at the same speed through air, they are often captured separately by means, respectively, of cameras and microphones. Video and audio signal are therefore processed separately. The hardware used to capture video and audio is not the same; the processing tools and method differ as well.

As far as video is concerned, in case of an analog camera, the analog signal is first digitized, then encoded to be streamed and/or broadcasted. In case of a digital camera, the signal is directly encoded to be streamed and/or broadcasted. Whichever the camera technology, processing (mostly coding) the generated video signal requires a certain period of time (called latency) which needs to be taken into account when broadcasting.

As far as audio is concerned, the audio signal s generated by various microphones are generally gathered and processed by a master mixing table where in the signals undergo several operations such as:.

A mixing table outputs processed, mixed audio data (generally stereo) which is subsequently synchronized with corresponding processed video data to generate a primary audio/video signal suitable for broadcasting. Synchronization is often provided by means of timestamps embedded both in the audio and video signals in order for them to be played simultaneously on a display system and on a sound system, respectively.

Despite the richness of the initial signals and the capabilities of the processing devices, the signal provided to the audience is remarkably poor, in the sense that it includes far less information than the raw captured signal.

The audio signal, in particular, usually includes arbitrary mixing choices from the sound engineer who runs the mixing table; that leaves the listener with only few settings such as volume, balance, and, depending upon the capabilities of the sound system, equalizing.

It is an object of the invention to enhance a spectator's experience during the broadcasting of a show.

It is another object of the invention to enhance the spectator's audio experience during the broadcasting of the show.

It is a further object of the invention to provide the audience with an audio signal that they may adapt to their preferences with unusual setting parameters.

It is known in the prior art document <CIT> an appliance for receiving and reading audio signals, which includes:
a receiver of high-frequency electromagnetic waves designed to capture radio broadcast signals including a plurality of audio tracks; a wireless communication interface, separate from the receiver and designed to receive at least one regulating parameter to be applied to at least one audio track of the captured signal; and a digital signal processor designed to apply the regulating parameter to the audio track and produce a mixed signal.

Also, <CIT> discloses a television receiver that receives an A/V signal from an event including a primary audio signal, and also receives a secondary audio signal of the same event such as a radio broadcast. The receiver synchronizes the secondary signal to the video signal and output the video signal with the secondary audio signal instead of with the primary audio signal.

In a preferred embodiment, as defined in claim <NUM>, the synchronizing step includes the operations of:.

Temporal comparison between the multichannel audio data from the secondary signal and the mixed audio data extracted from the primary audio/video signal may be achieved through a maximum likelihood estimation.

The method wherein the user managed conditioning settings are applied within a conditioning module and are obtained from a computerized user device, on which a virtual mixing table including virtual buttons for specifying a conditioning parameter for each channel of themultichannel audio data is displayed.

In a preferred embodiment, the userdevice provides the conditioning settings to the conditioning modulethrough a wireless link.

It is provided, according to a second aspect, as defined in claim <NUM>, an audio/Video playerfor producing and playing video and audio content, including:.

It is provided, according to a third aspect, an audio/video system including such an audio/video player, and a computerized user device providing the user managed conditioning settings, said computerized user device being implemented with a virtual mixing table including virtual buttons for specifying a conditioning parameter for each channel of the multichannelaudio data.

The user device is preferably in wireless communication with the audio/video player.

It is provided, according to a fourth aspect, an audio/video broadcasting network including:.

In a preferred embodiment, the audio/video broadcasting network further includes a computerized user device as presented hereinbefore.

The above and further objects and advantages of the invention will become apparent from the detailed description of preferred embodiments, considered in conjunction with the accompanying drawings.

The description of the present invention is made for an example of audio and video signals that come from a show, especially from live scenes. But the present invention doesn't exclude any other origin of audio and video signals, and is not limited to shows or live scenes.

On <FIG> is partly represented a recording studio <NUM> for producing video and audio data from, respectively, images and sound captured from a show <NUM> to be broadcasted (preferably live, although the show <NUM> may be broadcasted with a delay) to a distant audience including one or more viewers <NUM> (where the word "viewer" means a person having both eyesight and audition capabilities).

The show <NUM> takes place in a location called "scene" although this term should be interpreted in the broadest meaning, i.e. it might designate a room, a stage, a studio, an arena, a stadium, a hall, or even an open space. The show <NUM> involves a plurality of image and sound sources, such as musical instruments (here a drum set <NUM>, a guitar <NUM>, a bass guitar <NUM>), actors and/or singers <NUM>, speakers, players, animals, such list being non-exhaustive.

As can be seen on <FIG>, images from the show are captured by means of one or more cameras <NUM>, which may be of the analog or digital type, while sound from the show <NUM> is captured by means of several microphones <NUM>, <NUM>, <NUM>, <NUM> preferably at least one for each source <NUM>, <NUM>, <NUM>, <NUM>. One or more ambience microphones may be provided as well. For certain sources such as a drum set <NUM>, a plurality of microphones may be provided (e.g. one for each drum and/or for each cymbal).

Each camera <NUM> is connected to an audio/video mixing console <NUM> through a video link <NUM> which might be wired or wireless. The line linking the camera to the audio/video mixing console on <FIG> should be interpreted as representative of a wired or a wireless video link.

Images from the camera(s) <NUM> are timestamped, i.e. each image or group (or packet) of image is associated with a timestamp consisting ofa n incremental time information automatically generated by a clock, e.g. in HH:MM:SS format.

Each microphone <NUM>-<NUM> is connected to a master sound mixing console <NUM> through an audio link, which might be wired or wireless. The line linking each microphone <NUM>, <NUM>, <NUM>, <NUM> to the master sound mixing console <NUM> on <FIG> should be interpreted as representative of a wired or a wireless audio link.

The number of microphones <NUM>, <NUM>, <NUM>, <NUM> depicted on <FIG> should not be regarded as limitative. The number of microphones allowed on stage should be instead regarded as only limited by the number of tracks allowed to be connected to the master sound mixing console <NUM>, which may be equal to eight, twelve, twenty or even thirty-two (all those numbers non-limitative), depending upon the architecture or capabilities of the master sound mixing console <NUM>.

The master sound mixing console <NUM> is supposed to be managed by a sound engineer who is entitled to process the audio signal coming from the microphones <NUM>, <NUM>, <NUM>, <NUM> using one or more processing functions such as gaining, filtering, equalizing conditioning, mixing, digitizing to produce and output a mixed, master sound which, in most cases, is a stereo sound.

The mixed, master sound is timestamped, i.e. successive soundtracks or packets of the mixed master sound are associated with respective timestamps each consisting of an incremental information automatically generated by a clock, e.g. in HH:MM:SS format.

As depicted on <FIG>, the master sound mixing console <NUM> is linked to the audio/video mixing console <NUM> through an audio link <NUM> which might be wired or wireless. The line linking the master sound mixing console <NUM> to the audio/video mixing console <NUM> on <FIG> should be interpreted as representative of a wired or a wireless video link.

The images provided by the camera(s) <NUM> and the mixed, master sound are further processed within the audio/video mixing console <NUM> to produce a primary, audio/video signal A2/V including video data V and mixed audio data A2 synchronized (through the use of timestamps) with each other.

The primary audio/video signal A2/V is then broadcasted to the audience's premises, and more precisely to an audio/video player <NUM> which is described in detail hereinafter.

This broadcasting of the primary audio/video A2/V signal is preferably achieved through a primary, audio/video server <NUM> in network communication with the audio/video player <NUM> and configured to hostthe
primary, audio/video signal A2/V and to stream the primary audio/video signal A2/V to the audio/video player <NUM>.

As depicted on <FIG>, there is also provided in the studio <NUM> a slave sound mixing console <NUM> connected to the master sound mixing console <NUM>, through an audio link <NUM> which might be wired or wireless. The line linking the slave sound mixing console <NUM> to the master sound mixingconsole <NUM> on <FIG> should be interpreted as representative of a wiredor a wireless audio link.

The master mixing console <NUM> and the slave mixing console <NUM> are configured to respectively provide the slave sound mixing console <NUM> / receive from the master sound mixing console <NUM> a multichannel audiosignal. Should the multichannel audio signal be handed over by themaster sound mixing console <NUM> to the slave sound mixing console <NUM> through a single audio link <NUM>, all channels may be transferred by means of a time-division multiplexing technique. Should the signal be digital, the transfer might be achieved through packet transfer protocols (such as Internet Protocol or Real-time protocol).

The number of channels handed over to the slave sound mixing console <NUM> by the master sound mixing console <NUM> may be equal to the number of channels inputted in the master sound mixing console <NUM>. In certain show configurations, this number might be equal to the number of microphones <NUM>, <NUM>, <NUM>, <NUM> (or sources <NUM>, <NUM>, <NUM>, <NUM>). However, in case of a rock concert for example, where the drum set <NUM> is equipped with a series of microphones each linked to the master sound mixing console <NUM>, the latter might hand over to the slave mixing console <NUM> a single drum channel resulting from a sound mixing achieved on the master sound mixing console <NUM> with all tracks coming from the drum set <NUM>.

In one embodiment, the number of channels handed over to the slave sound mixing console <NUM> is equal to eight.

As the multichannel audio signal may have been conditioned at the master sound mixing console <NUM>, it is preferably further processed within the slave sound mixing console <NUM>, in order to apply predefined settings thereto, such as equalizing all channels to provide, in output, a secondary, audio signal including multichannel audio data A8 to be broadcasted to the audience's premises.

As depicted on the drawings, the broadcasting of the secondary audio signal (or the multichannel audio data A8) is preferably achieved through a secondary, audio server <NUM>, separate from the primary audio/video server <NUM>, in network communication with the audio/video player <NUM> and configured to host the secondary audio signal (i.e. the multichannel audio data A8) and to stream this secondary audio signal to the audio/video player <NUM>.

The audio/video player <NUM> is configured to produce and play video and audio content. The audio/video <NUM> player may be in the form of a box, linked to the servers <NUM>, <NUM> through a communication network such as a metropolitan area network (MAN) or even a wide area network (WAN), for example through Internet.

The audio/video player <NUM> includes a decoder module <NUM>configured to receive the primary audio/video signal A2/V from theprimary audio/video server <NUM> and to extract the mixed audio data A2 from the primary audio/video signal A2/V. It should be noted that the extracted mixed audio data A2 is timestamped (and thus might be played in synchronicity with the video data V from the primary audio/video signal).

The decoder module <NUM> may have the form of a computer chip of board, or it might have the form of a software module implemented on a shared processor.

The audio/video player <NUM> further includes a synchronizing module <NUM> configured to receive the multichannel audio signal A8 from thesecondary audio server <NUM>, to receive the mixed audio data A2 from thedecoder module <NUM> and to synchronize the multichannel audio data A8 from the secondary signal with the mixed audio data A2 extracted from the primary audio/video signal A2/V.

The synchronizing module <NUM> may have the form of a computer chip or board, or it might have the form of a software module implemented on a shared processor.

The synchronizing of the multichannel audio data A8 with the mixed audio data A2 may include the following operations:.

Temporal comparison between the multichannel audio data A8 and the mixed audio data A2 may be achieved through a maximum likelihood estimation.

For more details on audio signals synchronization, the skilled person may refer to the following documents, herewith incorporated by reference:.

Alignment the multichannel audio data A8 and the mixed audio data A2 is achieved through control commands CTRL such as forward/back, applied to the multichannel audio data A8 by the synchronizing module <NUM> back to the secondary audio server <NUM>.

The audio/video player <NUM> further includes a conditioning module <NUM> configured to:.

The display system <NUM> may be or include a screen, and the sound system <NUM> may be or include one or more loudspeakers. In a particular embodiment, the display system <NUM> and sound system <NUM> are integrated within a same such as a TV set.

In consequence, the conditioned audio signal A2', synchronized to the video data V, is played on the sound system in lieu of the mixed audio data A2 which is withdrawn from the primary audio/video signal A2/V and simply not played.

The conditioning module <NUM> may have the form of a computer chipo r board, or it might have the form of a software module implementedon a shared processor.

To determine the conditioning settings, there is provided a computerized (preferably mobile) user device <NUM> implemented with a virtual mixing table <NUM> including virtual buttons <NUM> for applying a conditioning parameter to each channel of the multichannel audio data A8.

The user device <NUM> may be a Smartphone, a tablet or even a computer, linked to the audio/video player (and more precisely to the conditioning module) through a network link <NUM>. This network link <NUM> may be wired, but it is preferably wireless, for example through a standard wireless protocol such as Wifi or Bluetooth®.

In a preferred embodiment, the virtual mixing table <NUM> includes a virtual button <NUM> for each channel of the multichannel audio data A8. In the depicted example, the virtual mixing table <NUM> includes eight virtual buttons <NUM> (in the form of sliding buttons <NUM>), one for each channel of an eight channel audio data A8.

Conditioning settings might include, for each channel, at least a sound level (or volume). The conditioning settings are introduced manually by the viewer <NUM> within the virtual mixing table <NUM> (and are hence relayed to the conditioning module <NUM>) while he/she is listening to the conditioned audio signal A2' played on the sound system <NUM>.

The time delay in which conditioning settings are relayed to the audio/video player <NUM> through the network link <NUM> and subsequently applied to the multichannel audio data A8 is imperceptible to the viewer <NUM>, who has therefore the sensation of an immediate (in other words, real-time) modification to the conditioned audio signal A2' played on the sound system <NUM> according to his/her settings.

Together, the audio/video player <NUM> and the user device <NUM> form an audio/video system <NUM> located in the viewer's premises.

Together, the audio/video player <NUM>, the primary audio/video server <NUM> and the secondary audio server <NUM> form an audio/video broadcasting network <NUM>, in which the user device <NUM> may be regarded as included.

Producing and playing video and audio content is achieved as follows.

There is provided the primary audio/video signal A2/V including the video data V and the mixed audio data A2 synchronized with the video data V. As disclosed hereinbefore, the primary audio/video signal A2/V is hosted on the primary audio/video server <NUM> to be broadcasted to the audio/video player <NUM> located at the viewer's premises.

There is also provided the secondary audio signal including multichannel audio data A8. As disclosed hereinbefore, the secondary audio signal (or the multichannel audio data A8) is hosted on the secondary audio server <NUM> to be broadcasted to the audio/video player <NUM>.

As the primary audio/video signal A2/V is received by the audio/video player <NUM>, the mixed audio data A2 is extracted therefrom by the decoder module <NUM>, and handed over to the synchronizing module <NUM>.

The multichannel audio data A8 received by the audio/video player <NUM> is synchronized by the synchronizing module <NUM> with the mixed audio data A2 extracted from the primary audio/video signal A2/V, as disclosed hereinbefore.

Conditioning settings are then applied to the synchronized multichannel audio data A8 by the conditioning module <NUM> as inputted by the viewer <NUM> in the virtual mixing table <NUM> and relayed by the user device <NUM>, to produce the conditioned audio signal A2', which remains synchronized with the video data V.

The conditioning module <NUM> then hands:.

Accordingly, the conditioned audio data A2', including the settings chosen in real-time by the viewer <NUM> through his/her virtual mixing console <NUM> available on the user device <NUM>, replaces the mixed audio data A2 initially broadcasted within the primary audio/video signal A2/V, whereby the viewer <NUM> is capable of adapting in real-time the conditioned audio data A2' played on his/her sound system <NUM> according to his/her preferences, even with unusual setting parameters.

Claim 1:
Method for producing and playing video and audio content, said method including the steps of:
• providing a primary, audio/video signal (A2/V) including video data (V) and mixed audio data (A2) synchronized with the video data (V);
• providing a secondary, audio signal including multichannel audio data (A8);
the primary audio/video signal and the secondary audio signal being captured from a show taking place in a scene and being to be broadcasted to at least one viewer's premises located distant to the scene, said primary audio/video signal and said secondary audio signal originating from a same recording source;
• receiving in said at least one viewer's premises the primary audio/video signal and extracting the mixed audio data (A2) from the primary audio/video signal (A2/V);
• receiving in said at least one viewer's premises the secondary audio signal and synchronizing the multichannel audio data (A8) from the secondary signal with the mixed audio data (A2) extracted from the audio/video primary signal (A2/V);
• applying user managed conditioning settings to the synchronized multichannel audio data (A8) to produce conditioned audio data (A2') synchronized to the video data (V);
• simultaneously playing:
∘ on a display system (<NUM>), only the video data (V) from the primary signal (A2/V);
∘ on a sound system (<NUM>), only the conditioned audio data (A2') synchronized with the video data (V), such that the conditioned audio signal (A2'), synchronized to the video data (V), is played on the sound system in lieu of the mixed audio data (A2) which is withdrawn from the primary audio/video signal (A2/V) and not played.