AUDIO PROCESSING

An apparatus comprising means for:   classifying an audio signal as at least a first audio signal for time synchronization with a video signal or a second audio signal not for time synchronization with a video signal;   processing the first audio signal for time synchronization with the video signal wherein the processing introduces a first delay for time synchronization with the video signal;   processing the second audio signal wherein the processing introduces a second delay shorter than the first delay.

TECHNOLOGICAL FIELD

Embodiments of the present disclosure relate to audio processing. Some relate to audio processing of an audio signal that is contemporaneous with a video signal.

BACKGROUND

In some applications audio and video are rendered contemporaneously.

A passive viewer of television, or some other audio-visual application, has a better experience if the video and its associated audio are synchronized.

If the television content comprises an interviewer and an interviewee, who are communicating live over a satellite link, then the return delay introduced by the satellite link can result in over-talking.

Similar problems can arise in teleconferences and with live streaming.

BRIEF SUMMARY

According to various, but not necessarily all, embodiments there is provided an apparatus comprising means for: classifying an audio signal as at least a first audio signal for time synchronization with a video signal or a second audio signal not for time synchronization with the video signal; processing the first audio signal for time synchronization with the video signal wherein the processing introduces a first delay for time synchronization with the video signal; processing the second audio signal wherein the processing introduces a second delay shorter than the first delay.

In some but not necessarily all examples, the apparatus comprises a first path comprising a first audio coder and a second path comprising a second audio coder, wherein the apparatus is configured to

direct the first audio signal along the first path to be processed, with the first delay, by the first audio coder and to direct the second audio signal along the second path to be processed, with the second delay, by the second audio coder.

In some but not necessarily all examples, the second path is optimized for minimum delay.

In some but not necessarily all examples, the first path is optimized for audio-video synchronization.

In some but not necessarily all examples, the apparatus is configured to process the video signal; wherein the first delay is adjusted to time synchronize the processed first audio signal with the processed video signal wherein an event in the processed video signal that produces a sound is time synchronized with the produced sound in the processed first audio signal.

In some but not necessarily all examples, the apparatus is configured to classify an audio signal as a second audio signal by classifying the audio signal as a speech signal.

In some but not necessarily all examples, the apparatus is configured to classify an audio signal as a first audio signal by classifying the audio signal as a non-speech signal.

In some but not necessarily all examples, the apparatus is configured to classify an audio signal as a second audio signal by classifying the audio signal as a conversational speech signal.

In some but not necessarily all examples, the apparatus is configured to classify an audio signal as a first audio signal by classifying the audio signal as a non-conversational speech signal.

In some but not necessarily all examples, the apparatus is configured to classify the audio signal using audio analysis of the audio signal and/or video analysis of the video signal.

In some but not necessarily all examples, the apparatus is configured to identify a sound source in the video signal, and classify an audio signal associated with the identified sound source as a first audio signal.

In some but not necessarily all examples, the apparatus is configured to identify a sound source as not being in the video signal, and classify the audio signal associated with the identified sound source as a second audio signal.

In some but not necessarily all examples, the apparatus is configured to process the second audio signal wherein the processing of the second audio signal introduces the first delay such that the processed first audio signal and the processed second audio signal are time synchronized.

According to various, but not necessarily all, embodiments there is provided a computer program that when run on a processor causes: classifying a received audio signal as a first audio signal or a second audio signal, wherein the second audio signal is to be processed with less delay than the first audio signal; processing the first audio signal wherein the processing introduces a first delay; processing the second audio signal wherein the processing introduces a second delay, wherein the second delay is shorter than the first delay.

According to various, but not necessarily all, embodiments there is provided a method comprising: classifying a received audio signal as a first audio signal or a second audio signal, wherein the second audio signal is to be processed with less delay than the first audio signal; processing the first audio signal wherein the processing introduces a first delay; processing he second audio signal wherein the processing introduces a second delay, wherein the second delay is shorter than the first delay.

According to various, but not necessarily all, embodiments there is provided examples as claimed in the appended claims.

DETAILED DESCRIPTION

The following examples relate to an apparatus100comprising means for: classifying212an audio signal10as at least a first audio signal11for time synchronization with a video signal30or a second audio signal12not for time synchronization40with a video signal30;

processing the first audio signal11for time synchronization40with the video signal30wherein the processing introduces a first delay21for time synchronization40with the video signal30; and

processing the second audio signal12wherein the processing introduces a second delay22shorter than the first delay21.

FIGS.1and4illustrate examples of the apparatus100. In these examples a classifier110performs the classification212. Examples of different classifiers110are illustrated inFIGS.2and3. The classifier inFIG.2classifies for speech. The classifier inFIG.3classifies for conversational speech.FIG.5Aillustrates an audio signal10(Voice 1) and video signal30that has associated audio which may or may not be the voice signal10. InFIG.5B, the audio signal10(Voice 1) has been classified as a first audio signal11for time synchronization with the video signal30and, after processing, has a first delay21for time synchronization40with the video signal30(and its associated audio signal). InFIG.5C, the audio signal10(Voice 1) has been classified as a second audio signal12as is not for time synchronization with the video signal30. The second audio signal12, after processing, has a second delay22, shorter than the first delay21.FIG.6illustrates a method200for classifying202, processing214the first audio signal11and processing206the second audio signal12.

FIG.1illustrates an apparatus100, for selectively delaying an audio signal10. The selective delay is based on classification of the audio signal10.

The apparatus100receives at least an audio signal10and a video signal30. In some but not necessarily all examples, the video signal30has an associated audio signal (not illustrated in this FIG) that provides audio for the video images.

The apparatus100comprises a classifier110configured to classify an audio signal10as a first audio signal11or a second audio signal12. The first audio signal11is to be time synchronized with the video signal30. The second audio signal12is not to be time synchronized with be time the video signal30.

The apparatus100comprises a synchronizer120for time-synchronizing an audio signal10classified as a first audio signal11with the video signal30. The processing performed by the synchronizer120on the first audio signal11introduces a first delay21to the first audio signal11which results in time synchronization of the audio signal10(classified as the first audio signal11) with the video signal30.

Time synchronization means that the audio is rendered at the correct corresponding moment in the video rendering. In some examples, the first delay21that achieves time synchronization is fixed. In some examples, the first delay21that achieves time synchronization is variable and, in some examples, dynamically adjusts to timing differences arising between the audio signal10(first audio signal11) and the video signal30. For example, in some examples the apparatus receives contemporaneously, the audio signal10and a video signal30, whereas in other examples the apparatus can respond to receiving the audio signal10and a video signal30with a relative time delay that may be fixed or variable.

In some but not necessarily all examples, the first delay21synchronizes the processed first audio signal11with the processed video signal30so that an event in the processed video signal30that produces a sound is time synchronized with that sound in the processed first audio signal11. In some but not necessarily all examples, the first delay21is adjusted to time synchronize the processed first audio signal11with the processed video signal30so that an event in the processed video signal30that produces a sound is time synchronized with that sound in the processed first audio signal11.

In some but not necessarily all examples, the synchronizer120receives control information111relating to audio encoding delay and video encoding delay and matches the sync between the first audio signal11and the video signal30by applying a correcting delay to one of the signals (assumed to be the first audio signal11that needs to be delayed). For example, the first audio signals11are delayed by value given by video_delay-audio_delay. The synchronizer120can additionally, take into account further mismatches in audio-video (AV) sync. For example, there can be delays relating to any processing prior to the synchronizer120including those incurred in capturing and/or delays relating to any processing after the synchronizer120. These delays may be received as control information111and can be created from image analysis of the video signal30and/or audio analysis of the first audio signal11or from metadata associated with the audio signal10and/or metadata associated with the video signal30.

In this example, but not necessarily all examples, the apparatus100comprises a video coder140that processes the video signal30and produces an encoded video signal104.

In this example, but not necessarily all examples, the apparatus100comprises an audio coder130that processes the audio signal10and produces an encoded audio signal102. In some examples, the same audio coder130is used to encode the audio signal10whether it is classified as a first audio signal11or classified as a second audio signal12.

In other examples, the apparatus100does not comprise an audio coder130. In still other examples, for example as illustrated, the apparatus100comprises multiple audio coders. A first audio coder130A processes the first audio signal11(the audio signal10when classified as the first audio signal11) and produces an encoded audio signal102and a second, different, audio coder130B processes the second audio signal12(the audio signal10when classified as the second audio signal12) and produces an encoded audio signal102.

In some but not necessarily all examples, the classifier110and/or the synchronizer120receive control information111that assists in the respective tasks of classification and synchronization.

In some but not necessarily all examples, the control information111is obtained by processing the audio signal10and/or the video signal. In some but not necessarily all examples, the control information111is alternatively or additionally obtained, as metadata, from the audio signal10and/or the video signal.

In this example, the apparatus100comprises a first path201and a second path202, the apparatus100is configured to direct the first audio signal11along a first path201to be processed, with the first dleay21, and to direct the second audio signal12along the second path202to be processed, with the second delay22.

In some but not necessarily all examples, the second path202is optimized for minimum delay. In some but not necessarily all examples, the first path201is optimized for audio-video synchronization, that is to achieve time synchronization between the first audio signal11and the video signal30.

The outputs from the paths201,202are encoded individually, or jointly for example as a downmixed signal, using a conversational speech and audio coder. In some examples, only the low-delay audio signal (second audio signal12) is encoded using a conversational coder, while the first audio signal11is encoded using a different coder, e.g., a generic audio coder with a longer algorithmic delay.

Thus, in some but not necessarily all examples, the first path201comprises a first audio coder103A and the second path202comprises a second audio coder1308.

The apparatus100is configured to direct the first audio signal11along the first path201to be processed, with the first delay21, by the first audio coder130A and to direct the second audio signal12along the second path202to be processed, with the second delay22, by the second audio coder130B.

In the example where a common audio coder130is used for the first audio signal11and the second audio signal12, then the first delay21and the second delay22can be measured and compared before or after the audio coder130. In the example where a different audio coders130A,130B, that introduce different delays, are used for the first audio signal11and the second audio signal12, then the first delay21and the second delay22can be measured and compared after the audio coders130A,130B.

In some but not necessarily all examples, the audio signal10may be signals representing different audio objects.

In some examples the apparatus100can be a preprocessor, for example an IVAS preprocessor, that classifies sound (audio signal10) into low-delay objects (second audio signal12) and video-delay objects (first audio signal11) and puts them to different paths201,202, for example, different streams for different audio coders130A,130B.

The acronym IVAS relates to immersive voice and audio services. This includes immersive voice and audio for virtual reality (VR). There is on-going activity in developing an IVAS audio coder. The multi-purpose IVAS audio coder is expected to handle the encoding, decoding, and rendering of speech, music, and generic audio. It will be expected to receive and process audio signals10in various formats, including stereo format, multi-channel format, object-based audio format, Ambisonics format, and the metadata-assisted spatial audio (MASA) format. The MASA format enables practical spatial audio capture using a smartphone or similar device. It uses audio signal10together with corresponding spatial metadata, control information111, (containing, e.g., directions and direct-to-total energy ratios in frequency bands).

The apparatus100can therefore be configured for use in an IVAS system, or other audio coding systems. The apparatus100can therefore be configured to receive and process an audio signal10in the MASA format, or other formats. The audio signal10can be, e.g., mono, stereo, or various spatial audio formats including audio objects and MASA.

Although the classifier110, in this example is a simple two class classifier with two different associated delays21,22. In other examples, the classifier can be a multi-class classifier. In some but not necessarily all examples, the different classes of the multi-class classifier are each associated with a different delay.

The classifier110can be any suitable audio classifier. There is significant existing literature on the classification of audio. The classification can for example be based on spectral analysis of audio time segments (audio frames). These frames can, for example be 20 ms. The spectral analysis can take different forms. It can for example be a mel spectral analysis. The classification can be based on audio models or on learned models. For example, machine learning can be used for classification. The use of machine earning for audio and visual classification is well documented. Visual classification approaches can be used if the audio signal10is represented as a spectrograph.

The classification can be based on the audio signal10that is the audio content and/or on the video signal30that is the video content. The classification can additionally or alternatively be based on contextual control information111.

The contextual control information can, for example, be metadata associated with the audio signal10and/or metadata associated with the video signal30.

The contextual control information can, for example, indicate a direction of a sound source that produces the audio signal10and/or a direction of video capture that produces the video signal30

FIG.2illustrates an example of a classifier110suitable for use in the apparatus100. The classifier110is configured to classify an audio signal10as a second audio signal12by classifying the audio signal10as a speech signal and configured to classify an audio signal10as a first audio signal11by classifying the audio signal10as a non-speech signal.

The classifier110comprises a speech detector112and decision logic114. If speech is detected in the audio signal10by the speech detector112, then the decision logic classifies the audio signal10as a second audio signal12. If speech is not detected in the audio signal10by the speech detector112, then the decision logic classifies the audio signal10as a first audio signal11.

The speech detector can be any suitable speech detector or voice activity detector (VAD). Examples of voice activity detectors are specified in various telecommunication standards.

In this example, the speech signal (the second audio signal12) is processed with a minimum delay.

FIG.3illustrates an example of a classifier110suitable for use in the apparatus100.

In this example, the classifier110is configured to classify an audio signal10as a second audio signal12by classifying the audio signal10as a conversational speech signal and configured to classify an audio signal10as a first audio signal11by classifying the audio signal10as not conversational speech.

The classifier110comprises a speech detector112as previously described with reference toFIG.2.

The classifier110also comprises a first-stage analyzer116for analysis of speech and/or audio. This can for example be used to detect speech that is or is not associated with the video or speech that is or is not conversational. In some examples, video analysis is used.

The classifier110also comprises a second-stage analyzer118for analysis of non-conversational audio-visual analysis. This can for example be used to detect whether speech is conversational. In some examples, video analysis is used.

The classifier110classifies the audio signal10as:

In comparison toFIG.2, it therefore sub-classifies speech into conversational speech, that is speech that is part of a live, two-way conversation and non-conversational speech that is speech that is not part of a live two-way conversation.

Speech can in some examples be classified as non-conversational speech because it is audio-visual speech, that is it is speech associated with the video signal30.

The classifier110classifies the audio signal10that has been classified as conversational speech as the second audio signal12. The classifier110otherwise classifies the audio signal10as the first audio signal11. The classifier110therefore classifies the audio signal10that has been classified as non-conversational speech as the first audio signal11. The classifier110therefore classifies the audio signal10that has been classified as non-speech as the first audio signal11.

In this example, the conversational speech signal (the second audio signal12) is processed with a minimum delay.

The classification of speech as conversational (or not) can be based on analysis of the audio signal10. In some but not necessarily all examples, the classification of speech as conversational (or not) can additionally be based on analysis of the video signal30.

The classification of speech as conversational (or not) can for example be based on a speech model e.g., detecting pauses, delays, discourse markers such as ‘um’, ‘er’ etc.

The classification of speech as conversational (or not) can for example be based on a content analysis model e.g., the meaning of the words used are not associated with the video.

The classification of speech as conversational (or not) can for example be based on a trained convolutional neural network (or other machine learning algorithm). The convolutional neural network is trained to recognized conversational speech through training examples.

In some examples, the classifier110is configured to identify a sound source in the video signal30, and classify an audio signal10associated with the identified video sound source as a first audio signal11.

In some examples, the video sound source in the video signal30may be taken to be a direction of video capture, or by detecting lip movement (or some other visual indicator of sound creation) some sub-portion of the camera field of view. An audio signal10can be associated with the video sound source in the video if a direction of audio capture for the audio signal10spatially corresponds (matches) the respective direction of video capture, or sub-portion of the camera field of view.

20Additionally or alternatively, an audio signal10can be associated with a video sound source in the video signal30if the timing of the lip movement in the video is contemporaneous with speech in the audio signal10or the visual indicator of sound creation in the video is contemporaneous with a sound or expected sound in the audio signal10.

In some examples, the classifier110is configured to identify a sound source as not being in the video signal30, and classify the audio signal10associated with the identified sound source as a second audio signal12.

Thus, in some examples, the classifier110classifies the audio signal10as sound associated with the video signal30and classifies the audio signal10as a first audio signal11or classifies the audio signal10as sound not associated with the video signal30and classifies the audio signal10as a second audio signal12.

The classifier110can also take into consideration other factors such as location of a sound source, an identifier of a sound source, language of a sound sources, emotion of a sound source etc.

FIG.4illustrates an example of an apparatus100as previously described. It may comprise a classifier110as previously described.

The apparatus100is configured to produce different output data streams (encoded audio and video signals102,104) for different end-uses. For example, different data output10streams are created in dependence upon whether or not a receiver of the data stream is an active speaking (or potentially speaking) participant in a conversation or is merely listening.

If the receiver is participating then there is a requirement for fast processing of conversational speech. Therefore, if the receiver is participating in the conversational audio the audio signal10for conversational speech is classified as a second audio signal12.

If the receiver is not participating then there is not a requirement for fast processing of conversational speech and audio video synchronization is prioritized. Therefore, if the receiver is not participating in the conversational audio the audio signal10for conversational speech is classified as a first audio signal11.

The system also provides for synchronization at the receiver by providing synchronization information106with the encoded audio and video signals102,104.

25The synchronization information106allows different encoded audio and video signals102,104received from different sources to be time-synchronized at the receiver.

This synchronization can be optional if the receiver is participating in the conversational speech as all the encoded audio and video signals102,104should be sent with minimum delay. However, it can still be used if desired.

This synchronization is particularly useful if the receiver is not participating in the conversational speech as all the encoded audio and video signals102,104are not then sent with a minimum delay and the delays may vary.

The ‘participant’ processing block122processes the first audio signal11as described in previous examples to obtain audio-video synchronization for the first audio signal11but not for the second audio signal12which is processed with less delay.

The ‘non-participant’ processing block124processes the audio signal10(this could include audio signals classified as first audio signals11or second audio signals12) to obtain audio video synchronization for the first audio signals11and the second audio signals12.

The apparatus100is therefore configured to operate in different modes for a receiver (listener).

In a participant mode the apparatus100is configured to process122the first audio signal11to introduce a longer first delay21and process the second audio signal12to introduce a shorter second delay22such that the processed first audio signal11and the processed second audio signal12are not simultaneous at the participant receiver (listener). This occurs when the second audio signal12relates to conversational speech and the receiver (listener) is participating in the conversational speech.

In a non-participant mode, the apparatus100is configured to process124both the first audio signal11and the second audio signal12(i.e. the audio signal10) to introduce a longer first delay21. Thus, the processing block124receives the audio signal10(first audio signal11and the second audio signal12) and is configured to process the first audio signal11to introduce a longer first delay21and process the second audio signal12to introduce the longer first delay21(not the shorter second delay22) such that the processed first audio signal11and the processed second audio signal12are simultaneous at the non-participant receiver (listener). This occurs when the second audio signal12relates to conversational speech but the receiver (listener) is not participating in the conversational speech.

Therefore, the apparatus100can be configured to detect when a receiver (listener) changes between participating in conversational speech and not-participating in conversational speech and in response to the detection can switch between the different modes. In some but not necessarily all examples, the longer first delay21can be used for audio-video synchronization.

FIG.5Aillustrates, on a time line, an audio signal10(Voice 1) and video signal30(Video 1) that has associated audio (Audio 1). The video signal30(Video 1) that has associated audio (Audio 1) together form an audio-visual signal (AV1). A portion of the associated audio (Audio 1), illustrated with dotted lines, is contemporaneous with the audio signal10(Voice 1).

There is a possibility of a further audio signal10(Voice 2), at a later time.

InFIG.5B, the audio signal10(Voice 1) has been classified as a first audio signal11for time synchronization with the video signal30and, after processing, has a first delay21for time synchronization40with the video signal30(and its associated audio signal). In this FIG, the audio signal10(Voice 1), the video signal30(Video1) and its associated audio (Audio 1) have all been delayed by the same delay—the first delay21. Time synchronization is maintained between the audio-visual signal and the audio signal10(Voice 1). The portion of the associated audio (Audio 1), illustrated with dotted lines, remains contemporaneous with the audio signal10(Voice 1).

InFIG.5C, the audio signal10(Voice 1) has been classified as a second audio signal12that is, not for time synchronization with the video signal30. In this FIG, the audio signal10(Voice 1) has been delayed by a short delay- the second delay22, whereas the AV signal comprising the video signal30(Video 1) and its associated audio (Audio 1) has been delayed by a longer delay—the first delay21. Time synchronization is not maintained between the audio-visual signal and the audio signal10(Voice 1). The portion of the associated audio (Audio 1), illustrated with dotted lines, no longer remains contemporaneous with the audio signal10(Voice 1).

The audio signal10(Voice 1) does not occur contemporaneously with Voice 2.

It will therefore be appreciated that, in at least some examples, the apparatus100comprises means for:

receiving contemporaneously multiple audio signals10;

classifying the received multiple audio signals10to identify first audio signals11and second audio signals12, wherein the second audio signals12are to be processed with less delay than the first audio signals11;

processing the first audio signals11wherein the processing introduces a first delay21;

processing the second audio signal12wherein the processing introduces a second delay22, wherein the second delay22is shorter than the first delay21such that the processed first audio signals11and the processed second audio signals12are no longer contemporaneous.

In some but not necessarily all examples, the longer first delay21can be used for time synchronization of the first audio signal11with the video signal30. Other reasons for having different delays include balancing bit rate, quality, and delay where, for example, quality and delay can be traded.

FIG.6illustrates an example of a method200.

The method200comprises at block212classifying a received audio signal10as a first audio signal11or a second audio signal12, wherein the second audio signal12is to be processed with less delay than the first audio signal11.

The method200comprises at block214processing the first audio signal11wherein the processing introduces a first delay21.

The method200comprises at block206processing the second audio signal12wherein the processing introduces a second delay22, wherein the second delay22is shorter than the first delay21.

FIG.7illustrates an example of a system comprising the apparatus100and one or more audio rendering devices150,152for rendering the encoded audio signals102, when based on the first audio signal11(referred to and labelled as the coded first audio signal161) and for rendering the encoded audio signals102when based on the second audio signal12(referred to and labelled as the coded second audio signal162).

The apparatus100is configured to cause rendering of the processed first audio signal11at a first audio rendering device150and rendering of the processed second audio signal12at a second audio rendering device152that is different to the first audio rendering device150.

For example, conversational audio (second audio signal12) can be rendered using devices headphones152(with transparency), while non-conversation audio signal (first audio signal11) can be rendered by a loudspeaker system150connected or associated with a video rendering device154(e.g., smartphone loudspeakers) for rendering the encoded video signal104or video signal30.

It will be appreciated from the forgoing that the apparatus100can be used in many applications including, but not limited to video conferencing, live streaming, conversational services and other applications where a low end-to-end delay can be preferable compared, for example, to maintaining audio and video time synchronization or other aspects of quality.

The apparatus100, in some examples, can achieve the lowest possible delay for conversational audio to make discussions as effortless and natural as possible. At the same time, especially audio that relates to what is seen in the video can be synchronized with the visual content, which leads to reduced viewer irritation.

For conversational use, the lowest possible audio delay can be achieved, in some examples. For many other audio content, it still remains important to provide AV sync. This may also include speech in some cases, e.g., when the talker is not a participant.

The media stream delivery can in embodiments be done in various ways. For example, the transmitting system can packetize low-delay content and AV synced content separately. This may allow for different priority levels. For example, considering 3GPP TS 23.203, we could associate the different content streams with QoS Class Identifiers (QCI) ‘1’ and ‘2’, respectively, or ‘1’ and ‘7’, respectively, for different packet delay budget, packet error loss rate, and resource type characteristics:‘1’—e.g., conversational voice‘2’—e.g., conversational video‘7’—e.g., voice, video (live streaming)

Alternatively, the content can be packetized commonly.

Furthermore, in some examples, different types of audio content can be forwarded for rendering by different types of presentation devices. For example, conversational audio signals10can be presented using headphones (with transparency), while non-conversation audio signals10can be presented by loudspeaker system connected with the video presentation screen (e.g., smartphone loudspeakers).

FIG.8illustrates an example of a controller400for the apparatus100. Implementation of a controller400may be as controller circuitry. The controller400may be implemented in hardware alone, have certain aspects in software including firmware alone or can be a combination of hardware and software (including firmware).

As illustrated inFIG.8the controller400may be implemented using instructions that enable hardware functionality, for example, by using executable instructions of a computer program406in a general-purpose or special-purpose processor402that may be stored on a computer readable storage medium (disk, memory etc) to be executed by such a processor402.

The processor402is configured to read from and write to the memory404. The processor402may also comprise an output interface via which data and/or commands are output by the processor402and an input interface via which data and/or commands are input to the processor402.

The memory404stores a computer program406comprising computer program instructions (computer program code) that controls the operation of the apparatus100when loaded into the processor402. The computer program instructions, of the computer program406, provide the logic and routines that enables the apparatus to perform the methods illustrated in the Figs. The processor402by reading the memory404is able to load and execute the computer program406.

The apparatus100therefore comprises:at least one processor402; andat least one memory404including computer program codethe at least one memory404and the computer program code configured to, with the at least one processor402, cause the apparatus100at least to perform:classifying a received audio signal10as a first audio signal11or a second audio signal12, wherein the second audio signal12is to be processed with less delay than the first audio signal11;processing the first audio signal11wherein the processing introduces a first delay21; and processing the second audio signal12wherein the processing introduces a second delay22, wherein the second delay22is shorter than the first delay21.

As illustrated inFIG.9, the computer program406may arrive at the apparatus100via any suitable delivery mechanism408. The delivery mechanism408may be, for example, a machine readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a Compact Disc Read-Only Memory (CD-ROM) or a Digital Versatile Disc (DVD) or a solid-state memory, an article of manufacture that comprises or tangibly embodies the computer program406. The delivery mechanism may be a signal configured to reliably transfer the computer program406. The apparatus100may propagate or transmit the computer program406as a computer data signal.

Computer program instructions for causing an apparatus to perform at least the following or for performing at least the following: classifying a received audio signal10as a first audio signal11or a second audio signal12, wherein the second audio signal12is to be processed with less delay than the first audio signal11; processing the first audio signal11wherein the processing introduces a first delay21; processing the second audio signal12wherein the processing introduces a second delay22, wherein the second delay22is shorter than the first delay21.

The computer program instructions may be comprised in a computer program, a non-transitory computer readable medium, a computer program product, a machine readable medium. In some but not necessarily all examples, the computer program instructions may be distributed over more than one computer program.

Although the processor402is illustrated as a single component/circuitry it may be implemented as one or more separate components/circuitry some or all of which may be integrated/removable. The processor402may be a single core or multi-core processor.

This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor and its (or their) accompanying software and/or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit for a mobile device or a similar integrated circuit in a server, a cellular network device, or other computing or network device. The blocks illustrated in theFIG.6(and others) may represent steps in a method and/or sections of code in the computer program406. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block may be varied. Furthermore, it may be possible for some blocks to be omitted.

The systems, apparatus, methods and computer programs may use machine learning which can include statistical learning. Machine learning is a field of computer science that gives computers the ability to learn without being explicitly programmed. The computer learns from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. The computer can often learn from prior training data to make predictions on future data. Machine learning includes wholly or partially supervised learning and wholly or partially unsupervised learning. It may enable discrete outputs (for example classification, clustering) and continuous outputs (for example regression). Machine learning may for example be implemented using different approaches such as cost function minimization, artificial neural networks, support vector machines and Bayesian networks for example. Cost function minimization may, for example, be used in linear and polynomial regression and K-means clustering. Artificial neural networks, for example with one or more hidden layers, model complex relationship between input vectors and output vectors. Support vector machines may be used for supervised learning. A Bayesian network is a directed acyclic graph that represents the conditional independence of a number of random variables.

The algorithms hereinbefore described may be applied to achieve the following technical effects rendering of audio with different delays.

As used here ‘module’ refers to a unit or apparatus that excludes certain parts/components that would be added by an end manufacturer or a user. The apparatus can be a module.

The above described examples find application as enabling components of:automotive systems; telecommunication systems; electronic systems including consumer electronic products; distributed computing systems; media systems for generating or rendering media content including audio, visual and audio visual content and mixed, mediated, virtual and/or augmented reality; personal systems including personal health systems or personal fitness systems; navigation systems; user interfaces also known as human machine interfaces; networks including cellular, non-cellular, and optical networks; ad-hoc networks; the internet; the internet of things; virtualized networks; and related software and services.