Methods and apparatus for decoding multiple independent audio streams using a single audio decoder

An embodiment of the present invention discloses a system and method for decoding multiple independent encoded audio streams using a single decoder. The system includes one or more parsers, a preprocessor, an audio decoder, and a renderer. The parser extracts individual audio frames from each input audio stream. The preprocessor combines the outputs of all parsers into a single audio frame stream and enables sharing of the audio decoder among multiple independent encoded audio streams. The audio decoder decodes the single audio frame stream and provides a single decoded audio stream. And the renderer renders the individual reconstructed audio streams from the single decoded audio stream.

TECHNICAL FIELD

An embodiment of the present invention relates to an audio signal processing system, and more specifically to a decoder for decoding multiple independent audio streams.

SUMMARY

Audio signal processing has become increasingly important in today's multimedia environment. Audio data processing is an important requirement for many multimedia applications, such as gaming and in telecommunications. One of the key elements in many digital audio information processing systems is the audio decoder. Generally, the audio decoder receives data in a compressed form and converts that data into a decompressed digital form.

FIG. 1illustrates an approach for decoding audio signals in an audio-video playback system. In an audio-video playback system, multiplexed audio-video streams are fed into the system for processing. The multiplexed streams are de-multiplexed and then fed into the audio and video subsystems. The multiplexed streams are of various formats such as MPEG2, PES, Windows Media files, AVI etc. Examples of audio-video playback system are Set Top Box, DVD player etc.

In an audio subsystem, the audio playback includes three stages: audio parsing, audio decoding, and audio rendering. In the audio parsing stage, the encoded data is extracted from its container (PES, File containers etc.) and constructed into a stream of audio frames. Each of these frames is an independent decodable unit. In addition to the compressed data, some metadata like PTS, etc is attached to these frames. In the audio decoding stage, audio frames are fed into a hardware/software decoder. This decoder decodes the compressed data and outputs PCM samples. Metadata could be used by the decoding stage or forwarded to the next stage. In addition to the original metadata some additional parameters such as sampling frequency, number of channels and bits per sample are attached to the decoded data by the decoding stage. In the audio rendering phase, PCM data is formatted (according to the requirements of the system's output interfaces) and output to the external interfaces exposed by the system.

U.S. Pat. No. 6,081,783, which is incorporated herein by reference, discloses an audio decoder operating on a received compressed audio data stream. The audio data stream has been compressed using an algorithm utilizing transform encoding and a bit allocation routine. The audio decoder includes a first digital signal processor and a second digital signal processor. The first digital signal processor executes a first set of operations on the received compressed audio data stream including parsing the compressed audio data stream, recovering data fields within the compressed audio data stream, calculating bit allocation, and passing frequency domain coefficients to shared memory. The second digital signal processor executes a second set of operations on the data provided by the first digital signal processor including performing inverse transform operations on the data passed from the first digital signal processor.

U.S. Pat. No. 6,233,562, which is incorporated herein by reference, discloses an audio decoding device, which decodes coded audio information with multiple channels. The audio decoding device includes a coded information memory section, an information transmission section, and an audio decoding section. The coded information memory section stores the coded audio information. The information transmission section reads the coded audio information stored at an arbitrary position in the coded information memory section. The audio decoding section decodes the coded audio information read by the information transmission section and outputs the resultant audio information in accordance with a time parameter.

The above-mentioned patents are known for decoding two independent audio streams. The limitation with said patents is that they need two independent audio decoders for decoding two independent audio streams simultaneously and this result in significant overhead and resource requirements.

Therefore, there is a need of a system for decoding multiple independent audio streams using a single decoder.

One embodiment of the present disclosure describes a system for decoding multiple independent encoded audio streams comprising a parser coupled to each input encoded audio stream for extracting individual audio frames, a preprocessor coupled to the outputs of all parsers for combining the outputs of said parsers into a single audio frame stream, an audio decoder coupled to the output of said preprocessor for decoding the single audio frame stream, and a renderer coupled to the output of said audio decoder for generating the individual output decoded audio streams from the single decoded stream.

This disclosure also describes an embodiment of a set-top-box comprising audio and video subsystems, said audio subsystem comprising a parser coupled to each input encoded audio stream for extracting individual audio frames, a preprocessor coupled to the outputs of all parsers for combining the outputs of said parsers into a single audio frame stream, an audio decoder coupled to the output of said preprocessor for decoding the single audio frame stream, and a renderer coupled to the output of said audio decoder for generating the individual output decoded audio streams from the single decoded stream.

This disclosure further describes an embodiment of a DVD player comprising audio and video subsystems, said audio subsystem comprising a parser coupled to each input encoded audio stream for extracting individual audio frames, a preprocessor coupled to the outputs of all parsers for combining the outputs of said parsers into a single audio frame stream, an audio decoder coupled to the output of said preprocessor for decoding the single audio frame stream, and a renderer coupled to the output of said audio decoder for generating the individual output decoded audio streams from the single decoded stream.

This disclosure also describes an embodiment of an audio processor capable of decoding multiple independent audio streams, said audio processor comprising a parser coupled to each input encoded audio stream for extracting individual audio frames, a preprocessor coupled to the outputs of all parsers for combining the outputs of said parsers into a single audio frame stream, an audio decoder coupled to the output of said preprocessor for decoding the single audio frame stream, and a renderer coupled to the output of said audio decoder for generating the individual output decoded audio streams from the single decoded stream.

This disclosure further describes an embodiment of a method for decoding multiple independent encoded audio streams comprising parsing each independent encoded audio stream for extracting individual audio frames, preprocessing the extracted individual audio frames for combining into a single encoded audio frame stream, allocating a shared audio decoder resource to decode said encoded audio frame stream when said decoder resource is available, producing a single decoded audio stream, and rendering the individual output decoded audio streams from said decoded audio stream.

DETAILED DESCRIPTION

One or more embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to these embodiments. The present invention can be modified in various forms. In the accompanying drawings, like reference numerals are used to indicate like components.

FIG. 2illustrates the block diagram of a system200for decoding multiple independent encoded audio streams according to an embodiment of the present invention. The system200includes one or more parsers201, a preprocessor202, an audio decoder203, and a renderer204. Each parser201extracts individual encoded audio frames from the audio stream. Preprocessor202, which is coupled to the outputs of all the parsers receives the extracted audio frames and combines them into a single pseudo interleaved audio frame stream. Audio decoder203decodes the received interleaved audio frame stream and provides a single pseudo-interleaved decoded audio stream. Renderer204receives the single pseudo-interleaved decoded audio stream, discards redundant decoded frames and segregates the remaining decoded audio frames into multiple decoded audio frame streams. In one embodiment of the present invention, the preprocessor202is software. The terms “preprocessor” and “audio driver” are used interchangeably in the description below.

FIG. 3illustrates the block diagram of the preprocessor202according to an embodiment of the present invention. Preprocessor202includes a replicator301and an interleaver302. Replicator301receives extracted individual audio frames from the output of the parser and repeats selected audio frames depending on a windowing filter. Replicator301may repeat all frames except the first and the last frame of the input audio stream. The number of replicas is determined by the number of samples required for the window filter to converge to the point where proper filtered samples are available. Interleaver302receives the repeated audio frames and combines them into a single pseudo-interleaved audio frame stream. After decoding, the pseudo-interleaved audio frame stream is applied to the window filter (which is an intrinsic part of the decoder203) for providing filtered audio frames.

Preprocessing Process

Decoding of a stream by the audio decoder203through the audio driver is initiated when two conditions are met: firstly, the audio driver has collected number of frames for decoding and secondly, the audio decoder is available for decoding.

The value of “K” is determined by the number of samples taken by a particular audio type for its windowing filter to converge to the value from where it starts giving proper output and number of PCM samples generated per frame.

So if,“K”=Number of continuous compressed frames to be collected“S”=Number of Samples per frame“N”=Number of Samples taken by windowing filter to converge

Then K will be given by the equation
K=┌N/S┐+nEq. 1
n>0 (where n is an integer)

Until both of the above conditions are met (in the same order), the audio driver waits. As the above said conditions are met, the following steps are taken to decode until the Ithframe where I is last of the collected “K” frames1. All the collected frames i.e. (I−K+1), (I−K+2), . . . , I are fed to the audio decoder in the same order as they were received.2. After the decoding process is completed, the audio decoder is released so that it can be used by the other instance of audio driver running in the system.3. The selected frames are I, I-1, . . . I-n+14. All the metadata received with the selected frames are attached to the current output buffers.5. Output buffers along with their associated metadata can be delivered to the next stage for further processing or playback.

FIG. 4illustrates a block diagram of the renderer204according to an embodiment of the present invention. Renderer204includes a demultiplexer401and a frame selector402. Demultiplexer401segregates the decoded audio frames into multiple decoded audio frame streams depending on the input. Frame selector402discards redundant decoded frames from the decoded audio frame stream.

Rendering Process

The audio driver has already collected K number of frames on any stream, out of which only X number of frames are useful. Hence, in the delivery process, Y numbers of frames are dropped to get the right frames.“K”=Number of Frames collected at an input to the decoder.“Y”=Number of Frames to be dropped at an output of the decoder.“S”=Number of Samples per frame“N”=Number of Samples taken by windowing filter to converge“X”=Number of frames to be delivered at the rendering stage.
Y=┌N/S┐Eq. 2

Now the Number of frames sent for post-rendering processing will be given by
X=K−YEq. 3

The audio decoder produces all the decoded frames, i.e. 1, 2 . . . K. Out of these the first Y frames are dropped and X frames are sent for further processing.

FIG. 5illustrates a mechanism for decoding multiple independent encoded audio streams according to an embodiment of the present invention. Consider two independent compressed encoded audio streams501,502. Each independent encoded audio stream is parsed for extracting individual audio frames from their containers (PES, File containers etc). Then the metadata (PTS, etc.) is extracted from the streams. Selected extracted audio frames are replicated to form audio streams503and504. Audio decoder505is time multiplexed between audio stream503and audio stream504(which thereby function as a single pseudo-interleaved audio stream fed to the audio decoder). Two decoded audio streams508,509are generated from the demultiplexed decoded audio streams506and507respectively by dropping a selected number of frames at the output of audio decoder505.

FIG. 6illustrates a method for decoding multiple independent encoded audio streams according to another embodiment of the present invention. The embodiment is related to decoding MPEG audio signals such as MPEG1 layer 2/3 or MPEG2. Consider a first parsed audio elementary stream601and a second parsed audio elementary stream602. The two streams are interleaved for creating a pseudo-interleaved stream603by replicating selected audio frames. Two consecutive frames of each independent stream601,602are copied alternatively to create the pseudo-interleaved stream603. Pseudo-interleaved stream603is provided to an MPEG decoder604. MPEG decoder604decodes the pseudo-interleaved stream603and producing a pseudo-interleaved decoded audio stream605. Then selected numbers of frames are dropped from the pseudo-interleaved decoded audio stream605for forming a first reconstructed audio stream606and a second reconstructed audio stream607. The selected numbers of frames refer to redundant frames, which are to be dropped. The MPEG1 Layer 2 frame size is 1152 samples. The audio decoder604needs 480 samples of previous frames to produce the output without any distortion.S=Number of Samples per frame=1152N=Number of Samples taken by windowing filter to converge=480n=1

Therefore as per equation 1 value of K (number of frames) will be
K=┌480/1152┐+1=2Number of frames to be dropped: Y=┌480/1152┐=1Number of frames to be sent for post-rendering processing:

Processing requirements for any audio type which may be decoded with the method described above may be calculated as follows:

P=MIPS required to decode a single stream of a particular audio type

K=Number of continuous compressed frames to be collected as derived from equation 1

If the processing requirement to decode a single MPEG 1 layer 2/3 or MPEG2 stream is P MIPS, then as per the equation 4 stream, the worst case processing requirement P′ will become
P′=2*PMIPS

FIG. 7illustrates a method for decoding multiple independent encoded audio streams according to yet another embodiment of the present invention. The embodiment demonstrates the applicability of decoding process for MPEG1 Layer 1. The process of decoding is almost similar to the process explained inFIG. 6but in this embodiment, four consecutive frames of each independent stream701,702are copied alternatively to create the pseudo interleaved stream703instead of two consecutive frames in the previous embodiment (FIG. 6).

The MPEG1 Layer1 frame size is 384 samples. The audio decoder needs 480 samples of previous frames to produce the output without any distortion.S=Number of Samples per frame=384N=Number of Samples taken by windowing filter to converge=480n=1

Therefore as per equation 1 value of K will be

If processing requirement to decode a single MPEG1 layer 1 stream is P MIPS then as per equation 2 stream worst case processing requirement P′ will become
P′=3*PMIPS
Optimizing Processing Requirements (MIPS/MHz)

As calculated in equation 2, processing power required for decoding a single stream using this algorithm will depend on the number of frames generated from a decoding sequence. Therefore if we increase the number of frames generated per decoding sequence then processing power requirement will come down. Therefore
P′opt=(P*(K+M))/(M+1)  Eq. 5
whereP=MIPS required to decode a single stream of a particular audio typeK=Minimum number of continuous compressed frames to be collectedM=Number of valid additional decoded frames generated per decoding sequence

Processing requirements for a stream may be reduced at the cost of memory (buffering) and system delay, in delivering decoded audio data.

Therefore as per equation 3, if we consider M=1 and K=3,

Thus,FIG. 7describes the method of reducing the processing requirement for MPEG 1 Layer 1 from 3*P to 2*P, by using M=1 and decoding 4 frames at a time instead of minimum 3 frames and getting 2 valid decoded frames.

FIG. 8illustrates a block diagram that discloses an application for a system200, decoding multiple independent encoded audio streams, according to an embodiment of the present invention used in a set-top-box800. The set-top-box800includes audio and video subsystems. The audio subsystem200includes a parser, a preprocessor, an audio decoder, and a renderer. According to another embodiment of the present invention, the system200is used in a DVD player.

FIG. 9illustrates a flow diagram of a method for decoding multiple independent encoded audio streams according to an embodiment of the present invention. At step901, each independent encoded audio stream is parsed for extracting individual audio frames. At step902, the extracted individual audio frames are preprocessed for combining into a single encoded audio frame stream. At step903, a shared audio decoder resource is allocated to decode the encoded audio frame stream when the decoder resource is available. At step904, a single decoded audio stream is produced. At step905, the individual output decoded audio streams are rendered from the decoded audio stream.

An embodiment of the present invention is related to a system for decoding multiple independent encoded audio streams and can be used in various applications, such as dual decoding set top boxes, dual decoding DVD players, set top boxes with audio description support, and multi site video conferencing systems.

An embodiment of the present invention offers several advantages. Firstly, an embodiment utilizes a single decoder for decoding multiple independent audio streams. Secondly, an embodiment is applicable to both hardware and software decoders. Thirdly, an embodiment supports multiplexed stream of various formats such as MPEG2, PES, Windows Media files, AVI etc.