MPEG-4 SYSTEMS COMPLIANT ARCHITECTURE

Abstract: The invention relates to an MPEG-4 Systems compliant architecture subdivided in key modules having a specific task managed by a multi-threading operating system. The MPEG-4 demultiplexer is partitioned into three levels of process. Each process is independent of each other, but synchronization primitives are however provided by the operating system, in the form of semaphores controlling the buffers so that their sizes are kept as small as possible. Application: all kinds of industrial MPEG-4 equipment (desk-tops, set-tops boxes).

Description: MPEG-4 systems compliant architecture. FIELD OF THE INVENTION The present invention relates to an MPEG-4 Systems compliant architecture. This invention has applications in relation with the MPEG-4 standard, for the streaming of MPEG-4 multimedia data. BACKGROUND OF THE INVENTION The Systems part of the MPEG-4 standard defines the architecture to access MPEG-4 multimedia data (video, audio, 2D and 3D graphics). MPEG-4 Systems is, more precisely, described in the following document "MPEG-4 Systems", ISO/TEC JTC1/3C29/WG11 N 1901, 21 November 1997. Such an architecture is complex, since it provides access to an arbitrary number of streams, such as audio and video streams for instance, originated from a large number of sources (networks, broadcasting, storage,...). Particularly, the standard specifies, for the synchronization of audio and video data, a mechanism allowing to introduce, by means of time-stamps, a temporal information into said different streams. Most of the concerned potential products will use network technology with any type of stability in terms of bitrate (for instance, mobile networks, Internet). On these networks, bitrate variations occur (for Internet, depending on the time in the day ; for mobile networks, depending on the distance to the nearest base station). As an MPEG-4 compliant architecture (the topology of which has to be handled with respect to a so- called "configuration" that may be fixed but generally evolves dynamically during the session) has to deal with a number of memory buffers that will be specifically organized, an optimal architecture implementation has to ensure an appropriate buffer management. The problems to be solved in this context then concern the exchange of information between the different modules of said architecture, in order to avoid drawbacks such as : - a break of the "pipe-line" effect, for instance losses in the stream that could lead to a desynchronization (a break in the audio stream or a desynchronization of the audio signals with respect to the video stream are very perceptible by the end user and therefore unacceptable) ; - a processing time that would be too long for interactive applications such as video conference ; - a waste of the memory resources if over-sized buffers are used. A compromise between the solutions to these problems is difficult to obtain, since the only way to reduce the effect of the network is to increase the size of the buffers, which however increases the delays and processing time, and so on. Moreover, it seems unrealistic to predict the size of the buffers at an early stage of the implementation of the architecture, as it was the case for previous applications (for example with the MPEG-2 standard in the field of TV broadcast). SUMMARY OF THE INVENTION It is therefore an object of the invention to obviate these drawbacks. To this end, the invention relates to an architecture characterized in that it is subdivided into key modules, a specific task managed by a multi-threading operating system being assigned to each part of said architecture, and the strategy for the buffer management being optimally based on specific synchronization primitives provided by the multi-threading operating system. An implementation of MPEG-4 Systems has to be able to instantiate dynamically the differents components of a configuration, whatever this one is very simple (for instance, a movie composed of an audio flow and a video flow) or very complex (for example, for an application of interactive multi-user virtual reality with teleconferencing). The technical solution according to the invention, that defines a generic architecture for every implementation of MPEG-4 Systems based on microprocessor(s) with a multi-threading operating system, is a reply to this problem. The proposed architecture, that uses basic primitives present in all multi-threading environments, is indeed modular, flexible, dynamic, extensible, and easy to implement and to manage. Moreover, multi-threading operating systems are optimized for the target processor and are built to be able to efficiently manage the synchronization of numerous tasks. BRIEF DESCRIPTION OF THE DRAWINGS These and other aspects of the invention will be now explained with reference to the following description, given in connection with the accompanying drawings, in which : - Fig.l depicts an implementation of an MPEG-4 Systems buffer partitioning model (on the so-called Win32 operating system); - Fig.2 illustrates the control management with semaphores, used to keep the different threads efficient. DESCRIPTION OF THE INVENTION The proposed solution addresses the following issues. First, key modules for an optimal buffer management are defined, which implies a specific generic partitioning of MPEG-4 Systems. Each of these modules is then implemented as a task that is managed by a multi-threading operating system (with the use, in order to manage the filling of the buffers in an optimal way, of specific synchronization primitives provided by said multi -threading operating system). Finally, the strategy for the buffer management is defined. For the definition of the MPEG-4 Systems key modules, it is proposed, in order to assign a specific task to each part of the MPEG-4 demultiplexer, to partition said demultiplexer into three different levels, shown in Fig.l : TransMux, FlexMux, Elementary Stream. A decoder buffer (audio, video,...) is also defined (and represented in Fig.l), in order to read Access Units (AUs) and produce frames that can be played (images, sounds), but this task is not part of the one carried out by the demultiplexer. The scheme of Fig.l has been implemented for instance on the Win32 operating system (= Windows 95), which provides multi-threading functionality. One thread is assigned to each process : - a TransMux thread (TMX), for reading data on the disk DK or on the network NW and storing a number N of FlexMux Packet Data Units (= FM-PDUs) ; - a FlexMux thread (FMX), for reading these FM-PDUs in an FM-PDU buffer, storing N Access unit Layer Packet Data Units (= AL-PDUs) and reconstructing N Access Units (= AUs), the FlexMux FMX and the Elementary Stream ELS being, in that implementation, organized in one single thread ; - a Decoder thread (DEC), for building N frames with the Access Units (AUs) for the decoders. Each of these threads is independent from the others. As shown in Fig.l, the buffers are implemented as queues, i.e. as FIFO (First-In, First-Out) memories, without any fixed length (it is consequently recommended to control the state of these buffers in order not to have huge buffers nor starving tasks). However the tasks have to watch at each other using so-called synchronization primitives, which is done by means of semaphores (or : counters) that keep the different threads efficient. Each of these semaphores, provided by the operating system, controls each respective buffer (FM-PDU, AL-PDU, AU) in the demultiplexer. For each decoder, there is also one semaphore controlling the decoder buffer, and one more for signalling to the TransMux to have to stop if the AU buffer becomes too large. These actions are depicted in Fig.2, that illustrates the control management with the semaphores. The steps (1) and (2) correspond to the following tests : (1) means "if N greater than 0, work", and (2) means "if N greater than Max, stop". Based on the synchronization facilities of the operating system, this control function is carried out according to the following buffer management strategy : (a) at run-time, the first operation is done by the TransMux TMX that reads data on the local disk DK or on the network NW ; (b) the FlexMux FMX watches at the TransMux TMX ; (c) the Elementary Stream ELS watches at the FlexMux FMX; (d) the real-time decoding process starts when AUs are present in the AU buffer ; (e) the regulation of the TransMux TMX is made by the AU feedback semaphore controlling that the AU buffer is not in a critical state. The principle of this buffer management strategy is to keep the buffer sizes as small as possible. When the number of AL-PDUs is sufficient to reconstruct an AU, it has to be done immediately. Thus there is always data present for decoding. Moreover, an important point is to consider the type of TransMux that is used. Two types of TransMuxes can be differentiated : slow ones (network access), and fast ones (disk access). The disk accesses are quick and stable in terms of latency, and, with such types of accesses, it is then not needed to store many data in the different buffers, since no loss would occur and the maximal critical value could be rather low. On the contrary, network accesses can introduce some latency (Internet, for example), so that it is important to store more data in the buffers in order not to let the decoder starve. In the case of more than one decoder, it must however be indicated that the TransMux has to be stopped only when all the AU buffers have reached the critical maximum value in order not to have starving decoding threads (it is therefore assumed that the multiplexing strategy has been adapted for that purpose on the sending side).

Claims: CLAMS : 1. An MPEG-4 Systems compliant architecture, characterized in that it is subdivided into key modules, a specific task managed by a multi-threading operating system being assigned to each part of said architecture, and the strategy for the buffer management being optimally based on specific synchronization primitives provided by the multi-threading operating system. 2. An architecture according to claim 1, wherein the MPEG-4 demultiplexer is partitioned into three different levels of process, the first one, TransMux, being provided for reading data on a local disk or a network and storing them by packet data units, the second one, FlexMux, being provided for reading said packet data units and storing access unit layer packet data units, and the third one for reconstructing access units. 3. An architecture according to claim 2, wherein each process is independent from each other, synchronisation primitives being however provided by said operating system in order to watch at each other, by means of semaphores controlling each buffer so that the buffer sizes are kept as small as possible.

Inventors: GENTRIC PHILIPPE, HERRMANN LAURENT, GENTRIC, PHILIPPE, HERRMANN, LAURENT
Applicants: KONINKL PHILIPS ELECTRONICS NV, PHILIPS SVENSKA AB, KONINKLIJKE PHILIPS ELECTRONICS N.V., PHILIPS AB
Application year: 1999
Main classification: H04N7
IPC codes: H04N7