Patent ID: 12206871

DETAILED DESCRIPTION

The present principles are directed to methods and apparatus for video usability information (VUI) for scalable video coding (SVC).

The present description illustrates the present principles. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the present principles and are included within its spirit and scope.

All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the present principles and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.

Moreover, all statements herein reciting principles, aspects, and embodiments of the present principles, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the present principles. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.

Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.

Reference in the specification to “one embodiment” or “an embodiment” of the present principles means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

It is to be appreciated that the use of the term “and/or”, for example, in the case of “A and/or B”, is intended to encompass the selection of the first listed option (A), the selection of the second listed option (B), or the selection of both options (A and B). As a further example, in the case of “A, B, and/or C”, such phrasing is intended to encompass the selection of the first listed option (A), the selection of the second listed option (B), the selection of the third listed option (C), the selection of the first and the second listed options (A and B), the selection of the first and third listed options (A and C), the selection of the second and third listed options (B and C), or the selection of all three options (A and B and C). This may be extended, as readily apparent by one of ordinary skill in this and related arts, for as many items listed.

Moreover, it is to be appreciated that while one or more embodiments of the present principles are described herein with respect to the MPEG-4 AVC standard, the present principles are not limited to solely this standard and, thus, may be utilized with respect to other video coding standards, recommendations, and extensions thereof, including extensions of the MPEG-4 AVC standard, while maintaining the spirit of the present principles.

As used herein, “high level syntax” refers to syntax present in the bitstream that resides hierarchically above the macroblock layer. For example, high level syntax, as used herein, may refer to, but is not limited to, syntax at the slice header level, Supplemental Enhancement Information (SEI) level, Picture Parameter Set (PPS) level, Sequence Parameter Set (SPS) level and Network Abstraction Layer (NAL) unit header level.

Scalable video coding (SVC) is an extension (Annex G) to the MPEG-4 AVC Standard. In SVC, a video signal can be encoded into a base layer and one or more enhancement layers constructed in a pyramidal fashion. An enhancement layer enhances the temporal resolution (i.e., the frame rate), the spatial resolution, or simply the quality of the video content represented by another layer or part thereof. Each layer together with all its dependent layers is one representation of the video signal at a certain spatial resolution, temporal resolution, and quality level. Each layer in combination with all its dependent layers that require decoding the video signal at a certain spatial resolution, temporal resolution and quality level are denoted by an interoperability point (IOP), also referred to as operation points. An SVC bitstream typically has multiple IOPs, due at least in part to the fact that the bitstream is scalable. Such a bitstream may be scalable spatially, temporally, and in Signal-to-Noise Ratio (SNR), for example. Sub-bitstreams, corresponding to the scalable aspects, may be extracted from the bitstream.

Turning toFIG.1, an exemplary scalable video encoder to which the present invention may be applied is indicated generally by the reference numeral100.

A first output of a temporal decomposition module142is connected in signal communication with a first input of an intra prediction for intra block module146. A second output of the temporal decomposition module142is connected in signal communication with a first input of a motion coding module144. An output of the intra prediction for intra block module146is connected in signal communication with an input of a transform/entropy coder (signal to noise ratio (SNR) scalable)149. A first output of the transform/entropy coder149is connected in signal communication with a first input of a multiplexer140.

A first output of a temporal decomposition module132is connected in signal communication with a first input of an intra prediction for intra block module136. A second output of the temporal decomposition module132is connected in signal communication with a first input of a motion coding module134. An output of the intra prediction for intra block module136is connected in signal communication with an input of a transform/entropy coder (signal to noise ratio (SNR) scalable)139. A first output of the transform/entropy coder139is connected in signal communication with a first input of a multiplexer130.

A second output of the transform/entropy coder149is connected in signal communication with an input of a 2D spatial interpolation module138. A second output of the motion coding module144is connected in signal communication with an input of the motion coding module134.

A first output of a temporal decomposition module122is connected in signal communication with a first input of an intra predictor126. A second output of the temporal decomposition module122is connected in signal communication with a first input of a motion coding module124. An output of the intra predictor126is connected in signal communication with an input of a transform/entropy coder (signal to noise ratio (SNR) scalable)129. An output of the transform/entropy coder129is connected in signal communication with a first input of a multiplexer120.

A second output of the transform/entropy coder139is connected in signal communication with an input of a 2D spatial interpolation module128. A second output of the motion coding module134is connected in signal communication with an input of the motion coding module124.

A first output of the motion coding module124, a first output of the motion coding module134, and a first output of the motion coding module144are each connected in signal communication with a second input of the multiplexer170.

A first output of a 2D spatial decimation module104is connected in signal communication with an input of the temporal decomposition module132. A second output of the 2D spatial decimation module104is connected in signal communication with an input of the temporal decomposition module142.

An input of the temporal decomposition module122and an input of the 2D spatial decimation module104are available as inputs of the encoder100, for receiving input video102.

An output of the multiplexer170is available as an output of the encoder100, for providing a bitstream180.

The temporal decomposition module122, the temporal decomposition module132, the temporal decomposition module142, the motion coding module124, the motion coding module134, the motion coding module144, the intra predictor126, the intra predictor136, the intra predictor146, the transform/entropy coder129, the transform/entropy coder139, the transform/entropy coder149, the 2D spatial interpolation module128, and the 2D spatial interpolation module138are included in a core encoder portion187of the encoder100.

Turning toFIG.2, an exemplary scalable video decoder to which the present invention may be applied is indicated generally by the reference numeral200. An input of a demultiplexer202is available as an input to the scalable video decoder200, for receiving a scalable bitstream. A first output of the demultiplexer202is connected in signal communication with an input of a spatial inverse transform SNR scalable entropy decoder204. A first output of the spatial inverse transform SNR scalable entropy decoder204is connected in signal communication with a first input of a prediction module206. An output of the prediction module206is connected in signal communication with a first input of a combiner230.

A second output of the spatial inverse transform SNR scalable entropy decoder204is connected in signal communication with a first input of a motion vector (MV) decoder210. An output of the MV decoder210is connected in signal communication with an input of a motion compensator232. An output of the motion compensator is connected in signal communication with a second input of the combiner230.

A second output of the demultiplexer202is connected in signal communication with an input of a spatial inverse transform SNR scalable entropy decoder212. A first output of the spatial inverse transform SNR scalable entropy decoder212is connected in signal communication with a first input of a prediction module214. A first output of the prediction module214is connected in signal communication with an input of an interpolation module216. An output of the interpolation module216is connected in signal communication with a second input of the prediction module206. A second output of the prediction module214is connected in signal communication with a first input of a combiner240.

A second output of the spatial inverse transform SNR scalable entropy decoder212is connected in signal communication with a first input of an MV decoder220. A first output of the MV decoder220is connected in signal communication with a second input of the MV decoder210. A second output of the MV decoder220is connected in signal communication with an input of a motion compensator242. An output of the motion compensator242is connected in signal communication with a second input of the combiner240.

A third output of the demultiplexer202is connected in signal communication with an input of a spatial inverse transform SNR scalable entropy decoder222. A first output of the spatial inverse transform SNR scalable entropy decoder222is connected in signal communication with an input of a prediction module224. A first output of the prediction module224is connected in signal communication with an input of an interpolation module226. An output of the interpolation module226is connected in signal communication with a second input of the prediction module214.

A second output of the prediction module224is connected in signal communication with a first input of a combiner250. A second output of the spatial inverse transform SNR scalable entropy decoder222is connected in signal communication with an input of an MV decoder230. A first output of the MV decoder230is connected in signal communication with a second input of the MV decoder220. A second output of the MV decoder230is connected in signal communication with an input of a motion compensator252. An output of the motion compensator252is connected in signal communication with a second input of the combiner250.

An output of the combiner250is available as an output of the decoder200, for outputting a layer0signal. An output of the combiner240is available as an output of the decoder200, for outputting a layer1signal. An output of the combiner230is available as an output of the decoder200, for outputting a layer2signal.

We propose to modify Network Abstraction Layer (NAL) Hypothetical Reference Decoder (HRD) parameters and Virtual Coding Layer (VCL) HRD parameters for the Hypothetical Reference Decoder (HRD) for Scalable Video Coding (SVC). However, we do not consider other information in VUI, in particular, bitstream restriction information. Therefore, in accordance with the present principles, modifications are proposed for additional VUI information in SVC.

Thus, in accordance with the present principles, we propose to modify MPEG-4 AVC Standard Video Usability Information (VUI) for Scalable Video Coding (SVC). In an embodiment, we define VUI for each interoperability point (IOP) of SVC. In particular, we consider the bitstream restriction information in VUI and how to modify the bitstream restriction information for SVC. We also consider how to use Supplemental Enhancement Information (SEI) messages to convey VUI information for the MPEG-4 AVC Standard compatible case.

Currently, the Hypothetical Reference Decoder parameters in the Video VUI are set for each IOP for Scalable Video Coding, but the other VUI information uses only one set of VUI parameter for the bitstream. Since the decoder is supposed to decode only subsets of the video signals (corresponding to IOPs), the size of the data corresponding to each IOP which is required to be transmitted and decoded varies from IOP to IOP. Therefore, we believe that besides Hypothetical Reference Decoder parameters, some other VUI information should be sent to each IOP or a set of IOPs for some information. In particular, we are considering bitstream restriction information in VUI.

In an embodiment, we use bitstream restriction information as an example. However, it is to be appreciated that other VUI information may also be used in accordance with the present principles. That is, given the teachings of the present principles provided herein, one of ordinary skill in this and related arts will contemplate these and various other types of VUI information to which the present principles may be applied, while maintaining the spirit of the present principles.

In one embodiment, we propose a set of IOPs share the same bitstream restriction information. For example, all layers with the same dependency_id share the same bitstream restriction information.

In another embodiment, we propose defining the bitstream restriction information for each IOP. An IOP can be uniquely identified by the combination of dependency_id, temporal_id and quality_id. Given i is the index of IOP, the bitstream restriction information includes the following syntax:bitstream_restriction_flag [i]—indicates that the bitstream restriction information for the current scalable layer is present in the SEI message. bitstream_restriction_flag[i] equal to 0 specifies that the bitstream restriction information for the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] is not present in the SEI message.motion_vectors_over_pic_boundaries_flag [i]—specifies the value of motion_vectors_over_pic_boundaries_flag of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i]. When the motion_vectors_over_pic_boundaries_flag[i] syntax element is not present, motion_vectors_over_pic_boundaries_flag value of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] shall be inferred to be equal to 1.max_bytes_per_pic_denom [i]—specifies the max_bytes_per_pic_denom value of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i]. When the max_bytes_per_pic_denom[i] syntax element is not present, the value of max_bytes_per_pic_denom of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] shall be inferred to be equal to 2.max_bits_per_mb_denom [i]—specifies the max_bits_per_mb_denom value of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i]. When the max_bits_per_mb_denom[i] is not present, the value of max_bits_per_mb_denom of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] shall be inferred to be equal to 1.log 2_max_mv_Iength_horizontal [i] and log 2_max_mv_Iength_vertical [i]—specify the values of log_2_max_mv_length_horizontal and log_2_max_mv_length_vertical of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i]. When log_2_max_mv_length_horizontal[i] is not present, the values of log_2_max_mv_length_horizontal and log_2_max_mv_length_vertical of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] shall be inferred to be equal to 16.num_reorder_frames [i]—specifies the value of num_reorder_frames of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i]. The value of num_reorder_frames[i] shall be in the range of 0 to max_dec_frame_buffering, inclusive. When the num_reorder_frames[i] syntax element is not present, the value of num_reorder_frames of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] shall be inferred to be equal to max_dec_frame_buffering.max_dec_frame_buffering [i] specifies the value of max_dec_frame_buffering of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i]. The value of max_dec_frame_buffering[i] shall be in the range of num_ref_frames[i] to MaxDpbSize (as specified in sub-clause A.3.1 or A.3.2 in the MPEG-4 AVC Standard), inclusive. When the max_dec_frame_buffering[i] syntax element is not present, the value of max_dec_frame_buffering of the scalable layer having the dependency_id equal to dependency_id[i], temporal_id equal to temporal_id[i] and quality_id equal to quality_id[i] shall be inferred to be equal to MaxDpbSize.

Moreover, TABLE 2 includes the proposed VUI syntax for this embodiment relating to defining bitstream restriction information for each IOP.

TABLE 2CDescriptorvui_parameters( ) {aspect_ratio_info_present_flag0u(1)if( aspect_ratio_info_present_flag ) {aspect_ratio_idc0u(8)if( aspect_ratio_idc = = Extended_SAR ) {sar_width0u(16)sar_height0u(16)}}overscan_info_present_flag0u(1)if( overscan_info_present_flag )overscan_appropriate_flag0u(1)video_signal_type_present_flag0u(1)if( video_signal_type_present_flag ) {video_format0u(3)video_full_range_flag0u(1)colour_description_present_flag0u(1)if( colour_description_present_flag ) {colour_primaries0u(8)transfer_characteristics0u(8)matrix_coefficients0u(8)}}chroma_loc_info_present_flag0u(1)if( chroma_loc_info_present_flag ) {chroma_sample_loc_type_top_field0ue(v)chroma_sample_loc_type_bottom_field0ue(v)}if( profile_idc == ‘SVC’) {num_layers_minus10ue(v)for( i = 0; i <= num_layers_minus1; i++ ) {temporal_level[ i ]0u(3)dependency_id[ i ]0u(3)quality_level[ i ]0u(2)timing_info_present_flag[ i ]0u(1)if( timing_info_present_flag[ i ] ) {num_units_in_tick[ i ]0u(32)time_scale[ i ]0u(32)fixed_frame_rate_flag[ i ]0u(1)}nal_hrd_parameters_present_flag[ i ]0u(1)if( nal_hrd_parameters_present_flag[ i ] )hrd_parameters( )vcl_hrd_parameters_present_flag[ i ]0u(1)if( vcl_hrd_parameters_present_flag[ i ])hrd_parameters( )if( nal_hrd_parameters_present_flag[ i ] | |vcl_hrd_parameters_present_flag[ i ] )low_delay_hrd_flag[ i ]0u(1)pic_struct_present_flag[ i ]0u(1)bitstream_restriction_flag [i]0u(1)if( bitstream_restriction_flag [i] ) {motion_vectors_over_pic_boundaries_flag [ i ]0u(1)max_bytes_per_pic_denom [ i ]0ue(v)max_bits_per_mb_denom [ i ]0ue(v)log2_max_mv_length_horizontal [ i ]0ue(v)log2_max_mv_length_vertical [ i ]0ue(v)num_reorder_frames [ i ]0ue(v)max_dec_frame_buffering [ i ]0ue(v)}}} else {timing_info_present_flag0u(1)if( timing_info_present_flag ) {num_units_in_tick0u(32)time_scale0u(32)fixed_frame_rate_flag0u(1)}nal_hrd_parameters_present_flag0u(1)if( nal_hrd_parameters_present_flag )hrd_parameters( )vcl_hrd_parameters_present_flag0u(1)if( vcl_hrd_parameters_present_flag )hrd_parameters( )if( nal_hrd_parameters_present_flag | |vcl_hrd_parameters_present_flag )low_delay_hrd_flag0u(1)pic_struct_present_flag0u(1)}bitstream_restriction_flag0u(1)if( bitstream_restriction_flag ) {motion_vectors_over_pic_boundaries_flag0u(1)max_bytes_per_pic_denom0ue(v)max_bits_per_mb_denom0ue(v)log2_max_mv_length_horizontal0ue(v)log2_max_mv_length_vertical0ue(v)num_reorder_frames0ue(v)max_dec_frame_buffering0ue(v)}}

Turning toFIG.3, an exemplary method for encoding Video User Information (VUI) is indicated generally by the reference numeral300.

The method300includes a start block305that passes control to a decision block310. The decision block310determines whether or not profile_idc is equal to SVC. If so, then control is passed to a function block315. Otherwise, control is passed to a function block350.

The function block315sets a variable M equal to the number of layers −1, and passes control to a function block320. The function block320writes the variable M to the bitstream, and passes control to the function block325. The function block325sets a variable i equal to zero, and passes control to a function block330. The function block330writes layer i's temporal level, dependency_id, and quality_level to the bitstream, and passes control to a function block335. The function block335writes layer i's timing information and HRD parameters to the bitstream, and passes control to a function block340. The function block340writes layer i's bitstream restriction information to the bitstream, and passes control to a decision block345. The decision block345determines whether or not the variable i is equal to the variable M. If so, control is passed to an end block399. Otherwise, control is passed to a function block360.

The function block350writes timing information and HRD parameters to the bitstream, and passes control to a function block355. The function block355writes bitstream restriction information to the bitstream, and passes control to the end block399.

The function block360increments the variable i by one, and returns control to the function block330.

Turning toFIG.4, an exemplary method for decoding Video User Information (VUI) is indicated generally by the reference numeral400.

The method400includes a start block405that passes control to a decision block410. The decision block410determines whether or not profile_idc is equal to SVC. If so, the control is passed to a function block415. Otherwise, control is passed to a function block450.

The function block415reads a variable M from the bitstream, and passes control to a function block420. The function block420sets the number of layers equal to M+1, and passes control to a function block425. The function block425sets a variable i equal to zero, and passes control to a function block430. The function block430reads layer i's temporal_level, dependency_id, and quality_level from the bitstream, and passes control to a function block435. The function block435reads layer i's timing information and HRD parameters from the bitstream, and passes control to a function block440. The function block440reads layer i's bitstream restriction information from the bitstream, and passes control to a decision block445. The decision block445determines whether or not the variable i is equal to the variable M. If so, the control is passed to an end block499. Otherwise, control is passed to a function block460.

The function block450reads timing information and HRD parameters from the bitstream, and passes control to a function block455. The function block455reads bitstream restriction information from the bitstream, and passes control to the end block499.

The function block460increments the variable i by one, and returns control to the function block430.

SVC required the base layer to be compatible with the MPEG-4 AVC Standard. However, the MPEG-4 AVC Standard compatible bitstream may include several temporal layers. According to an embodiment of the present principles, we propose to use high level syntax to convey the bitstream restriction information for different temporal layers in an MPEG-4 AVC Standard compatible layer. In one embodiment, a Supplemental Enhancement Information (SEI) message is used. Of course, the present principles are not limited solely to the use of SEI messages with respect to high level syntax and, thus, other high level syntaxes may also be used in accordance with the present principles, while maintaining the spirit of the present principles. TABLE 3 illustrates a proposed AVC temporal Video User Information (VUI) Supplemental Enhancement Information (SEI) message, in accordance with an embodiment of the present principles. The following syntax definitions apply to the syntaxes set forth in TABLE 3.

TABLE 3CDescriptoravc_temporal_vui ( payloadSize ) {num_of_temporal_layers_in_base_layer_minus10ue(v)for( i = 0; i < num_of_temporal_layers_in_base_layer_minus1; i++){temporal_level[ i ]0u(3)bitstream_restriction_flag0u(1)if( bitstream_restriction_flag ) {motion_vectors_over_pic_boundaries_flag0u(1)max_bytes_per_pic_denom0ue(v)max_bits_per_mb_denom0ue(v)log2_max_mv_length_horizontal0ue(v)log2_max_mv_length_vertical0ue(v)num_reorder_frames0ue(v)max_dec_frame_buffering0ue(v)}}}

Turning toFIG.5, an exemplary method for encoding an MPEG-4 AVC Standard temporal Supplemental Enhancement Information (SEI) message is indicated generally by the reference numeral500.

The method500includes a start block505that passes control to a function block510. The function block510sets a variable M equal to the number of temporal subsets in the base layer −1, and passes control to a function block515. The function block515writes the variable M to the bitstream, and passes control to a function block520. The function block520sets a variable i equal to zero, and passes control to a function block525. The function block525writes layer i's temporal_level to the bitstream, and passes control to a function block530. The function block530writes layer i's bitstream restriction information to the bitstream, and passes control to a decision block535. The decision block535determines whether or not the variable i is equal to the variable M. If so, the control is passed to an end block599. Otherwise, control is passed to a function block540. The function block540increments the variable i by one, and returns control to the function block525.

Turning toFIG.6, an exemplary method for decoding an MPEG-4 AVC Standard temporal Supplemental Enhancement Information (SEI) message is indicated generally by the reference numeral600.

The method600includes a start block605that passes control to a function block610. The function block610reads a variable M from the bitstream, and passes control to a function block615. The function block615sets the number of temporal subsets in the base layer equal to the variable M+1, and passes control to a function block620. The function block620sets a variable i equal to zero, and passes control to a function block625. The function block625reads layer i's temporal_level from the bitstream, and passes control to a function block630. The function block630reads layer i's bitstream restriction information from the bitstream, and passes control to a decision block635. The decision block635determines whether or not the variable i is equal to the variable M. If so, the control is passed to an end block699. Otherwise, control is passed to a function block640.

The function block640increments the variable i by one, and returns control to the function block625.

TABLE 4 shows another implementation of the bitstream restriction information in Scalability information SEI message.

TABLE 4CDescriptorscalability_info( payloadSize ) {temporal_id_nesting_flag5u(1)quality_layer_info_present_flag5u(1)priority_id_setting_flag5u(1)num_layers_minus15ue(v)for( i = 0; i <= num_layers_minus1; i++ ) {layer_id[ i ]5ue(v)priority_id[ i ]5u(6)discardable_flag[ i ]5u(1)temporal_id[ i ]5u(3)dependency_id[ i ]5u(3)quality_id[ i ]5u(4)...bitstream_restriction_info_present_flag[ i ]5u(1)...if( bitstream restriction info_present_flag[ i ] ) {motion_vectors_over_pic_boundaries_flag[ i ]5u(1)max_bytes_per_pic_denom[ i ]5ue(v)max_bits_per_mb_denom[ i ]5ue(v)log2_max_mv_length_horizontal[ i ]5ue(v)log2_max_mv_length_vertical[ i ]5ue(v)num_reorder frames[ i ]5ue(v)max dec frame buffering[ i ]5ue(v)}...}}

A description will now be given of some of the many attendant advantages/features of the present invention, some of which have been mentioned above. For example, one advantage/feature is an apparatus that includes an encoder for encoding video signal data into a bitstream. The encoder specifies video user information, excluding hypothetical reference decoder parameters, in the bitstream using a high level syntax element. The video user information corresponds to a set of interoperability points in the bitstream relating to scalable video coding.

Another advantage/feature is the apparatus having the encoder as described above, wherein the encoder specifies the video user information for each of the interoperability points in the bitstream relating to scalable video coding, including the set of interoperability points, using the high level syntax element.

Yet another advantage/feature is the apparatus having the encoder as described above, wherein the encoder specifies the video user information for each of the interoperability points relating to scalable video coding at a layer compatible with the International Organization for Standardization/International Electrotechnical Commission Moving Picture Experts Group-4 Part 10 Advanced Video Coding standard/International Telecommunication Union, Telecommunication Sector H.264 recommendation, using the high level syntax element.

Still another advantage/feature is the apparatus having the encoder as described above, wherein the video user information includes bitstream restriction information.

Moreover, another advantage/feature is the apparatus having the encoder wherein the video user information includes bitstream restriction information as described above, wherein the bitstream restriction information includes at least one of a motion_vectors_over_pic_boundaries_flag syntax element, a max_bytes_per_pic_denom syntax element, a max_bits_per_mb_denom syntax element, a log_2_max_mv_length_horizontal syntax element, a log_2_max_mv_length_vertical syntax element, a num_reorder_frames syntax element, and a max_dec_frame_buffering syntax element.

Further, another advantage/feature is the apparatus having the encoder as described above, wherein the high level syntax element corresponds to at least at one of a slice header level, a sequence parameter set level, a picture parameter set level, a network abstraction layer unit header level, and a level corresponding to a supplemental enhancement information message.

Also, another advantage/feature is the apparatus having the encoder as described above, wherein a profile_idc syntax element is used to differentiate the bitstream for scalable video coding or for compliance with the International Organization for Standardization/International Electrotechnical Commission Moving Picture Experts Group-4 Part 10 Advanced Video Coding standard/International Telecommunication Union, Telecommunication Sector H.264 recommendation.

Additionally, another advantage/feature is the apparatus having the encoder as described above, wherein the encoder writes a dependency_id syntax element, a temporal_level syntax element, and a quality_level syntax element to the bitstream for at least each of the interoperability points in the set.

Moreover, another advantage/feature is the apparatus having the encoder as described above, wherein the encoder writes a temporal_level syntax element and a quality_level syntax element to the bitstream for at least each of the interoperability points in the set.

Further, another advantage/feature is the apparatus having the encoder as described above, wherein the encoder writes a temporal_level syntax element to the bitstream for at least each of the interoperability points in the set.

These and other features and advantages of the present principles may be readily ascertained by one of ordinary skill in the pertinent art based on the teachings herein. It is to be understood that the teachings of the present principles may be implemented in various forms of hardware, software, firmware, special purpose processors, or combinations thereof.

Most preferably, the teachings of the present principles are implemented as a combination of hardware and software. Moreover, the software may be implemented as an application program tangibly embodied on a program storage unit. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPU”), a random access memory (“RAM”), and input/output (“I/O”) interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit.

It is to be further understood that, because some of the constituent system components and methods depicted in the accompanying drawings are preferably implemented in software, the actual connections between the system components or the process function blocks may differ depending upon the manner in which the present principles are programmed. Given the teachings herein, one of ordinary skill in the pertinent art will be able to contemplate these and similar implementations or configurations of the present principles.

Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the present principles is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present principles. All such changes and modifications are intended to be included within the scope of the present principles as set forth in the appended claims.