Patent Description:
Digital video compression technologies, for example high efficiency video coding (HEVC) that enhances the efficiency of digital video communication, distribution and/or consumption continue to be developed. In comparison with the traditional digital video services (such as TV signals), video applications may be deployed in heterogeneous environments. Such heterogeneity may exist on the client side as well as the network side. Scalable video coding mechanisms may be used to encode a video signal once at the highest resolution, but may enable decoding from subsets of the streams depending on the specific rate and resolution required by certain applications and/or supported by the client device.

Video compression technologies may be provided that may include scalable video coding techniques to improve the experience of an end user and/or quality of service. For example, the scalable video coding may be implemented with high level syntax designs via one or more parameter sets. However, the parameter sets and the syntax elements that may be signaled may inefficiently consume valuable communication bandwidth and/or processing resources. The document <NPL><NPL>) discloses the signalling of a list of reference picture layer IDs.

Systems, methods, and instrumentalities are provided to implement video data processing. A video coding device may receive a plurality of video representation format subsets. The video coding device may receive a count of the plurality of video representation format subsets. The plurality of video representation format subsets may correspond to a plurality of layers. For example, a first video representation format subset of the plurality of video representation format subsets may correspond to a first layer and a second video representation format subset of the plurality of video representation format subsets may correspond to a second layer, or to multiple layers.

Each of the plurality of video representation subsets may include one or more video representation parameter values. The parameter values may include at least one of an indication of picture width in luma samples, an indication of picture height in luma samples, an indication of bit depth of one or more samples of a luma array, an indication of bit depth of one or more samples of a chroma array, or an indication of a chroma format index.

The video coding device may receive a video representation format subset index associated with a current layer. For example, the video representation format subset index may be received in a cross-layer parameter set and/or a sequence parameter set (SPS).

The video coding device may determine, using the video representation format subset index, one of the plurality of video representation format subsets associated with the current layer.

A video encoding device may compare a number of active reference layers with a number of direct reference layers. The video encoding device may determine whether to include an indication of the active reference layers in a slice-level header based on the comparing of the number of active reference layers with the number of direct reference layers.

The video encoding device may include the indication of a picture that may be used for inter-layer prediction, e.g., if that the number of active reference layers is not equal to the number of direct reference layers. The video encoding device may skip the indication of a picture that may be used for inter-layer prediction, e.g., if the number of active reference layers is equal to the number of direct reference layers.

A video decoding device may receive a bitstream comprising one or more layers. On a condition that the number of active reference layers in the received bitstream is not equal to the number of direct reference layers, the video decoding device may receive an inter-layer prediction layer syntax element. The inter-layer prediction layer syntax element indicates a list of reference picture layer IDs that may be used by a current picture of a current layer for inter-layer prediction.

The video decoding device may derive the inter-layer prediction layer syntax element, e.g., if the number of active reference layers is equal to the number of direct reference layers. The inter-layer prediction layer syntax element may be inferred from a layer ID of a direct reference layer of the current layer.

A detailed description of illustrative embodiments will now be described with reference to the various figures. Although this description provides a detailed example of possible implementations, it should be noted that the details are intended to be exemplary and in no way limit the scope of the application.

In a video coding system, on a client device side, the N-screen scenario, that is, consuming video content on devices with varying screen sizes and/or display capabilities, for example smart phones, tablets, PCs, HDTVs, and the like, is expected to continue. On the communications network side, video may be transmitted across one or more of the Internet, WiFi networks, mobile communications networks (e.g., <NUM>, <NUM>, etc.), and the like, or any combination thereof.

To improve the user experience (e.g., for an end user of a client device) and/or video quality of service, scalable video coding may be implemented. With scalable video coding, a video signal may be encoded once at a highest resolution. Such a video signal may be decoded from one or more subsets of one or more video streams associated with the video signal, for example in accordance with a specific rate and/or resolution that may be required by a particular application and/or that may be supported by a client device. A resolution may include one or more video parameters, such as spatial resolution (e.g., picture size), temporal resolution (e.g., frame rate), and video quality (e.g., subjective quality such as MOS and/or objective quality such as PSNR, SSIM, or VQM). Other video parameters that may be used include chroma format (e.g., YUV420, YUV422, or YUV444), bit-depth (e.g., <NUM>-bit or <NUM>-bit video), complexity, view, gamut (e.g. color gamut), and/or aspect ratio (e.g., <NUM>:<NUM> or <NUM>:<NUM>).

Video standards may include tools and/or profiles that may support scalability modes. For example, high efficiency video coding may be configured to support scalable video coding. An example scalable extension to HEVC may support one or more of spatial scalability (e.g., a scalable bitstream may include respective signals at more than one spatial resolution), quality scalability (e.g., a scalable bitstream may include respective signals at more than one quality level), and standard scalability (e.g., a scalable bitstream may include a base layer coded using H. <NUM>/AVC and one or more enhancement layers coded using HEVC). Scalable video may be extended to 3D video. For example, multi-view scalability may be implemented (e.g., a scalable bitstream may include both 2D and 3D video signals). It should be appreciated that while aspects of scalable HEVC design may include the use of spatial and/or quality scalability, for example as described herein, that the techniques described herein may be applied to one or more other types of scalability.

Scalable extensions of HEVC (SHVC) may be implemented in accordance with a reference index based framework. A reference index based framework may keep operations at block level and/or below block level intact, such that single layer codec logics may be reused within a scalable coding system that employs such a framework. A reference index based framework may simplify the design of a scalable codec. Such a framework may support different types of scalabilities, for example by incorporating high level syntax signaling and/or inter-layer processing modules to achieve coding efficiency. High level syntax changes may be implemented to support inter-layer processing and/or multi-layer signaling of SHVC, for example. Such syntax changes may be implemented in accordance with a reference index based framework, for example.

Scalable video coding may support one or more layers (e.g., multiple layers). Each such layer may be designed to enable one or more of spatial scalability, temporal scalability, SNR scalability, or other types of scalability. A scalable bit stream, for example, may include mixed scalability layers, and one or more respective enhancement layers may depend on one or more lower layers in order to be decoded. An inter-layer process may generate an inter-layer reference picture (ILR) sample and/or motion field information, for example to enhance the prediction accuracy of one or more enhancement layers.

Several parameter sets may be specified for an HEVC implementation and/or for one or more corresponding extensions. For example, a video parameter set (VPS) may include one or more syntax elements that may be shared by multiple layers. A VPS may include information used for bitstream extraction, capability exchange, and/or session negotiation (e.g., maximum number of layers and/or one or more of profile, tier, and level information).

A sequence parameter set (SPS) may include information that may be common to one or more coded slices (e.g., all coded slices) in a coded video sequence, such as a series of video pictures spanning an interval of time. Such information may include one or more of picture resolution, bit depth, coding block size, and the like.

A picture parameter set (PPS) may include picture level information. Such information may include one or more of an initial quantization value, coding tools enable and/or disable flags, and the like. Information carried in the PPS may remain unchanged through a relatively long duration, for example a duration of multiple pictures, such that the information may not be updated frequently. Information that may be changed on the slice level may be included in a slice header.

One or more parameter sets, such as an VPS, an SPS and/or an PPS may be transmitted out-of-band (e.g., using a reliable channel, as in some application scenarios). A high level syntax design may allow multiple layers to refer to a single SPS (e.g., the same SPS). This may be used for multi-view and/or SNR scalability, for example. For spatial scalability, one or more layers (e.g., each layer) may refer to respective different SPSs, for example due to different video resolutions. If one or more parameters in an SPS (e.g., a majority of the parameters) are identical across multiple layers, it may be desirable to save bitrate by removing such redundancy. One or more such parameters may be shared among the multiple layers.

In an example approach to save bitrate, SPS to SPS prediction may be implemented, which may be used to predict one or more enhancement layer SPS parameters, such as a scaling list, a reference picture set (RPS), etc., for example from the SPS parameters of a base layer and/or another dependent layer. Such SPS to SPS prediction may introduce parameter set dependency among different layers.

In another example to save bitrate, VPS to SPS prediction may be implemented, which may relocate one or more shared parameters across layers to the VPS, and may predict one or more shared SPS parameters (e.g., SPS parameters of each layer) based on corresponding parameters in the VPS.

Design criteria for VPS and/or SPS implementations for HEVC extensions may include one or more of the following. The VPS may include one or more parameters that may be useful for bitstream extraction and/or capability exchange. Decoded picture buffer (DPB) related parameters may be included in a VPS extension.

A parameter set (e.g., an inter-layer parameter set (IPS)) may be implemented that may aggregate one or more high level syntax elements that may be shared among multiple layers. One or more layers (e.g., each layer) may refer to one or more IPS parameters, which may save corresponding overhead bits.

An IPS may be provided in a scalable HEVC video coding system. For example, because IPS may not be carried in the base layer, the size of the IPS may not impact the base layer sub-stream. IPS may provide high level signaling efficiency, for example by facilitating the prediction of one or more shared parameters across multiple layers. An implementation of an IPS may remove parsing dependency in a video coding system, for example because one or more parameters that may typically be placed in different parameter sets may be included in the same IPS, such that the parsing of each parameter may not rely on a parsing result from another different parameter set.

IPS may be applicable to one or more enhancement layers in a scalable coding system, such that the nuh_layer_id value of an IPS NAL unit may not be zero (<NUM>) for a conforming bitstream. For example, a conforming bitstream may have the nuh_layer_id of one or more IPS NAL units (e.g., all IPS NAL units) equal to one (<NUM>).

<FIG> are syntax tables that illustrate an example IPS. As illustrated in <FIG>, an IPS may include one or more parameters and may be designed for the purpose of coding multiple layers. Such parameters may include, for example, max_sublayer_for_ilp_plus1 and direct_dependency_type. Because one or more layers may share the same or very similar RPSs, the IPS may include RPSs related to the one or more layers.

One or more parameters that may serve a similar purpose or similar purposes, and that may be present in the SPS, may be grouped into respective subsets that may include one or more of a video format subset, a coding parameter subset, a scaling list subset, a scaled offset subset, or a VUI subset. In an IPS, one or more subsets (e.g., each subset) may have respective pluralities of parameter values. This may allow an enhancement layer to refer to a plurality of parameter values by indexing into the IPS and a subset. For example, a first video format set (e.g., <NUM>) may specify 720p format and a second video format set (e.g., <NUM>) may specify 1080p format. For a mixed spatial and/or SNR scalability coding system with four (<NUM>) layers (e.g., where layer-<NUM> is a 720p layer and layers <NUM>, <NUM>, and <NUM> are 1080p layers), a base layer (e.g., layer-<NUM>) SPS may refer to ips_video_format_subset(<NUM>), while enhancement layers (e.g., layers <NUM>, <NUM>, and <NUM>) may refer to ips_video_format_subset(<NUM>). In such an example, a reduced (e.g., minimal) number of syntax elements may be signaled to cover parameters used by multiple layers.

The following may apply to entries in the example IPS syntax tables shown in <FIG>. The syntax element ips_inter_layer_view_parameter_set_id may identify the IPS for reference by other syntax elements. The syntax element num_video_format_subsets may specify the number of video format syntax structures (ips_video_format_subset). The syntax element num_coding_param_subsets may specify the number of coding parameter syntax structure (ips_coding_param_subset). The syntax element num_pcm_param_subsets may specify the number of PCM coding parameter syntax structure (ips_pcm_param_subset). The syntax element num_scaling_list_subsets may specify the number of scaling list structure (ips_scaling_list_subset). The syntax element num_scaled_ref_layer_offset_subset may specify the number of scaled reference layer offset structures
(ips_scaled_ref_layer_offset_subset). The syntax element num_vui_param_subsets may specify the VUI parameter structure (ips_vui_param_subset).

One or more video representation formats may be grouped into a subset. One or more subsets may be signaled in a parameter set (e.g., an IPS). The subsets may be referenced by one or more layers. For example a first layer may reference a first subset. One or more layers may reference a second subset. Each of the layers may refer to the index of the subsets to derive the video representation syntax values. One or more subsets, e.g., in an IPS, may be implemented to further save the bits signaling IPS syntax elements (e.g., overhead bits). For example, an absolute parameter value limited to the first set of parameter values of a given subset may be signaled. For one or more subsequent sets of parameter values, respective differential values between a current set of parameter values and a previous set of parameter values may be signaled. To illustrate, ips_video_formatsubset(<NUM>) may indicate 720p format (pic_width_in_luma_samples may be set to <NUM> and pic_height_in_luma_samples may be set to <NUM>), and ips_video_format_set(<NUM>) may indicate 1080p format (pic_width_in_luma_samples may be set to <NUM> and pic_height_in_luma_samples may be set to <NUM>). Rather than signaling both <NUM> and <NUM>, respective differential values between ips_video_format_set(<NUM>) and ips_video_format_set(<NUM>) may be signaled. In accordance with the example, differential values of <NUM> and <NUM> for width and height, respectively, may be signaled in ips_video_format_set(<NUM>).

A corresponding syntax table of such an IPS may be as is illustrated in <FIG>, while the descriptor type may be changed to ue(v) or se(v), for example, for those parameters that may be signaled as differential values. The value of the relevant parameter may be derived as follows.

For example, if S(i) is the i-th set of parameters for a given subset, the variable P(i, X) is the parameter X in the i-th parameter set S(i), and/or the variable ParamValueInIPS(i, X) is the value signaled for P(i, X) in IPS. The variable ParamValue(i, X) of parameter X in the i-th parameter subset, P(i, X), may be derived from the parameter X in the (i-<NUM>)-th parameter subset, P(i-<NUM>, X), for example, as follows.

The SPS and/or its extension may be simplified, for example as depicted in <FIG> and <FIG>, respectively. Rather than carrying similar syntax elements in the SPS for one or more enhancement layers, the enhancement layer SPS syntax table may be simplified, for example by including reference to an IPS parameter set index.

One or more syntax elements of the SPS (e.g., all such syntax elements) may be kept intact for the base layer (nuh_layer_id = <NUM>), as represented by the shaded entries in <FIG>. This may allow for backward compatibility, for example with a single layer HEVC specification. Example syntax elements that may be added to the SPS in accordance with an implementation of an IPS are represented by italicized text in <FIG> and <FIG>.

The following may apply to entries in the simplified SPS and extension syntax tables shown in <FIG> and <FIG>. As illustrated in <FIG> and <FIG>, the syntax element sps_inter_layer_view_parameter_set_id may specify the value of the ips_inter_layer_view_parameter_set_id of an active parameter set (e.g., an IPS). The syntax element ips_video_format_subsets_index signaled in a parameter set may specify the index, into the list of video representation format syntax structures included in the active parameter set. The syntax element ips_video_format_subsets_index may specify the representation format syntax structure that may apply to the layers that refer to this SPS. The range of ips_video_format_subsets_index may be from <NUM> to num_video_format_subsets_index, exclusive. The syntax element ips_coding_param_subsets_index may specify the index, into the list of coding parameter syntax structures included in the active IPS. The range of ips_coding param_subsets index may be from <NUM> to num_coding_param_subsets_index, exclusive. ips_scaling_list_subsets_index may specify the index, into the list of scaling list syntax structures included in the active IPS. The range of ips_scaling_list_subsets_index may be from <NUM> to num_scaling_list_subsets_index, exclusive. ips_pcm_param_subsets_index may specify the index, into the list of PCM parameter syntax structures included in the active IPS. The range of ips_pcm_param_subsets_index may be from <NUM> to num_pcm_param_subsets_index, exclusive. ips_vui_param_subsets_index may specify the index, into the list of VUI syntax structures included in the active IPS. The range of ips_vui_param_subsets_index may be from <NUM> to num_vui_param_subsets_index, exclusive. ips_scaled_ref_layer_offset_subset_index may specify the index, into the list of video format syntax structures included in the active IPS. The range of ips_scaled_ref_layer_offset_subsets_index may be from <NUM> to num_scaled_ref_layer_offset_subsets_index, exclusive.

These example syntax structure indexes may allow an enhancement layer to derive a plurality of parameter values, for example by indexing into an IPS and a particular subset. For example, to derive the pic_width_in_luma_samples of a layer (e.g., an enhancement layer (EL)), the EL may locate the associated active IPS by the IPS identification (sps_inter_layer_view_parameter_set_id), e.g., present in the given EL's SPS. Using the value of the index ips_video_format_subsets_index in the EL SPS, the EL may locate the particular video format subset present in the associated active IPS,
ips_video_formatsubset(ips_video_format_subsets _index). The value of pic_width_in_luma_samples of EL may be derived from the corresponding parameter value of pic_width_in_luma_samples present in ips_video_format_subset (ips_video_format_subsets_index), for example directly in accordance with the first example IPS signaling method described herein. Alternatively, the value of pic_width_in_luma_samples may be derived from the ParamValue (ips_video_format_subsets _index, pic_width_in_luma_samples), for example in accordance the second example IPS signaling method described herein. The value of pic_height_in_luma_samples of this EL may be derived in a similar manner. <FIG> illustrates such an example to derive the parameters from IPS. Values of one or more other parameters in one or more other parameter subsets may be derived in a similar manner.

An IPS raw byte sequence payload (RBSP) may include one or more parameters that may be referred to by one or more SPS RBSPs. Each IPS RBSP may initially be considered not active, for example at the start of operation of a decoding process. During operation of an example decoding process, at most one IPS RBSP may be considered active. The activation of any IPS RBSP may result in the deactivation of a previously-active IPS RBSP.

When an IPS RBSP is not already active and it is referred to by activation of an SPS RBSP (e.g., in which sps_inter_layer_view_parameter_set_id is equal to the ips_inter_layer_view_parameter_set_id value), the IPS RBSP may be activated. The activated IPS RBSP may be called the active IPS RBSP until it is deactivated, for example, as a result of the activation of another IPS RBSP. An IPS RBSP, with that particular value of ips_interlayer_view_parameter_subset_id, may be available to the decoding process prior to its activation.

A slice header may include information that may change from one slice to other, and picture related information that may be small or relevant for some of the slice and/or picture types. In a video coding standard, e.g., scalable extensions of high efficiency video coding (HEVC) (SHVC), the syntax elements designed for inter-layer prediction, may include sample prediction and motion prediction may have inherent redundancy. The bit cost of slice header may be reduced by removing certain redundancy, for example, through adjusting some signaling logics.

Table <NUM> illustrates an example of an VPS extension syntax table. Table <NUM> illustrates an example of slice header syntax used in a video coding standard, e.g., SHVC.

As illustrated in Table <NUM>, max_one_active_ref_layer_flag may be signaled in VPS extension to specify, e.g., if one or more pictures from one layer or more than one layers may be used for inter-layer prediction in the scalable system. This flag may be used to impose the restriction to allow inter-layer reference pictures from one layer in the scalable system. Such a restriction may be desirable when scalable profiles and/or levels are defined. Depending on the setting of max_one_active_ref_layer_flag, the syntax element num_inter_layer_ref_minus1 in the slice header (e.g., as illustrated in Table <NUM>) may or may not be signaled in the slice header. When max_one_active_ref_layer_flag is equal to <NUM>, num_inter_layer_ref_pic_minus1 may be inferred to be <NUM> and therefore may not be signaled, and one reference layer's layer ID may be signaled in the slice header, otherwise (e.g., when max_one_active_ref_layer_flag is <NUM>), num_inter_layer_ref_pic_minus1 may be signaled. The num_inter_layer_ref_pic_minus1 flag, in the slice header, may be followed by layer IDs of (num_inter__layer_ref_pics_minus <NUM> + <NUM>) layers.

The max_one_active_ref_layer_flag may be replaced by max_num_active_ref_layers _minus <NUM> flag. The descriptor type of the max_num_active_ref_layers_minus1 flag may be ue(v) (e.g., as illustrated in Table <NUM>). The syntax element may indicate the maximum reference layers that may be used in the decoding process to serve the purpose of capability exchange. The appropriate profile and/or level constraints may be defined. The syntax element may be more flexible than a <NUM>-bit flag.

When max_num_active_ref_layers_minus1 is equal to <NUM>,
num_inter_layer_ref_pics_minus1 may not be signaled (e.g., may be omitted) in the slice header. In case at most one reference layer is allowed for inter-layer prediction, the bit cost of such syntax element in VPS extension may be the same as the original max_one_active_ref_layer_flag (e.g., <NUM> bit). In such a case, a video decoding device may infer the inter-layer prediction layer syntax element. Table <NUM> illustrates an example of a slice segment header.

The variable NumActiveRefLayerPics may be derived based on max_num_active_ref_layers_minus1. For example, the varialble may be derived as follows:
<IMG>
<IMG>.

As illustrated in Table <NUM>, inter_layer_pred_layer_idc may be signaled, e.g., when number of active reference layers (e.g., NumActiveRefLayerPics) is not the same as the number of direct reference layers (e.g., NumDirectRefLayers[num_layer_id]). inter_layer_pred_layer_ide may not be present in the bitstream, e.g., when number of active reference layers (e.g., NumActiveRefLayerPics) is the same as the number of direct reference layers (e.g., NumDirectRefLayers[num_layer_id]). For example, inter_layer_pred_layer_ide in a slice header may be redundant. In such a case the indication of a picture that be used for inter-layer prediction may be skipped. In such a case, inter_layer_pred_layer_idc may be derived or inferred from RefLayerId as illustrated in Pseudo Code <NUM>. The value of the variable NumDirectRefLayers may be provided in a standard, e.g., SHVC.

If a slice is not a dependent slice, for error resilience considerations, slice header may be provided for each of the slices in a picture. Since a picture may include one or more slices and the slice header may be provided for each of the slices, the bit cost of slice header may be more of a concern than the bit cost of other parameter sets, e.g., an SPS (Sequence parameter set), an PPS (Picture parameter set), etc. These parameter sets may be provided less frequently than the slice header.

In a video coding standard, e.g., SHVC, the variables such as NumActiveRefLayerPics, inter_layer_pred_layer_idc and collocated_ref_layer_idx may be the same for each of the slices of a coded picture. Therefore, instead of the slice header, the syntax elements, e.g., inter _layer_pred_enable_flag, num_inter_layer_ref_pics_minus1, inter_layer_pred_layer_ide, inter_layer_sample_pred_only_flag, alt_collocated_indicate_flag and collocated_ref_layer_idx, in SPS extension, PPS, APS or IPS may be sent so that the same syntax may not be duplicated for each slices within a picture. Table <NUM> illustrates syntax elements that may be signaled in SPS extension.

The conditions that rely on the parameter value signaled in other parameter sets may be modified when relocating the syntax elements from slice header to the parameter set, e.g., to avoid parsing dependency among parameter sets. Table <NUM> illustrates an example of an SPS extension syntax table when the relevant syntax elements are relocated from slice header into SPS extension.

In some applications the inter layer prediction related signaling (e.g., inter_layer_pred_enabled_flag, inter_layer_sample_pred_only_flag, etc.) may be changed from slice to slice or from picture to picture. For such applications, sending the syntax elements in the slice header may incur undesired signaling overhead. The flags may be added in SPS extension (or PPS or IPS), e.g., to indicate whether or not the inter layer prediction related syntax elements may be present in the slice_segment header. Table <NUM> illustrates an example of syntax elements that may be signaled in the SPS extension. Table <NUM> illustrates an example of the corresponding slice header.

The sample_prediction_present_slice_present_flag equal to <NUM> may indicate that the inter layer sample prediction related syntax elements such as inter_layer_pred_enable_flag, num_inter_layer_ref_pics_minus1,
inter_layer_pred_layer_idc, inter_layer_sample_pred_only_flag may be present in the slice header. The sample_prediction_present_slice_present_flag equal to <NUM> may indicate that the relative sample prediction syntax elements may not be present in the slice segment header. When not present, the values of the syntax elements may be inferred based on one or more variables. For example, Pseudo Code <NUM> provides an example of how the values of the syntax elements may be inferred.

The variables, NumSamplePredRefLayers, NumSamplePredLayers and SamplePredEnabledFlag may be provided in a video coding standard, e.g., SHVC. The sample prediction syntax element inter_layer_sample_pred_only may be inferred to be equal to <NUM> when sample_prediction_slice_present_flag is equal to <NUM>. The motion_prediction_slice_present_flag equal to <NUM> may indicate that the inter layer motion prediction related syntax elements, e.g., alt_collocated_indication_flag, collocated_ref_layer_idx, etc. may be present in slice header. The motion_prediction_slice_present_flag equal to <NUM> may indicate that the inter layer motion prediction related syntax elements may not be present in the enhancement layer slice segment header. The value of these syntax elements may be inferred based on one or more variables. For example, Pseudo Code <NUM> provides an example of how the values of the syntax elements may be inferred. The NumMotionPredRefLayers and/or MotionPredRefLayerId may be provided by a video coding standard, e.g., SHVC. <IMG>
<IMG>.

As illustrated in Pseudo Code <NUM>, the inter layer motion information (e.g., instead of temporal motion information) may be used for temporal motion vector prediction (TMVP), if at least one motion prediction reference layer is available. The collocated reference layer may be set to the motion prediction reference layer closest to the current enhancement layer. Other motion prediction reference layer may be specified as the collocated reference layer. For example, instead of the closest reference layer, the lowest motion prediction reference layer, MotionPredRefLayerId[nuh_layer_id][<NUM>], may be used as the default collocated reference layer for the TMVP.

The syntax element inter_layer_sample_pred_only_flag equal to <NUM> may indicate that the inter prediction using temporal reference pictures in the EL may not be allowed when decoding of the current picture. The reference picture lists L0 and L1 may not include a temporal reference picture. The inter_layer_sample_pred_only_flag may be signaled in each of the slices regardless of the slice's network abstraction layer (NAL) unit type. Instantaneous decoder refresh (IDR) picture of enhancement layer (EL) may be a picture without inter prediction, using temporal reference pictures. The inter_layer_sample_pred_only_flag may be determined based on IDR NAL unit in the EL. The condition (e.g., (nal_unit_type! = IDR_W_RADL && nal_unit_type != IDR_N_LP)) as illustrated in the Table <NUM> may be applied.

When inter_layer_sample_pred_only_flag is equal to <NUM>, the reference pictures available may be the inter-layer reference pictures. Since in SHVC the motion vectors from inter layer reference pictures may be equal to zero, the temporal motion vector prediction (TMVP) may be bypassed and each of the syntax elements in the slice header related to TMVP may be skipped. The inter_layer_sample_pred_only_flag may be utilized to skip motion prediction signaling.

A WTRU may determine reference pictures for inter-layer prediction without the use of the inter_layer_sample_pred_only_flag. For example, the WTRU may not receive the inter_layer_sample_pred_only_flag, but the WTRU may determine reference pictures that may be utilized for inter-layer prediction. The inference of inter-layer prediction, for example, without temporal prediction, by a WTRU may not rely on the inter_layer_sample_pred_only_flag. If the inter_layer_sample_pred_only_flag is not signaled in the bitstream (e.g., and/or not received by the WTRU), then the WTRU may infer temporal reference pictures. For example, the WTRU may detect that temporal reference pictures are not used for the current slice, for example, by examining the RPS (e.g., the flag used_by_curr_pic_flag, used_by_curr_pic_s0_flag and used_by_curr_pic_s1_flag in RPS may be set to <NUM>). The temporal motion vector prediction (TMVP) process may be bypassed, for example, if temporal reference pictures are not used for coding of the current slice. For example, other related syntax elements may be skipped (e.g., may also be skipped).

The slice_temporal_mvp_enabled_flag may be signaled based on sps_temporal_mvp_enabled_flag (e.g., as provided in SHVC) and/or the inter_layer_sample_pred_only_flag for the enhancement layer (e.g., nuh_layer_id > <NUM>). Table <NUM> illustrates an example of such signaling. For example, the variable InterRefEnabledInRPLFlag may be derived as follows: If NumSamplePredRefLayers[ nuh_layer_id ] and NumActiveRefLayerPics is greater than <NUM>, InterRefEnabledInRPLFlag may be set equal to !inter_layer_sample_pred_only_flag; otherwise, InterRefEnabledInRPLFlag may be set equal to <NUM>.

To condition slice_temporal_mvp_enabled_flag on inter_layer_sample_pred_only_flag, the signaling of inter_layer_sample_pred_only_flag and sample prediction syntax structure (e.g., as illustrated in Table <NUM>) may be determined or signalled prior to slice_temporal_mvp_enabled_flag. When slice_temporal_mvp_enabled flag is not signaled (for example because inter_layer_sample_pred_only_flag is set equal to <NUM>), slice_temporal_mvp_enabled_flag may be inferred to be equal to <NUM>.

When the slice_temporal_mvp_enabled_flag is <NUM>, the syntax elements such as alt_collocated_indication_flag, collocated_ref_layer _idx, collocated_from_10_flagand/or collocated_ref_idx may be skipped (e.g., as illustrated in Table <NUM>).

A signaling order where slice_temporal_mvp_enabled_flag may be signaled prior to inter_layer_sample_pred_only_flag, may be kept. A condition (InterRefEnabledInRPLFlag) may be applied for signaling the TMVP parameters as is illustrated in Table <NUM>. The derivation of InterRefEnabledInRPLFlag may be specified in a video coding standard, e.g., SHVC. The variable InterRefEnabledInRPLFlag may be derived as follows: if NumSamplePredRefLayers[ nuh_layer_id ] is greater than <NUM> and NumActiveRefLayerPics is greater than <NUM>, InterRefEnabledInRPLFlag may be set equal to !inter_layer_sample_pred_only_flag; otherwise, InterRefEnabledInRPLFlag may be set equal to <NUM>. The value of slice_temporal_mvp_enabled_flag may be changed as follows: if InterRefEnabledInRPLFlag is equal to <NUM>, slice_temporal_mvp_enabled_flag may be set equal to <NUM>.

The derivation process for temporal luma motion vector prediction may be changed, and the variables mvLXCol and availableFlagLXCol may be derived. For example, slice_temporal_mvp_enabled_flag is equal to <NUM> or InterRefEnabledInRPLFlag is equal to <NUM>, the components of mvLXCol (e.g., both components) may be set equal to <NUM>, and availableFlagLXCol may be set equal to <NUM>.

A video coding device (e.g., based on an SHVC coding standard) may signal one or more syntax elements (e.g., two syntax elements), alt_collocated_indication_flag and/or collocated_ref_layer_idx in the slice header to indicate the reference layer for inter-layer motion prediction. The original temporal motion vector prediction (TMVP) may use the syntax elements collocated_from_10_flag, and collocated_ref_idx to indicate the reference picture used for the TMVP. The same signaling may be applied to inter-layer motion prediction, so that the redundant syntax elements alt_collocated_indication_flag and collocated_ref_layer_idx may be not be signaled (e.g., may be omitted). Table <NUM> illustrates an example general slice_segment header syntax.

The alt_collocated_indication_flag and collocated_ref_layer_idx may be provided by a video coding standard, e.g., HEVC. Some of the inter-layer reference pictures used for TMVP may not be used for sample prediction. The index of the inter layer reference picture in the reference picture list may be used to indicate the inter-layer collocated reference picture for TMVP. The index of the inter layer reference picture in the reference picture list may not be used for inter layer sample prediction. A bitstream restriction may be imposed that a reference index that corresponds to an inter layer reference picture may not be referred to by any prediction blocks of the enhancement layer picture.

Systems and/or methods for signaling SNR scalability may be provided. In such systems and/or methods, such signaling may differentiate between different scalabilities. For example spatial scalability may be differentiated from SNR scalability, and vice-versa. In an example embodiment, an indicator, flag, or other identifier or information may differentiate SNR from spatial scalability. Additionally, based on an indicator such as a SNR scalability indictor, such signaling may indicate whether to invoke or perform a resampling process (e.g., for a sample) and/or motion prediction.

As described herein, existing international video standards such as MPEG-<NUM> Video, H. <NUM>, MPEG4 Visual and H. <NUM> each may include or have tools and/or profiles that support scalability modes. However, HEVC may not currently support such scalable coding that may be supported by those existing standards. As such, HEVC may be extended to support such scalability coding including at least one of the following: spatial scalability (i.e., the scalable bitstream may signals at more than one spatial resolution), quality scalability (i.e., the scalable bitstream includes signals at more than one quality level), and standard scalability (i.e., the scalable bitstream includes a base layer coded using H. <NUM>/AVC and one or more enhancement layers coded using HEVC). In example embodiments, the quality scalability that may be supported by HEVC may also include SNR scalability. Additionally, as 3D video becomes more popular nowadays, additional extensions of scalability (e.g., the scalable bitstream including 2D and/or 3D video signals) may further be provided and/or used (e.g., as described or defined in MPEG).

A common specification for the scalable and multi-view extensions of HEVC may include a reference index base framework for the scalable extensions of HEVC (SHVC). In such a framework, syntax, semantics and decoding processes for SHVC may be provided. The reference index based framework may keep one or more operations of a block level and below intact such that the existing single layer codec logics may be reused within the scalable coding system. The framework may simplify the scalable codec design, and may further be flexible to support different types of scalabilities by incorporating high level syntax signaling and inter-layer processing module to achieve coding efficiency. Various new high level syntax changes may be provided and/or to support inter-layer processing and the multi-layer signaling of SHVC.

To signal such scalabilities in HEVC, systems and/or methods as described herein may be provided. For example, spatial scalability may be differentiated from SNR scalability, and vice-versa. In an example embodiment, an indicator, flag, or other identifier or information may differentiate SNR from spatial scalability. Additionally, based on an indicator such as a SNR scalability indictor, such signaling may indicate wither to invoke or perform a resampling process (e.g., for a sample) and/or motion prediction.

Scalable video coding may support multiple layers, and each layer may support spatial scalability, temporal scalability, SNR (Signal-to-Noise Ratio) scalability, and/or any other type of scalability. A scalable bit stream may have mixed scalability layers and each enhancement layer may depend on one or more lower layers to be decoded. An inter-layer process may generate the inter-layer reference picture (ILR) sample and/or motion field information to enhance or improve the enhancement layer coding efficiency.

SNR scalability signaling may be provided and/or used. For spatial scalability, video may be coded at different resolutions and at different layers. For example, a base layer video may have 720p resolution and an enhancement layer may have 1080p resolution. Additionally, for SNR scalability, video resolution may be the same across multiple layers, but different layers may be coded at different qualities. For example the base layer may be coded at 33dB, whereas the enhancement layer may be coded at 36dB. In SHVC, a syntax element such as scalability_mask, may be included in a parameter set (e.g., a Video Parameter Set (VPS)) to differentiate between multiview scalability and spatial/SNR scalability (e.g., as shown in Table <NUM>).

However, currently in SHVC, the scalability_mask syntax may not differentiate between spatial and SNR scalability. For example, the spatial scalability and SNR scalability may be two different kinds of scalabilities that may use different codec operations and memory allocations. Some examples of these differences may be as follows. The re-sampling process for reference layer picture samples and reference layer motion field may be used for spatial scalability but may not be used for SNR scalability. Additionally, some inter-layer filters such as a fixed alternative resampling filter (e.g., being evaluated in core experiment SCE3) may achieve improved performance gain on SNR scalability, but may not be applicable to spatial scalability. An application may use single-loop design (e.g., which may have been supported by SVC, the scalable extension of H. <NUM>) for SNR scalability and not spatial scalability. A sampling grid (e.g., currently undergoing in core experiment SCE1) may address particular issues related to spatial scalability, but not SNR scalability.

Systems, methods, and instrumentalities are described herein that may differentiate between spatial and SNR scalability in the high level syntax such that that the encoder and decoder operations may be configured and/or initialized according to the relevant coding tools that may be supported.

For example, currently in SHVC, SNR scalability may be inferred from a scale factor such as a ScaleFactorX that may be specified in a resampling process (e.g., such as the resampling process described in G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>). When the ScaleFactorX is equal to <NUM>, the scalability may be the SNR scalability. The scalability may be derived after parsing the SPS and SPS extension. Other signaling options may be provided to address the signaling of SNR scalability and the resampling process such that redundant codec operations and memory allocation and/or access may be reduced or avoided (e.g., by avoiding parsing).

Separate scalability dimension may be assigned to the spatial and SNR scalability, e.g., to differentiate between spatial and SNR scalability. Table <NUM> illustrates an example embodiment of a modified scalability dimension table where spatial and SNR scalability may have distinct or separate values.

As shown in Table <NUM>, besides ViewId and DependencyId, a variable SnrId[layer_id_in_nuh[i]] may be provided and/or used as the SNR identifier of the i-th layer. According to an example embodiment, the SnrId may be derived as follows:
<IMG>.

Additionally, e.g., to differentiate between spatial and SNR scalability, an SNR scalability flag may be provided and/or used. For example, an SNR scalability flag may be added in a parameter set extension (e.g., a Video Parameter Set (VPS) extension) to indicate the SNR scalability as shown in Table <NUM>.

According to an example embodiment, if or when the SNR_scalability_flag is equal to <NUM>, the scalability between layers with a nuh_layer_id equal to a layer_id_in_nuh[ i ] and the nuh_layer_id equal to layer_id_in_nuh[ j ] may specify or may indicate the SNR scalability. If or when the SNR_scalability_flag is equal to <NUM> , the scalability between layers with the nuh_layer_id equal to layer_id_in_nuh[i] and the nuh_layer_id equal to laycr_id_in_nuh[j] may not be the SNR scalability (e.g., may not specific or indicate SNR scalability). Additionally, if or when SNR_scalability_flag is not provided, it may be inferred to be equal to <NUM>.

As described herein, decoding may be performed, for example, as part of a resampling process. A decoding process associated with an example resampling process may be performed as follows. If PicWidthInSamplesL is equal to RefLayerPicWidthInSamplesL, PicHeightInSamplesL is equal to RefLayerPicHeightInSamplesL, and each of the values of ScaledRefLayerLeftOffset, the ScaledRefLayerTopOffset, ScaledRefLayerRightOffset and/or ScaledRefLayerBottomOffset are equal to <NUM>, rsPicSample may be set to rlPicSample and rsPicMotion may be set equal to rlPicMotion, e.g., when alt_collocated_indication_flag may be equal to <NUM>. rsPic may be derived as follows. The picture sample resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with the sample values of rlPicSample as an input, and with the resampled sample values of rsPicSample as an output. When the alt_collocated_indication_flag is equal to <NUM>, the picture motion field resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with rlPicMotion as an input, and with the resampled motion field of rsPicMotion as an output.

Using the SNR_scalability_flag, described herein, the example resampling process may be provided as follows. For example, if the SNR_scalability_flag is set to <NUM>, rsPicSample may be set to be equal to rlPicSample. Additionally, when the alt_collocated_indication_flag may be equal to <NUM> and if the SNR_scalability_flag is set to <NUM>, rsPicMotion may be set to rlPicMotion. rsPic may be derived as follows. The picture sample resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with the sample values of rlPicSample as an input, and with the resampled sample values of rsPicSample as an output. When the alt_collocated_indication flag may be equal to <NUM>, the picture motion field resampling process (e. g, as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with rlPicMotion as input, and with the resampled motion field of rsPicMotion as output.

In example embodiments, one or more exceptions may be provided and/or present. One such exception may include or may be hybrid standard scalability where the base layer (BL) may be AVC coded. The video coding size may be different between HEVC EL and AVC BL. For example, when an AVC base layer (BL) and an HEVC enhancement layer (EL) may both encode a 1920x1080 video, the decoded BL reference picture size may be 1920x1088 while the enhancement coding picture size may be 1920x1080 (e.g., this is because AVC and HEVC standards apply different padding processes). Although the resampling of the luma and/or chroma sample may not be necessary, the decoded reference picture (1920x1088) may not be used directly to predict the EL picture (1920x1080) and the corresponding cropped region may be copied into the ILR picture.

There are multiple methods that may be used as described herein to address such an exception. For example, in a method, the SNR_scalability_flag may be restricted to l when both the BL and EL video coding size may be the same regardless the values of one or more scaled offsets. Such restriction may be provided and/or imposed in the form of a bitstream conformance restriction to ensure that the encoder may set the value of SNR_scalability_flag properly. In that case, the SNR_scalability_flag may be set to <NUM> for the above 1920x1080 hybrid standard scalability and the inter-layer reference picture may be derived from the 1920x1088 AVC base layer picture according to the resampling process (e.g., as specified in SHVC WD G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>).

In a method, the SNR_scalability_flag may be set to <NUM> when the scale factor, ScaleFactorX (e.g., as specified in G. <NUM>) may be equal to <NUM>. In such method, the resampling process may be further modified to cover the special cases as follows. In addition to the rlPic and rsPic (e.g., as defined in SHVC WD G. <NUM>), another cropped reference layer picture rcPic may be added in the resampling process. The variables CroppedRefLayerPicWidthInSamplesL and CroppedRefLayerPicWidthinSamplesL may be set to be equal to the width and height of rcPic in units of luma samples respectively. The variable rcPicSample may further be defined as a group of sample arrays specifying the sample values of rcPic of the luma and chroma components. Additionally, the rcPicMotion may be defined as a group of variable arrays specifying the compressed motion field of rcPic.

The variables RefLayerPicWidthInSamplesL and RefLayerPicHeightInSamplesL may be set equal to the width and height of the decoded reference layer picture rlPic in units of luma samples respectively. The luma samples location [xP][yP] may specify the top-left sample of the rlPic. Additionally, the variable rcLeftStart, rcRightEnd, rcTopStart and rcBottomEnd may be derived as follows:
rcLeftStart = xP + ScaledRefLayerLeftOffset rcRightEnd = xP + (RefLayerPicWidthInSamplesL - <NUM>) + ScaledRefLayerRightOffset rcTopStart = yP + ScaledRefLayerTopOffset rcBottomEnd = yP + (RefLayerPicHeightInSamplesL - <NUM>) + ScaledRefLayerBottomOffset.

The rcPic may be derived by cropping the rlPic with the top-left position (rcLeftStart, rcTopStart) and bottom-right position (rcRightEnd, rcBottomEnd). <FIG> illustrates an example of cropping. As illustrated in <FIG>, rcPic may be derived from rlPic when scaled offsets may not be zero.

The resampling process may be provided as follows. If SNR_scalability_flag may be set to <NUM>, the rsPicSample may be set equal to rcPicSample and when the alt_collocated_indication_flag may be equal to <NUM>,the rsPicMotion may be set equal to rcPicMotion. The rsPic may be derived as follows. A picture sample resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with the sample values of rlPicSample as an input and with the resampled sample values of rsPicSample as an output. The picture motion field resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with rlPicMotion asan input and with the resampled motion field of rsPicMotion as an output, e.g., when the alt_collocated_indication_flag is equal to <NUM>.

Spatial and SNR scalability may be distinguished, e.g., to avoid unnecessary resampling operations and/or memory allocation. An additional or extra syntax element may be signaled in a parameter set extension (e.g., a Video Parameter set (VPS) extension, as illustrated in Table <NUM>) to indicate whether the resampling process may be bypassed or not.

A resampling_buffer_enable_flag[ i ][ j ] that may be equal to <NUM> may indicate or specify that the resampling process between the i-th layer and the j-th layer may be bypassed and no resampling buffer may be allocated. A resampling_buffer_enable_flag that may be equal to <NUM> may indicate or specify that a related buffer for the resampling process of picture sample or motion values may be invoked. When a resampling_buffer_enable_flag may not be present, the default value may be <NUM>. The resampling process may be modified as follows. If the resampling_buffer_enable_flag may be set to <NUM>, the rsPicSample may be set equal to rlPicSample and when the alt_collocated_indication_flag may be equal to <NUM>, the rsPicMotion may be set equal to rlPicMotion. The rsPic may be derived as follows. A picture sample resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with the sample values of rlPicSample as an input, and with the resampled sample values of rsPicSample as an output. Additionally, when the alt_collocated_indication_flag may be equal to <NUM>, a picture motion field resampling process (e.g., as specified in subclause G. <NUM> of JCTVC-M1008, SHVC Working Draft, April <NUM>) may be invoked with rlPicMotion as an input, and with the resampled motion field of rsPicMotion as an output.

The resampling_buffer_enable_flag may not be coupled with a direct_dependency_type syntax element. The resampling_buffer_enable_flag may be signaled (e.g., signaled independently) as illustrated in Table <NUM>). The resampling_buffer_enable_flag may offer the flexibility such that the resampled reference picture and motion may be used for a purpose other than sample prediction and/or motion prediction. For example, the resampled motion may be used to generate hybrid inter-layer reference picture.

The resampling_buffer_enable_flag may not be coupled with a direct_dependency_type syntax element. The resampling_buffer_enable_flag may be signaled (e.g., signaled independently) as SNR_scalability_flag and resampling_buffer_enable_flag, may be placed in SPS or SPS extensions or any other appropriate parameter set as well. Table <NUM> illustrates an example embodiment of signaling the SNR_scalability flag in an SPS extensions. The syntax element num_SNR_scalability_flags as illustrated in Table <NUM> may indicate the number of signaled flags. The value of num_SNR_scalability_flags may be equal to the number of reference layers of the current enhancement layer. The value of num_scaled_ref_layer_offsets may be equal to the number of reference layers of the current enhancement layer. As illustrated in Table19, the syntax elements num_SNR_scalability_flags and num_scaled_ref_layer_offsets may be combined and signaled as one syntax element, e.g., num_ref_layers.

With SNR scalability signaling, as described herein, the codec operation may signal one or more syntax elements to save number of bits signaled and thereby increase efficiency. For example, the SNR_scalability_flag, as described herein (e.g., that may be added to a VPS extension as described above), may be used in various application scenarios.

For example, sampling grid shift signaling may be provided and/or used, e.g., with a SNR_scalability_flag. The sampling grid shift information may be sampled using various techniques and/or methods (e.g., as proposed in JCTVC-M0465, "Signaling of phase offset in up-sampling process and chroma sampling location," April, <NUM>). For example, sampling grid information may be signaled in an SPS extension with a phase offset present flag and luma and/or chroma phase offset parameters. Since sampling phase shift may be applicable to spatial scalability, the proposed SNR_scalability_flag may be used as a condition to avoid unnecessary syntax elements present in the bitstream for the SNR scalability. Table <NUM> illustrates an example syntax table that may use SNR_scalability_flag signaling as a condition to determine whether extra sampling grid parameters may be signaled.

A scaled reference layer offset may also be provided and/or used. For example, scaled ref layer offset syntax elements that may be signaled in SPS extension may be used to align the base layer and the enhancement layer. In SNR scalability, the scaled offsets may zero and, as such, by conditioning the signaling of these offsets on the SNR_scalability_flag, extra bits may be saved by skipping the scaled offset signaling for SNR scalable layers. Table <NUM> illustrates an example of such signaling.

The semantic of the num_ref_layer that may not be accurate to match the number of dependent layers (e.g., reference layers) signaled in VPS where the num_ ref_layers may specify the number of sets of scaled reference layer offset parameters that may be present in the SPS. In an embodiment, the value of num_ ref_layers may be in the range of <NUM> to <NUM>, inclusive. The semantics may be modified as follows. The num_ ref_layers may specify or indicate the number of sets of scaled reference layer offset parameters that may be present in the SPS. The value of num_ref_layers may be equal to NumDirectRefLayers[ nuh_layer_id] (e.g., as a part of btistream conformance restriction, and as specified in F. <NUM> of the JCTVC-M1008, SHVC Working Draft, April <NUM>).

Inter-layer filtering signaling may be provided and/or used. For example, a switchable integer position filter (e.g. such as the filter described in JCTVC-M0273, Non-SCE4 Switchable filter on integer position, April <NUM> that may be provided in Core Experiment on inter layer filtering (SCE3)) may achieve a performance gain for SNR scalability, but not for spatial scalability. As such, various filter switching methods may be provided and/or used. For example, a filter switch may be signaled in a slice header, and, in an additional embodiment, both a ILR and filtered ILR may be inserted into a reference picture list (RPL) for inter-layer prediction. For example, the one bit syntax element in the slice header may be bypassed when the SNR_scalability_flag is set to <NUM>, e.g., since such filter may not improve the performance for spatial scalability scenarios. In another example, the number of active inter-layer reference picture may be reduced, when the SNR_scalability _flag is set to <NUM>, e.g., because the filtered ILR picture may not be added into the reference picture sets and the reference picture lists. Signaling the SNR scalability indicator may simplify the reference picture list construction process and may save DPB memory size in advance for spatial scalability case. The SNR_scalability_flag may be referred in the resampling process to bypass the switchable integer position filter (e.g. such as the filter described in JCTVC-M0273, Non-SCE4 Switchable filter on integer position, April <NUM>) for the spatial scalability (SNR_scalability_flag is set to <NUM>) so as to reduce the codec operation complexity and memory allocation.

Multiple flags may be signaled in a parameter set (e.g., a Video Parameter Set). Each of the flags may indicate whether a resampling process associated with a layer (e.g., a base layer and a dependent enhancement layer) of a scalable bitstream needs to be performed. On a condition that a flag indicates that the resampling process is not needed, allocation of a resampling buffer may be by-passed. On a condition that the flag indicates that the resampling process is needed, one or more resampling buffers may be allocated and resampling of one or more of a reference picture sample or motion associated with a reference picture sample may be invoked.

The signaling described herein may be used, for example, in any of the networks or suitable network elements described herein. For example, the signaling described herein may be implemented in accordance with scalable video coding implemented by devices (e.g., video streaming devices) associated with a wireless communication system, such as the example wireless communication system <NUM> and/or components thereof illustrated in <FIG>.

<FIG> depicts a diagram of an example communications system <NUM> in which one or more disclosed embodiments may be implemented and/or may be used.

As shown in <FIG>, the communications system <NUM> may include wireless transmit/receive units (WTRUs) 102a, 102b, 102c, and/or 102d (which generally or collectively may be referred to as WTRU <NUM>), a radio access network (RAN) <NUM>/<NUM>/<NUM>, a core network <NUM>/<NUM>/<NUM>, a public switched telephone network (PSTN) <NUM>, the Internet <NUM>, and other networks <NUM>, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of the WTRUs 102a, 102b, 102c, and/or 102d may be any type of device configured to operate and/or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, and/or 102d may be configured to transmit and/or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, consumer electronics, and the like.

The communications systems <NUM> may also include a base station 114a and a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, and/or 102d to facilitate access to one or more communication networks, such as the core network <NUM>/<NUM>/<NUM>, the Internet <NUM>, and/or the networks <NUM>. By way of example, the base stations 114a and/or 114b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like.

The base station 114a may be part of the RAN <NUM>/<NUM>/<NUM>, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a and/or the base station 114b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown). In another embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.

The base stations 114a and/or 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, and/or 102d over an air interface <NUM>/<NUM>/<NUM>, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface <NUM>/<NUM>/<NUM> may be established using any suitable radio access technology (RAT).

For example, the base station 114a in the RAN <NUM>/<NUM>/<NUM> and the WTRUs 102a, 102b, and/or 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface <NUM>/<NUM>/<NUM> using wideband CDMA (WCDMA).

In another embodiment, the base station 114a and the WTRUs 102a, 102b, and/or 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface <NUM>/<NUM>/<NUM> using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A).

In other embodiments, the base station 114a and the WTRUs 102a, 102b, and/or 102c may implement radio technologies such as IEEE <NUM> (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard <NUM> (IS-<NUM>), Interim Standard <NUM> (IS-<NUM>), Interim Standard <NUM> (IS-<NUM>), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

The base station 114b in FIG. 10A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like. Thus, the base station 114b may not be required to access the Internet <NUM> via the core network <NUM>/<NUM>/<NUM>.

The RAN <NUM>/<NUM>/<NUM> may be in communication with the core network <NUM>/<NUM>/<NUM>, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, and/or 102d. For example, the core network <NUM>/<NUM>/<NUM> may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown in <FIG>, it will be appreciated that the RAN <NUM>/<NUM>/<NUM> and/or the core network <NUM>/<NUM>/<NUM> may be in direct or indirect communication with other RANs that employ the same RAT as the RAN <NUM>/<NUM>/<NUM> or a different RAT. For example, in addition to being connected to the RAN <NUM>/<NUM>/<NUM>, which may be utilizing an E-UTRA radio technology, the core network <NUM>/<NUM>/<NUM> may also be in communication with another RAN (not shown) employing a GSM radio technology.

The core network <NUM>/<NUM>/<NUM> may also serve as a gateway for the WTRUs 102a, 102b, 102c, and/or 102d to access the PSTN <NUM>, the Internet <NUM>, and/or other networks <NUM>. For example, the networks <NUM> may include another core network connected to one or more RANs, which may employ the same RAT as the RAN <NUM>/<NUM>/<NUM> or a different RAT.

Some or all of the WTRUs 102a, 102b, 102c, and/or 102d in the communications system <NUM> may include multi-mode capabilities, i.e., the WTRUs 102a, 102b, 102c, and/or 102d may include multiple transceivers for communicating with different wireless networks over different wireless links.

<FIG> depicts a system diagram of an example WTRU <NUM>. As shown in <FIG>, the WTRU <NUM> may include a processor <NUM>, a transceiver <NUM>, a transmit/receive element <NUM>, a speaker/microphone <NUM>, a keypad <NUM>, a display/touchpad <NUM>, non-removable memory <NUM>, removable memory <NUM>, a power source <NUM>, a global positioning system (GPS) chipset <NUM>, and other peripherals <NUM>. Also, embodiments contemplate that the base stations 114a and 114b, and/or the nodes that base stations 114a and 114b may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (HeNB), a home evolved node-B gateway, and proxy nodes, among others, may include some or all of the elements depicted in <FIG> and described herein.

While <FIG> depicts the processor <NUM> and the transceiver <NUM> as separate components, it may be appreciated that the processor <NUM> and the transceiver <NUM> may be integrated together in an electronic package or chip.

The transmit/receive element <NUM> may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface <NUM>/<NUM>/<NUM>.

Thus, in one embodiment, the WTRU <NUM> may include two or more transmit/receive elements <NUM> (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface <NUM>/<NUM>/<NUM>.

For example, the power source <NUM> may include one or more dry cell batteries (e.g., nickelcadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

<FIG> depicts a system diagram of the RAN <NUM> and the core network <NUM> according to an embodiment. As noted above, the RAN <NUM> may employ a UTRA radio technology to communicate with the WTRUs 102a, 102b, and/or 102c over the air interface <NUM>. As shown in <FIG>, the RAN <NUM> may include Node-Bs 140a, 140b, and/or 140c, which may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and/or 102c over the air interface <NUM>. The Node-Bs 140a, 140b, and/or 140c may each be associated with a particular cell (not shown) within the RAN <NUM>. The RAN <NUM> may also include RNCs 142a and/or 142b. It will be appreciated that the RAN <NUM> may include any number of Node-Bs and RNCs while remaining consistent with an embodiment.

As shown in <FIG>, the Node-Bs 140a and/or 140b may be in communication with the RNC 142a. Additionally, the Node-B 140c may be in communication with the RNC142b. The Node-Bs 140a, 140b, and/or 140c may communicate with the respective RNCs 142a, 142b via an Iub interface. The RNCs 142a, 142b may be in communication with one another via an Iur interface. Each of the RNCs 142a, 142b may be configured to control the respective Node-Bs 140a, 140b, and/or 140c to which it is connected. In addition, each of the RNCs 142a, 142b may be configured to carry out or support other functionality, such as outer loop power control, load control, admission control, packet scheduling, handover control, macrodiversity, security functions, data encryption, and the like.

The MSC <NUM> and the MGW <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to circuit-switched networks, such as the PSTN <NUM>, to facilitate communications between the WTRUs 102a, 102b, and/or 102c and traditional land-line communications devices.

The SGSN <NUM> and the GGSN <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to packet-switched networks, such as the Internet <NUM>, to facilitate communications between and the WTRUs 102a, 102b, and/or 102c and IP-enabled devices.

<FIG> depicts a system diagram of the RAN <NUM> and the core network <NUM> according to an embodiment. As noted above, the RAN <NUM> may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, and/or 102c over the air interface <NUM>.

The RAN <NUM> may include eNode-Bs 160a, 160b, and/or 160c, though it will be appreciated that the RAN <NUM> may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, and/or 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and/or 102c over the air interface <NUM>. In one embodiment, the eNode-Bs 160a, 160b, and/or 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

Each of the eNode-Bs 160a, 160b, and/or 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. As shown in <FIG>, the eNode-Bs 160a, 160b, and/or 160c may communicate with one another over an X2 interface.

The MME <NUM> may be connected to each of the eNode-Bs 160a, 160b, and/or 160c in the RAN <NUM> via an S1 interface and may serve as a control node. For example, the MME <NUM> may be responsible for authenticating users of the WTRUs 102a, 102b, and/or 102c, bearer activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, and/or 102c, and the like. The MME <NUM> may also provide a control plane function for switching between the RAN <NUM> and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.

The serving gateway <NUM> may be connected to each of the eNode-Bs 160a, 160b, and/or 160c in the RAN <NUM> via the S1 interface. The serving gateway <NUM> may generally route and forward user data packets to/from the WTRUs 102a, 102b, and/or 102c. The serving gateway <NUM> may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, and/or 102c, managing and storing contexts of the WTRUs 102a, 102b, and/or 102c, and the like.

The serving gateway <NUM> may also be connected to the PDN gateway <NUM>, which may provide the WTRUs 102a, 102b, and/or 102c with access to packet-switched networks, such as the Internet <NUM>, to facilitate communications between the WTRUs 102a, 102b, and/or 102c and IP-enabled devices.

For example, the core network <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to circuit-switched networks, such as the PSTN <NUM>, to facilitate communications between the WTRUs 102a, 102b, and/or 102c and traditional land-line communications devices. In addition, the core network <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to the networks <NUM>, which may include other wired or wireless networks that are owned and/or operated by other service providers.

<FIG> depicts a system diagram of the RAN <NUM> and the core network <NUM> according to an embodiment. The RAN <NUM> may be an access service network (ASN) that employs IEEE <NUM> radio technology to communicate with the WTRUs 102a, 102b, and/or 102c over the air interface <NUM>. As will be further discussed below, the communication links between the different functional entities of the WTRUs 102a, 102b, and/or 102c, the RAN <NUM>, and the core network <NUM> may be defined as reference points.

As shown in <FIG>, the RAN <NUM> may include base stations 180a, 180b, and/or 180c, and an ASN gateway <NUM>, though it will be appreciated that the RAN <NUM> may include any number of base stations and ASN gateways while remaining consistent with an embodiment. The base stations 180a, 180b, and/or 180c may each be associated with a particular cell (not shown) in the RAN <NUM> and may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and/or 102c over the air interface <NUM>. In one embodiment, the base stations 180a, 180b, and/or 180c may implement MIMO technology. Thus, the base station 180a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a. The base stations 180a, 180b, and/or 180c may also provide mobility management functions, such as handoff triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like. The ASN gateway <NUM> may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network <NUM>, and the like.

The air interface <NUM> between the WTRUs 102a, 102b, and/or 102c and the RAN <NUM> may be defined as an R1 reference point that implements the IEEE <NUM> specification. In addition, each of the WTRUs 102a, 102b, and/or 102c may establish a logical interface (not shown) with the core network <NUM>. The logical interface between the WTRUs 102a, 102b, and/or 102c and the core network <NUM> may be defined as an R2 reference point, which may be used for authentication, authorization, IP host configuration management, and/or mobility management.

The communication link between each of the base stations 180a, 180b, and/or 180c may be defined as an R8 reference point that includes protocols for facilitating WTRU handovers and the transfer of data between base stations. The communication link between the base stations 180a, 180b, and/or 180c and the ASN gateway <NUM> may be defined as an R6 reference point. The R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 102a, 102b, and/or 102c.

The MIP-HA may be responsible for IP address management, and may enable the WTRUs 102a, 102b, and/or 102c to roam between different ASNs and/or different core networks. The MIP-HA <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to packet-switched networks, such as the Internet <NUM>, to facilitate communications between the WTRUs 102a, 102b, and/or 102c and IP-enabled devices. The AAA server <NUM> may be responsible for user authentication and for supporting user services. The gateway <NUM> may facilitate interworking with other networks. For example, the gateway <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to circuit-switched networks, such as the PSTN <NUM>, to facilitate communications between the WTRUs 102a, 102b, and/or 102c and traditional land-line communications devices. In addition, the gateway <NUM> may provide the WTRUs 102a, 102b, and/or 102c with access to the networks <NUM>, which may include other wired or wireless networks that are owned and/or operated by other service providers.

Although not shown in <FIG>, it should, may, and/or will be appreciated that the RAN <NUM> may be connected to other ASNs and the core network <NUM> may be connected to other core networks. The communication link between the RAN <NUM> the other ASNs may be defined as an R4 reference point, which may include protocols for coordinating the mobility of the WTRUs 102a, 102b, and/or 102c between the RAN <NUM> and the other ASNs.

Claim 1:
A video encoding method according to scalable extensions of high efficiency video coding, SHVC, comprising:
comparing a number of active reference layers with a number of direct reference layers, wherein the number of active reference layers indicates a number of reference layers that can be used for inter-layer prediction by a current picture of a current layer; and
on a condition that the number of active reference layers is not equal to the number of direct reference layers, generating a bitstream comprising one or more layers and including, in a slice level header of the bitstream, a list of reference picture layer IDs that can be used for inter-layer prediction by the current picture of the current layer;
on a condition that the number of active reference layers is equal to the number of direct reference layers, generating a bitstream comprising one or more layers and not including any indication of a list of reference picture layer IDs that can be used for inter-layer prediction by the current picture of the current layer.