Video coding using temporally coherent dynamic range mapping

A more efficient co-use of dynamic range mapping on the one hand and temporal prediction on the other hand such as, for example, in order to code HDR frame sequences, is achieved by exploiting the concept of weighted prediction in order to transition from the mapping parameter from the reference frame to the currently temporally predicted frame. By this measure, the temporal prediction does not fail and despite the frame-wise variation in the dynamic range mapping, encoding efficiency is, thus, maintained. As a favorable side aspect, weighted temporal prediction is already within the capabilities of existing video coding stages such as, for example, the H.264/AVC.

BACKGROUND OF THE INVENTION

The present application is concerned with video coding such as for use with HDR sequences.

So far, most image and video coding applications can cover only a luminance range of about 2 orders of magnitude (low dynamic range (LDR)) [1]. However, the human visual system (HVS) allows us to adapt to light conditions that can cover a range of more than ten orders of magnitude and to perceive about five orders of magnitude simultaneously [2]. With an increasing number of applications that can profit from a representation of the full HDR luminance (e.g., CGI, special effects productions, HDR displays), there will be an increasing demand in HDR video coding methods. Using a standard coding method, like H.264/AVC, will allow for a seamless transition from LDR towards HDR video coding without much additional effort. Note that the term HDR refers to the representation of real luminance values throughout this work and not to a tone-mapped LDR representation, what is sometimes called HDRI.

Since the most natural representation of HDR data, floating-point numbers, does not result in a good compression and is also costly to handle, several authors proposed a suitable mapping from floating-point luminance values to integer luma values [3, 4, 5, 6]. These luminance-to-luma mappings have in common that the associated loss in precision is below the tolerance of the HVS and no distortion is therefore perceived. They further have in common, that they apply a conversion of the HDR image data to the CIELUV color space [1] before further processing. That is, the data is represented by a luminance component Y and the chromacity components (u′, v′). The advantage of the (u′, v′) color representation is

that it is perceptually uniform. That is, equal offsets in this representation represent equal perceptual color differences and therefore they can be linearly mapped to integer values with a bit depth of, e.g., 8 bit. Such a mapping from the perceivable (u′, v′) interval [0, 0.62] to integer values in the range [0, 255] introduces a maximum absolute quantization error of 0.00172 which is well below the visible threshold.

Since the HVS obeys to the Weber-Fechner law, for a large luminance range, in most works a logarithmic mapping of the luminance Y to luma code values is performed [3, 5, 6]. This results in a constant relative quantization error leading to a perceptually uniform representation of the luminance. E.g., in [3] Larson proposed the following luminance-to-luma mapping (Log Luv transform):

It maps the real-valued luminances in the interval [5.44×10−20, 1.84×1019] to 15 bit integer luma values in the range [0, 215−1] and vice versa. That is, about 38 orders of luminance magnitude are represented with a relative step size of 0.27%. This is well below the visible quantization threshold of about 1% [1].

However, the dynamic range covered by such a mapping is far beyond the range of what the HVS can simultaneously perceive. Furthermore, there exists no natural image data that spans such high dynamic ranges. Whereas for lossless image compression of data that can undergo further image processing steps this extremely high range and fidelity might be useful, for lossy video encoding that is intended for being watched by human observers, it is not. Consequently, there is no need to reserve bits to represent luminance values that are not perceivable or that do not occur in the source image or video frame. Since this would degrade the compression efficiency, e.g., in HDR still image coding with the TIFF library [3], a scaling factor can be used to scale the source image to an appropriate range before the Log Luv transform. In a similar Log Luv approach [6], scaling has been applied to each individual frame of a video sequence in order to exploit the full range of possible luma code values for a given bit depth.

However, like many HDR video coding methods, the latter is just a straightforward extension of HDR image coding to individual video frames. Therefore, the approach lacks some video specific aspects what significantly degrades the compression efficiency. Most notably, mapping the luminance values of successive frames to different code values with an individual scaling significantly harms the temporal coherence of the sequence. Consequently the temporal motion compensated prediction in the H.264/AVC video coder mostly fails.

Naturally, this is also true for other temporally predicting coders and also for sample values other than luminance values.

SUMMARY

According to an embodiment, a video encoder for encoding a first sequence of frames the sample values which are represented in a first format covering a first dynamic range, may have a sample value converter configured to convert the sample values of the frames of the first sequence from the first format into a second format having a second dynamic range lower than the first dynamic range, using a mapping function which maps a portion out of the first dynamic range, which is settable by a mapping parameter, to the second dynamic range, so as to acquire a second sequence of frames; a video encoding stage configured to encode the second sequence of frames by weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence of a reconstructed version of the second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter, as a reference; and a parameter setter for setting the weighting parameter and the offset parameter depending on the mapping parameter for the second frame of the first sequence corresponding to the second frame of the second sequence, and setting the mapping parameter for a first frame of the first sequence corresponding to the first frame of the second sequence, depending on the mapping parameter for the second frame of the first sequence, the weighting parameter and the offset parameter.

According to another embodiment, a video decoder for reconstructing a first sequence of frames, the sample values of which are represented in a first format covering a first dynamic range, from a data stream, may have a video decoding stage configured to reconstruct, from the data stream, a second sequence of frames the sample values of which are represented in a second format covering a second dynamic range lower than the first dynamic range, by a weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter comprised by the data stream, as a reference; a parameter setter configured to set a mapping parameter for the first frame of the second sequence depending on a mapping parameter for the second frame of the second sequence, the weighting parameter and the offset parameter, and a sample value reconverter configured to convert the sample values of the frames of the second sequence from the second format to the first format using a mapping function which maps the second dynamic range onto a portion out of the first dynamic range which is set by the mapping parameter for the respective frame of the second sequence.

According to another embodiment, a method for encoding a first sequence of frames the sample values which are represented in a first format covering a first dynamic range may have the steps of converting the sample values of the frames of the first sequence from the first format into a second format having a second dynamic range lower than the first dynamic range, using a mapping function which maps a portion out of the first dynamic range, which is settable by a mapping parameter, to the second dynamic range, so as to acquire a second sequence of frames; encoding the second sequence of frames by weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence of a reconstructed version of the second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter, as a reference; and setting the weighting parameter and the offset parameter depending on the mapping parameter for the second frame of the first sequence corresponding to the second frame of the second sequence, and setting the mapping parameter for a first frame of the first sequence corresponding to the first frame of the second sequence, depending on the mapping parameter for the second frame of the first sequence, the weighting parameter and the offset parameter.

According to another embodiment, a method for reconstructing a first sequence of frames, the sample values of which are represented in a first format covering a first dynamic range, from a data stream may have the steps of reconstructing, from the data stream, a second sequence of frames the sample values of which are represented in a second format covering a second dynamic range lower than the first dynamic range, by a weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter comprised by the data stream, as a reference; setting a mapping parameter for the first frame of the second sequence depending on a mapping parameter for the second frame of the second sequence; the weighting parameter and the offset parameter, and converting the sample values of the frames of the second sequence from the second format to the first format using a mapping function which maps the second dynamic range onto a portion out of the first dynamic range which is set by the mapping parameter for the respective frame of the second sequence.

Another embodiment may have a data stream from which a first sequence of frames, the sample values of which are represented in a first format covering a first dynamic range, is reconstructable, wherein the first sequence is encoded into the data stream indirectly via a second sequence of frames the sample values of which are represented in a second format covering a second dynamic range lower than the first dynamic range, the second sequence being encoded into the data stream by a weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter, as a reference, wherein the weighting parameter and the offset parameter are comprised be the data stream such that a mapping parameter for the first frame of the second sequence depends on a mapping parameter for the second frame of the second sequence, the weighting parameter and the offset parameter, and the sample values of the frames of the second sequence is converted from the second format to the first format using a mapping function which maps the second dynamic range onto a portion out of the first dynamic range which is set by the mapping parameter for the respective frame of the second sequence, reconstruct the first sequence.

According to another embodiment, a computer readable digital storage medium may have stored thereon a computer program having a program code for performing, when running on a computer, a method for encoding a first sequence of frames the sample values which are represented in a first format covering a first dynamic range, the method having the steps of converting the sample values of the frames of the first sequence from the first format into a second format having a second dynamic range lower than the first dynamic range, using a mapping function which maps a portion out of the first dynamic range, which is settable by a mapping parameter, to the second dynamic range, so as to acquire a second sequence of frames; encoding the second sequence of frames by weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence of a reconstructed version of the second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter, as a reference; and setting the weighting parameter and the offset parameter depending on the mapping parameter for the second frame of the first sequence corresponding to the second frame of the second sequence, and setting the mapping parameter for a first frame of the first sequence corresponding to the first frame of the second sequence, depending on the mapping parameter for the second frame of the first sequence, the weighting parameter and the offset parameter.

According to another embodiment, a computer readable digital storage medium may have stored thereon a computer program having a program code for performing, when running on a computer, a method for reconstructing a first sequence of frames, the sample values of which are represented in a first format covering a first dynamic range, from a data stream, having the steps of reconstructing, from the data stream, a second sequence of frames the sample values of which are represented in a second format covering a second dynamic range lower than the first dynamic range, by a weighted temporal prediction of a first frame of the second sequence using a second frame of the second sequence, weighted by a weighting parameter and offset by an offset parameter comprised by the data stream, as a reference; setting a mapping parameter for the first frame of the second sequence depending on a mapping parameter for the second frame of the second sequence; the weighting parameter and the offset parameter, and converting the sample values of the frames of the second sequence from the second format to the first format using a mapping function which maps the second dynamic range onto a portion out of the first dynamic range which is set by the mapping parameter for the respective frame of the second sequence.

A basic idea underlying the present invention is that a more efficient co-use of dynamic range mapping on the one hand and temporal prediction on the other hand such as, for example, in order to code HDR frame sequences, may be achieved by exploiting the concept of weighted prediction in order to transition the mapping parameter from the reference frame to the currently temporally predicted frame. By this measure, the temporal prediction does not fail and despite the frame-wise variation in the dynamic range mapping, encoding efficiency is, thus, maintained. As a favorable side aspect, weighted temporal prediction is already within the capabilities of existing video coding stages such as, for example, the H.264/AVC.

DETAILED DESCRIPTION OF THE INVENTION

Before the embodiments of the present invention are described in more detail below with respect to the figures, it should be noted that equal elements occurring within different ones of these figures, are indicated using equal reference signs, and accordingly, a description of these elements with respect to one figure is also applicable with respect to another figure as long as the specific details brought forward with respect to the latter do not teach to the contrary.

FIG. 1shows a video encoder10according to an embodiment of the present invention. The video encoder10is configured to encode a first sequence12of frames14the sample value16of which are represented in a first format covering a first dynamic range. For example, the frame sequence12may be a video such as an HDR video, and the sample values16may represent a spatial sampling of the luminance distribution of the individual frames14. The first format in which the sample values16are represented may be a floating point format. Detailed examples will be outlined below. However, it should be noted that the type of information spatially sampled by the sample values16is not restricted to luminance. Rather, other types of information could be the object of the sample values16instead. For example, frames14could represent depth maps, and accordingly, the sequence12could represent a temporal sampling of a depth map of a certain scene or the like.

The video encoder10comprises a sample value converter18, a video encoding stage20and a parameter setter22. Sample value converter18and video encoding stage20are connected between an input24and an output26of video encoder10wherein the input24is configured to receive frame sequence12while output26is for outputting the data stream resulting from encoding sequence12by video encoder10. Parameter setter22has an input connected to input24and outputs connected to parameter inputs of sample value converter18and video encoding stage20, respectively. As indicated by a dashed line28, parameter setter22may also output side information contributing to the data stream26as will be outlined in more detail further below.

The sample value converter18is configured to convert the sample values16of the frames14of the first sequence12from the first format into a second format having a second dynamic range lower than the first dynamic range. Thus, sample value converter18forwards to the video encoding stage20a second sequence30of frames32which completely corresponds to sequence12except for the sample values16having been converted from the first format into the second format. Accordingly, each frame32corresponds to a respective frame14of sequence12, with frames32being arranged within sequence30in the same order as the corresponding frames14within sequence12.

The second format may be, for example, an integer format in which, for example, the sample values34of frames32are represented in, for example, PCM coded form using a binary code. For example, the sample values34may be represented by n bits with n, for example, being equal to 8, 9 or 10. In case of eight bits, for example, the second format would, thus, merely cover a sample value range of about two orders of magnitude (102≈28), and in case of ten bits, for example, the second format would, thus, merely cover a sample value range of about three orders of magnitude (103≈210). Compared thereto, the first format by way of which the sample values16are represented, covers a greater, or even far greater dynamic range. As mentioned above, and in accordance with the more detailed embodiments outlined below, the first format may be a floating-point format. However, it should be noted that the first format could also be an integer format with using, however, more bits than the second format.

In order to convert the sample values of the frames14of the first sequence12from the first format into the second format, sample value converter18uses a mapping function36which maps a portion38out of the first dynamic range40to the second dynamic range42. In particular, the sample value converter18is configured such that the portion38which the mapping function36maps to the dynamic range42corresponding to the second format, is settable by a mapping parameter44which is set by parameter setter22as will be outlined in more detail below, on a frame-wise basis. In the specific embodiments outlined in more detail below, the mapping function36represents a linear mapping function between the first dynamic range40in logarithmic domain to the second dynamic range in linear domain. However, other strictly monotonic functions may also be used instead of this type of function. As will become clearer from the further description below, portion38is set by parameter setter22on a frame-by-frame basis so as to capture substantially all information contained within the respective frame14in the first format. Briefly spoken, parameter setter22seeks to position and dimension—or scale—portion38within the first dynamic range40such that all perceptually relevant samples within the respective frame14have their sample value16within that portion38so that all these sample values are correctly mapped—without being clipped—to the second dynamic range of the second format42. An exemplary distribution44of sample values16within a current frame is exemplarily shown inFIG. 1. In the example ofFIG. 1, this distribution is completely contained within portion38. As will be outlined in more detail below, the distribution44may merely represent the distribution of sample values16within a certain part of frame14such as a center portion thereof as such a center portion is most likely to contain the most important portion in the scene of a video content.

As is obviously clear, the distribution of sample values16within the first dynamic range40may change from frame to frame, and accordingly,FIG. 1shows with dotted lines46exemplarily a distribution of another frame14within sequence12. As exemplarily shown inFIG. 1, this distribution46may, for example, be displaced relative to and/or be narrower than distribution44of the current frame. Accordingly, parameter setter22may have set the mapping parameter44for the frame with sample value distribution46differently from the mapping parameter45defining portion48. For example, sample value distribution46may set the mapping parameter for these frames such that portion48approximates a portion of the first dynamic range40occupied by distribution46, i.e., such that portion48is as small as possible but still covers the range of distribution46, with the same applying to portion38with respect to distribution44.

Thus, sequence30substantially corresponds to sequence12with the sample values, however, being represented in another format. Viewing sequence30, however, would result in an unpleasant impression as the sample values34of one frame within sequence30would be defined with respect to another luminance portion than sample values34within another frame of the same sequence. For example, the afore-mentioned frames of sequence12would have the sample values16mapped to sample values34residing within portions38and48, respectively. Thus, a sample value34of, for example, one in one frame would very likely correspond to another actual luminance value than a sample value of one within a different frame of sequence30. Without additional measures, video encoding stage20would, thus, not be able to perform a usual temporal prediction using, for example, motion-compensated prediction as the needed motion vector search would most likely not be successful.

In particular, video encoding stage20is configured to encode the second sequence30of frames32by weighted temporal prediction of a first frame of the second sequence30using a second frame of the second sequence30or a reconstructed version of the second frame of the second sequence30, weighted by a weighting parameter and offset by an offset parameter, as a reference. In other words, video encoding stage20may temporally predict a current frame32of sequence30by motion-compensated prediction and with using another, previously encoded frame32of sequence30as a reference. The motion-compensated prediction may be performed on a block-by-block basis. Motion prediction data such as motion vectors and reference frame index are inserted into the data stream as side information, along with the weighting/offset parameters mentioned below. Each temporally predicted block may have associated therewith a motion vector which video encoding stage20determines by determining a best match of the content of the current block of the current frame32within the reference, i.e. the reference frame weighted and sample-value-offset by parameters50, with trying various displacements (motion-vectors) relative to the position corresponding to the position of the block in the current frame. In order to restrict the search overhead, video encoding stage20restricts the search to some search range.

As will become clearer below, due to the fact that video encoding stage20uses weighted temporal prediction, it is possible for parameter setter22to adapt the reference frame to the current frame with respect to the difference in the associated mapping portion48and38, respectively.

Before describing the functionality of the video encoder ofFIG. 1in accordance with specific embodiments in more detail below, an embodiment for an implementation of the video encoding stage20is described with respect toFIG. 2. In accordance with the embodiment ofFIG. 2, the video encoding stage20comprises a residual coder60, an entropy encoder62, a residual reconstructor64, a temporal predictor66, a subtracter68, an adder70, a further adder72, and a weighter or multiplier74. Subtracter68, residual coder60and entropy encoder62are connected, in the order mentioned, between an input76of video encoding stage20, which, in turn, is connected to an output of sample value converter18, and an output78of video encoding stage20which, in turn, is connected to output26of video encoder10. Residual reconstructor64has an input connected to the output of residual coder60. A first input of adder70is connected to an output of residual reconstructor64. Multiplier74, adder72and temporal predictor66form a loop and are serially connected, in the order mentioned, between an output of adder70and a further input thereof. Concurrently, the serial connection of multiplier74, adder72and temporal predictor66is connected to a further, subtractive input of subtracter68. The values applied to the further inputs of adder72and multiplier74, respectively, are determined by weighting an offset parameters50entering at a parameter input80of video encoding stage20.

In operation, a current frame enters input76while a temporal prediction of the current frame is applied at the subtractive input of subtracter68. The prediction residual82resulting from subtracting the temporal prediction84from the current frame is coded by residual coder60. Residual coder60may, for example, subject residual signal82to a transform, such as a spectrally decomposing transform, wherein residual coder60may perform this transform on a block-by-block basis. Additionally or alternatively, residual coder60may apply a quantization onto residual signal82to reduce the information content contained within the prediction residual82to be encoded into the data stream. Residual coder60may use a quantizer step-size parameter as parameter for the quantization which may additionally be changeable from frame to frame as is illustratively shown by the dashed arrow86. At the output of residual coder60, thus, a lossy coded version88of the prediction residual is obtained. Same is coded into the data stream at output78by entropy encoder62in a lossless way.

Residual reconstructor64recovers a reconstructed version90of the prediction residual at its output connected to a first input of adder70. At the other input of adder70, the result of the temporal prediction84for the current frame enters, and accordingly, adder70combines reconstructed residual90and temporal prediction84to yield a reconstructed version of the current frame forming the basis of the temporal prediction for the next frame. As will be outlined in more detail below, multiplier74multiplies or scales each sample value of reconstructed version70depending on a weighting parameter (ŵ, log WD), and adder72adds an offset depending on the offset parameter ô to each thus scaled sample value. By this measure, the sample values of the reconstructed version70are displaced to a corresponding luminance position within portion38of the current frame to be temporally predicted next. Accordingly, at the output of adder72, a weighted and offset reference frame92results, based on which temporal predictor66performs the temporal prediction using, for example, motion prediction. For example, temporal predictor66uses for a certain block of the current frame, a potentially interpolated and displaced (according to a motion-vector) portion out of reference frame92as a prediction for this block of the current frame currently entering input76.

Thus, as video encoding stage20uses weighted temporal prediction, temporal predictor66uses the reconstructed version of a previously encoded frame in a sample-value-weighted and sample-value-offset from92rather than directly, as output by adder70. Thereby, the difference in the positioning and dimensioning of portions38and48between these frames is balanced. In even other words, the balancing is guaranteed by parameter setter22which, in turn, sets the weighting and offset parameters50entering at input80appropriately.

Thus, returning toFIG. 1again, parameter setter22may be configured to determine an occupied portion of the first dynamic range40within which the sample values16of the current frame of the first sequence12are distributed, with then setting the weighting and offset parameters50such that the portion38set by the mapping parameter55for the current frame approximates the occupied portion. In even other words, parameter setter22may firstly inspect distribution44in order to determine an interesting occupied portion of the first dynamic range40. Then, parameter setter22may set the weighting and offset parameters50of the current frame such that the application of these parameters50onto the sample values of the reconstructed version70effectively leads to displacement and scaling of the portion48of the frame of which the reconstructed version70represents a reconstruction, to yield a portion38approximating the occupied portion defined by distribution44.

In this regard, it should be noted that internally, video encoding stage20may uses a higher dynamic range, such as a higher number of bits, in order to represent the scaled and sample-value-offset reference frame resulting from the application of the weighting and offset parameters at input80onto the reconstruction70of the reference frame, i.e. for reference frame92, as compared to the dynamic range of sequence30, so that the application of these parameters does not lead to any clipping problems. The number of representation bits may be increased by two, for example.

Thus, in even further detail, parameter setter22may be configured to set the offset parameter ô according to a deviation between an upper bound, or a deviation between a lower bound, of the portions38,48set by the mapping parameters for the current and reference frames, respectively, and set the weighting parameter (log WD, ŵ) according to a deviation between the length of the portions38,48, respectively. In specific embodiments outlined further below, for example, the sample value converter18is configured to convert the sample values Ŷ of the frames14of the first sequence12from the first format into the second format according to
b·(logm(Ŷ)−a)
wherein b and a are comprised by the mapping parameter45and are related to a lower bound Ŷminand upper bound Ŷmaxof the portion38out of the first dynamic range40, Ŷminto Ŷmax, according to

b=2n-1logm⁡(Y^max/Y^min),a=logm⁡(Y^min)
wherein logmis a logarithmic function to a base m, and n is an integer indicating a number of integer representation bits of the second format. If so, the parameter setter may be configured to determine an occupied portion of the first dynamic range within which the sample values16of the first frame14of the first sequence12are distributed, and set the weighting parameter and the offset parameter such that

The precision and range of the weighting and offset parameter might be limited, for example, by the video coding stage20, which operates, for example, in accordance with H.264/AVC. If so, the parameter setter may be configured to determine an occupied portion of the first dynamic range within which the sample values16of the first frame14of the first sequence12are distributed, and set the weighting parameter and the offset parameter such that

Further, as will also be discussed with respect to the detailed embodiments outlined below, the video encoding stage20and the residual coders60may be configured to use a quantizer step-size parameter in encoding the second sequence30of frames32and the parameter setter22may be configured to set the quantizer step-size parameter for the frames32of the sequence30depending on the length of the respective portion38,48, set for the respective frame. By this measure, it is possible to harmonize the quantization noise in its temporal variation as it would other wise occur if using a static quantizer step-size parameter due to the temporal variation of the length of portions38and48, respectively. The parameter setter22may be configured to encode the quantizer step-size parameter into the data stream differentially to a quantized step-size parameter for a starting frame of the second sequence such as the I frame of an IPPPPP . . . sequence.

After having described an embodiment for a video encoder, with respect toFIG. 3, a video decoder100in accordance with an embodiment is described below. The video decoder is for reconstructing a sequence102of frames104the sample values106of which are represented in a first format covering a first a dynamic range, from a data stream, such as one generated by the video encoder ofFIG. 1. The format in which values106are represented, may be the format underlying sample values16. However, this is not mandatory.

The video decoder100comprises a video decoding stage108, a parameter setter110and a sample value reconverter112. Further, the video decoder100comprises an input114at which the afore-mentioned data stream enters video decoder100, and an output116for outputting sequence102. Between input114and output116, the video decoding stage108and the sample value reconverter112are serially connected in the order mentioned. Parameter setter110is connected between input114and a parameter input of sample value reconverter112.

With respect toFIG. 4it is shown that the video decoding stage108may be implemented substantially similar to components64,70,74,72, and66of the encoder ofFIG. 2. In particular, video decoding stage108may comprise a residual reconstructor118, an adder120, a temporal predictor122, a scaler/multiplier124and a further adder126. Residual reconstructor118and adder120may be serially connected between an input128of video decoding stage108which, in turn, is connected to input114, and an output130of video decoding stage which, in turn, is connected to sample value reconverter112. In form of a loop, multiplier124, adder126and temporal predictor122are serially connected in the order mentioned between an output of adder120and a further input thereof. The values applied to the further inputs of multiplier124and adder126are controlled according to the weighting and offset parameters which the video decoding stage108derives from the data stream entering input128.

Thus, after having described the internal structure of the video decoder100and the video decoding stage108, respectively, in accordance with an embodiment, their mode of operation thereof is described in more detail below.

As already mentioned above, the video decoder100is for decoding the data stream generated, for example, by the video encoder ofFIG. 1. The data stream has been derived from sequence30in the lower dynamic range format and using the weighting and offset parameters50which the video encoding stage20inserted into the data stream as side information. Accordingly, the video decoder has access to the weighting and offset parameters50used at the encoding side and is able to emulate the reconstruction at the encoding side using the parameters finally chosen at the encoding side by way of, for example, some rate/distortion optimization.

In particular, the video decoding stage108is configured to reconstruct, from the data stream entering input114, the second sequence30′ of frames32′ which corresponds to sequence30ofFIG. 1apart from the coding loss such as the quantization loss introduced by video encoding stage20. The sample values34′ of frames32′ are, accordingly, also represented in the second format covering the second dynamic range42which is lower than the dynamic range of the final reconstructed sequence102. Just as it was the case with the video encoding stage20, the video decoding stage108performs the reconstruction by a weighted temporal prediction of a current frame of the second sequence30′ using a reference frame of the second sequence30′, weighted by a weighting parameter and offset by an offset parameter, both comprised by the data stream entering input114, as a reference. The parameter setter110, in turn, is configured to set the mapping parameter132for the current frame of the second sequence30′ depending on a mapping parameter for the reference frame of the second sequence30′, and the weighting parameter and the offset parameter50of the current frame. The sample value reconverter112, in turn, is configured to convert the sample values34′ of the frames32′ of the second sequence30′ from the second format to the first format using a mapping function which is inverse to the mapping function used by sample value converter18, which maps the second dynamic range42onto the portion out of the first dynamic range such as40, which is set by the mapping parameter for the respective frame of the second sequence.

Imagine, for example, residual reconstructor118of video decoding stage108currently reconstructs a residual for a current frame32′, the reconstruction being indicated by134inFIG. 4. Obviously, residual134will correspond to the one occurring during encoding at reference sign88inFIG. 2. Adder120combines this residual134with the temporal prediction136of the current frame as output by temporal predictor122so as to achieve the reconstructed version138of the current frame, i.e. frame32′. Imagine further, that this reconstructed version138serves as a reference frame for a subsequently decoded frame of frame sequence30′. Then, the weighting parameter (log WD, ŵ) and the offset parameter ô would be contained within the data stream for that subsequently decoded frame, and accordingly, the sample values34′ of the reference frame32′ would be offset and scaled in stages124and126before actually being used as the reference140in the temporal prediction performed by the predictor122. This mirrors the functionality at the encoding side. The temporal predictor122uses motion vectors contained within the data stream to obtain the temporal prediction136from reference140. As the dynamic range, or the number of bits, used for representing reference140is higher than the dynamic range of the original claim sequence30, the reconstruction of which is to be output at130of video decoding stage108, potentially clipping effects which could otherwise occur because of the application of the weighting and offset parameters50in stages124and126, are effectively avoided.

Thus, the sequence30′ output by the video decoding stage108represents a reconstruction of the frame sequence30input into the video encoding stage20at the encoding side. The sample value reconverter112transfers this sequence30′ into a meaningful sequence102by mapping the sample values of frames32′ onto a common format having enough dynamic range in order to accommodate the dynamic range originally contained in the original material12. This format might be the format of the sample values16of sequence12, but may also deviate therefrom. In order to obtain the portion within this common dynamic range which the sample values34′ of a respective frame32′ cover, the sample value reconverter112sequentially applies the chain of weighting/offset parameters associated with these frames32′. In particular, for a current frame, the sample value reconverter112determines this portion, i.e. the position and dimension thereof, by applying the weighting and offset parameters for the current frame onto the position and dimension of the portion previously determined for the reference frame of the current frame. By this measure, the sample value reconverter112recovers portions38and48shown inFIG. 1sequentially.

Thus, in yet other words, the mapping parameter mentioned above may define a length Ŷmax,k−Ŷmin,kof the portion38out of the first dynamic range40and a lower bound ŷmin,k, or an upper bound Ŷmax,k, of the portion38out of the first dynamic range for the current frame32′, and the parameter setter110may be configured to set this mapping parameter132for the current frame of the second sequence30′ by modifying a length Ŷmax,l−Ŷmin,lof the portion48out of the first dynamic range40, defined by the mapping parameter for the reference frame l of sequence30′, depending on the weighting parameter (log WD, ŵ) for the current frame k to derive the length Ŷmax,k−Ŷmin,kof the portion38defined by the motion parameter132for the current frame k, and by modifying a lower or upper bound Ŷmax,lof the portion48out of the first dynamic range40, defined by the mapping parameter for the reference frame l depending on the offset parameter ô for the current frame k, to derive the lower or upper bound Ŷmin/max,kas defined by the mapping parameter132of the current frame. Accordingly, the parameter setter110is steered by the weighting and offset parameters50contained within the data stream entering input114to act like parameter setter22controlling video encoding stage20and sample value converter18.

At his moment it should be noted that the parameter setter110ofFIG. 3is drawn to be merely connected to sample value reconverter112, whereas parameter setter is drawn to control both the sample value converter and the video encoding stage20, respectively. The seeming discrepancy between encoding and decoding site stems from the afore-mentioned fact that encoder's video encoding stage20is not able freely choose the weighting/offset parameters. Rather, same are prescribed from outside, namely by parameter setter22which, in turn, has to take the original signal and it's distribution44and46, respectively, into account when setting these weighting/offset parameters. Parameter setter110, however, is steered by the result of this choice via the side information contained in the data stream arriving via input110, and thus, video decoding stage108may use the weighting/offset parameter information contained within the data stream independently from the parameter setter's evaluation of the same information, namely the weighting/offset parameter information, and accordingly, no control path leading from the parameter setter110to the video decoding stage108is needed. However, according to an alternative embodiment, parameter setter110assumes responsibility for both settings and controls video decoding stage108accordingly from outside. In the latter case, a control path would lead from the parameter setter110to the video decoding stage108.

As has already been noted above, the more detailed description of an embodiment outlined below, will use a logarithmic mapping function between both formats, i.e. a linear mapping function between the first format a logarithmic domain and a second format a logarithmic domain. Accordingly, the sample value reconverter112may be configured to convert the sample values Ln34′ of the frames32′ of the second sequence30′ from the second format into the first format according to
m(Ln+ε)b−1+a
wherein b and a are comprised by the mapping parameter and are related to a lower bound Yminand upper bound Ymaxof the portion38out of the first dynamic range, Yminto Ymax, according to

b=2n-1logm⁡(Y^max/Y^min),a=logm⁡(Y^min)
wherein logmis a logarithmic function to a base m, and n is an integer indicating a number of integer representation bits of the second format.
If so, the parameter setter112may be configured to compute a and b such that

Further, similar to the above description, the video decoding stage108may be configured to use a quantizer step-size parameter in reconstructing the second sequence of frames, and the parameter setter may be configured to set the quantizer step-size parameter for the frames of the second sequence depending on a length of the portion out of the first dynamic range, as set for the respective frames of the second sequence. In this regard, the parameter setter110may be configured to decode the quantized step-size parameter from the data stream differentially to a quantized step-size parameter for a starting frame of the second sequence.

As also described above, although the sample values of the frames of the first sequence have been assumed to be luminance floating-point values, and the sample values of the frames of the second sequence of luma have been assumed to be integer values, other possibilities do also exist.

FIG. 5shows an exemplary portion of a data stream being transmitted from encoding side to decoding side in accordance with the embodiments outlined above with respect toFIGS. 1 to 4. It follows from the above discussion, that the data stream150has the first sequence102of frames, the sample values of which are represented in a first format covering a first dynamic range, encoded therein in a reconstructable form. In particular, the first sequence is encoded into the data stream150indirectly via a second sequence30of frames32the sample values of which are represented in a second format covering a second dynamic range42lower than the first dynamic range, the second sequence being encoded into the data stream by a weighted temporal prediction of a first frame of the second sequence30′ using a second frame of the second sequence30′, weighted by a weighting parameter and offset by an offset parameter, as a reference, wherein the weighting parameter and the offset parameter are comprised be the data stream such that a mapping parameter132for the first frame of the second sequence30′ depends on a mapping parameter for the second frame of the second sequence30′, the weighting parameter and the offset parameter, and the sample values34′ of the frames32′ of the second sequence is converted from the second format to the first format using a mapping function which maps the second dynamic range42onto a portion out of the first dynamic range40which is set by the mapping parameter for the respective frame of the second sequence, reconstruct the first sequence. In other words, data stream may by structured into frame portions152each associated with a respective one of the frames30′ and104, respectively. Each frame30′ may be coded into the data stream150in unit of blocks. Each frame portion152may include motion prediction data154including, for example, a motion vector. Additionally, each frame portion152data may include the weighting and offset parameters50for the respective claim. The data stream may be coded such that the motion prediction data154of each frame portion refers back156to the frame portion immediately preceding in time t, i.e. when arranging the frame portions152along the presentation time axis. That is, each frame may be a P frame using the immediately preceding frame as reference frame, and the portion out of the common dynamic range40may be updated using this dependency chain. Merely, the overall first, i.e. starting, frame158of the frame sequence, may be an I frame, or the starting frames of each GOP, i.e. group of (immediately preceding) pictures. This starting frame158may have incorporated therein an explicit coding160of the mapping parameters for this first frame158. Alternatively, even this explicit coding160may be unnecessary. Further, each frame152, or each frame152but the starting frame158, may have encoded therein a quantizer step-size parameter162, prescribing the quantizing step size to be used in dequantizing in residual reconstructor118and being set in dependency on the length of portion38. In particular, the quantizer step-size parameter162may have been coded into data stream in a differential manner using the (explicitly or implicitly determined) quantizer step-size parameter of the starting frame portion158as a reference.

After having described, by use of rather general terms, embodiments for an encoding and decoding apparatus, more detailed embodiments representing concrete implementations of the above embodiments, are outlined below. In accordance with the concrete implementation details outlined below, a frame-wise adaptive luminance-to-luma mapping is used to perform the transition between the video de/encoding stage and the sample value re/conversion, respectively. In accordance with the embodiments outlined below, the weighted prediction tool of H.264/AVC is exploited to maintain the temporal coherence. In other words, in accordance with the embodiments outlined below, the video encoding stage and the video decoding stage of the above embodiments act like H.264 conform entities, i.e. video encoding stage20generates an H.264 conform data stream and a video decoding stage108is implemented in conformity with the H.264/AVC standard. The data stream ofFIG. 5may even be completely H.264/AVC conform. The weighted prediction tool is, thus, in accordance with the following embodiments not only exploited to maintain the temporal coherence, but, at the same time, to transmit the adaptive mapping parameters used for the sample value conversion. Further, an example will be given as to how to adapt the quantization parameter (QP) for each frame dependent on the adaptive mapping.

Thus, in the following, implementation details with respect to the above-outlined embodiments ofFIGS. 1-5are presented below by use of mathematical equations in more detail. After that, in Section 2, experimental results using these implementation details, are presented.

1.1. Dynamic Range Adaptive Luminance Mapping

In the following we re-visit the luminance-to-luma mapping for video coding applications. The trade-off between the representable luminance range [Ymin, Ymax], the luma bit depth n and the associated relative precision can be seen in the following more general formulations of the luminance-to-luma mapping functions:

This linear relationship between the logarithm of the luminance Y and the luma space L is also depictedFIG. 6.FIG. 6shows an adaptive logarithmic luminance-to-luma mapping: different ranges for different frames l and k result in different mapping functions. Consequently, different luma values can represent the same luminance value.

Obviously, the mapping achieves the highest fidelity when Yminand Ymaxequals the minimum and maximum luminance of the current video frame, respectively. That is, if the existing luminance values in a video frame are mapped to the full luma range by the mapping function with the steepest possible slope. However, since the dynamic ranges can vary from one frame to the next (even in a static scene, due to noise), such a straightforward adaptation would break the temporal coherence of the video sequence and prevent an efficient temporal prediction. The next section will present an adaptive mapping that takes such effects into account.

Consider that two consecutive frames k and l=k+1 of an HDR video sequence exhibit different luminance ranges [Ymin,k, Ymax,k] and [Ymin,l, Ymax,l], respectively. Obviously, using the extrema of each frame in (2) will result in a different mapping for each frame. That is, the same luminance value Ŷ=Yk=Ylin frame k and l will be mapped to different luma values Ln,kand Ln,l, respectively as exemplified inFIG. 1. Plugging (3) into (2) using a different mapping for frame k and l, respectively, yields:

Apparently, the relation of two luma values Ln,kand Ln,lstemming from the same luminance value Ŷ is entirely defined by a scale w and an offset o. w and o can be easily derived from the ranges [Ymin,k, Ymax,k] and [Ymin,l, Ymax,l].

H.264/AVC is the first international video coding standard defining the syntax for a weighted prediction (WP) tool [7]. The original intention of WP is to enhance the coding efficiency for fade-in and fade-out sequences where motion compensated prediction usually fails. It allows to explicitly signal a weight parameter ŵ and an offset parameter ô per slice. The parameters can be used to weight and shift the reference frame for enhancing the temporal prediction. Equation (4) shows that a change of the dynamic range of successive frames merely results in a weighting w and shifting o of identical luminance values in the luma space. Therefore, the WP syntax of H.264/AVC is perfectly suited to allow for an efficient temporal prediction despite any changes in the luminance range. Consider, e.g., the case that a nearly static scenery is recorded by an HDR capable camera facing the bright sun. When the sun is now abruptly covered by a cloud, the dynamic range will change by several orders of magnitude whereas the luminance values of all the foreground objects will approximately remain constant. If we can use the WP tools to adapt the luma values of the reference frame, it allows for a perfect temporal prediction of the foreground pixels that stem from the same luminance values. Furthermore, the WP parameter information is sufficient to convey any needed side information for a frame-wise adaptation of the luminance-to-luma mapping as it will be shown in the following.

In H.264/AVC the precision and dynamic range of ŵ and ô is limited. Both parameters can take on integer values between −128 and 127. The precision of ŵ is confined by a quantization interval of ½log WD, where log WD is signaled explicitly and can take on integer values from 0 to 7. Consequently, a higher log WD value leads to a more fine-grained representation of the parameter ŵ. It also means that more bits are needed for coding the weighting factors and a narrowing of the range of the effective scaling [7]. The step size of the offset parameter is defined by 2n-8in order to take into account the bit depth n of the luma representation in the H.264/AVC coder. Consequently, in order to allow for a perfect temporal prediction of unchanged luminance values from one frame to the next, it is needed to quantize the change of the adaptive mapping function in such a way that it can be represented by the H.264/AVC WP parameters ŵ and ô.

That is, given the dynamic luminance range covered by the mapping function of frame k,

[Ŷmin,k, Ŷmax,k], we have to find the minimum Ŷmax,land the maximum Ŷmax,lthat fulfill

The latter two inequalities assure that the luminance range covered by the adaptive mapping covers at least the range of luminance range present in the current frame, [Ymin,l, Ymax,l].

In practice, parameter setter22may find the solution to this problem by solving (5) and (6), setting Ŷmax,l=Ymax,land Ŷmin,l=Ymin,land rounding towards zero. This yields the initial values for ŵ and ô and (5) and (6) w.r.t. Ŷmin,land Ŷmax,l, respectively can be solved:

If the results violate one of the conditions in (7), parameter setter22may decrease ŵ or increase ô by 1, respectively and re-calculate (8) and (9).

After finding the best luminance range [Ŷmin,l, Ŷmax,l], of frame l w.r.t. frame k, the parameter setter22and the sample value converter18may use these values for the mapping in (2). Furthermore, the weight and offset parameters ŵ and ô are readily available for usage in the weighted temporal prediction of the H.264/AVC video encoder20. Finally, it can be seen from the relations in (5) and (6) that these parameters fully suffice to exactly recover the luminance range of the current frame38given the range of the previous frame48. No additional side information is needed for the adaptive mapping when the mapping of the first frame (and possibly IDR frames) covers the maximal visible dynamic range. Otherwise, the range for the first frame may be signaled explicitly to the decoder as illustrated by dashed line28. In general, however, the scheme according to Section 1 avoids that the float-valued scaling information has to be transmitted as side information for each frame, otherwise complicating standard conformant coding and increasing bit rate.

In accordance with the above measures, for each frame, different luminance ranges are mapped to luma code values. Therefore, using the identical QP during the H.264/AVC encoding process, would lead to a varying quantization of the luminance space, depending on the mapping. In other words, even though the encoder might use a constant quantization, the effective quantization will largely vary across time, leading to strong variations in quality and bit rate. Therefore, in accordance with an embodiment, the coding stages20and108take the luminance mapping range into account and find a suitable ΔQP for each frame, accordingly. Here, ΔQP denotes a QP offset for the current frame w.r.t. the reference QP that is used to encode the first frame. It can be easily seen inFIG. 1that, in order to introduce the same effective quantization to the luminance values, the quantizer step sizes Qstep,land Qstep,kof the current frame l and an arbitrary reference frame k have to be related according to

Taking into account the fact that, per definition Qstepapproximately doubles when the QP value is increased by 6 units we can state:
Qrell,k≈2ΔQPl,k/6⇄ΔQPl,k=round(6 log2(Qrell,k)).  (11)

In this work, we use the first frame of a sequence as reference frame for calculating the QP offset values for each frame. That is, an arbitrary frame l will be quantized with QP=QP1+ΔQP1,1.

2. EXPERIMENTAL RESULTS

For evaluating the temporally coherent luminance-to-luma mapping of Section 1, we performed coding experiments with three HDR test sequences: Panorama, Tunnel, and Sun. All sequences have a resolution of 640×480 pixel and a frame rate of 30 fps. The panorama test sequence was generated by panning a 8000×4000 pixel HDR panorama image. It and shows dark interior areas as well as very bright sun reflections from outside a window. Its overall dynamic range is of the order of 1010:1. Both, Tunnel and Sun were taken from inside a driving car with an HDR video camera and are freely available from Max-Planck Institute [8]. The former one shows a drive through a dark tunnel, the latter one shows a drive on a highway facing the bright sun. The overall dynamic range represented in these sequences is 105:1 and 107:1, respectively. In our experiments we use two metrics to evaluate the quality of the decoded HDR videos: the HDR visible difference predictor (VDP) [9] and the perceptually uniform peak signal-to-noise ratio (PU PSNR) [10]. The former one estimates the percentage of pixels in a pair of images that an observer will notice to be different with a probability of more than 75%. The latter metric is a straightforward extension of the common PSNR metric to HDR. For LDR images it is assumed that the gamma corrected pixel code values are perceptually uniform, that is, equal error amplitudes are equally visible in bright and dark regions of an image. However, this assumption does not hold for HDR images and therefore, the code values are to be scaled to a perceptually uniform space before meaningful PSNR values can be calculated [10].

For encoding the sequences, they are first transformed from RGB floating-point values to the Log Luv space and then encoded with the H.264/AVC reference software JM 17.2. The luma component is encoded with a bit depth of 12 bit/sample, the u′ and v′ components are subsampled by a factor of two vertically and horizontally and encoded with 8 bit/sample. We use the same configuration of the H.264/AVC high profile with 8×8 transform, IPPP GOP structure, intra frame period of 15, and CABAC enabled for all experiments. A fixed reference QP is selected for each encoder run and no rate-control is enabled. However, the frame-wise QP may deviate from this reference QP as described in Sec. 1.3. After decoding the sequences, they are mapped back to RGB floating-point values and their quality is evaluated according to the metrics described before.

In particular,FIG. 7shows the coding results for three cases: temporally coherent mapping according to Section 1 (“proposed”), frame-wise adaptation for each frame without temporal coherence (“frame-wise”) [6], and constant mapping of the whole visual luminance range [10−4, 108] (“visual range”). Upper row: visible difference predictor (VDP). Lower row: perceptually uniform peak signal-to-noise ration (PU PSNR).

FIG. 7shows the coding results for all test sequences in terms of the VDP averaged over all decoded frames (upper row) and in terms of mean PU PSNR of the luminance component (lower row). In particular,FIG. 7shows the coding results for three cases: temporally coherent mapping (“proposed”), frame-wise adaptation for each frame w/o temporal coherence (“frame-wise”) [6], and constant mapping of the whole visual luminance range [10-4, 108] (“visual range”). Upper row: visible difference predictor (VDP). Lower row: perceptually uniform peak signal-to-noise ratio (PU PSNR).

The proposed method (“proposed”) is compared with two reference methods inFIG. 7: straightforward frame-wise adaptation of the luminance-to-luma mapping to the dynamic range of each frame without taking into account the temporal coherence (“frame-wise”) [6], and constant mapping of the whole perceivable luminance range [10-4, 108] (“visual range”). In the latter case, the luminance range of the mapping function might exceed the range of occurring luminances in many HDR video sequences. However, in a real-time coding application it is not possible to narrow the mapping range to the absolute luminance range of a sequence, because this would request the processing of the whole sequence before encoding.FIG. 7clearly shows that the proposed mapping significantly outperforms the reference methods for all test sequences. It is worth noting here that the VDP metric is a threshold metric that only offers an estimate about if a pixel is perceived erroneous or not. It does not state how annoying this error is for an observer. Thus, e.g., the results inFIG. 7(a)can be interpreted as follows: if we allow about 1% of the pixels to be perceived erroneously, with the proposed mapping, we only need a bit rate of less than 2500 kbits/s. This is a reduction of about 50% (25%) compared to the 5000 kbits/s (3250 kbits/s) we have to spend to achieve same VDP value in the “visual range” (“frame-wise”) scenario. Likewise, huge rate savings can be observed for the Tunnel and Sun test sequences inFIGS. 7(b) and (c).

As expected, the PU PSNR results inFIGS. 7(d)-(f)depict similar performance characteristics as the VDP results for all sequences. Furthermore, they allow a quantitative conclusion of the gain in quality that can be achieved with the proposed method for a large range of bit rates. E.g., for the Panorama sequence the PU PSNR value of the proposed method exceeds the PU PSNR value of the “visual range” mapping by 3 dB at 3250 kbits/s (cf.FIG. 7(d)). This means that the mean squared error in the perceptually uniform luminance space is halved at the same bit rate and the visual quality is increased significantly.

It is worth noting, that for the Panorama sequence the frame-wise adaptive mapping has a very detrimental effect on the coding efficiency compared to the non-adaptive “visual range” mapping. This sequence exhibits very large and fast variations of its dynamic range and therefore, in the case of the frame-wise adaptive mapping, the temporal prediction fails (cf.FIGS. 7(a),(d)). On the other hand, it can be observed inFIGS. 7(b) and (e)that the proposed method performs almost identical to the “frame-wise” mapping. In this sequence, the temporal changes of the dynamic range are very smooth. In our experiments we further observed that for the “frame-wise” mapping there exist strong temporal variations of the bit rate and quality whenever the dynamic range changes significantly. This negative effect could be circumvented by the temporally coherent quantization and mapping of the proposed method.

In Section 1, thus, an adaptive luminance-to-luma mapping has been proposed that allows the compression of floating-point high dynamic range video data with the state-of-the-art H.264/AVC video coding standard. Unlike other methods the mapping is adapted to the dynamic range of each frame. Nevertheless, temporal coherence is sustained by exploiting the weighted prediction tools of H.264/AVC and by applying a frame-wise adaptation of the quantization parameter in accordance with the mapping function. No additional side information is needed and significant bit rate savings of up to 50% compared to non-adaptive methods can be observed at the same quality.

Finally, it should be noted that all the details presented in Sections 1-3, could also vary in some sense. For example, neither the weighting/offset parameters mentioned with respect toFIGS. 1-5, nor the weighting/offset parameters mentioned in Sections 1-3, are restricted to those of the H.264/AVC standard, i.e. log WD, ŵ and ô. The weighting/offset parameters could be transmitted in form of different syntax elements. In particular, it is not needed to split up the transmission of the weighting parameter into two syntax element entities log WD, ŵ. Similarly, it should be noted that the sequence30and30′, respectively, could be coded in form of an IPPPP . . . sequence—or in form of IPPPP . . . GOPs—with using the respective immediately preceding frame as reference frame. The first I frame could represent a starting frame as mentioned in Section 1.3 referring to which the quantization parameter may be readjusted. However, all the embodiments outlined above are not restricted to such a type of sequence. Even B frames could be used within the coding scheme in video encoding stage20and video decoding stage108when taking additional measures in the parameter setter22into account in order to fulfill the constraints posed by both weighting/offset parameters for the current frame with respect to the two reference frames, that is, by taking into account the weighting/offset parameters of the reference frame and the weighting/offset parameters of the other reference frame of the current frame with both parameter pairs being transmitted within the data stream.

Further, as already noted above, instead of a luminance-to-luma mapping, another mapping could be the subject of the embodiments outlined above. In other words, the sample values could pertain to other information than luminance. Further, the implementation of the video encoding stage20and the video decoding stage108ofFIGS. 2 and 4are to be understood merely as being of illustrative nature. For example, the entropy encoder62responsible for entropy coding the residual signal88could be left off. Similarly, an entropy decoder129could optionally connect it between input128and residual reconstructor118of video decoding stage108ofFIG. 4.

The inventive data stream can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

In even other words, embodiments suitable for an efficient compression of high dynamic range video (HDR) sequences have been described. In order to obtain a coded representation that is compatible with the H.264/AVC video coding standard, the float-valued HDR values are mapped to a suitable integer representation. The mapping used is adapted to the dynamic range of each video frame. Furthermore, to compensate for the associated dynamic contrast variation across frames, a weighted prediction method and quantization adaptation are introduced.

From another point of view, above embodiments are an improvement of the Adaptive-Log Luv transform also described in EP10151074.1, the description of which is incorporated herewith for further details. Basically, an adaptive logarithmic mapping of float to integer values similar to that in EP10151074.1 has been used. The parameters of this mapping, however, are no longer totally free. Instead they are, in accordance with the above embodiments, restricted to fit the characteristics of the H.264/AVC video codec and especially the weighted prediction (WP) tool of H.264/AVC. With these restrictions, the following benefits were gained: (1) The WP tool can be used to ensure temporal coherence. (2) The H.264/AVC syntax for WP can be used to signal the parameters of the Log Luv mapping, thus removing the need for additional side information. IN the above description, it has been shown how to adapt the quantization parameter of the H.264/AVC coder dependent on the adaptive mapping.

REFERENCES