Inter-layer correlation considered R-D models and Lagrange multiplier for SVC MGS coding
Traditional rate-distortion (R-D) model-based Lagrange multiplier \(\lambda \) that is employed by H.264/SVC does not consider the inter-layer correlation. This paper presents new R-D models and \(\lambda \) which exploits inter-layer correlation for coding modes that perform residual prediction in H.264/SVC medium-grain quality scalability (MGS) coding. We have observed that in MGS coding, the prediction error of modes performing residual prediction is approximately equal to the reconstruction error of the corresponding macroblock in the reference layer. Based on that observation, we investigate the distribution \(f_r \) of transformed residual prediction error signals and prove that \(f_r \)is related to the quantization step of the corresponding macroblock in the reference layer. In such a case, both the conventional \(\lambda \) and the R-D models in the literature that are derived independently of inter layers are not much fit for residual prediction modes any more. Thus, we build more appropriate R-D models depending on the derived distribution and develop a new \(\lambda \) from the R-D models. Experimental results show that when residual prediction is enabled, the proposed scheme by using the new Lagrange multiplier achieves an average PSNR gain of 0.47 dB and up to more than 1 dB over the scheme using the conventional Lagrange multiplier.
lagrange multiplier rate-distortion h.264/svc inter layer correlation