## 3 Proposed Lagrange Multiplier Selection Algorithm
### Adaptive Lagrange Multiplier Selection
Based on the observation the variance of each coefficient can be estimated by:  
\[\sigma_{q}^{2}=\beta_{q}\sigma_{t}^{2}, \tag{28}\]  
where \(\beta_{q}\) is a parameter related to the wavelet transform, and \(\sigma_{t}^{2}\) the variance of the residual pixel values before wavelet transform. Therefore, Equation (27) can then be rewritten as:  
\[\mathrm{H}(\mathrm{X},\mathrm{Y})=-\sum\limits_{\mathrm{x},\mathrm{y}\in \mathrm{A}_{X},\mathrm{A}_{Y}}\mathrm{p}_{XY}(\mathrm{x},\mathrm{y})\log[ \mathrm{p}_{XY}(\mathrm{x},\mathrm{y})]. \tag{20}\]0  
Moreover, the variance of the residual pixel value before wavelet transform \(\sigma_{t}^{2}\) can be further approximated using the mean absolute difference (MAD) by \(\sigma_{t}^{2}\approx 2\text{MAD}^{2}\). Consequently, the distortion \(\text{d}_{i}\) for the \(i\) th subband can be expressed as:  
\[ \begin{array}{ll}\text{d}_{i}&=\text{k}^{\prime}\cdot\frac{\tau\text{e}}{6} \cdot\left(\prod\limits_{q=1}^{\text{Q}}\beta_{q}\sigma_{t}^{2}\right)^{1/ \text{Q}}\cdot 2^{-\eta\tau_{i}}\cdot\omega_{i}\\ &=\text{k}^{\prime}\cdot\frac{\tau\text{e}}{6}\cdot\left(\prod\limits_{q=1}^ {\text{Q}}\beta_{q}\cdot 2\text{MAD}^{2}\right)^{1/\text{Q}}\cdot 2^{-\eta\tau_{i}} \cdot\omega_{i}\\ &=\text{k}\cdot\frac{\tau\text{e}}{6}\cdot\text{MAD}^{2}\cdot 2^{-\eta\tau_{i}} \cdot\omega_{i}\end{array} , \tag{30}\]  
with \(\text{k}=2\text{k}^{\prime}\left(\prod\limits_{q=1}^{\text{Q}}\beta_{q} \right)^{1/\text{Q}}\) a parameter related to the wavelet transform decomposition scheme used. According to Equations (26)-(30), it is easy to compute the Lagrange multiplier \(\lambda_{i}\) of the \(i\)th subband as:  
\[\lambda_{i}=-\left.\frac{\partial\mathrm{d}_{i}}{\partial\mathrm{r}_{i}}\right|_ {\mathrm{r}_{i}=\mathrm{R}_{i}},\quad(i=1,2,\ldots\mathrm{K}), \tag{31}\]  
Correspondingly, Equation (31) can be written as:  
\[\left\{ \begin{array}{c}\lambda_{1}=-\omega_{1}\cdot 2^{-\mathrm{n}}\cdot \left.\frac{\partial\mathrm{d}_{1}}{\partial\mathrm{r}_{1}}\right|_{\mathrm{r }_{1}=\mathrm{R}_{1}}\\ \lambda_{2}=-\omega_{2}\cdot 2^{-\mathrm{n}}\cdot\left.\frac{\partial\mathrm{d}_{2}}{ \partial\mathrm{r}_{2}}\right|_{\mathrm{r}_{2}=\mathrm{R}_{2}}\\ \vdots\\ \lambda_{\mathrm{K}}=-\omega_{\mathrm{K}}\cdot 2^{-\mathrm{n}}\cdot\left.\frac{ \partial\mathrm{d}_{2}}{\partial\mathrm{r}_{\mathrm{K}}}\right|_{\mathrm{r}_ {\mathrm{K}}=\mathrm{R}_{\mathrm{K}}}\end{array} \right., \tag{32}\]  
Based on Equations (30)-(32), a more practical Lagrange multiplier expression is then derived as:  
\[\lambda_{i}=\varphi\mathrm{MAD}^{2}2^{-\mathrm{n}\mathrm{r}_{i}}\omega_{i}, \tag{33}\]  
with \(\varphi=6.2\) an empirical constant suitable for different sequences. By means of Lagrange multiplier \(\lambda_{i}\), the RDO objective function Equation (7) can be rewritten as:  
\[ \begin{array}{rl}\mathrm{J}(\mathrm{F}_{i})&=\mathrm{D}_{i}(\mathrm{F}_{i}, \mathrm{M}_{i})+\lambda_{i}\cdot\mathrm{R}_{i}(\mathrm{F}_{i},\mathrm{M}_{i}) \\ &=\sum\limits_{i=1}^{\mathrm{K}}\omega_{i}\mathrm{d}_{i}(\mathrm{r}_{i})+ \lambda_{i}(\sum\limits_{i=1}^{\mathrm{K}}\mathrm{R}_{i}-\mathrm{R}_{\mathrm{ T}})\\ &=\sum\limits_{i=1}^{\mathrm{K}}\left[\sum\limits_{t=1}^{\mathrm{T}}\prod \limits_{j=1}^{t-1}\omega_{i,t}^{(j)}\mathrm{d}_{i,t}^{(t)}(\mathrm{r}_{i})+ \prod\limits_{t=1}^{\mathrm{T}}\omega_{i,t}^{(t)}\mathrm{d}_{i,t}^{(\mathrm{T })}(\mathrm{r}_{i})\right]+\lambda_{i}\cdot(\sum\limits_{i=1}^{\mathrm{K}} \mathrm{R}_{i}-\mathrm{R}_{\mathrm{T}})\end{array} , \tag{34}\]  
where \(\omega_{i,t}^{(j)}\mathrm{d}_{i,t}^{(t)}(\mathrm{R}_{i})\) denote the weighted distortion term for the high-frequency subbands, and \(\omega_{i,t}^{(t)}\mathrm{d}_{i,t}^{(\mathrm{T})}(\mathrm{R}_{i})\) the weighted distortion term for the low-frequency subbands. Only the frame mode with the minimum Lagrange cost \(\mathrm{J}(\mathrm{F}_{i})\) is finally chosen as the best frame mode.