# Per-Pixel Manifold-Based Color Calibration Technique
## 4 Formulation in 2D Pixel Grid  
Suppose we have an image of size \(W\) by \(H\), and we can indicate the pixel \(p\) position by its \(u,v\) coordinates in the image. Moreover, suppose that we introduce 2 smoothness constraints: one vertical and one horizontal. It means that pixel \(p_{v}\) and \(q_{v}\) have two neighboring pixels in which the \(v\) coordinate differs only by \(1\). \[p_{v}=u,v\quad q_{v}=u,v+1 \tag{35}\]  
Similarly, pixel \(p_{h}\) and \(q_{h}\) have two neighboring pixels whose u coordinate differs only by \(1\). \[p_{h}=u,v\quad q_{h}=u+1,v \tag{36}\]Moreover, we will assume that condition (34) is reflective and must hold for both pixels \(p\) and \(q\), so:  
\[o=p\quad o=q \tag{37}\]  
will lead to  
\[ \begin{bmatrix}LL^{p}&-LL^{p}\\ -LL^{p}&LL^{p}\end{bmatrix} \cdot \begin{bmatrix}AA^{p}\\ AA^{q}\end{bmatrix} = \begin{bmatrix}0\\ 0\end{bmatrix}  \tag{38}\]  
\[ \begin{bmatrix}LL^{q}&-LL^{q}\\ -LL^{q}&LL^{q}\end{bmatrix} \cdot \begin{bmatrix}AA^{p}\\ AA^{q}\end{bmatrix} = \begin{bmatrix}0\\ 0\end{bmatrix}  \tag{39}\]  
Combining both (38) and (39) together results in  
\[ \begin{bmatrix}LL^{p}+LL^{q}&-(LL^{p}+LL^{q})\\ -(LL^{p}+LL^{q})&LL^{p}+LL^{q}\end{bmatrix} \cdot \begin{bmatrix}AA^{p}\\ AA^{q}\end{bmatrix} = \begin{bmatrix}0\\ 0\end{bmatrix}  \tag{40}\]  
After making such assumptions, we can provide an explicit form of AA, \(\mathcal{RL}\) vectors, and \(\mathcal{LL}\) matrix for any given pixel \(p=u,v\) based on (31), (32), and (40). As the conditions \(S_{1}\) and \(S_{2}\) need to be satisfied simultaneously, we introduce two coefficients \(\alpha\) and \(\beta\) to control the strength of the smoothness prior to \(S_{2}\) in both directions on the 2D grid. The values of coefficients \(\alpha\) and \(\beta\) were experimentally adjusted for universally good results among the tested sequences. The value of \(0.1\) was found to work well in our experiments, independent of the sequence contents and scene complexity. It needs to be stressed, however, that different resolutions of the sequences may require a slight modification of the values, but we found during the experiments that the method is not sensitive to the actual value of the coefficients \(\alpha\) and \(\beta\). \[\text{AA}^{p=u,v}= \begin{bmatrix}0\\ 0\\ \vdots\\ 0\\ AA^{p=u,v}\\ 0\\ 0\\ \vdots\\ 0\\ 0\\ 0\\ \vdots\\ 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \end{bmatrix}  \tag{41}\]  
stemming from slight differences between the color characteristics of the cameras and/or lighting conditions are reduced significantly. The PSNR value measured against a real view is a commonly used measure of quality for virtual view synthesis research. For color correction algorithms, however, a more significant figure of merit would be the comparison between the virtual view synthesized only from the left and only from the right view. Table 3 shows the values of the error (measured as a mean value of SSD for pixel values) for the virtual view generated from the left view calculated against the virtual view generated from the right view for three cases: no color correction applied, the global color correction applied according to the method described in [33], and the proposed method. A visual comparison of the differences between the virtual view generated from the left view and the virtual view generated from the right view is presented in Figures 2 and 3. The proposed method outperforms significantly the global color correction method. In order to test our method in a flash/no-flash scenario, we use the images from [4], as supplied in [34]. The proposed method of estimating per-pixel color transfer manifold can be used to correct images captured with flash to its low-light ambient counterpart, to match the no-flash colors (Figure 4). Figure 3: Comparison of the differences between the virtual view generated from the left view and the virtual view generated from the right view: **Left column—**original VSRS, **right column—**modified VSRS. From the top: Breakdancer, Ballet, Soccer.