# Ascertaining the Ideality of Photometric Stereo Datasets under Unknown Lighting
## 4 Dealing with Nonideal Data
```
0: PS data matrix \(M\) of size \(p\times q\) (\(q\) pictures, each with \(p\) pixels)
0: Set \(\mathcal{S}\) containing the indices of the images to be kept in the dataset
1: Compute the compact SVD \(M=U\Sigma V^{T}\), with \(V=[\mathbf{v}_{1},\dots,\mathbf{v}_{q}]\), \(\mathbf{v}_{i}\in\mathbb{R}^{q}\)
2:\(\widetilde{L}=[\mathbf{v}_{1},\mathbf{v}_{2},\mathbf{v}_{3}]^{T}\)
3:\(\mathcal{S}=\{1,\dots,q\}\)
4:\(k=0\), \(\mu_{0}=0\), \(L=\widetilde{L}\)
5:repeat
6:\(k=k+1\)
7:for\(i=1,2,3\)do
8:\(H_{:,i}=(L_{i,:}^{T})^{2}\) (componentwise)
9:endfor
10:\(H_{:,\mathcal{A}}=2(L_{1,:}^{T})*(L_{2,:}^{T})\) (componentwise)
11:\(H_{:,\mathcal{S}}=2(L_{1,:}^{T})*(L_{3,:}^{T})\) (componentwise)
12:\(H_{:,\mathcal{S}}=2(L_{2,:}^{T})*(L_{3,:}^{T})\) (componentwise)
13:for\(i=1,\dots,q\)do
14: Let \(\widetilde{H}\) be \(H\) with column \(i\) removed
15:\(\mathbf{g}=\arg\min\|\widetilde{H}\mathbf{g}-\mathbf{1}\|\)
16: Construct matrix \(G\) from \(\mathbf{g}\)
17:\(\lambda_{i}=\min\{\text{eigenvalues of }G\}\)
18:endfor
19:\(\lambda_{\ell}=\max\{\lambda_{1},\dots,\lambda_{q}\}\)
20:if\(k=1\) and \(\lambda_{\ell}\leq 0\)then
21: Stop iteration: unrecoverable breakdown
22:endif
23: Store in \(t\) the \(\ell\)th element of \(\mathcal{S}\)
24: Remove \(t\) from \(\mathcal{S}\)
25:\(q=q-1\)
26:\(\mu_{k}=\lambda_{\ell}\)
27:if fast version then
28:\(L=\widetilde{L}_{:,\mathcal{S}}\) (\(L\) contains the columns of \(\widetilde{L}\) with indices in \(\mathcal{S}\))
29:else
30: Remove column \(\ell\) from \(M\)
31: Compute the compact SVD \(M=U\Sigma V^{T}\), with \(V=[\mathbf{v}_{1},\dots,\mathbf{v}_{q}]\)
32:\(L=[\mathbf{v}_{1},\mathbf{v}_{2},\mathbf{v}_{3}]^{T}\)
33:endif
34:until\(\mu_{k}<\mu_{k-1}\) or \(q=6\)
35:\(\mathcal{S}=\mathcal{S}\cup\{t\}\)
```  
**Algorithm 1** Removal of \"unideal\" images from a PS dataset: linear approach. At the beginning, we construct the Rank-3 factor \(L_{3}\) from (5), here denoted as \(\widetilde{L}\), and initialize the indices set \(\mathcal{S}\) to \(\{1,\dots,q\}\). Inside the main loop, at Lines 7-12, the matrix \(H\) with rows indexed in \(\mathcal{S}\) is constructed. Then, each row is iteratively removed from \(H\), and the smallest eigenvalue of the corresponding matrix \(G\) is stored (see Lines 13-18). The largest among the found eigenvalues identifies the index to be removed from \(\mathcal{S}\) (Line 24), that is, the image to be excluded. At this point, the corresponding column is removed from \(M\), and its compact SVD decomposition is recomputed (see Lines 30-32). The iteration continues until the sequence of selected eigenvalues stops increasing, or the number of images \(q\) falls below six. We explored the possibility of reducing the computational cost by avoiding recomputing the SVD decomposition of \(M\) and extracting the matrix \(L\) at each iteration from the initial factorization (see Line 28). The resulting algorithm, denoted as \"fast\" in Algorithm 1, is less accurate, as will be shown in the following. The nonlinear model for identifying the light location, outlined in Section 3, also leads to a procedure for excluding unideal images from a redundant dataset. Indeed, we noticed that in the presence of images that deviate from ideality and that would lead to a nonpositive definite matrix \(G\) in the Hayakawa procedure, the Gauss-Newton method diverges, producing a meaningless solution. The reason is that the Jacobian (11) becomes rank deficient as the iterates reach a neighborhood of the solution. Indeed, the \(q\times 6\) matrix \(J(\mathbf{r}^{(k)})\) (\(q\geq 6\)) should have Rank 6, but as the iteration progresses, it may happen that its numerical rank falls below this value. Denoting by \(\gamma_{i}\), \(i=1,\ldots,6\), the singular values of \(J(\mathbf{r}^{(k)})\), we propose to use the ratio between the sixth and the fifth singular values  
\[\eta=\frac{\gamma_{6}}{\gamma_{5}} \tag{12}\]  
as an indicator of nonideality. This leads to the numerical procedure reported in Algorithm 2.