SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xingyu Jiang $^{1,2}$  Jiayi Ma $^{2*}$
$^{1}$ Huazhong University of Science and Technology,  $^{2}$ Wuhan University
{jiangxy998,jyma2010}@gmail.com
In this paper, we contribute to solving a threefold problem: outlier rejection, true model reasoning and parameter estimation with a unified optimization modeling. To this end, we first pose this task as a sparse subspace recovering problem, to search a maximum of independent bases under an over-embedded data space. Then we convert the objective into a continuous optimization paradigm that estimates sparse solutions for both bases and errors. Wherein a fast and robust solver is proposed to accurately estimate the sparse subspace parameters and error entries, which is implemented by a proximal approximation method under the alternating optimization framework with the "optimal" sub-gradient descent. Extensive experiments regarding known and unknown model fitting on synthetic and challenging real datasets have demonstrated the superiority of our method against the state-of-the-art. We also apply our method to multi-class multi-model fitting and loop closure detection, and achieve promising results both in accuracy and efficiency.
Code is released at: https://github.com/StaRainJ/DSP.
Geometric model estimation is a fundamental problem in computer vision, serving as the core of many high-level tasks including structure-from-motion [28, 39], SLAM [33, 9], and data alignment [24, 27]. The models to specific data can be explained as line/ellipse/circle for 2D point sets, plane/cylinder/sphere for 3D point sets, or fundamental/homography/affine model for point correspondences extracted from two-view images, etc. Estimating the parameters of a predefined model from inliers only is well studied [21]. But real-world inputs unavoidably contain severe noises and outliers, posing great challenges for accurate model estimation. Besides, another key problem rarely considered is how to recovery the model parameters without knowing the model type.
Recent researches mainly focus on proposing robust estimators to tackle the impact of noise and outliers, such as regarding it as a Consensus Maximization (CM) problem. This problem can be well solved with Sample Consensus (SAC) methods, such as RANSAC [19] and its variants[11, 38, 3, 6, 22]. They commonly sample a smallest inlier set, to best fit a given geometry model following a hypothesize-and-verify strategy [28]. Inside this loop, the model is actually estimated by Direct Linear Transform (DLT) method with a Least-Square (LS) solution. SAC methods can provide probabilistic guarantee of hitting an all-inlier subset. However, this scheme succeeds only if given a predefined model type with sufficient time budget [52]. Another popular strategy is to pose it in a Global Optimization framework, which formulates the fitting task as a subspace learning problem [45, 37, 13, 17, 18, 14]. Specifically, the representative method, Dual Principal Component Pursuit (DPCP) [45], tries to minimize an  $\ell_1$  co-sparse objective on the sphere. This is further applied to estimate specific models [12, 18], which typically embeds the model as a single hyperplane, or the intersection of multiple hyperplanes, and optimizes globally. This type of methods are admitted with
efficient implementations and strong theoretical guarantees, thus arising great research interest in recent years. Deep Learning has also stimulated numerous methods for geometric model learning, which extract deep geometric cues from sparse points using multilayer perceptrons [49, 43, 50], convolutional [51] or graph networks [40]. They can fast output potential inliers once trained, but still require robust estimator such as RANSAC as postprocess for accurate model estimation.
The above mentioned methods can only work on a fact that: one is certain about the true geometric models, then uses them to guide the formulation for parameter estimation. But in fact, we can always fit a model less constrained to obtain higher inlier count [31, 37]. Specifically, for homography data, estimating a fundamental matrix may return more consensus points, but many are outliers. While homography estimation can only find the largest plane structure in 3D scene, missing considerable inliers for full motion data. Fitting an unknown model for heavily contaminated data is much challenging and is in general NP hard. Existing methods solve it with model selection criteria [1, 42, 44], which first fit all possible models, then select the best one with a geometric verification, such as widely used GRIC metric [44, 32]. This is what is done in SfM or SLAM pipeline for fundamental matrix and homography identification [39]. However, such strategy is limited by the greedy selection strategy thus requiring huge computational burden. In addition, the used insufficient information would easily cause wrong selection of the true model for constrained motions of camera [37, 31, 32].
In this paper, we will simultaneously solve i) outlier rejection, ii) true model reasoning and iii) parameter estimation in a unified optimization modeling. To this end, we start with introducing a common paradigm for exact model fitting, then derive our sparse subspace learning theory, which makes it possible to estimate the true model without knowing the model type, and robust to outliers. On this basis, we convert the objective into a continuous optimization paradigm that estimates sparse solutions for both bases and outlier entries. Wherein a fast and robust solution is proposed, that is based on superiorities of projected sub-gradient method (PSGM) and alternating optimization strategy. Finally, the true model and the inliers are directly extracted from our solutions. Contributions: i) We are the first to propose a continuous optimization modeling for geometric model fitting with unknown model type and dominant outliers. ii) We propose sparse subspace recovery theory, which is a novel formulation for model reasoning and fitting. iii) We integrate the proximal approximation strategy and sub-gradient descent method into the alternating optimization paradigm, which solves our dual sparsity problem with a convergence rate of  $\mathcal{O}(1 / k^2)$ . iv) Extensive experiments on known/unknown model fitting and two visual applications are designed to validate the superiority of our method.
This paper aims to explore a valid solution for geometric model reasoning and robust fitting. Before this, we first give a brief review of the widely-used solution for exact model estimation, which helps to derive our concerned unknown model fitting problem.
2.1 Geometry Preliminaries and New Insights
In the community of model fitting, the objective with geometric error is extremely hard to optimize due to the highly non-linear nature. In contrast, if the data are properly normalized, using linearized error (i.e., algebraic error) to construct objective would show great efficiency to find the optimal solution [18]. For algebraic objective, DLT is known as an efficient method. To be specific, suppose we are given a set of feature correspondences  $S = \{\mathbf{s}_i = (\mathbf{p}_i,\mathbf{p}_i^{\prime})\}_{i = 1}^{N}$ , where  $\mathbf{p}_i = (u_i,v_i,1)^\top$  and  $\mathbf{p}_i^{\prime} = (u_i^{\prime},v_i^{\prime},1)^{\top}$  are column vectors denoting the homogeneous coordinates of feature points extracted from two-view images. Our goal is to recover the underlying geometric structure including Fundamental matrix, Homography and Affine. This two-view problem is essential in 3D vision applications, and is the main focus of this paper.
Fundamental Matrix  $\mathbf{F} \in \mathbb{R}^{3 \times 3}$  describes the entire epipolar geometry  $\mathbf{p}_i^{\prime \top} \mathbf{F} \mathbf{p}_i = 0$ . It is suggested to be represented by single equation living in the polynomial space
$$
\Phi_ {\mathbf {F}} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} v e c (\mathbf {F}) = 0, \tag {1}
$$
$$
\Phi_ {\mathbf {F}} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} = \left(u _ {i} ^ {\prime} u _ {i}, u _ {i} ^ {\prime} v _ {i}, u _ {i} ^ {\prime}, v _ {i} ^ {\prime} u _ {i}, v _ {i} ^ {\prime} v _ {i}, v _ {i} ^ {\prime}, u _ {i}, v _ {i}, 1\right), \tag {2}
$$
is the embedding of correspondence  $(\mathbf{p}_i, \mathbf{p}_i')$  under epipolar constraint, and  $vec(\mathbf{F}) \in \mathbb{R}^9$  is the vector form of matrix  $\mathbf{F}$  in row-major order.  $N \geq 8$  correspondences can uniquely determine  $\mathbf{F}$  up to scale.
Homography  $\mathbf{H} \in \mathbb{R}^{3 \times 3}$  describes the pure rotation or plane projection, which claims that  $\mathbf{p}_i'$  and  $\mathbf{H}\mathbf{p}_i$  are co-linear, implying  $[\mathbf{p}_i']_{\times} \mathbf{H}\mathbf{p}_i = \mathbf{0}$ , where  $[\mathbf{p}_i']_{\times}$  is a skew-symmetric matrix. In DLT solution, it usually converts to the following constraint:
$$
\Phi_ {\mathbf {H}} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} v e c (\mathbf {H}) = \mathbf {0}, \tag {3}
$$
where  $vec(\mathbf{H}) \in \mathbb{R}^9$ , similarly. And  $\Phi_{\mathbf{H}}(\mathbf{p}_i, \mathbf{p}_i')$  denotes the homography embedding:
$$
\Phi_ {\mathbf {H}} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} = \left[ \begin{array}{c c c c c c c c} u _ {i} & v _ {i} & 1 & 0 & 0 & 0 & - u _ {i} ^ {\prime} u _ {i} & - u _ {i} ^ {\prime} v _ {i} & - u _ {i} ^ {\prime} \\ 0 & 0 & 0 & u _ {i} & v _ {i} & 1 & - v _ {i} ^ {\prime} u _ {i} & - v _ {i} ^ {\prime} v _ {i} & - v _ {i} ^ {\prime} \end{array} \right]. \tag {4}
$$
This constraint suggests that,  $\mathbf{H}$  can be estimated from given at least 4 correspondences.
Affine Matrix  $\mathbf{A} \in \mathbb{R}^{3 \times 3}$  is the degraded case of  $\mathbf{H}$ , with the last row of  $\mathbf{A}$  being  $[0,0,1]$ . This model implies a linear transformation  $\mathbf{p}_i' = \mathbf{A}\mathbf{p}_i$ . The affine constraint is also represented as:
$$
\Phi_ {\mathbf {A}} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} v e c (\mathbf {A}) = \mathbf {0}, \tag {5}
$$
where  $\text{vec}(\mathbf{A}) = [a_{11}, a_{12}, a_{13}, a_{21}, a_{22}, a_{23}, 1]^\top$ , and
$$
\Phi_ {\mathbf {A}} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} = \left[ \begin{array}{c c c c c c c} u _ {i} & v _ {i} & 1 & 0 & 0 & 0 & - u _ {i} ^ {\prime} \\ 0 & 0 & 0 & u _ {i} & v _ {i} & 1 & - v _ {i} ^ {\prime} \end{array} \right] \tag {6}
$$
is the affine embedding. Using Eq. (5), the solution can be given by at least 3 correspondences.
Given sufficient inliers, DLT method provides an efficient solution for geometric model fitting, which converts to solving the following minimal least-square problem:
$$
\min  _ {\theta} \| \mathbf {M} ^ {\top} \theta \| _ {2} ^ {2}, \quad s. t. \quad \| \theta \| _ {2} = 1, \tag {7}
$$
where  $\mathbf{M}$  is the data embedding matrix for specific model  $\mathcal{M}$ , i.e.,  $\mathbf{m}_i = \Phi_{\mathcal{M}}(\mathbf{p}_i, \mathbf{p}_i')$  with  $\mathcal{M} \in \{\mathbf{F}, \mathbf{H}, \mathbf{A}\}$ , as defined in Eqs. (2), (4) and (6).  $\theta = \text{vec}(\mathcal{M}) \in \mathbb{R}^D$  indicates the parameter vector of  $\mathcal{M}$ , and  $\|\theta\|_2 = 1$  restricts it in a sphere thus avoiding the trivial solution. The optima solution  $\theta^*$  is exactly the right singular vector corresponding to the smallest singular value of  $\mathbf{M}$ . However, DLT only works on the outlier-free case, due to the use of  $\ell_2$  norm. Thus, it is usually applied in SAC paradigm by sampling a clean subset to fit a given model. Another practical formulation is using latent variable inside (7) to indicate the inlier, or using truncated or  $\ell_1$  loss [18]. In contrast, the  $\ell_1$  objective is easier to optimize due to the convex nature by solving
$$
\min  _ {\theta} \| \mathbf {M} ^ {\top} \theta \| _ {1}, \quad s. t. \quad \| \theta \| _ {2} = 1. \tag {8}
$$
This is actually a subspace learning problem as introduced in DPCP series [45, 52, 13], which can be efficiently solved by a projected sub-gradient method (PSGM) [13]. Besides, [18] gives comprehensive comparisons of above mentioned losses under different model cases. These works also demonstrate the robust property of (8), which roughly states that, the estimation task can even tolerate  $O(m^2)$  outliers ( $m$  is the inlier number).
New Insights. Above formulations almost work on knowing the exact information of model type, such that the embedding matrix  $\mathbf{M}$  is correct, i.e., choosing  $\mathcal{M}$  from  $\{\mathbf{F},\mathbf{H},\mathbf{A}\}$  to generate  $\Phi_{\mathcal{M}}$ . Practically, predefining a correct model is much difficult. First, the camera type and motion, or the scene in shooting is not always known. Besides, we cannot manually define the correct model for each image pair in large-scale vision tasks. In this regard, this paper tends to simultaneously seek the true model and robustly estimate the model parameters, by proposing a general continuous optimization framework with dual sparsity constraints.
2.2 Dual Sparsity Formulation
In this paper, we aim to estimate the model parameters without knowing the model type. That means we should find a common data embedding  $\widetilde{\Phi}(\cdot)$  thus avoiding exact  $\Phi_{\mathcal{M}}(\cdot)$  for each model. Instead it converts to constructing model embedding  $\Psi(\mathcal{M})$ , to generate a common constraint as:
$$
\widetilde {\Phi} \left(\mathbf {p} _ {i}, \mathbf {p} _ {i} ^ {\prime}\right) ^ {\top} \Psi (\mathcal {M}) = \mathbf {0}, \quad \mathcal {M} \in \{\mathbf {F}, \mathbf {H}, \mathbf {A} \}. \tag {9}
$$
Eq. (9) can provide great advantages for our model reasoning and fitting task. First, it avoids the requirement for exact model type to construct  $\Phi_{\mathcal{M}}(\cdot)$  before estimation. Moreover, the solution would have the form of  $\Psi (\mathcal{M})$ , which directly guides the identification of different models. Eq. (9) essentially relies on the following Sparse Subspace Recovery (SSR) theory.
(a) Point model:  $x - t = 0$
(b) 2D embedd. & solution of (a)
(c) 3D embedd. & sparse solution of (a)
Figure 1: Illustration of fitting a point model  $(x = t)$ . (a) original points, (b) embedding under DLT solution, (c) embedding under our SSR theory.  $\mathbf{x}^*$  or  $-\mathbf{x}^*$  is the optimal solution.
Theorem 1 (SSR) Geometric model fitting can be seen as a subspace recovery problem represented by the intersection of multiple sparse hyperplanes under an over embedded data space.
To better interpret the proposed Theorem 1, we first give an example on 2D point set fitting. Given a set of clean 2D points  $S = \{\mathbf{s}_i = (x_i,y_i)\}_{i = 1}^N$  that are sampled from a structure of line  $(L)$ , parabola  $(P)$  or ellipse  $(E)$ , i.e.,  $\mathcal{M}\in \{L,P,E\}$ . The model parameters can be estimated by exact data embedding  $\Phi_{\mathcal{M}}(x_i,y_i)$  with DLT method (7). For ellipse estimation,  $\Phi_E(x_i,y_i) = [1,x_i,y_i,x_i^2,y_i^2 ]^\top$ , while for a line,  $\Phi_L(x_i,y_i) = [1,x_i,y_i]^\top$ . But in our sparse theory, with a higher-order embedding, such as fixing  $\widetilde{\Phi} (x_i,y_i) = \Phi_E(x_i,y_i)$ , we can obtain a sparse solution  $\theta = [a,b,c,0,0]^\top$  for a line model, and  $\theta = [a,b,c,d,0]^\top$  or  $[a,b,c,0,e]^\top$  for a parabola model. See Fig. 2, or refer to Append.D and Fig. 6 for details. Thus, the SSR task can be formulated as:
$$
\min  _ {\mathbf {x}} \| \mathbf {x} \| _ {0}, \quad \text {s . t .} \quad \mathbf {M} ^ {\top} \mathbf {x} = \mathbf {0}, \quad \| \mathbf {x} \| _ {2} = 1, \tag {10}
$$
where  $\mathbf{M}$  is the over embedding for input data with  $\mathbf{m}_i = \widetilde{\Phi}(x_i, y_i) = \Phi_E(x_i, y_i)$ , and we denote  $\mathbf{x}$  as a sparse basis that indicates a hyperplane under SSR theory. The  $\ell_0$  norm in the objective suggests to use less parameters to fit the given data. Note that,  $\mathbf{m}_i$  can be composed by more polynomials such as  $x_i y_i$ ,  $x_i^3$ ,  $y_i^3$  or higher-order forms, but it is not necessary for the model pool  $\{L, P, E\}$ . Problem (10) can be illustrated by a point model fitting  $x = t$  as in Fig. 1. The middle plot shows that, the solution  $\mathbf{x}^*$  or  $-\mathbf{x}^*$  is ideally vertical to the embedding vector  $\tilde{\mathbf{m}}_i$  of inliers. With higher-order embedding  $[x_i, 1, x_i^2]^\top$ , the solution would be any vector located in the plane  $\mathcal{P}$  that is vertical to  $\tilde{\mathbf{m}}_i$  and through the origin. However, with sparsity constraint, the solution reduces to be the intersection of plane  $\mathcal{P}$  and one of coordinate planes  $\{\pi_{xoy}, \pi_{xoz}, \pi_{yoz}\}$ . Since  $x_i^2$  causes higher complexity, the final solution is exactly  $\mathcal{P} \cap \pi_{xoy}$ , i.e.,  $\mathbf{x}^* = \frac{\pm 1}{\sqrt{1 + t^2}} [1, -t, 0]^\top$ .
Denoting  $G_{\mathcal{M}} = (D,d,r,s)$  as the geometry relationship for a model  $\mathcal{M}$ , where  $D$  is the ambient dimension of embedded data,  $d$  is the subspace dimension,  $r$  denotes the number of basis to be estimated, while  $s$  means the minimum sparsity in each basis (i.e., the least number of zero parameters), the relations for 2D models are  $G_{L} = (5,4,1,2)$ ,  $G_{P} = (5,4,1,1)$ , and  $G_{E} = (5,4,1,0)$ . Next, we emphatically introduce the new formulation for two-view models, which is outlined in Appendix D.
Extending to two-view geometry, our SSR task can directly benefit from (8), which is a special case in subspace recovery [46, 52, 14], with  $d = D - 1$ . In subspace learning theory, the learning problem is significantly easier when the relative dimension  $d / D$  is small [17, 18]. For such purpose, and together with our sparse theory, we can obtain the following new formulations. First, for  $\mathbf{F}$  estimation, since its geometric constraint derives only one single hyperplane living in a 9-dimensional space, thus Eq. (9) has the same form with Eq. (1). And the geometric relation is clearly  $G_{\mathbf{F}} = (9,8,1,0)$ .
Proposition 1 Given  $n \geq 4$  clean correspondences conforming to a homography transformation  $\mathbf{H}$ , the model estimation can be converted to recovering a subspace with dimension no more than 6, which consists of 3 sparse hyperplanes with sparsity  $s \geq 3$  under the 9-D embedding  $\widetilde{\Phi} = \Phi_F(\mathbf{p}_i, \mathbf{p}_i')$ .
Proof. Homography model actually derives  $r = 3$  equations, and each of them indicates a sparse hyperplane with  $\widetilde{\Phi} (\mathbf{p}_i,\mathbf{p}_i')^\top \Psi (\mathbf{H}) = \mathbf{0}$ , where  $\Psi (\mathbf{H})\in \mathbb{R}^{9\times 3}$  is the embedding of model  $\mathbf{H}$ , which consists of three sparse intersected hyperplanes (i.e., orthometric bases) as:
$$
\Psi (\mathbf {H}) = \left[ \begin{array}{l} \mathbf {0} _ {1 \times 3}, \mathbf {h} _ {3} ^ {\top}, - \mathbf {h} _ {2} ^ {\top} \\ - \mathbf {h} _ {3} ^ {\top}, \mathbf {0} _ {1 \times 3}, \mathbf {h} _ {1} ^ {\top} \\ \mathbf {h} _ {2} ^ {\top}, - \mathbf {h} _ {1} ^ {\top}, \mathbf {0} _ {1 \times 3} \end{array} \right] ^ {\top}, \tag {11}
$$
with  $\mathbf{H} = [\mathbf{h}_1^\top ;\mathbf{h}_2^\top ;\mathbf{h}_3^\top ]$ . Since  $\Psi (\mathbf{H})$  is of rank 3, the dimension of subspace  $d\leq 6$ . And the solution for each basis would have at least  $s = 3$  zeros, thus the geometric relation is  $G_{\mathbf{H}} = (9,6,3,3)$ .
Proposition 2 Given  $n \geq 3$  clean correspondences conforming to an affine transformation  $\mathbf{A}$ , the model estimation can be converted to recovering a subspace with dimension no more than 7, which consists of 2 sparse hyperplanes with sparsity  $s \geq 5$  under the 9-D embedding  $\widetilde{\Phi} = \Phi_F(\mathbf{p}_i, \mathbf{p}_i')$ .
Proof. Eq. (5) derives  $r = 2$  linear equations, and each of them indicates a sparse hyperplane with  $\widetilde{\Phi} (\mathbf{p}_i,\mathbf{p}_i')^\top \Psi (\mathbf{A}) = \mathbf{0}$ , where  $\Psi (\mathbf{A})\in \mathbb{R}^{9\times 2}$  is the embedding of model  $\mathbf{A}$ , which consists of two sparse intersected hyperplanes as:
$$
\Psi (\mathbf {A}) = \mathbf {A} _ {9 \times 2} ^ {\prime} = \left[ \begin{array}{l} 0, 0, a _ {1 1}, 0, 0, a _ {1 2}, - 1, 0, a _ {1 3} \\ 0, 0, a _ {2 1}, 0, 0, a _ {2 2}, 0, - 1, a _ {2 3} \end{array} \right] ^ {\top}. \tag {12}
$$
Because  $\Psi (\mathbf{A})$  is of rank 2, the dimension of subspace is no more than 7. And the solution for each basis would have at least  $s = 5$  zeros, thus the geometric relation is  $G_{\mathbf{A}} = (9,7,2,5)$ .
The above two propositions have well interpreted Theorem 1. To be specific,  $\mathbf{H}$  and  $\mathbf{A}$  estimation can be performed as a hyperplane fitting problem with Eq. (3) and Eq. (5). Alternatively, they can be respectively solved as recovering a 6-dimensional subspace and a 7-dimensional subspace in a common 9-dimensional data embedding space. For the purpose of our unknown model fitting, the advantages of this conversion have been explained in Eq. (9). Besides, the low-relative dimension  $d / D$  significantly makes the subspace learning task easier [17, 18]. In the following, we will give a general formulation for unknown model fitting, then explore its efficient solution.
Formulation for Unknown Model Fitting. With no outliers and noise, unknown model fitting can convert to finding all  $r$  independent sparse bases  $\mathbf{X} = [\mathbf{x}_1,\mathbf{x}_2,\dots ,\mathbf{x}_r]\in \mathbb{R}^{D\times r}$  by solving
$$
\min  _ {\mathbf {X} \in \mathbb {R} ^ {D \times r}} \| \mathbf {X} \| _ {0}, \quad s. t. \quad \mathbf {M} ^ {\top} \mathbf {X} = \mathbf {0}, \quad \operatorname {r a n k} (\mathbf {X}) = r, \tag {13}
$$
where  $\mathbf{M}$  is the over embedding matrix of data with  $\mathbf{m}_i = \tilde{\Phi}(\mathbf{s}_i)$ . Rank  $r$  constraint of  $\mathbf{X}$  asks for  $r$  independent bases. But  $r$  is generally unknown in advance, thus its estimation is also necessary.
Proposition 3 Given sufficient inliers with no noise, if the conformed model derives  $r$  independent bases, then the solution of SSR satisfies  $\max(\mathrm{rank}(\mathbf{X})) = r$ .
Proof. Denoting  $\mathbf{X}^{*} = [\mathbf{x}_{1},\mathbf{x}_{2},\dots ,\mathbf{x}_{r}]\in \mathbb{R}^{D\times r}$  the optimal solution, any subset of  $\mathbf{X}^{*}$  is also a feasible solution with rank no more than  $r$ . For  $\forall \mathbf{x}_{r + 1}$  with  $\| \mathbf{x}_{r + 1}\| _2 = 1$ ,  $(\mathbf{X}^{*})^{\top}\mathbf{x}_{r + 1} = \mathbf{0}$ , we have  $\mathbf{M}^{\top}\mathbf{x}_{r + 1}\neq \mathbf{0}$ . Hence,  $\mathbf{x}_{r + 1}$  is not a valid basis, and we can conclude  $\max (\mathrm{rank}(\mathbf{X})) = r$ .
Proposition 3 suggests that the target of unknown model fitting can be converted to finding a maximum number of orthometric bases with sparsity constraint. Considering the outliers and noise, the ideal constraints can be written as  $\mathbf{M}^{\top}\mathbf{X} - \mathbf{G} - \mathbf{E} = \mathbf{0}$ , where  $\mathbf{G}$  and  $\mathbf{E}$  are noise and outlier entries, respectively. Following a common perception,  $\mathbf{G}$  is assumed to be Gaussian, which suggests to be constricted by Frobenius norm  $\| \mathbf{G}\| _F$ . While outlier matrix  $\mathbf{E}$  is sparse in row, and any  $\| \mathbf{E}_{i,:}\| _2\neq 0$  indicates that the  $i$ -th sample can be considered as an outlier, thus we use  $\ell_{2,0}$  to constrain this property. Then our problem is transformed into a multi-objective optimization task, including minimizing the fitting error, the parameter number, the outlier number, and maximizing the basis number. Hence, the general formulation for robust model reasoning and fitting can be written as a unified form:
$$
\min  _ {\mathbf {X}, \mathbf {E}, r} \frac {1}{2} \| \mathbf {G} \| _ {F} ^ {2} + \lambda \| \mathbf {X} \| _ {0} + \gamma \| \mathbf {E} \| _ {2, 0} - \tau \operatorname {r a n k} (\mathbf {X}), \tag {14}
$$
$$
s. t. \mathbf {M} ^ {\top} \mathbf {X} - \mathbf {G} - \mathbf {E} = \mathbf {0}, \quad \mathbf {X} ^ {\top} \mathbf {X} = \mathbf {I} _ {r \times r},
$$