# Least Squares Problems with Absolute Quadratic Constraints

Authors: R. Schöne; T. Hanning
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

This paper analyzes linear least squares problems with absolute quadratic constraints. We develop a generalized theory following Bookstein's conic-fitting and Fitzgibbon's direct ellipse-specific fitting. Under simple preconditions, it can be shown that a minimum always exists and can be determined by a generalized eigenvalue problem. This problem is numerically reduced to an eigenvalue problem by multiplications of Givens' rotations. Finally, four applications of this approach are presented.


---

## Body

## 1. Introduction

The least squares methods cover a wide range of applications in signal processing and system identification [1–5]. Many technical applications need robust and fast algorithms for fitting ellipses to given points in the plane. In the past, effective methods were Bookstein's conic-fitting or Fitzgibbon's direct ellipse-specific fitting, where an algebaic distance with a quadratic constraint is minimized [6, 7]. In this paper, we develop an extended theory of minimization of least squares with a quadratic constraint based on the ideas of Bookstein and Fitzgibbon. Thereby, we show the existence of a minimal solution and present the uniqueness regarding to the smallest positive generalized eigenvalue. So, arbitrary conic fitting problems with quadratic constraints are possible.LetA∈ℝn×m be matrix with n≥m≥2, C∈ℝm×m be symmetric matrix, and d∈ℝ a real value. We consider the problem of finding a vector x∈ℝm which minimizes the function F:ℝm→ℝ defined by(1.1)F(x):=‖Ax‖22subjecttoxtCx=d.
The side condition xtCx=d introduces an absolute quadratic constraint. The problem (1.1) is not a special case of Gander's optimization as presented in [8], because in our case C is a real symmetric matrix in contrast to the approach of Gander, where the side condition considers real symmetric matrices CtC which are positive definite. For our considerations, we require the following three assumptionsAssumption 1.1.
By replacingC by (-C), we consider d≥0. For d=0, the trivial solution x=0∈ℝm fulfills (1.1). Therefore, we demand d>0.Assumption 1.2.
The setN:={x∈ℝm,xtCx=d} is not empty, that is, the matrix C has not been less than one positive eigenvalue. If we assume that C has only nonpositive eigenvalues, it would be negative semidefinite and
(1.2)xtCx≤0
holds. With d>0 it follows that the set N would be empty in this case.Assumption 1.3.
In the following, we setS=AtA and assume that S is regular. S is sometimes called scatter matrix.In the following two sections, we introduce the theoretical basics of this optimization. The main result is the solution of a generalized eigenvalue problem. Afterwards, we reduce this system numerically to an eigenvalue problem. In Section5, we present four typical applications for conic fitting problems with quadratic constraints. These approximations contain the ellipse fitting of Fitzgibbon, the hyperbola fitting of O'Leary, the conic fitting of Bookstein, and an optical application of shrinked aspheres [6, 7, 9, 10].


## 2. Existence of a Minimal Solution

Theorem 2.1.
IfS∈ℝm×m is regular, then there exists a global minimum to the problem (1.1).Proof.
The real regular matrixS=AtA is symmetric and positive definite. Therefore, a Cholesky decomposition S=RtR exists with a regular upper triangular matrix R∈ℝm×m. In (1.1), we are looking for a solution x∈ℝm minimizing
(2.1)F(x)=‖Ax‖2=xtSx=xtRtRxsubjecttoxtCx=d.
With R regular we substitute x by R-1y for y∈ℝm. Thus, we obtain an equivalent problem to (2.1), where we want to find a vector y∈ℝm, minimizing
(2.2)F(y)=yty=‖y‖2subjecttoyt(R-1)tCR-1y=d.
Now, we define G:ℝm→ℝ with G(y)=yt(R-1)tCR-1y-d and look for a solution y on the zero-set NG of G with minimal distance to the point of origin. Let y0∈NG≠∅ and Kr0m¯(0) be the closed sphere of ℝm in 0 with radius r0=∥y0∥. Because of y0∈NG∩Kr0m¯(0) and G being continuous, the set
(2.3)NG⋂Kr0m¯(0)={y∈Kr0m¯(0),G(y)=0}
is nonempty, closed and bounded. Therefore, for the continuous function F(y)=∥y∥2 exists a minimal value yM on NG∩Kr0m¯(0) with
(2.4)min⁡y∈NG∩Kr0m¯(0)F(y)≤r02.
For all y from NG∖Kr0m¯(0), it is F(y)>r02. So, yM is a minimal value of F in NG. By the equivalence of (2.1), and (2.2) the assumption follows.


## 3. Generalized Eigenvalue Problem

The minimization problem in (1.1) induces a generalized eigenvalue problem. The following theorem is already proven by Bookstein and Fitzgibbon for the special case of ellipse-fitting [6, 7].Theorem 3.1.
Ifxs is an extremum of F(x) subject to xtCx=d, then a positive λ0∈ℝ exists with
(3.1)(S-λ0C)xs=0,
that is,  xs is an eigenvector to the generalized eigenvalueλ0 and
(3.2)F(xs)=λ0⋅d
holds.Proof.
LetG:ℝm→ℝ be a defined as G(x):=d-xtCx. For G(x)=0 and d>0 follows x≠0. Further, G is continuously differentiable with dG/dx=-2Cx≠0 for all x of the zero-set of G. So, if xs is a local extremum of F(x) subject to G(x)=0, then it is rank(dG/dx)(xs)=1. Since F is also a continuously differentiable function in ℝm with m>1, it follows by using a Lagrange multiplier [11]: if xs is a local extremum of F(x) subject to G(x)=0, then a λ0∈ℝ exists, such that the Lagrange function ϕ:ℝm+1→ℝ given as
(3.3)ϕ(x,λ)=F(x)+λG(x)=xtAtAx+λ(d-xtCx)
has a critical point in (xs,λ0). Therefore, xs fulfills necessarily the equations:
(3.4)gradxϕ(x,λ)=2Sx-2λCx=0,(3.5)dϕ(x,λ)dλ=xtCx-d=0.
The first equation describes a generalized eigenvalue problem with
(3.6)(S-λ⋅C)x=0.
With d>0, xs≠0 and xs fulfills (3.6), λ must be a generalized eigenvalue, and xs is a corresponding eigenvector to λ of (3.6), so that (S-λC) is a singular matrix.  If λ0 is an eigenvalue and x0≠0 a corresponding eigenvector to λ0 of (3.6), then every vector αx0 is also a solution of (3.6) for λ0. Now, we are looking for α, such that xs=αx0 satisfies (3.5). For λ0≠0 and (3.4), it follows
(3.7)d=α2x0tCx0=α2x0tSx0λ0.
Because the left side and the numerator are positive, the denominator must also be chosen positive, that is, only positive eigenvalues solve (3.4) and (3.5). By the multiplication with λ0,
(3.8)d⋅λ0=α2x0tSx0=xstSxs=F(xs)
follows and xs=α·x0 fulfills the constraint G(xs)=0.Remark 3.2.
Letx0 be a generalized eigenvector to a positive eigenvalue λ0 of problem (3.6). Then
(3.9)x1,2=±dλ0x0tSx0x0
are solutions of (3.8).Lemma 3.3.
IfS is regular and C is symmetric, then all eigenvalues of (3.1) are real-valued and different to zero.Proof.
Withdet(S)≠0,λ0≠0 in (3.1). The Cholesky decomposition S=RtR with a regular upper triangular matrix R yields (3.1)
(3.10)(RtR-λ0C)xs=Rt(I-λ0(Rt)-1CR-1)Rxs=0.
With R invertible and the substitution μ0=λ0-1,ys=Rxs, we obtain an eigenvalue problem to the matrix (Rt)-1CR-1:
(3.11)(μ0I-(Rt)-1CR-1)ys=0.
Furthermore, we have
(3.12)((Rt)-1CR-1)t=(R-1)tCt((Rt)-1)t=(Rt)-1CtR-1=(Rt)-1CR-1.
Therefore, the matrix (Rt)-1CR-1 is symmetric and all eigenvalues μ0 are real. With λ0=μ0-1 for μ0≠0 follows the proposition.Remark 3.4.
Because ofS regular and λ≠0 in
(3.13)(S-λ0C)xs=0,
we can consider the equivalent problem with μ0=λ0-1 instead of (3.11):
(3.14)(μ0I-S-1C)xs=0.
This system is called inverse eigenvalue problem to (3.1). Here, the eigenspaces to the generalized eigenvalue λ0 in (3.1) and to the eigenvalue 1/λ0 in (3.14) are identical. Therefore, the generalized eigenvectors in (3.1) are perpendicular.Definition 3.5.
The set of all eigenvalues of a matrixC is called spectrum σ(C). We call the set of all eigenvalues to the generalized eigenvalue problem in (3.1) also spectrum and denote σ(S,C). σ+(S,C) is defined as the set of all positive values in σ(S,C).Remark 3.6.
In case ofrg(C)<m=rg(S), the inverse problem in (3.14) has eigenvalue 0 with multiplicity rg(S)-rg(C). Otherwise, for μ≠0 and μ∈σ(S-1C) follows 1/μ∈σ(S,C). Analogously, for μ∈σ((Rt)-1CR-1) with μ≠0,1/μ∈σ(S,C).The following lemma is a modified result of Fitzgibbon [7].Lemma 3.7.
The signs of the generalized eigenvalues of (3.1) are the same as those of C.Proof.
WithS being nonsingular, every generalized eigenvalue λ0 of (3.1) is not zero. Therefore, it follows for the equivalent problem (3.11) that μ0=λ0-1 is also a positive eigenvalue to (R-1)tCR-1, where R is an upper triangular matrix to the Cholesky decomposition of S. With Sylvester's Inertia Law, we know that the signs of eigenvalues of the matrices (R-1)tCR-1 are the same as those of C.For the following proofs, we need the lemma of Lagrange (see, e.g., [12]).Lemma 3.8 (Lemma of Lagrange).
ForM⊂ℝn, f:M→ℝ, g=(g1,…,gk):M→ℝk, and Ng={x∈M,g(x)=0∈ℝk}, let λ∈ℝk, so that xs∈Ng is a minimal value of the function Φλ:M→ℝ with
(3.15)Φλ(x)=f(x)+∑i=1kλigi(x).
Then xs is a minimal solution of f in Ng.Definition 3.9.
Letλ* be the smallest positive value of σ+(S,C) and xs* a corresponding generalized eigenvector to λ* to the constraint xs*tCxs*=d.Lemma 3.10.
LetS-λC be a positive semidefinite matrix for λ∈σ+(S,C). Then a generalized eigenvector xs corresponding to λ is a local minimum of (1.1).Proof.
We considerΦ:ℝm→ℝ with
(3.16)Φ(x)=xtSx+λ(d-xtCx).
With gradxΦ(x)=2(Sx-λCx) holds gradxΦ(xs)=0 and since the Hessian matrix (S-λC) of Φ is positive semidefinite, the vector xs is a minimal solution of Φ. Then, xs minimizes also F(x) subject to xtCx=d by the Lemma 3.8.Remark 3.11.
In fact, in Lemma3.10, we require only a semidefinite matrix, because xs≠0 fulfills xst(S-λC)xs=0 in (3.1).Lemma 3.12.
The matrix(S-λ*C) is positive semidefinite.Proof.
Letμ be an arbitrary eigenvalue of ((λ*)-1I-(Rt)-1CR-1), where R is the upper triangular matrix of the Cholesky decomposition from S. With
(3.17)det⁡((λ*)-1I-(Rt)-1CR-1-μI)=det⁡(((λ*)-1-μ)I-(Rt)-1CR-1)=0,
it follows that ((λ*)-1-μ) is an eigenvalue of (Rt)-1CR-1. This value is corresponding in (3.11) with the inverse eigenvalue of problem (3.1). Furthermore, it yields
(3.18)1λ*-μ≤max⁡{1λ,λ∈σ+(S,C)}=1min⁡{λ,λ∈σ+(S,C)}=1λ*.
So μ≥0 follows, that is, ((λ*)-1I-(Rt)-1CR-1) is positive semidefinite, and for y∈ℝm we obtain
(3.19)yt((λ*)-1I-(Rt)-1CR-1)y≥0.
By setting y=Rx and with regular R, we get
(3.20)0≤xt((λ*)-1RtR-C)x=xt((λ*)-1S-C)x.
With λ*>0, xt(S-λ*C)x≥0 for x∈ℝm, that is,  S-λ*C is positive semidefinite.Theorem 3.13.
For the smallest valueλ* in σ+(S,C) exists a corresponding generalized eigenvector xs*, which minimizes F(x) subject to xtCx=d.Proof.
The matrix(S-λ*C) is positive semidefinite by Lemma 3.12, and with Lemma 3.10 it follows that xs* is a local minimum of problem (1.1). Furthermore, we know by Theorem 3.1 that if xs is a local extremum of F(x) subject to xtCx=d, then a positive value λs exists with
(3.21)(S-λsC)xs=0
and it is F(xs)=λsd. Because of the existence of a minimum xE in Theorem 2.1 a value λE∈σ+(S,C) exists in problem (3.21) regarding to xE. Otherwise, for an arbitrary local minimum xs,
(3.22)F(xs*)=λ*d=min⁡{λd,λ∈σ+(S,C)}≤λsd=F(xs).
So, λ*=λE follows and xs* is a minimum of F(x) subject to xtCx=d.Example 3.14.
We minimizeF:ℝ2→ℝ with F(ξ1,ξ2)=ξ12+ξ22 subject to ξ12-ξ22=1. So, we have d=1, S the identity matrix I2∈ℝ2×2, and C∈ℝ2×2 a diagonal matrix with values −1 and 1. Then, we get the following generalized eigenvalue problem:
(3.23)det⁡(I2-λC)=(1-λ)(-1-λ)=0
with eigenvalues 1 and −1. Because of Theorem 3.1, we consider a generalized eigenvector (α,0)t with α∈ℝ∖{0} only for λ=1. Then, (1,0)t and (-1,0)t are solutions subject to ξ12-ξ22=1. This result is conform to the geometric interpretation, since we are looking for x=(ξ1,ξ2)t on the hyperbola ξ12-ξ22=1 with minimal distance to the origin.


## 4. Reduction to an Eigenvalue Problem of Dimensionrg(C)

In numerical applications, a generalized eigenvalue problem is mostly reduced to an eigenvalue problem, for example, by multiplication withS-1. Thus, we obtain the inverse problem (3.14) from (3.1) (see, e.g., [13]). But, S may be ill-conditioned, so that a solution of (3.14) may be numerical instable. Therefore, we present another reduction of (3.1).Many times,C is a sparse matrix with r:=rank(C)≤rank(S). This symmetric matrix C is diagonalizable in C=PtDP with P orthogonal and D diagonal. Further, we assume that the first r diagonal entrees in D are different to 0. For the characteristic polynomial in (3.1), it follows(4.1)p(λ)=det⁡(S-λC)=det⁡(PSPt-λD)=0.
The order of p is r. We decompose these matrices in (4.2)PSPt=(S1S2S2tS3),D=(D1000)
with S1,D1∈ℝr×r, S2∈ℝr×(m-r), and S3∈ℝ(m-r)×(m-r). Now, we eliminate S2 in PSPt by multiplications with Givens rotations Gk∈ℝm×m,k=1,…,l, so that it follows: (4.3)Gl⋯G2G1PSPt=(Σ10Σ2Σ3),Gl⋯G2G1D=(Δ10Δ20)
with Σ1,Δ1∈ℝr×r, Σ2,Δ2∈ℝ(m-r)×r, and Σ3∈ℝ(m-r)×(m-r). In (4.1), we achieve with orthogonal Gk,k=1,…,l(4.4)p(λ)=det⁡(PSPt-λD)=det⁡(Gl⋯G2G1(PSPt-λD))=det⁡(Σ1-λΔ10Σ2-λΔ2Σ3)=det⁡(Σ3)det⁡(Σ1-λΔ1)=0.
Because of p(0)=det(PSPt)=det(S)≠0 and p(0)=det(Σ3)det(Σ1), the submatrices Σ1,Σ3 are regular and the generalized eigenvalues of det(Σ1-λΔ1) are different to zero. So, with y∈ℝr(4.5)(Σ1-λΔ1)y=0
can be transformed to an equivalent eigenvalue problem with(4.6)(Δ1-1Σ1-λI)y=0.
This system can be solved by finding the matrix X with Δ1X=Σ1 using the Gaussian elimination and determining the eigenvalues of X computing the QR algorithm [13]. Because all steps are equivalent, we have σ(Δ1-1Σ1)=σ(S,C), that is,  the eigenvalues of (3.1) and (4.6) are the same.With Theorem3.13, we are looking for the smallest value λ*∈σ+(S,C) and a corresponding generalized eigenvector xs* to minimize the problem (3.1). So, (4.7)0=(S-λ*C)xs*=(SPt-λ*CPt)Pxs*
yields. By substitution of ys* for Pxs*, we obtain(4.8)0=Gl⋯G2G1(PSPt-λ*D)ys*=(Σ1-λ*Δ10Σ2-λ*Δ2Σ3)ys*.
We decompose ys* into the subvectors ys,r*∈ℝr and ys,m-r*∈ℝm-r with (ys*)t=(ys,r*|ys,m-r*)t. Then, ys,r* is a generalized eigenvector for λ* of the problems (4.5) and (4.6).Letys,r* be an eigenvector to the smallest positive eigenvalue λ* of (4.6). Since Σ3 is regular, it follows in (4.8) (4.9)ys,m-r*=Σ3-1(Σ2-λ*Δ2)ys,r*and a generalized eigenvectorxs* for λ* in (3.1) is given as: (4.10)xs*=Pt(ys,r*Σ3-1(Σ2-λ*Δ2)ys,r*).


## 5. Applications in Conic Fitting

### 5.1.   Fitzgibbon's Ellipse Fitting

First, we would like to find an ellipse for a given set of points inℝ2. Generally, a conic in ℝ2 is implicitly defined as the zero set of f:ℝ6×ℝ2→ℝ to a constant parameter a=(α1,…,α6)t∈ℝ6: (5.1)f(a,ξ,η)=α1ξ2+α2ξη+α3η2+α4ξ+α5η+α6.
The equation f(a,ξ,η)=0 can also be written with x=(ξ,η)t as (5.2)0=xtAx+btx+cwithA=(α1α2/2α2/2α3),b=(α4α5),c=α6.
The eigenvalues λ1,λ2 of A characterize a conic uniquely [14]. Thus, we need 4λ1λ2=4α1α3-α22>0 for ellipses in f(a,x)=0. Furthermore, every scaled vector μa with μ∈ℝ∖{0} describes the same zero-set of f. So, we can impose the constraint for ellipses with 4α1α3-α22=1. For n(n≥6) given points (ξi,ηi)t∈ℝ2, we want to find a parameter a∈ℝ6, which minimizes  F:ℝ6→ℝ with(5.3)F(a)=∑i=1nf(a,ξi,ηi)2subject to4α1α3-α22=1.
This ellipse fitting problem is established and solved by Fitzgibbon [7]. With the following matrices D∈ℝn×6, C∈ℝ6×6(5.4)D=(ξ12ξ1η1η12ξ1η11ξ22ξ2η2η22ξ1η21⋮⋮⋮⋮⋮⋮ξn2ξnηnηn2ξnηn1),C=(0020000-10000200000000000000000000000),
and F(a)=∑i=1nf(a,ξi,ηi)2=∥Da∥22, we achieve the equivalent problem:(5.5)min⁡a∈R6‖Da‖22subjecttoatCa=1.
For S=DtD, we have a special case of (1.1). Assuming S is a regular matrix and since the eigenvalues of C are −2, −1, 0, and 2, by lemma 3.3 we know that the generalized eigenvalue problem (5.6)(S-λC)a=0
has exactly one positive solution λ*∈ℝ. Because of Theorem 3.13 a corresponding generalized eigenvector a* to λ* minimizes the problem (5.5) and a* consists of the coefficients of an implicit given ellipse.A numerically stable noniterative algorithm to solve this optimization problem is presented by Halir and Flusser [15]. In comparison with Section 4, their method uses a special block decomposition of the matrices D and C.


### 5.2. Hyperbola Fitting

Instead of ellipses, O'Leary and Zsombor-Murray want to find a hyperbola to a set of scattered dataxi∈ℝ2 [9]. A hyperbola is a conic, which can uniquely be characterized by 4λ1λ2=4α1α3-α22<0 [14]. So, we consider the constraint α22-4α1α3=1 and obtain the optimization problem: (5.7)min⁡a∈R6‖Da‖22subjecttoat(-C)a=1
with D and C being chosen in 5.1. The matrix (-C) has two positive eigenvalues. In this case, a solution is given by a generalized eigenvector to the smallest value in σ+(S,-C). But O'Leary and Zsombor-Murray determine the best hyperbolic fit by evaluation of κi=α2,i2-4α1,iα3,i, where the eigenvector ai=(ai,1,…,ai,6)t is associated to a positive value of σ+(S,-C).


### 5.3. Bookstein's Conic Fitting

In Bookstein's method, the conic constraint is restricted to(5.8)2(trace2A-2det⁡A)=2(λ12+λ22)=2α12+α22+2α32=1,
where λ1,λ2 are the eigenvalues of A in f [6]. There, it is λ1,2∈[-2/2,2/2] and at least one of them different to 0. But the constraint (5.8) is not a restriction to a class of conics. Here, we determine an arbitrary conic, which minimizes(5.9)F(a)=∑i=1nf(a,ξi,ηi)2subjectto2α12+α22+2α32=1.
The resulting data matrix D∈ℝn×6 is the same as for Fitzgibbon's problem. The constraint matrix C∈ℝ6×6 has a diagonal shape with the entrees (2,1,2,0,0,0), that is, all eigenvalues of C are nonnegative. In the case of a regular matrix S, the problem (5.9) is solved for a generalized eigenvector to the smallest value in σ+(S,C).


### 5.4. Approximation of Shrinked Aspheres

After the molding process in optical applications, the shrinkage of rotation-symmetric aspheres is implicitly defined forx=(ξ,ζ)t in (5.10)α1ζ2+α2ζ+α3+α4ξ2=0subjecttoα22-4α1α3=4r2,
where r∈ℝ∖{0} and a=(a1,…,a4)t are aspheric-specific constants [10]. For i=1,…,n with n≥4, the scattered data xi=(ξi,ζi)t∈ℝ2 of a shrinked asphere are given in this approximation problem. Here, we are looking for the conic parameter a=(α1,…,α4)t for a fixed value rref, which minimizes (5.11)F(a)=∑i=1n(α1ζi2+α2ζi+α3+α4ξi2)2subjecttoα22-4α1α3=4rref2.
Analogously to Fitzgibbon, we have the matrices D∈ℝn×4 and C∈ℝ4×4 with (5.12)D=(ζ12ζ11ξ12ζ22ζ21ξ22⋮⋮⋮⋮ζn2ζn1ξn2),C=(00-200100-20000000)
and with F(a)=∥Da∥22 we get the following optimization problem:(5.13)min⁡a∈R4‖Da‖22subjecttoatCa=4rref2.
This is also an application of (1.1). The matrix C has the eigenvalues -2,0,1, and 2. So, the generalized eigenvalue problem in (3.1) with regular S=DtD∈ℝ4×4 has two positive values in σ+(S,C). With Theorem 3.13, a generalized eigenvector a*∈ℝ4 to the smaller of both values solves (5.13).The coefficientsαi in the problems (5.5) and (5.13) correspond not to the same  monomialsξkζl. Hence, we have different matrices D and C.


## 5.1.   Fitzgibbon's Ellipse Fitting

First, we would like to find an ellipse for a given set of points inℝ2. Generally, a conic in ℝ2 is implicitly defined as the zero set of f:ℝ6×ℝ2→ℝ to a constant parameter a=(α1,…,α6)t∈ℝ6: (5.1)f(a,ξ,η)=α1ξ2+α2ξη+α3η2+α4ξ+α5η+α6.
The equation f(a,ξ,η)=0 can also be written with x=(ξ,η)t as (5.2)0=xtAx+btx+cwithA=(α1α2/2α2/2α3),b=(α4α5),c=α6.
The eigenvalues λ1,λ2 of A characterize a conic uniquely [14]. Thus, we need 4λ1λ2=4α1α3-α22>0 for ellipses in f(a,x)=0. Furthermore, every scaled vector μa with μ∈ℝ∖{0} describes the same zero-set of f. So, we can impose the constraint for ellipses with 4α1α3-α22=1. For n(n≥6) given points (ξi,ηi)t∈ℝ2, we want to find a parameter a∈ℝ6, which minimizes  F:ℝ6→ℝ with(5.3)F(a)=∑i=1nf(a,ξi,ηi)2subject to4α1α3-α22=1.
This ellipse fitting problem is established and solved by Fitzgibbon [7]. With the following matrices D∈ℝn×6, C∈ℝ6×6(5.4)D=(ξ12ξ1η1η12ξ1η11ξ22ξ2η2η22ξ1η21⋮⋮⋮⋮⋮⋮ξn2ξnηnηn2ξnηn1),C=(0020000-10000200000000000000000000000),
and F(a)=∑i=1nf(a,ξi,ηi)2=∥Da∥22, we achieve the equivalent problem:(5.5)min⁡a∈R6‖Da‖22subjecttoatCa=1.
For S=DtD, we have a special case of (1.1). Assuming S is a regular matrix and since the eigenvalues of C are −2, −1, 0, and 2, by lemma 3.3 we know that the generalized eigenvalue problem (5.6)(S-λC)a=0
has exactly one positive solution λ*∈ℝ. Because of Theorem 3.13 a corresponding generalized eigenvector a* to λ* minimizes the problem (5.5) and a* consists of the coefficients of an implicit given ellipse.A numerically stable noniterative algorithm to solve this optimization problem is presented by Halir and Flusser [15]. In comparison with Section 4, their method uses a special block decomposition of the matrices D and C.


## 5.2. Hyperbola Fitting

Instead of ellipses, O'Leary and Zsombor-Murray want to find a hyperbola to a set of scattered dataxi∈ℝ2 [9]. A hyperbola is a conic, which can uniquely be characterized by 4λ1λ2=4α1α3-α22<0 [14]. So, we consider the constraint α22-4α1α3=1 and obtain the optimization problem: (5.7)min⁡a∈R6‖Da‖22subjecttoat(-C)a=1
with D and C being chosen in 5.1. The matrix (-C) has two positive eigenvalues. In this case, a solution is given by a generalized eigenvector to the smallest value in σ+(S,-C). But O'Leary and Zsombor-Murray determine the best hyperbolic fit by evaluation of κi=α2,i2-4α1,iα3,i, where the eigenvector ai=(ai,1,…,ai,6)t is associated to a positive value of σ+(S,-C).


## 5.3. Bookstein's Conic Fitting

In Bookstein's method, the conic constraint is restricted to(5.8)2(trace2A-2det⁡A)=2(λ12+λ22)=2α12+α22+2α32=1,
where λ1,λ2 are the eigenvalues of A in f [6]. There, it is λ1,2∈[-2/2,2/2] and at least one of them different to 0. But the constraint (5.8) is not a restriction to a class of conics. Here, we determine an arbitrary conic, which minimizes(5.9)F(a)=∑i=1nf(a,ξi,ηi)2subjectto2α12+α22+2α32=1.
The resulting data matrix D∈ℝn×6 is the same as for Fitzgibbon's problem. The constraint matrix C∈ℝ6×6 has a diagonal shape with the entrees (2,1,2,0,0,0), that is, all eigenvalues of C are nonnegative. In the case of a regular matrix S, the problem (5.9) is solved for a generalized eigenvector to the smallest value in σ+(S,C).


## 5.4. Approximation of Shrinked Aspheres

After the molding process in optical applications, the shrinkage of rotation-symmetric aspheres is implicitly defined forx=(ξ,ζ)t in (5.10)α1ζ2+α2ζ+α3+α4ξ2=0subjecttoα22-4α1α3=4r2,
where r∈ℝ∖{0} and a=(a1,…,a4)t are aspheric-specific constants [10]. For i=1,…,n with n≥4, the scattered data xi=(ξi,ζi)t∈ℝ2 of a shrinked asphere are given in this approximation problem. Here, we are looking for the conic parameter a=(α1,…,α4)t for a fixed value rref, which minimizes (5.11)F(a)=∑i=1n(α1ζi2+α2ζi+α3+α4ξi2)2subjecttoα22-4α1α3=4rref2.
Analogously to Fitzgibbon, we have the matrices D∈ℝn×4 and C∈ℝ4×4 with (5.12)D=(ζ12ζ11ξ12ζ22ζ21ξ22⋮⋮⋮⋮ζn2ζn1ξn2),C=(00-200100-20000000)
and with F(a)=∥Da∥22 we get the following optimization problem:(5.13)min⁡a∈R4‖Da‖22subjecttoatCa=4rref2.
This is also an application of (1.1). The matrix C has the eigenvalues -2,0,1, and 2. So, the generalized eigenvalue problem in (3.1) with regular S=DtD∈ℝ4×4 has two positive values in σ+(S,C). With Theorem 3.13, a generalized eigenvector a*∈ℝ4 to the smaller of both values solves (5.13).The coefficientsαi in the problems (5.5) and (5.13) correspond not to the same  monomialsξkζl. Hence, we have different matrices D and C.


## 6. Conclusion

In this paper, we present a minimization problem of least squares subject to absolute quadratic constraints. We develop a closed theory with the main result that a minimum is a solution of a generalized eigenvalue problem corresponding to the smallest positive eigenvalue. Further, we show a reduction to an eigensystem for numerical calculations. Finally, we study four applications about conic approximations. We analyze Fitzgibbon's method for direct ellipse-specific fitting, O'Leary's direct hyperbola approximation, Bookstein's conic fitting, and an optical application of shrinked aspheres. All these systems are attribute to the general optimization problem.



---

*Source: 312985-2011-09-15.xml*