2015 IEEE International Conference on Computer Vision

Multiple-hypothesis afﬁne region estimation with anisotropic LoG ﬁlters
Takahiro Hasegawa
Chubu University

Mitsuru Ambai, Kohta Ishikawa
Denso IT Laboratory, Inc.

tkhr@vision.cs.chubu.ac.jp{manbai, kishikawa}@d-itlab.co.jp

Gou Koutaki
Kumamoto University

Yuji Yamauchi, Takayoshi Yamashita, Hironobu Fujiyoshi
Chubu University

koutaki@cs.kumamoto-u.ac.jp

yuu@vision.chubu.ac.jp, {yamashita, hf}@cs.chubu.ac.jp

Abstract
We propose a method for estimating multiple-hypothesis
afﬁne regions from a keypoint by using an anisotropic
Laplacian-of-Gaussian (LoG) ﬁlter. Although conventional
afﬁne region detectors, such as Hessian/Harris-Afﬁne, iterate to ﬁnd an afﬁne region that ﬁts a given image patch,such iterative searching is adversely affected by an initialpoint. To avoid this problem, we allow multiple detections
from a single keypoint. We demonstrate that the responsesof all possible anisotropic LoG ﬁlters can be efﬁciently computed by factorizing them in a similar manner to spectral
SIFT. A large number of LoG ﬁlters that are densely sampled in a parameter space are reconstructed by a weighted
combination of a limited number of representative ﬁlters,
called “eigenﬁlters”, by using singular value decomposition. Also, the reconstructed ﬁlter responses of the sampledparameters can be interpolated to a continuous representation by using a series of proper functions. This results in
efﬁcient multiple extrema searching in a continuous space.
Experiments revealed that our method has higher repeatability than the conventional methods.

Figure 1. Comparison of multiple afﬁne region estimation results.
Left column: detection from binary image. Central column: keypoint localization offsets. Right column: detection from grayscale
image.

space constructed by Gaussian kernels of different sizes.
The Hessian-Laplace and Harris-Laplace keypoint detectors[14] also estimate scale using the same Gaussian scale spaceas LoG and DoG. In addition, the Speeded-Up Robust Features (SURF) [2] keypoint detector approximates secondorder derivative ﬁlters by using box ﬁlters. The drawbackof these scale-invariant methods, however, is that they lose
their invariance once anisotropic transformations occur. To
overcome this limitation, detecting keypoints that are invariant under projective transformations are needed.
Afﬁne region estimation, which detects a region invariant to projective transformations, determines afﬁne parameters (σx , σy , θ) by convolving anisotropic Gaussian or LoG
ﬁlters. The Hessian-Afﬁne and Harris-Afﬁne methods [15]estimate an afﬁne region by using the iterative convolution
processing of an anisotropic Gaussian ﬁlter. This processing often fails, however, to detect corresponding afﬁne regions between a pair of images when one of them suffersfrom image deformation or keypoint offset. Furthermore, inmany cases, image textures can give multiple afﬁne regions
for a single keypoint. Conventional afﬁne region detectors

1. Introduction
Keypoint matching is a fundamental task in applications
such as object recognition, image mosaicking, large-scale
image searching and 3D reconstruction. It involves ﬁnding the correspondence between keypoints in a set of images, each having a different viewing angle, and other imaging conditions. A common matching technique that is robust across conditions is scale-invariant keypoint detection
and matching [10, 11, 12, 14, 2]. Scale-invariant keypoint
detectors, such as Laplacian-of-Gaussian (LoG) [10, 11]
and Difference-of-Gaussian (DoG) [12], estimate scale bydetermining the parameters at which the response valueof the LoG or DoG becomes an extremum from a scale
1550-5499/15 $31.00 © 2015 IEEE
DOI 10.1109/ICCV.2015.74585

troduced by Mikolajczyk et al. [15]. In the following, we
review conventional afﬁne region estimation methods andpresent our method for estimating multiple afﬁne regions.

are designed to ﬁnd only one afﬁne region, even if multiple
regions exist and would fail to match keypoints properly between a pair of images.
A solution to this problem is to detect all afﬁne regions for each keypoint by exhaustively convolving the LoG
ﬁlters with all keypoints. The naive implementation ofthis scheme is intractable for most cases, though, because
anisotropic LoG ﬁlters have three parameters (σx , σy , θ).
In this study, we efﬁciently calculate the response valuesof the anisotropic LoG ﬁlter by using the spectral SIFT [7].
The spectral SIFT detector solves eigenvalue problems for
Gaussian scale space and LoG space and decomposes each
set of ﬁlters into compact linear combinations of eigenﬁlters. It enables scale-continuous ﬁlter responses to be calculated from only a few of the eigenﬁlter convolution results.
Spectral SIFT formulates the eigenproblem in the form ofa continuous integral equation and solves it by approximating a discrete ﬁlter decomposition without an unreasonableamount of error. To approximate a large number of ﬁlters,
the separable ﬁlter [24] uses tensor decomposition. In our
proposed method we simply apply singular value decomposition (SVD) to the afﬁne space, which is composed ofseveral thousand different anisotropic LoG ﬁlters. We showthat the afﬁne space is well reconstructed from few (dozens)eigenﬁlters. This makes it tractable to compute all possible
LoG ﬁlter responses. Furthermore, our proposed method
can estimate multiple afﬁne regions for a single keypointby ﬁnding the multiple extrema of anisotropic LoG ﬁlter responses. We successfully estimated multiple afﬁne regions
under varying image conditions, as shown in Fig. 1.

2.1. Afﬁne region detectors
Afﬁne-invariant keypoint detection can be thought of asa generalization of scale estimation. The edge-based region(EBR) detector from Tuytelaars and colleagues treats a parallelogram region as an afﬁne region in which each vertex
is the point common to two edges crossing a corner determined by the Harris detector [26]. Tuytelaars et al. [27] also
proposed the intensity extrema-based region (IBR) detector
using intensity information in the periphery of a keypoint.
The IBR method observes intensity values along rays emanating from a keypoint and determines the difference between those values and the intensity value of the keypoint.
An afﬁne region is determined by ﬁtting an ellipse to theshape that joins the positions where that difference takes
on a maximum value on each ray. The Maximally Stable
Extremal Regions (MSER) approach [13] is another region
estimation method, which uses a watershed algorithm to determine connected regions having similar intensity values
within an image. It estimates afﬁne-invariant regions by ﬁtting ellipses to the regions so determined. Obdrzalek andMatas [19] extended the MSER approach to extract multiple regions based on parameters such as outline, curvatureand so on.
Mikolajczyk et al. proposed Hessian-Afﬁne and HarrisAfﬁne [15] methods, which extend isotropic regions detected by Hessian/Harris-Laplace to afﬁne regions. The procedure involves calculating a second-moment matrix withrespect to an isotropic region detected by Hessian/HarrisLaplace. They determine an afﬁne region by performing
convolution operations on the keypoint while repeatedly
varying the shape of an anisotropic Gaussian ﬁlter according to the second-moment matrix.
Afﬁne SIFT (ASIFT) [18] generates a set of sample
views of the initial image patches, obtained by varying theorientation of the two camera axes. Next, it applies the SIFT
descriptor to all images thus generated. The method of Lepetit and Fua [8] also generates a set of all possible appearances under different viewing conditions; it trains randomized trees, using the set to recognize keypoints.
Each of the above methods determines only a single
afﬁne region per keypoint. This limitation often results in
low repeatability; see Fig. 1. If the Hessian-Afﬁne method(shown in the top row) starts from different initial points, as
shown in the upper left and center of this ﬁgure, the afﬁne
estimation results converge to different locations. In contrast, our idea is to allow multiple afﬁne regions, which
can improve repeatability even in difﬁcult situations, suchas noisy localization and complicated shading, as shown inthe bottom row of Fig. 1.

2. Related works
A variety of corner detection methods have been proposed, including Moravec corner detection [17], Harris corner detection [5], Good Features to Track [23], Smallest
Univalue Segment Assimilating Nucleus (SUSAN) [25],and Features from Accelerated Segment Test (FAST) [21].
Scale-space ﬁltering has been studied since the 1980s[3]. Perona introduced the approximation of rotationaland scale-space ﬁlters by SVD [20], which is based on
the Steerable ﬁlter that uses a linear combination of rotation ﬁlters [4]. Scale-invariant keypoint detectors thatgenerate isotropic support regions to compute descriptors
have also been designed, such as LoG [10, 11], DoG [12],Hessian/Harris-Laplace [14] and SURF [2]. While these detectors compute scale-invariant keypoints using scale-space
theory, some more recent methods simply build an image
pyramid and apply a corner detector for each layer to copewith scale change [9, 22, 1, 6], which results in extremely
fast keypoint detection. While there has been much researchon estimating isotropic and scale invariant local regions,little work has been directed at estimating afﬁne regions
since the proposal of the Hessian/Harris-Afﬁne method in586

In this study, we propose a method for determining multiple afﬁne regions by using the anisotropic LoG ﬁlter. A
major challenge of this idea is that a very large number ofanisotropic LoG ﬁlters must be convolved with a given image patch to exhaustively search the multiple extrema of ﬁlter responses. This requires a massive amount of computational time. Our ﬁnding is that the responses of all possible
anisotropic LoG ﬁlters can be efﬁciently computed by factorizing them in a similar manner to spectral SIFT.

2.2. Contribution
We propose a method for accurately detecting afﬁne regions from an image keypoint by using a combination of theeigenﬁlters of anisotropic LoG ﬁlters. Our method has thefollowing features.

Figure 2. Approximation of anisotropic LoG ﬁlters by SVD. The
LoG ﬁlter is deﬁned by a linear combination of eigenﬁlters andeigenfunctions.

1. Compaction of ﬁlter parameters by replacing
anisotropic LoG ﬁlters with major eigenﬁlters
At the detection stage, only a small set of ﬁlters—
which we call ‘eigenﬁlters’—is used to convolvean image, instead of a large number of different
anisotropic LoG ﬁlters.

3.1. Approximation of anisotropic LoG space by
SVD
We generate anisotropic LoG ﬁlters beforehand using:
∂2
∂2
G(Σ)
+
G(Σ),
∂x2
∂y 2
 T −1 
1
x Σ x
√
,
G(Σ) =exp −2
2π det Σ

 2


cos θ
sin θ σx 0 cos θ − sin θΣ=,
0 σy2 sin θ− sin θ cos θ
cos θ

2. Accurate afﬁne parameter interpolation
The eigenfunctions, produced by SVD of anisotropic
LoG ﬁlters, are ﬁtted by continuous functions, which
can be used to interpolate accurate afﬁne parameters.

LoG(σx , σy , θ) =3. Detection of multiple afﬁne regions
Even when two or more afﬁne shapes are superimposed, the method accurately estimates the afﬁne parameters for each shape, as is demonstrated in Fig. 1.
Multiple region detection has not been very successful
to date, although overlapping elliptical shapes occurs
frequently in natural images.

(3)
(4)
(5)

where G(Σ) denotes an anisotropic Gaussian ﬁlter andx denotes the position(x, y) from the center of the ﬁlter. The ﬁlter parameters are set as follows: scale σx
in the x direction {1.6, 1.7, · · · , 3.2}, scale σy in the y
direction {1.6, 1.7, · · · , 3.2}, and ﬁlter rotational angle θ{0◦ , 5◦ , · · · , 175◦ }. The ﬁlter size D of an anisotropic LoG
ﬁlter is 361 (19 × 19) and the number of different ﬁlters N
is 4,913. These 4,913 anisotropic LoG ﬁlters can be decomposed into three matrices by applying SVD,

3. Proposed method
We propose a method for estimating multiple afﬁne regions based on Koutaki et al.’s [7] idea of spectral SIFT.
Spectral SIFT derives eigensolutions for Gaussian spaceand LoG space expressed by
 2

∂
∂2
LoG(σ) =G(σ) + 2 G(σ) σ 2
∂x2
∂y

 2
x2 + y 2 − 2σ 2 2
x + y2,(1)=
σ exp −
2πσ 6
2σ 2

 2
x + y2
1
.
(2)
exp
−
G(σ) =
2πσ 2
2σ 2

L = USVT ,(6)

where matrix L ∈ RD×N consists of the above 4,913
LoG ﬁlters L(σx , σy , θ) ∈ RD vectorized to form eachcolumn, matrices U ∈ RD×D and V ∈ RN ×N are Ddimensional and N -dimensional orthogonal matrices, respectively, and matrix S ∈ RD×N is a rectangular diagonal matrix having singular values. Fi ∈ RD and Pi ∈RN are i-th column vectors of U = [F1 , F2 , · · · , FD ]and V = [P1 , P2 , · · · , PN ] respectively. This SVD ofanisotropic LoG ﬁlters is shown in Fig. 2. Now, denoting
the row vectors of the product SVT as {φ1 , φ2 , · · · , φD },the anisotropic LoG ﬁlter L(σx , σy , θ) for afﬁne parameters(σx , σy , θ) can be deﬁned as

Equations (1) and (2) represent isotropic ﬁlters and thus
cannot be used to estimate afﬁne regions. Therefore,
we employ anisotropic ﬁlters to determine afﬁne regions.
Determining the eigenﬁlters and eigenfunctions (i.e., theeigensolutions) of the anisotropic ﬁlters by SVD is an efﬁcient means of deﬁning the afﬁne regions.
587

Figure 3. Singular values of diagonal matrix S. From indices 15
and above, the singular values are very close to zero.

L(σx , σy , θ) =Dφn [σx , σy , θ]Fn

Figure 5. Workﬂow of the anisotropic LoG ﬁlter response calculation; parameter values: σx = 1.4, σy = 1.6, θ = 45◦ .

n=1

≈

14


φn [σx , σy , θ]Fn .

(7)

n=1

After examining the singular values of matrix S (shown in
Fig. 3), we take the major 14 eigenﬁlters only because thecumulative sum of their singular values reaches 96.7% of
the total. This means that an anisotropic LoG ﬁlter can beadequately approximated by 14 eigenﬁlters. In the above,φn [σx , σy , θ] denotes the scalar value of the vector φ of theparameters σx , σy , θ. It is referred to here as an eigenfunction. In addition, Fn can be treated as a two-dimensional
ﬁlter and is therefore an eigenﬁlter. All 14 eigenﬁlters andeigenfunctions are shown in Fig. 4. An eigenfunction is avalue in the (σx , σy , θ) 3D space. The eigenfunctions shownin the ﬁgure are for θ = 45◦ .

Figure 6. Examples of eigenfunctions with ﬁxed parameters. The
upper plots are the eigenfunction values for each θ. The eigenfunctions for ﬁxed σx , σy are shown in the lower plots.

computationally expensive convolution processing only has
to be performed 14 times, making it efﬁcient. The workﬂow
for calculating the response value of an anisotropic LoG ﬁlter for parameters σx = 1.4, σy = 1.6, θ = 45◦ is shown in
Fig. 5.

3.2. Calculation of response value of an anisotropic
LoG ﬁlter
We now calculate the response value R of an anisotropic
LoG ﬁlter using eigenﬁlter Fn and eigenfunction φn [·]. The
response value R of an anisotropic LoG ﬁlter can be calculated by performing a convolution operation between Eq.
(7) and patch image I, which is the image corresponding to
the keypoint neighborhood (19 × 19). Here, the result ofconvolution between the patch image and an eigenﬁlter isgiven by qn = I ∗ Fn . Applying the distributive law to Eq.
(8) gives14
φn [σx , σy , θ]Fn
(8)
R(σx , σy , θ) ≈ I ∗3.3. Continuous function ﬁtting of eigenfunctions
The response values of 4,913 ﬁlters can be approximatedby 14 eigenfunctions using Eq. (9). These eigenfunctions,however, take on discrete values, thus the response values
can only be approximated for the 4,913 combinations of ﬁlter parameters generated before decomposition. We consider the ﬁtting of eigenfunctions to continuous functions
to solve this problem. As shown in Fig. 4, eigenfunction
φ[·] is a value in a 3D parameter space. The task, therefore,
is to solve the error minimization problem between eigenfunction φ[·], obtained by SVD, and the ﬁtting model ϕ(·),
based on a 3D function. The ﬁtting model uses a continuous function in power series form for the scale parametersin question. However, the eigenfunction for rotational angle
θ is a periodic function waveform that considers the shapeof the eigenﬁlter. Numerical values of eigenfunction φ5 areshown in Fig. 6. Eigenﬁlter F5 changes shape according tothe ﬁlter’s rotational angle θ. This causes the eigenfunction

n=1

≈

14


φn [σx , σy , θ]qn .

(9)

n=1

Convolution between the patch image and the 14 eigenﬁlters can be performed in advance as in Eq. (9). The response values for a desired ﬁlter can be easily calculated bysimply changing eigenfunction values. In our approach, the588

Figure 4. Eigenﬁlters and eigenfunctions. The eigenfunctions have θ = 45◦ .

values also to change according to θ (upper part of Fig. 6).
Furthermore, the eigenfunctions become periodic when ﬁxing parameters σx , σy (lower part of Fig. 6). For this reason,
the eigenfunction ﬁtting model uses a continuous function
consisting of trigonometric functions of θ. Accordingly, ﬁtting model ϕ(·) takes the form

combination (σx , σy , θ) at which a ﬁlter response value isan extremum. It is therefore necessary to detect multiple
extrema in ﬁlter response values in the 3D parameter space.
For the ellipsoidal patterns in Fig. 1, one afﬁne region is determined with respect to the scale directions σx , σy . In the θ
direction, however, the existence of multiple afﬁne regions
can be considered. For this situation, therefore, we determine one afﬁne region in the σx , σy directions and estimate
multiple afﬁne regions in the θ direction. The method usedhere to search for extrema is illustrated in Fig. 7. First, after discretizing the θ axis, Newton’s method is employed tosearch for different θ in the extrema in the 2D space (σx , σy )(denoted by a black ×-mark in the ﬁgure). Next, using these2D extrema for different θ, extrema are detected in the 3D
space (σx , σy , θ) (denoted by a blue ◦-mark in the ﬁgure).
When there are more than one extremum in 3D space, allextrema with a response value that is close to the largest extremum are adopted as the afﬁne regions of the keypoint. In
the example of Fig. 7, three extrema exist in 3D space but
only two extrema are used as the afﬁne regions. The resultsof keypoint afﬁne estimation using our proposed method areshown in Fig. 8. These results demonstrate that the proposed method determines multiple afﬁne regions for a single keypoint. It is also clear that these afﬁne regions are thesame as for a pair of images.

ϕ(σx , σy , θ)
=

f 
g
e 


(αijk σxi σyj sin kθ + βijk σxi σyj cos kθ), (10)i=0 j=0 k=1

where α and β are unknown coefﬁcients for which the squareerror E between the ﬁtting model ϕ(·) and discrete eigenfunction φ[·] (obtained by SVD) is minimal;E=(φ[σx , σy , θ] − ϕ(σx , σy , θ))2 . (11)
σx

σy

θ

Fitting eigenfunctions in this way enables the ﬁlter response
value R to be represented as a continuous function,
R(σx , σy , θ) ≈14
ϕn (σx , σy , θ)qn .

(12)

n=1

This makes it possible to calculate ﬁlter response values
for any parameters (σx , σy , θ). By applying the gradientmethod, it is possible to search for the extrema of response
values efﬁciently because the continuous function of theeigenfunction ϕ(·) is differentiable. The process of searching for the extrema is described next.

3.5. Simple pattern testing
In Fig. 9, the left part of the simple test pattern in each
subﬁgure depicts ellipses and intersecting ellipses drawn inblack, while the right part depicts the same shapes after gradation processing. The parameter range of each ellipse is25-30 pixels for the major axis, 16-19 pixels for the minor
axis, and 0◦ − 60◦ for the rotational angle. As shown in Fig.

3.4. Estimation of multiple afﬁne regions
Given that multiple afﬁne regions exist in an input image, there will also be multiple instances of the parameter589

rounding the combined shapes. It is insensitive to changes
in intensity and consequently failed in detecting the elliptical patterns in the right part of the test pattern (Fig. 9(b)).
The proposed method, however, detected all ellipses—even
those in an intersecting pattern, as it is designed to estimate
multiple afﬁne regions. The regions that it detected in theleft part of the test pattern were thus the same as those in
the right part, as shown in Fig. 9(d). The results of estimating afﬁne regions by using the original LoG ﬁlter priorto SVD are shown in Fig. 9(c). It can be seen that theseresults agree with those produced by the proposed method,
thus the proposed method approximates the original LoG
ﬁlter.

4. Experiments
Figure 7. Searching for a multiple-hypothesis afﬁne region.

This experiment evaluates the repeatability of afﬁneregion detection between two images and compares the results of the proposed method, Hessian-Afﬁne, MSER, andDoG.

4.1. Datasets
For this evaluation experiment, we used conventional
Afﬁne Covariant Regions Datasets1 and Spectrum magazine datasets created by us. The Afﬁne Covariant Regions
Datasets consist of image datasets covering eight scenes(Grafﬁti, Wall, Boat, Bark, Bikes, Trees, Leuven and UBC).
Each scene comprises six images, each of which represents a certain type of change in the appearance of thatscene. These changes in appearance are summarized in Table 1. The Spectrum magazine datasets consist of image

Figure 8. Result of afﬁne regions estimation with the proposed
method. Red afﬁne regions are estimated to be the same region ineach of two images.

Table 1. Change in appearance for each scene type.

Change in appearance
Projective transformation
Rotation and scale change
Blur
Other

Scene
Grafﬁti, Wall
Boat, Bark
Bikes, Trees
Leuven, UBC

datasets from three magazines (Spectrum1, Spectrum2 andSpectrum3). Each magazine has image sets corresponding
to pitch angle rotations of 0◦ , 10◦ , 20◦ and 30◦ , and eachpitch angle set consists of 73 images having rotations in rollangle. Examples of images in the Spectrum1 dataset are
shown in Fig. 10. Homography matrices H between image
pairs were determined beforehand for the above datasets.

Figure 9. Results of the afﬁne region estimation for a simple test
pattern. The input pattern depicts simple ellipses and intersecting
ellipses drawn in black and gradation.

4.2. Repeatability
9(a), the Hessian-Afﬁne method can detect simple ellipses
but cannot correctly detect intersecting ellipses. Thus, forseveral elliptical shapes, this method detected different ellipses between the left and right parts of the test pattern. The
MSER method, by comparison, ﬁts ellipses using region
partitioning and therefore predicted elliptical patterns sur-Repeatability is calculated as the ratio of the number of corresponding points (corresp. points) in apair of images to the minimum number of keypoints(keypoints1, keypoints2) detected in those two images;
1 http://www.robots.ox.ac.uk/˜vgg/data/data-aff.html

590

Figure 10. Example from the Spectrum magazine dataset.

i.e.,
Repeatability =corresp. points
× 100. (13)min(keypoints1, keypoints2)

Figure 13. Examples of keypoint matching by Hessian-Afﬁne andproposed method.

When determining corresponding points, we calculate theerror between the two images in the overlapping of the corresponding keypoint afﬁne regions [16] by using the homography matrix H,
A1 ∩ HiT A2 Hi
overlap error = 1 −× 100, (14)A1 ∪ HiT A2 Hi

matching using SIFT descriptors with the Hessian-Afﬁneand the proposed method are shown in Fig. 13. It is clearthat a high matching rate can be achieved with the proposed
method.

4.4. Computational time
We compare the processing times per image (640 × 480
pixels) of the Hessian-Afﬁne, Original LoG, and proposed
method. The computer used in the experiment had a 3.33GHz Intel Xeon X5470 CPU and 32 GB of RAM. All code
was written in C++. The results of the experiment are listed
in Table 2. The proposed method was 87.2 times faster

where A1 and A2 are keypoint afﬁne regions detected in thetwo images and Hi is a linear homography matrix. If theoverlap error of corresponding afﬁne regions is less than
the threshold T , the corresponding points are counted. In
this experiment, we set T = 40% and compared repeatability among the proposed method, Hessian-Afﬁne, MSER,
and DoG. To put the evaluations of the proposed methodand Hessian-Afﬁne on equal footing, we used the same keypoints used in the estimation of afﬁne regions in both methods.

Table 2. Computational time (s) per image (640 × 480 pixels).

Total time

4.3. Experiment results

Hessian-Afﬁne
4.091

Original LoG
198.654

Proposed
2.277

than the Original LoG. This was possible because it approximates the convolution processing of 4,913 ﬁlters with only
14 eigenﬁlters.

The repeatability results obtained when using theAfﬁne Covariant Regions Datasets and Spectrum magazine
datasets are shown in Figures 11 and 12, respectively. The
results show that the proposed method has higher repeatability than do the conventional methods. We attribute thisto the proposed method estimating multiple afﬁne regions,
thereby reducing error in the estimation of afﬁne regions
between a pair of images. Thus the afﬁne region estimation
employed in the proposed method exhibits high accuracy.
For the Afﬁne Covariant Regions Datasets, these results
also show that repeatability improved by using the proposed method even for datasets that did not involve a projective transformation (namely, Boat, Bark, Bike, Trees, Leuven, and UBC). This indicates that the proposed method
was capable of determining invariant afﬁne regions for various types of changes in appearance. Examples of keypoint

5. Conclusion
We proposed a method for estimating multiplehypothesis afﬁne regions using anisotropic LoG ﬁlters.
This method efﬁciently calculates the response values ofanisotropic LoG ﬁlters by applying singular value decomposition. We showed that estimating multiple afﬁne regions for each keypoint in an image improves repeatability.
The proposed method also represents ﬁlter response values
in continuous function form, enabling response values forany parameter to be found. These features make for highaccuracy afﬁne region estimation. In future work, we plan
to increase the efﬁciency of extrema searching to increase
the speed of high-speed afﬁne region estimation.
591

Figure 11. Repeatability of the Proposed, Hessian-Afﬁne, MSER, and DoG methods operating on the Afﬁne Covariant Regions Datasets.

Figure 12. Repeatability of the Proposed, Hessian-Afﬁne, MSER, and DoG methods operating on the Spectrum magazine datasets.

References[2] H. Bay, T. Tuytelaars, and L. V. Gool. SURF: Speeded-Up
Robust Features. Computer Vision and Image Understanding, 110(3):346–359, 2008.

[1] M. Ambai and Y. Yoshida. Card: Compact and real-time descriptors. In International Conference on Computer Vision,
pages 97–104, 2011.

[3] J. Crowley and A. Parker. A representation for shape based592

[21] E. Rosten, R. Porter, and T. Drummond. FASTER and better:
A machine learning approach to corner detection. PatternAnalysis and Machine Intelligence, IEEE Transactions on,
32:105–119, 2010.
[22] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. ORB:
An Efﬁcient Alternative to SIFT or SURF. In International
Conference on Computer Vision, pages 2564–2571, 2011.
[23] J. Shi and C. Tomasi. Good Features to Track. In Conference
on Computer Vision and Pattern Recognition, pages 593 –
600, 1994.
[24] A. Sironi, B. Tekin, R. Rigamonti, V. Lepetit, and P. Fua.
Learning Separable Filters. Pattern Analysis and MachineIntelligence, pages 94–106, 2015.
[25] S. M. Smith and J. M. Brady. Susan & mdash; a new approach to low level image processing. International Journalof Computer Vision, 23(1):45–78, 1997.
[26] T. Tuytelaars and L. V. Gool. Content-based Image Retrieval
based on Local Afﬁnely Invariant Regions. In International
Conference on Visual Information Systems, pages 493–500,
1999.
[27] T. Tuytelaars and L. V. Gool. Wide baseline stereo matchingbased on local, afﬁnely invariant regions. In British Machine
Vision Conference, pages 412–425, 2000.

on peaks and ridges in the difference of low pass transform.
Pattern Analysis and Machine Intelligence, 6(2):156–170,
1984.
[4] W. Freeman and E. Adelson. The design and use of steerable
ﬁlters. Pattern Analysis and Machine Intelligence, 13:891–
906, 1991.
[5] C. Harris and M. Stephens. A combined corner and edge
detector. In Alvey Vision Conference, pages 147–151, 1988.
[6] T. Hasegawa, Y. Yamauchi, M. Ambai, Y. Yoshida, andH. Fujiyoshi. Keypoint Detection by Cascaded FAST. In
International Conference on Image Processing, pages 5611–
5615, 2014.
[7] G. Koutaki and K. Uchimura. Scale-space Processing Using
Polynomial Representations. In Conference on Computer Vision and Pattern Recognition, pages 2744–2751, 2014.
[8] V. Lepetit and P. Fua. Keypoint recognition using randomized trees. Pattern Analysis and Machine Intelligence,
28(9):1465–1479, 2006.
[9] S. Leutenegger, M. Chli, and R. Siegwart. Brisk: Binary
robust invariant scalable keypoints. In International Conference on Computer Vision, pages 2548–2555, 2011.
[10] T. Lindeberg. Scale-space theory: A basic tool for analysing
structures at different scales. Journal of Applied Statistics,
pages 224–270, 1994.
[11] T. Lindeberg. Feature Detection with Automatic Scale Selection. International Journal of Computer Vision, 30:79–116,
1998.
[12] D. G. Lowe. Distinctive Image Features from ScaleInvariant Keypoints. International Journal of Computer Vision, 60:91–110, 2004.
[13] J. Matas, O. Chum, M. Urban, and T. Pajdla. Robust Wide
Baseline Stereo from Maximally Stable Extremal Regions.
In British Machine Vision Conference, pages 36.1–36.10,
2002.
[14] K. Mikolajczyk and C. Schmid. Indexing Based on Scale Invariant Interest Points. In International Conference on Computer Vision, pages 525–531, 2001.
[15] K. Mikolajczyk and C. Schmid. Scale & Afﬁne Invariant
Interest Point Detectors. International Journal of Computer
Vision, 60(1):63–86, 2004.
[16] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman,
J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool. A Comparison of Afﬁne Region Detectors. International Journal ofComputer Vision, 65(1-2):43–72, 2005.
[17] H. Moravec. Rover visual obstacle avoidance. In International Joint Conference on Artiﬁcial Intelligence, pages 785–
790, 1981.
[18] J.-M. Morel and G. Yu. ASIFT: A New Framework for Fully
Afﬁne Invariant Image Comparison. SIAM Journal on Imaging Sciences, 2(2):438–469, 2009.
[19] S. Obdrzalek and J. Matas. Object Recognition Using Local
Afﬁne Frames on Maximally Stable Extremal Regions. In
Toward Category-Level Object Recognition, pages 85–108,
2006.
[20] P. Perona. Deformable Kernels for Early Vision. PatternAnalysis and Machine Intelligence, 17:488–499, 1991.

593