Repository: scikit-image/scikit-image

Issue:
SIFT wrong octave values for keypoints
Thank you for including SIFT in skimage. I think there is a small problem with how the octave values are calculated for keypoints. If there are no keypoint candidates (local extremas) found in one octave it is skipped.  See:
https://github.com/scikit-image/scikit-image/blob/ed642e2bc822f362504d24379dee94978d6fa9de/skimage/feature/sift.py#L338

Later, when the octave indices are calculated, the lower octave is not included. See:
https://github.com/scikit-image/scikit-image/blob/ed642e2bc822f362504d24379dee94978d6fa9de/skimage/feature/sift.py#L421
So, keypoints in higher octaves have indices too low. The empty octave has to be included for the correct indices.

I hope this makes sense.

Edit: there is also a typo  here:
https://github.com/scikit-image/scikit-image/blob/ed642e2bc822f362504d24379dee94978d6fa9de/skimage/feature/sift.py#L154
it should be scales


Discussion:
Hey @longjoke,
thank you for opening this issue!

> I think there is a small problem with how the octave values are calculated for keypoints. If there are no keypoint candidates (local extremas) found in one octave it is skipped. See:

I am not an expert on the SIFT algorithm, so I'll ask for help: @grlee77 @faberno @AlexanderZeilmann could y'all help me here? Thank you! :slightly_smiling_face:

>  it should be scales

For our SIFT implementation, we used the following paper:
```
    The implementation here closely follows the
    detailed description in [3]_, including use of the same default parameters.
    .. [3] I. R. Otero and M. Delbracio. "Anatomy of the SIFT Method",
           Image Processing On Line, 4 (2014), pp. 370–396.
           :DOI:`10.5201/ipol.2014.82`
```
Maybe there is a difference in notation from the implementation you're referring to. Could you point me to a reference that describes the algorithm as you see it?
Thanks again @longjoke!

Thanks for the reply. I think there was a misunderstanding regarding the second issues. The attribute is called scales, just the documentation contains a typo. The sigmas attribute is just above. The implementation is great.
@longjoke oh, thanks for clarifying! I see it. Would you like to fix it opening a PR? :slightly_smiling_face:
Thanks again!
Hey @longjoke, thanks a lot for pointing this out. It's definitely an issue, but luckily an easy fixable one :D
I think adding `extrema_pos.append(np.empty((0, 2)))` to `if keys.size == 0:` should do the job, right?

Also thanks for finding the typo!

Resolving patch:
diff --git a/skimage/feature/sift.py b/skimage/feature/sift.py
--- a/skimage/feature/sift.py
+++ b/skimage/feature/sift.py
@@ -151,7 +151,7 @@ class SIFT(FeatureDetector, DescriptorExtractor):
         Subpixel-precision keypoint coordinates as ``(row, col)``.
     sigmas : (N, ) array
         The corresponding sigma (blur) value of a keypoint.
-    sigmas : (N, ) array
+    scales : (N, ) array
         The corresponding scale of a keypoint.
     orientations : (N, ) array
         The orientations of the gradient around every keypoint.
@@ -335,6 +335,7 @@ def _find_localize_evaluate(self, dogspace, img_shape):
             # find extrema
             keys = _local_max(np.ascontiguousarray(octave), threshold)
             if keys.size == 0:
+                extrema_pos.append(np.empty((0, 2)))
                 continue
 
             # localize extrema
@@ -413,13 +414,14 @@ def _find_localize_evaluate(self, dogspace, img_shape):
             extrema_scales.append(keys[border_filter, 2])
             extrema_sigmas.append(sigmas[border_filter])
 
-        if not extrema_pos:
+        octave_indices = np.concatenate([np.full(len(p), i)
+                                         for i, p in enumerate(extrema_pos)])
+
+        if len(octave_indices) == 0:
             raise RuntimeError(
                 "SIFT found no features. Try passing in an image containing "
                 "greater intensity contrasts between adjacent pixels.")
 
-        octave_indices = np.concatenate([np.full(len(p), i)
-                                         for i, p in enumerate(extrema_pos)])
         extrema_pos = np.concatenate(extrema_pos)
         extrema_scales = np.concatenate(extrema_scales)
         extrema_sigmas = np.concatenate(extrema_sigmas)
@@ -443,12 +445,15 @@ def _compute_orientation(self, positions_oct, scales_oct, sigmas_oct,
         key_count = 0
         for o, (octave, delta) in enumerate(zip(gaussian_scalespace,
                                                 self.deltas)):
+            gradient_space.append(np.gradient(octave))
+
             in_oct = octaves == o
+            if not np.any(in_oct):
+                continue
             positions = positions_oct[in_oct]
             scales = scales_oct[in_oct]
             sigmas = sigmas_oct[in_oct]
 
-            gradient_space.append(np.gradient(octave))
             oshape = octave.shape[:2]
             # convert to octave's dimensions
             yx = positions / delta


Test patch: