Repository: opencv
Issue: Sometimes focal length is negative during bundle adjustment, it makes crash.
Labels: bug,auto-transferred,priority: normal,affected: 2.4,category: stitching

Transferred from http://code.opencv.org/issues/1545

```
|| p x on 2012-01-13 06:14
|| Priority: Normal
|| Affected: branch '2.4'
|| Category: stitching
|| Tracker: Bug
|| Difficulty: None
|| PR: 
|| Platform: None / None
```
## Sometimes focal length is negative during bundle adjustment, it makes crash.

```
I detected that, this problem occurs in void BundleAdjusterRay::calcError(Mat &err) in motion_estimators.cpp

After some iterations, f1 or f2 may be negative, so mult = sqrt(f1 * f2) is NaN.
It makes the program crash in a further call of SVD to a NaN matrix.

Please investigate it.
```
## History
##### None on 2012-02-12 18:56

```
-   Description changed from I detected that, this problem occurs in
    void [[BundleAdjusterRay]]::calcError... to I detected that, this
    problem occurs in void BundleAdjusterRay::calcError(Mat... More
```
##### Alexander Shishkov on 2012-02-17 08:59

```
-   Category set to stitching
```
##### Andrey Kamaev on 2012-03-24 15:23

```
-   Priority changed from Blocker to Normal
-   Assignee changed from Vadim Pisarevsky to Alexey Spizhevoy
```
##### Alexey Spizhevoy on 2012-03-26 06:51

```
p x,

Could you attach here the dataset where that error occurs? Provide us, please, all the information to reproduce the bug. 

Thanks.
```
##### Anatoly Baksheev on 2012-03-31 23:52

```
I think this reporter doesn't present at this tracker. Try to find his contacts on trac.
```
##### p x on 2012-04-01 00:24

```
I am trying to repro this bug and if it still exists now, some test cases will be given. Thank you for your reply.
```
##### Alexey Spizhevoy on 2013-01-10 06:24

```
no response from the submitter
-   Status changed from Open to Cancelled
```
##### Martin Ljungqvist on 2013-04-30 08:51

```
I have happened to reproduce this problem in BundleAdjusterRay::calcError(Mat &err) using the sample code stitching_detailed.cpp.

As described above; after some iterations, if f1 or f2 is negative, then mult = sqrt(f1 * f2) is NaN. This makes the program fail, the output image being just one pixel. If this 'mult' variable is removed then it works and the resulting image looks ok, but the refined focal lengths become quite high. This corresponds to the comments about bundle adjustment and residual weights in "Image alignment and stitching - A tutorial" by Richard Szeliski 2006, page 50, where this weight variable is explained.

This problem happens when running stitching_detailed.cpp with two images being very similar and hardly no movement of the camera but small changes inside the image. All settings default. The iteration in BundleAdjusterRay makes the focal lengths f1 and f2 to approach zero in this case, when one of them might eventually become a small negative value. So either the 'mult' weight variable could perhaps be changed in some way, or e.g the stitching_detailed algorithm could somewhere take precautions if the images are too similar.

Please send me a message if you need images for reproducing the problem.
```
##### Anna Kogan on 2013-05-07 08:20

```
Hello Martin, 
Please attach the images for us to reproduce the issue. Thank you!
-   Affected version set to branch '2.4'
-   Status changed from Cancelled to Open
```
##### Martin Ljungqvist on 2013-05-07 14:08

```
Hello Anna,
Please find the attached files.
-   File vid_00001.pgm added
-   File vid_00002.pgm added
```
##### Martin Ljungqvist on 2013-05-13 13:41

```
Some speculation if BundleAdjusterRay() would assume not too similar images.. The function leaveBiggestComponent() sorts out images that are not matching at all or are too similar, depending on their confidence. The confidence is calculated in BestOf2NearestMatcher::match(), where there is a threshold for confidence which is set to 3 in the code. The attached images above give 2.66 confidence and fail. But some images can have 2.7 and still be stitched. So confidence alone seems not to discriminate. This is an interesting problem of how and where to handle this.
```