Repository: opencv
Issue: Broken Epipolar Geometry and Ruined Fundamental Matrix
Labels: bug,category: calib3d,Hackathon


- OpenCV => 4.5.4.58
- Operating System / Platform => Windows 10 64bit
- Compiler => PyCharm

**##### Detailed description**

I am working through the OpenCV tutorials and I’m currently working on calculating the Epipolar lines from two images. However, even though I’m using the exact code from the OpenCV tutorial and the same test images I’m getting very strange results.

The F matrix looks completely ruined. These values are definitely not correct.
```
[[-1.60622769e-05  1.36431322e-04 -1.08742243e-04]
 [ 4.08336438e-05 -3.65632851e-06 -6.03366278e-02]
 [-1.82580135e-03  9.77636192e-03  1.00000000e+00]]
```
nearly 5000 keypoints per image, but only 182 “inliers”, also sounds broken.

I would expect those match points to have the same color for each match. that’s what the code says… so something up to and including the matching must have gone wrong.

I tried this code using both SIFT and ORB feature detection algorithms and both result in the same problem.

Broken Result:
![image](https://user-images.githubusercontent.com/10361187/141693535-6f134399-5fd1-4c31-9d26-9a611bfe03fb.png)

Tutorial Result:
![image](https://user-images.githubusercontent.com/10361187/141693600-a5276e25-9f1d-4d13-a6df-bef9820e266b.png)



**##### Steps to reproduce**
Run code below.
.py

```
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

def drawlines(img1, img2, lines, pts1, pts2):
    """img1 - image on witch we draw the epilines for the points in img2
       lines - corresponding epilines"""
    r, c = img1.shape
    img1 = cv.cvtColor(img1, cv.COLOR_GRAY2BGR)
    img2 = cv.cvtColor(img2, cv.COLOR_GRAY2BGR)
    for r,pt1,pt2 in zip(lines, pts1, pts2):
        color = tuple(np.random.randint(0, 255, 3).tolist())
        x0, y0 = map(int, [0, -r[0]/r[1]])
        x1, y1 = map(int, [c, -(r[2] + r[0]*c/r[1])])
        img1 = cv.line(img1, (x0, y0), (x1, y1), color, 1)
        img1 = cv.circle(img1, tuple(pt1), 5, color, -1)
        img2 = cv.circle(img2, tuple(pt2), 5, color, -1)
    return img1, img2


if __name__ == '__main__':

    img1 = cv.imread('left.jpg', 0)         # queryingImage # left image
    img2 = cv.imread('right.jpg', 0)        # trainImage # right image

    orb = cv.ORB_create(nfeatures=5000)

    # find the keypoints and descriptors with SIFT
    kp1, des1 = orb.detectAndCompute(img1, None)
    kp2, des2 = orb.detectAndCompute(img2, None)

    # FLANN parameters
    FLANN_INDEX_LSH = 6
    index_params = dict(algorithm=FLANN_INDEX_LSH, table_number=6, key_size=12, multi_probe_level=1)
    search_params = dict(checks = 25)

    flann = cv.FlannBasedMatcher(index_params, search_params)
    matches = flann.knnMatch(des1, des2, k=2)

    good = []
    pts1 = []
    pts2 = []

    matchesMask = [[0, 0] for i in range(len(matches))]
    # ratio test as per Lowe's paper
    for i, (m,n) in enumerate(matches):
        if m.distance < 0.8*n.distance:
            good.append(m)
            pts1.append(kp1[m.queryIdx].pt)
            pts2.append(kp2[m.trainIdx].pt)
            matchesMask[i] = [1, 0]
            print("Number of good matches: ",+ len(good))

    draw_params = dict(matchColor=(0, 255, 0), singlePointColor=(255, 0, 0), matchesMask=matchesMask, flags=0)
    img3 = cv.drawMatchesKnn(img1, kp1, img2, kp2, matches, None, **draw_params)
    cv.imshow("Feature Detection", img3)
    cv.waitKey(0)

    pts1 = np.int32(pts1)
    pts2 = np.int32(pts2)
    F, mask = cv.findFundamentalMat(pts1, pts2, cv.FM_LMEDS) #F = fundamental matrix
    print(F)

    # We select only inlier points
    pts1 = pts1[mask.ravel() == 1]
    pts2 = pts2[mask.ravel() == 1]
    print("NUmber of pts1 inlier points: ",+len(pts1))
    print("NUmber of pts2 inlier points: ", +len(pts2))
    # Find epilines corresponding to points in right image (second image) and
    # drawing its lines on left image
    lines1 = cv.computeCorrespondEpilines(pts2.reshape(-1, 1, 2), 2, F)
    lines1 = lines1.reshape(-1, 3)
    img5, img6 = drawlines(img1, img2, lines1, pts1, pts2)

    # Find epilines corresponding to points in left image (first image) and
    # drawing its lines on right image
    lines2 = cv.computeCorrespondEpilines(pts1.reshape(-1, 1, 2), 1, F)
    lines2 = lines2.reshape(-1, 3)
    img3, img4 = drawlines(img2, img1, lines2, pts2, pts1)

    plt.subplot(121), plt.imshow(img5)
    plt.subplot(122), plt.imshow(img3)
    plt.show()
```