魚眼レンズの補正

# 魚眼レンズの補正

## はじめに

研究で使用しているドローンが魚眼レンズなため、取得した画像を補正する必要が出てきた。

参照[1]から一般的な補正は、`rectilinear補正`らしく、opencvでできるみたいなので触ってみた。

## 環境構築

pythonのopencvモジュール`cv2`には魚眼レンズのキャリブレーションモジュールがない

そのため、`opencv c++`の環境を構築する。

> でも`opencv3系`でやってる人がいたのでやっぱりそっちでやった。参照[4]

キャリブレーションに必要なチェッカー柄をA4でコピー。


![チェッカーボード-1024x717.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/423218/25eee79e-4a56-2995-045c-afc376b91716.png)


魚眼カメラで様々な角度からチェッカー柄を撮影する（20枚程度）。

## 実行

```python:calibration.py
import cv2
# assert cv2.__version__[0] == '3', 'The fisheye module requires opencv version >= 3.0.0'
import numpy as np
import os
import glob

CHECKERBOARD = (6,9)

print(CHECKERBOARD)
subpix_criteria = (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 0.1)
calibration_flags = cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC+cv2.fisheye.CALIB_CHECK_COND+cv2.fisheye.CALIB_FIX_SKEW
objp = np.zeros((1, CHECKERBOARD[0]*CHECKERBOARD[1], 3), np.float32)
objp[0,:,:2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)
_img_shape = None
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
images = glob.glob('source/*.jpg')
for fname in images:
    img = cv2.imread(fname)
    if _img_shape == None:
        _img_shape = img.shape[:2]
    else:
        assert _img_shape == img.shape[:2], "All images must share the same size."
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    # Find the chess board corners
    ret, corners = cv2.findChessboardCorners(gray, CHECKERBOARD, cv2.CALIB_CB_ADAPTIVE_THRESH+cv2.CALIB_CB_FAST_CHECK+cv2.CALIB_CB_NORMALIZE_IMAGE)
    # If found, add object points, image points (after refining them)
    if ret == True:
        objpoints.append(objp)
        cv2.cornerSubPix(gray,corners,(3,3),(-1,-1),subpix_criteria)
        imgpoints.append(corners)

print('img end')
N_OK = len(objpoints)
K = np.zeros((3, 3))
D = np.zeros((4, 1))
rvecs = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(N_OK)]
tvecs = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(N_OK)]
rms, _, _, _, _ = \
    cv2.fisheye.calibrate(
        objpoints,
        imgpoints,
        gray.shape[::-1],
        K,
        D,
        rvecs,
        tvecs,
        calibration_flags,
        (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
    )
print("Found " + str(N_OK) + " valid images for calibration")
print("DIM=" + str(_img_shape[::-1]))
print("K=np.array(" + str(K.tolist()) + ")")
print("D=np.array(" + str(D.tolist()) + ")")

```

実行すると3つのパラメータが出力される。
DIM：画角
K：半径方向の歪み係数
D：円周方向の歪み係数

```python:undistort.py
import numpy as np
import cv2
import sys
import os

# You should replace these 3 lines with the output in calibration step
DIM=##
K=##
D=##

def undistort(img_path):
    img = cv2.imread(img_path)
    h,w = img.shape[:2]
    map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), K, DIM, cv2.CV_16SC2)
    undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)
    # cv2.imshow("undistorted", undistorted_img)
    # cv2.waitKey(0)
    # cv2.destroyAllWindows()
    cv2.imwrite('out_' + os.path.basename(img_path), undistorted_img)
if __name__ == '__main__':
    for p in sys.argv[1:]:
        undistort(p)
```

実行例

![Bebop2_20201027162227+0900.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/423218/63f99ee4-b2d4-3b08-9c5a-aef76c4f3eb6.jpeg)

![out_Bebop2_20201027162227+0900.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/423218/36671323-067d-a230-a987-04794c704f73.jpeg)



## 参照

1. [魚眼カメラで物体検出するなら歪み補正いる？ - Qiita](https://qiita.com/chin_self_driving_car/items/1efcb36d844751213d51)

2. [GitHub - opencv/opencv: Open Source Computer Vision Library](https://github.com/opencv/opencv)

3. [macでC++のOpenCVを動かす。 - Qiita](https://qiita.com/uyuutosa/items/c55e4a13a1e388ae2e8f)

4. [Calibrate fisheye lens using OpenCV — part 1 \| by Kenneth Jiang | Medium](https://medium.com/@kennethjiang/calibrate-fisheye-lens-using-opencv-333b05afa0b0)