Repository: opencv
Issue: warpAffine: correct coordinate system, documentation and incorrect usage
Labels: category: imgproc,category: documentation,RFC,future

##### System information (version)
Tested with
- OpenCV => 3.4.1
- Operating System / Platform => Linux
- Compiler => gcc

##### Detailed description

After some tests I found out that the coordinate system in warpAffine is translated by 0.5 pixels, in other words the topleft origin pixel area goes from -0.5 to +0.5 on both x and y axis. This is not intuitive and without proper documentation yield to incorrect results. Example code, tutorial and regression test code in OpenCV are using incorrectly the warpAffine function. 

##### Example 1

I want to scale the source image by a factor of 10. Without applying a translation correction the resulting image is moved off by over 5 pixels. 
```.cpp
// define an intuitive scaling matrix: NOT OK. Image is translated
double scale = 10;
cv::Matx23d matrix(scale, 0,       0, 
                   0,     scale,   0);

// the corrected scaling matrix: We have to translate by scale/2-0.5. 
// Not intuitive at all and not documented.
double scale = 10;
cv::Matx23d matrix(scale, 0,       scale/2-0.5), 
                   0,     scale,   scale/2-0.5);

// Apply matrix 
cv::Mat destImage;
cv::Mat image = imread("helloworld.png");
destSize = cv::Size(image.cols*scale, image.rows*scale);
cv::warpAffine(image, destImage, matrix, destSize, cv::INTER_CUBIC, 
      cv::BORDER_CONSTANT, cv::Scalar(255, 255, 255));
cv::imwrite("helloworldScaled.png", destImage);
```

##### Example 2

I want to rotate the image by 45 degrees around the center. If the original image was symmetric then the resulting image must be symmetric too. The center of the image is not (cols/2, rows/2) but (cols/2-0.5, rows/2-0.5)
```.cpp
// Wrong call used almost everywhere. The resulting image is no longer symmetric. 
// Resulting image is translated by 0.5 pixels.
cv::Point2f centerImage(image.cols/2., image.rows/2.);
cv::Matx23d = cv::getRotationMatrix2D(centerImage, 45, 1), cv::INTER_LINEAR);

// Correct call. 
cv::Point2f centerImage(image.cols/2.-0.5, image.rows/2.-0.5);
cv::Matx23d = cv::getRotationMatrix2D(centerImage, 45, 1), cv::INTER_LINEAR);
```
With a rotation of 90 degrees the whole image is translated by 1 pixel in the wrong call. It's a small error but still an error. Note: The problem is not the getRotationMatrix2D function. 

##### Solution 1.

Maintain the -0.5, -0.5 design and update documentation and code to reflect this. Examples of where in OpenCV a correction is needed:

- In the opencv_test code modules/imgproc/perf/perf_warp.cpp there are multiple calls to warpAffine / getRotationMatrix2D using the wrong center of the image. I suppose the test doesn't fail because a certain amount of tolerance is accepted. Still the calls are wrong.
https://github.com/opencv/opencv/blob/master/modules/imgproc/perf/perf_warp.cpp

- The sample code samples/cpp/tutorial_code/ImgTrans/Geometric_Transforms_Demo.cpp uses a wrong call to getRotationMatrix2D.
https://github.com/opencv/opencv/blob/master/samples/cpp/tutorial_code/ImgTrans/Geometric_Transforms_Demo.cpp

- in the tutorial the wrong call to getRotationMatrix2D is used to Rotate by 90 degrees. The resulting image is translated by 1 pixel.
https://docs.opencv.org/3.4.0/da/d6e/tutorial_py_geometric_transformations.html

- More wrong usages can found on opencv.org, not counting the large amount of wrong usages on the web. For a rotation developers typically uses getRotationMatrix2D(image.width/2.f, image.height/2.f, angle, 1) which is the wrong call.

- Actually in my own code as a workaround I apply a 0.5 translation to the matrix before calling warpAffine, so that I can use the (0,0)-(1,1) coordinate system. The fix can be easily computed before using the resulting matrix in warpAffine:
```.cpp
cv::Matx23d fixMatrix(cv::Matx23d matrix)
{
    matrix(0,2) += (matrix(0,0) + matrix(0,1) - 1) / 2;
    matrix(1,2) += (matrix(1,0) + matrix(1,1) - 1) / 2;
    return matrix;
}
```
The fix above could be documented too, for whoever like me prefers to use the (0,0)-(1,1) based coordinate system. Advantages: The matrices have a intuitive definition, especially scaling. I don't need to apply a correction on the getRotationMatrix2D or getAffineTransform. All coordinates are intuitively area based instead of point based and can be shared in different scenarios without the need of a conversion / correction: image pixel coordinates, vector/shape object coordinates, mathematical coordinates.

##### Solution 2.

The amount of incorrect usages even within OpenCV denotes that the design of (-0.5,-0.5) to (0.5,0.5) for the topleft pixel area is not intuitive. A second solution would be to update warpAffine and to use (0,0) to (1,1) as the area for the topleft pixel. Then the sample code, test code and documentation doesn't need to be updated because that would be the intuitive usage of warpAffine.

##### Complete Sample Code showing the issues

```.cpp
#include <opencv2/opencv.hpp>
#include <stdio.h>

cv::Mat createSymmetricTestImage(int width, int height)
{
    cv::Mat image(width, height, CV_8UC3, cv::Scalar(63, 0, 0));
    cv::Point pos1(width/4, height/4);
    cv::Point pos2(width - pos1.x - 1, height - pos1.y - 1);
    cv::rectangle(image, pos1, pos2, cv::Scalar(0, 255, 255), 1);
    return image;
}

cv::Matx23d fixMatrix(cv::Matx23d matrix)
{
    matrix(0,2) += (matrix(0,0) + matrix(0,1) - 1) / 2;
    matrix(1,2) += (matrix(1,0) + matrix(1,1) - 1) / 2;
    return matrix;
}

cv::Matx23d getAffineTransformRotate90(cv::Size size)
{
    cv::Point2f srcTri[3];
    cv::Point2f dstTri[3];
    /// Set your 3 points to calculate the  Affine Transform
    /// Use area based coordinates ranging from 0,0 to width,height
    srcTri[0] = cv::Point2f( 0,0 );
    srcTri[1] = cv::Point2f( size.width, 0 );
    srcTri[2] = cv::Point2f( 0, size.height);
    /// Apply rotation 90 degrees CCW
    dstTri[0] = srcTri[2];
    dstTri[1] = srcTri[0];
    dstTri[2] = cv::Point2f( size.width, size.height);
    /// Get the Affine Transform
    return getAffineTransform( srcTri, dstTri );
}

void doWarpAffine(const std::string fileNameToSave, const cv::Mat &image, cv::Size destSize, const cv::Matx23d &matrix, int interpolation)
{
    printf("%20s, Matrix2x2: [ %9lf, %9lf] [ %9lf, %9lf ], Traslation: [ %9lf, %9lf ]\n", fileNameToSave.c_str(), matrix(0,0), matrix(0, 1), matrix(1,0), matrix(1,1), matrix(0,2), matrix(1,2));
    cv::Mat dest, dest2;
    cv::warpAffine(image, dest, matrix, destSize, interpolation /*| cv::WARP_INVERSE_MAP*/, cv::BORDER_CONSTANT, cv::Scalar(0, 0, 255));
    cv::imwrite(fileNameToSave.c_str(), dest);
}

void doWarpAffineTest(const std::string fileNameToSavePrefix, const cv::Mat &image, cv::Size destSize, const cv::Matx23d &matrix, int interpolation)
{
    cv::Matx23d fixedMatrix = fixMatrix(matrix);
    if (fixedMatrix == matrix) {
        // fixed matrix is equal to original one
        doWarpAffine(fileNameToSavePrefix + ".png",     image, destSize, matrix,      interpolation);
    } else {
        // fixed matrix is different
        doWarpAffine(fileNameToSavePrefix + "Orig.png", image, destSize, matrix,      interpolation);
        doWarpAffine(fileNameToSavePrefix + "Fix.png",  image, destSize, fixedMatrix, interpolation);
    }
}

int main(int, char **)
{
    // create a symmetric Test Image
    cv::Mat image = createSymmetricTestImage(17,17);
    cv::imwrite("SourceImage.png", image);
    cv::Point2f centerImage(image.cols/2., image.rows/2.);
    cv::Point2f originImage(0, 0);
    cv::Size destSize = cv::Size(image.cols, image.rows);

    //------------------------------------------------
    // Test Identity
    cv::Matx23d identityMatrix(1, 0, 0, 0, 1, 0);
    doWarpAffineTest("Identity", image, destSize, identityMatrix, cv::INTER_CUBIC);

    //------------------------------------------------
    // Test Rotaiton
    // 45 degrees Rotation around the center of the image
    doWarpAffineTest("Rotate45", image, destSize, cv::getRotationMatrix2D(centerImage, 45, 1), cv::INTER_LINEAR);
    // 90 degrees Rotation around the center of the image
    doWarpAffineTest("Rotate90", image, destSize, cv::getRotationMatrix2D(centerImage, 90, 1), cv::INTER_LINEAR);

    //------------------------------------------------
    // Test Traslation by 1 pixel
    double traslate = 1;
    destSize = cv::Size(image.cols+traslate, image.rows+traslate);
    cv::Matx23d traslateMatrix(1, 0, traslate, 0, 1, traslate);
    doWarpAffineTest("Traslate", image, destSize, traslateMatrix, cv::INTER_LINEAR);

    //------------------------------------------------
    // Test Scaling. 10x bigger.
    double scale = 10;
    destSize = cv::Size(image.cols*scale, image.rows*scale);
    cv::Matx23d scaleMatrix(scale, 0, 0, 0, scale, 0);
    doWarpAffineTest("Scale", image, destSize, scaleMatrix, cv::INTER_CUBIC);

    //------------------------------------------------
    // Test Shear
    // 
    double shear = 1;
    destSize = cv::Size(image.cols*(1+shear)+0.5, image.rows);
    cv::Matx23d shearMatrix(1, shear, -0.5, 0, 1, 0);
    doWarpAffineTest("Shear", image, destSize, shearMatrix, cv::INTER_CUBIC);

    //------------------------------------------------
    // Test getAffineTransform with Rotate 90 degrees
    destSize = cv::Size(image.cols, image.rows);
    cv::Matx23d warp_mat = getAffineTransformRotate90(cv::Size(image.cols, image.rows));
    doWarpAffine("GetAffine.png", image, destSize, fixMatrix(warp_mat), cv::INTER_LINEAR);

    return 0;
}
```
Scale without correction:
![scaleorig](https://user-images.githubusercontent.com/3232744/41507958-f99d86da-723c-11e8-85ee-601156ab94cf.png)

Scale with correction:
![scalefix](https://user-images.githubusercontent.com/3232744/41507960-049c0a7a-723d-11e8-92af-105a738b26f7.png)

Rotation 45 degrees (typical usage):
![rotate45orig](https://user-images.githubusercontent.com/3232744/41507968-32540cf6-723d-11e8-8463-643896454ad8.png)

Rotation 45 degrees with fix:
![rotate45fix](https://user-images.githubusercontent.com/3232744/41507983-5a8c5bec-723d-11e8-818d-96b72ffaa001.png)

Rotate 90 degrees (typical usage). Image is translated down by 1 pixel:
![rotate90orig](https://user-images.githubusercontent.com/3232744/41507993-7349dd9e-723d-11e8-97d2-e58fb931c986.png)

Rotate 90 degrees with fix:
![rotate90fix](https://user-images.githubusercontent.com/3232744/41508003-b5ce3a8e-723d-11e8-8d9b-766149a4df69.png)