Repository: kornia/kornia

Issue:
warp_perspective produced wrong results with integer input
### Describe the bug

got a runtime error when I test warp_perspective

### Reproduction steps

run
```bash


def test_warp_perspective()        :
    import kornia as K
    import torch

    input = torch.zeros(1,3,1000,1000).cuda()

    print(input.shape)

    h, w = 64, 128  # destination size
    points_src = torch.tensor([[
    [100, 100], [200, 100], [200, 200], [100, 200],
    ]])

    # points_src = torch.tensor([[
    # [125., 150.], [562., 40.], [562., 282.], [54., 328.],
    # ]])

    points_dst = torch.tensor([[
        [0., 0.], [w - 1., 0.], [w - 1., h - 1.], [0., h - 1.],
    ]])
    print(points_src, points_dst)
    # compute perspective transform
    M: torch.tensor = K.get_perspective_transform(points_src, points_dst)

    print(input.shape, M)
    # warp the original image by the found transform
    img_warp: torch.tensor = K.warp_perspective(input, M.cuda(), dsize=(h, w))
    print(img_warp.shape)

if __name__ == "__main__":
    test_warp_perspective()


```
  File "test2.py", line 33, in <module>
    test_warp_perspective()
  File "test2.py", line 29, in test_warp_perspective
    img_warp: torch.tensor = K.warp_perspective(input, M.cuda(), dsize=(h, w))
  File "/usr/local/lib/python3.6/dist-packages/kornia/geometry/transform/imgwarp.py", line 96, in warp_perspective
    dst_norm_trans_src_norm: torch.Tensor = normalize_homography(M, (H, W), (h_out, w_out))  # Bx3x3
  File "/usr/local/lib/python3.6/dist-packages/kornia/geometry/transform/homography_warper.py", line 378, in normalize_homography
    src_pix_trans_src_norm = _torch_inverse_cast(src_norm_trans_src_pix)
  File "/usr/local/lib/python3.6/dist-packages/kornia/utils/helpers.py", line 50, in _torch_inverse_cast
    return torch.inverse(input.to(dtype)).to(input.dtype)
RuntimeError: inverse_cuda: For batch 0: U(1,1) is zero, singular U.
```
```


### Expected behavior

no RuntimeError

### Environment

```shell
>>> torch.__version__
'1.9.1+cu102'
>>> kornia.__version__
'0.5.11'
```
```


### Additional context

_No response_


Discussion:
my mistake.  points_src  should be float
```python
points_src = torch.tensor([[
    [100., 100.], [200., 100.], [200., 200.], [100., 200.],
    ]])
```
ignore this. I was wrong.
~~~~
@hal-314 Can you take this into account in #1304? Should we somehow convert it internally?
@shijianjian I think the problem here was different - more on having collinear points where svd cannot find a a proper solution
Ah, I have checked the code. It will convert the dtypes automatically to ```float32``` and ```float64```. Then I am confused that why floating points worked but not integers.

@edgarriba I did not get it. How could floating points effect on collinear points?

https://github.com/kornia/kornia/blob/1686165ef37f0b6b55290933fa6d64d7137712c7/kornia/utils/helpers.py#L88-L102
If I'm not mistaken the original error: ` RuntimeError: linalg_solve: For batch 0: U(8,8) is zero, singular U.` was because the system doesn't have solution not because of the data type. /cc @ducha-aiki 
> If I'm not mistaken the original error: ` RuntimeError: linalg_solve: For batch 0: U(8,8) is zero, singular U.` was because the system doesn't have solution not because of the data type. /cc @ducha-aiki

after fixing it, it also has the error:

```
RuntimeError: inverse_cuda: For batch 0: U(1,1) is zero, singular U.
```

I update the error message now and sorry for the confusion
get_perspective_transform got wrong result when points_src is type of int
```python
torch.int64 tensor([[[   1,    0, -127],
         [   0,    0,  -63],
         [   0,    0,    1]]])
```

when points_src is type of float: the result seems normal
```
         torch.float32 tensor([[[ 1.2700e+00,  0.0000e+00, -1.2700e+02],
         [-8.3447e-09,  6.3000e-01, -6.3000e+01],
         [ 5.0144e-10, -1.0169e-10,  1.0000e+00]]])

```

6.3000e-01 vs 0


now it makes more sense - we should investigate that and add corner case tests for that function.

For the integer case - it's easy to add a condition check whether inputs are in floating point.
@edgarriba  what is the decision here? Throw away integer and to only allow dtypes in [float16, float32, float64]?
the solver only works with floating points similarly to other library functions.
I would say that we should raise an error to the user saying that if input is not floating point then the method won't work as expected.
@ShiyangZhang Mind sending the PR for raising an error?
not this time
> get_perspective_transform got wrong result when points_src is type of int
> 
> ```python
> torch.int64 tensor([[[   1,    0, -127],
>          [   0,    0,  -63],
>          [   0,    0,    1]]])
> ```
> 
> when points_src is type of float: the result seems normal
> 
> ```
>          torch.float32 tensor([[[ 1.2700e+00,  0.0000e+00, -1.2700e+02],
>          [-8.3447e-09,  6.3000e-01, -6.3000e+01],
>          [ 5.0144e-10, -1.0169e-10,  1.0000e+00]]])
> ```
> 
> 6.3000e-01 vs 0

@edgarriba @shijianjian I believe the issue is out [here](https://github.com/kornia/kornia/blob/master/kornia/geometry/transform/imgwarp.py#L263), in the case of an integer tensor being passed `M` would also have the `dtype` of `torch.int64`. Although `X` would be of type `torch.float32`. When we run `
M[..., :8] = torch.squeeze(X, dim=-1)` 
`X` would be cast to `torch.int64` and then stored back to `M`. Hence I think as a correction we should replace the line 
`M = torch.ones(batch_size, 9, device=src.device, dtype=src.dtype)` with
`M = torch.ones(batch_size, 9, device=src.device, dtype=X.dtype)`.
By doing this I find the result to be the same in case of int and float input, 
Let me know if this sounds good
@anirudhb11 looks like you found it - Please, send a PR to fix.

Resolving patch:
diff --git a/kornia/geometry/transform/imgwarp.py b/kornia/geometry/transform/imgwarp.py
--- a/kornia/geometry/transform/imgwarp.py
+++ b/kornia/geometry/transform/imgwarp.py
@@ -219,6 +219,9 @@ def get_perspective_transform(src, dst):
     if not isinstance(dst, torch.Tensor):
         raise TypeError(f"Input type is not a torch.Tensor. Got {type(dst)}")
 
+    if not src.dtype == dst.dtype:
+        raise TypeError(f"Source data type {src.dtype} must match Destination data type {dst.dtype}")
+
     if not src.shape[-2:] == (4, 2):
         raise ValueError(f"Inputs must be a Bx4x2 tensor. Got {src.shape}")
 


Test patch:
diff --git a/.github/workflows/tests_cpu.yml b/.github/workflows/tests_cpu.yml
--- a/.github/workflows/tests_cpu.yml
+++ b/.github/workflows/tests_cpu.yml
@@ -5,7 +5,7 @@ on:
     branches: [master]
   pull_request:
     branches: [master]
-    types: [open, review_requested, ready_for_review]
+    types: [open, reopened, synchronize, review_requested, ready_for_review]
   schedule:
     - cron: "0 4 * * *"
 
diff --git a/test/geometry/transform/test_imgwarp.py b/test/geometry/transform/test_imgwarp.py
--- a/test/geometry/transform/test_imgwarp.py
+++ b/test/geometry/transform/test_imgwarp.py
@@ -279,6 +279,28 @@ def test_crop(self, batch_size, channels, device, dtype):
         patch_warped = kornia.geometry.warp_perspective(patch, dst_trans_src, (dst_h, dst_w))
         assert_close(patch_warped, expected)
 
+    @pytest.mark.parametrize("batch_size", [1, 5])
+    def test_crop_src_dst_type_mismatch(self, device, dtype, batch_size):
+        # generate input data
+        src_h, src_w = 3, 3
+        dst_h, dst_w = 3, 3
+
+        # [x, y] origin
+        # top-left, top-right, bottom-right, bottom-left
+        points_src = torch.tensor(
+            [[[0, 0], [0, src_w - 1], [src_h - 1, src_w - 1], [src_h - 1, 0]]], device=device, dtype=torch.int64
+        )
+
+        # [x, y] destination
+        # top-left, top-right, bottom-right, bottom-left
+        points_dst = torch.tensor(
+            [[[0, 0], [0, dst_w - 1], [dst_h - 1, dst_w - 1], [dst_h - 1, 0]]], device=device, dtype=dtype
+        )
+
+        # compute transformation between points
+        with pytest.raises(TypeError):
+            kornia.geometry.get_perspective_transform(points_src, points_dst).expand(batch_size, -1, -1)
+
     def test_crop_center_resize(self, device, dtype):
         # generate input data
         dst_h, dst_w = 4, 4