Repository: opencv
Issue: Problem with cv2.triangulatePoints
Labels: bug,category: python bindings,category: calib3d

### System Information

// Example for python user
OpenCV Version: 4.8.1
Operation System MacBook Pro, M1 Pro, macOS Monterey 12.5.1
Python version: 3.11.6

### Detailed description

I encountered two problems using cv2.triagulatedPoints() regarding datatypes:

a) If either points1 or points2 arguments contain elements of type int, or float16 instead of float or np.float64, then the execution of the method will result in memory corruption as seen here:

python3 triangulation_problem_minimal_example.py
Python(18412,0x102efc580) malloc: Heap corruption detected, free list is damaged at 0x600003b29ad0
*** Incorrect guard value: 3436605322615303626
Python(18412,0x102efc580) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      python3 triangulation_problem_minimal_example.py

b) If either there projection_matrix1 or projection_matrix2 contain elements of type int, or float16 instead of float or np.float64, then the execution of the method will result in non-deterministic results, as here:

Execution 1 output:

Triangulated 3D points
[[ 1.65432616e+04  2.52449782e+04 -6.09420163e+04]
 [-8.96032626e+19 -1.15577024e+19  1.34218479e+19]
 [-6.53786864e+19 -9.62109348e+18  1.27982918e+19]]
 
 Execution 2 output:
 Triangulated 3D points
[[ 1.65432616e+004  2.52449782e+004 -6.09420163e+004]
 [-4.21918487e+226 -5.44222181e+225  6.31999953e+225]
 [-4.44215232e+263 -6.53704824e+262  8.69579441e+262]]

Execution 3 output:

Triangulated 3D points
[[ 1.65432616e+04  2.52449782e+04 -6.09420163e+04]
 [-1.63136122e+00 -1.40321303e+00 -3.20917197e+00]
 [-2.51931305e+00 -1.61646291e+00 -2.82647260e+00]]
 
 
 I would argue that there might be a problem with missing type conversion. If there is the intension to avoid type conversation, maybe at least the types could be asserted and an exception thrown so that it is clear that the problem arises from type mismatch. That would make debugging the issue much easier.
 
Hope this helps!
 



### Steps to reproduce

```
#
# tringulation_problem_minimal_example.py
# 2023-10-12
#

import cv2
import numpy as np 


def failing_code1():
    '''
        This code will result immediate in a memory corruption error.

        Problem here is that each element of image_point1 and image_point2 are of type np.int.
    '''
    projection_matrix1 = np.array([[1, 0, 0, 0],
                                    [0, 1, 0, 0],
                                    [0, 0, 1, 0]])
    
    projection_matrix1 = np.array(projection_matrix1, dtype=np.float64)

    projection_matrix2 = np.array([[0.8, 0.2, 0.3, -1],
                                    [0.1, 0.9, 0.4, 2],
                                    [0.2, 0.1, 0.9, 3]])

    # Create 3 arbitrary points
    image_points1 = np.array([[10*i, 2*i] for i in range(3)])  # Image point in the first camera's image
    image_points2 = np.array([[10*i, 2*i] for i in range(3)]) # Image point in the second camera's image

    # Triangulate the 3D point
    homogeneous_points = cv2.triangulatePoints(projection_matrix1, projection_matrix2, image_points1.T, image_points2.T)

    print(homogeneous_points)

    cartesian_points = (homogeneous_points[:3, :] / homogeneous_points[3, :]).T
    
    print("Triangulated 3D points")
    print(cartesian_points)


def failing_code2():
    '''
        This code will not fail on a run and produce a similar result if run several times within the same execution, 
        i.e. for example in a for loop (for i in range(10): failing_code(), would give 10 times the same output)

        However, if the program is executed 10 times, it can sporadically produce different results on executions.

        Problem here is that each element of projection_matrix1 is of type np.int.
    '''
    projection_matrix1 = np.array([[1, 0, 0, 0],
                                    [0, 1, 0, 0],
                                    [0, 0, 1, 0]])

    projection_matrix2 = np.array([[0.8, 0.2, 0.3, -1],
                                    [0.1, 0.9, 0.4, 2],
                                    [0.2, 0.1, 0.9, 3]])

    # Create 3 arbitrary points
    image_points1 = np.array([[10.0*i, 2.0*i] for i in range(3)])  # Image point in the first camera's image
    image_points2 = np.array([[10.0*i, 2.0*i] for i in range(3)]) # Image point in the second camera's image

    # Triangulate the 3D point
    homogeneous_points = cv2.triangulatePoints(projection_matrix1, projection_matrix2, image_points1.T, image_points2.T)

    cartesian_points = (homogeneous_points[:3, :] / homogeneous_points[3, :]).T
    
    print("Triangulated 3D points")
    print(cartesian_points)

def working_code1():
    '''
        This code works, as projection_matrix1 is converted to dtype=np.float64 and image_points1 and image_point2 entries are initialized as type np.float. 
    '''
    projection_matrix1 = np.array([[1, 0, 0, 0],
                                    [0, 1, 0, 0],
                                    [0, 0, 1, 0]])
    
    projection_matrix1 = np.array(projection_matrix1, dtype=np.float64)

    projection_matrix2 = np.array([[0.8, 0.2, 0.3, -1],
                                    [0.1, 0.9, 0.4, 2],
                                    [0.2, 0.1, 0.9, 3]])

    # Create 3 arbitrary points
    image_points1 = np.array([[10.0*i, 2.0*i] for i in range(3)])  # Image point in the first camera's image
    image_points2 = np.array([[10.0*i, 2.0*i] for i in range(3)]) # Image point in the second camera's image

    # Triangulate the 3D point
    homogeneous_points = cv2.triangulatePoints(projection_matrix1, projection_matrix2, image_points1.T, image_points2.T)

    cartesian_points = (homogeneous_points[:3, :] / homogeneous_points[3, :]).T
    
    print("Triangulated 3D points")
    print(cartesian_points)

def working_code2():
    '''
        This code works, as projection_matrix1 is converted to dtype=np.float64 and image_points1 and image_point2 entries are initialized as type np.float. 
    '''
    projection_matrix1 = np.array([[1, 0, 0, 0],
                                    [0, 1, 0, 0],
                                    [0, 0, 1, 0]])
    
    projection_matrix1 = np.array(projection_matrix1, dtype=np.float64)

    projection_matrix2 = np.array([[0.8, 0.2, 0.3, -1],
                                    [0.1, 0.9, 0.4, 2],
                                    [0.2, 0.1, 0.9, 3]])

    # Create 3 arbitrary points
    image_points1 = np.array([[10*i, 2*i] for i in range(3)])  # Image point in the first camera's image
    image_points2 = np.array([[10*i, 2*i] for i in range(3)]) # Image point in the second camera's image

    image_points1 = np.array(image_points1, dtype=np.float64)
    image_points2 = np.array(image_points2, dtype=np.float64)

    # Triangulate the 3D point
    homogeneous_points = cv2.triangulatePoints(projection_matrix1, projection_matrix2, image_points1.T, image_points2.T)

    cartesian_points = (homogeneous_points[:3, :] / homogeneous_points[3, :]).T
    
    print("Triangulated 3D points")
    print(cartesian_points)



if __name__ == '__main__':
    # working_code1()
    # working_code2()
    # failing_code1()
    failing_code2()

```

### Issue submission checklist

- [X] I report the issue, it's not a question
- [X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
- [X] I updated to the latest OpenCV version and the issue is still there
- [X] There is reproducer code and related data files (videos, images, onnx, etc)