Repository: opencv
Issue: Memory leak in UMat on Vega3 GPU
Labels: bug,category: ocl

### System Information

I'm using just the latest debian bullseye, so, in C++, it's a bit outdated (but tested in 4.7.0 in python):

OpenCV: 4.5.1
OS: Debian Bullseye (11)
Compiler: gcc Debian 10.2.1-6
Mesa 20.3.5

And to confirm it's not fixed in a newer version of OpenCV, I did it in python too:

Python version: 3.9.2
Python opencv version: 4.7.0

### Detailed description

So, upfront, my "GPU" is a Vega3 (AMD integrated into CPU thing), it doesn't really have OpenCL support, though clinfo prints stuff claiming it's there, I know it not to be there, I'm pretty sure clBuildProgram() always fails on this "GPU". From my point of view, OpenCV should be falling back to SW and using the CPU.

When clBuildProgram() is run, I always see this error:

> OpenCL program build log: imgproc/resize
Status -11: CL_BUILD_PROGRAM_FAILURE
-D INTER_NEAREST -D T=uchar3 -D T1=uchar -D cn=3 -D AMD_DEVICE
fatal error: cannot open file '/usr/lib/clc/gfx909-amdgcn-mesa-mesa3d.bc': No such file or directory

That message is always printed before it starts to leak

However, instead, it seems that UMat.release() and related leaks the data backing it instead of releasing it. I found this originally because my OpenCV program was just sucking up memory endlessly until the system crashed. Debugging it has identified UMat.release() and related functions as the culprit, mostly with the help of Valgrind (which reports it as possibly lost, but I can assure it's causing memory problems). 

Here are some examples of the valgrind output I see:

```
==332118== 1,440,000 bytes in 10,000 blocks are possibly lost in loss record 2,678 of 2,681
==332118==    at 0x4838DEF: operator new(unsigned long) (vg_replace_malloc.c:342)
==332118==    by 0x55A1A3C: _M_clone (std_function.h:161)
==332118==    by 0x55A1A3C: _M_manager (std_function.h:196)
==332118==    by 0x55A1A3C: std::_Function_handler<void (clover::event&), std::function<void (clover::event&)> (anonymous namespace)::soft_copy_op<void*, clover::buffer*>(clover::command_queue&, void*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, clover::buffer*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&)::{lambda(clover::event&)#1}>::_M_manager(std::_Any_data&, std::_Function_handler<void (clover::event&), std::function<void (clover::event&)> (anonymous namespace)::soft_copy_op<void*, clover::buffer*>(clover::command_queue&, void*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, clover::buffer*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&)::{lambda(clover::event&)#1}> const&, std::_Manager_operation) (std_function.h:283)
==332118==    by 0x55AE8AF: function (std_function.h:593)
==332118==    by 0x55AE8AF: clover::event::event(clover::context&, clover::ref_vector<clover::event> const&, std::function<void (clover::event&)>, std::function<void (clover::event&)>) (event.cpp:31)
==332118==    by 0x55AE9FF: clover::hard_event::hard_event(clover::command_queue&, unsigned int, clover::ref_vector<clover::event> const&, std::function<void (clover::event&)>) (event.cpp:130)
==332118==    by 0x55A3ED2: clover::intrusive_ref<clover::hard_event> clover::create<clover::hard_event, clover::command_queue&, int, clover::ref_vector<clover::event>&, std::function<void (clover::event&)> >(clover::command_queue&, int&&, clover::ref_vector<clover::event>&, std::function<void (clover::event&)>&&) [clone .isra.0] (pointer.hpp:241)
==332118==    by 0x55A46AE: clEnqueueReadBuffer (transfer.cpp:343)
==332118==    by 0x4A256A2: cv::ocl::OpenCLAllocator::map(cv::UMatData*, cv::AccessFlag) const (ocl.cpp:5706)
==332118==    by 0x4A735A6: cv::UMat::getMat(cv::AccessFlag) const (umatrix.cpp:1032)
==332118==    by 0x4A74A0D: cv::UMat::setTo(cv::_InputArray const&, cv::_InputArray const&) (umatrix.cpp:1303)
==332118==    by 0x4A751DA: cv::UMat::operator=(cv::Scalar_<double> const&) (umatrix.cpp:1310)
==332118==    by 0x4A75FA6: cv::UMat::zeros(int, int, int) (umatrix.cpp:1405)
==332118==    by 0x1091CC: main (opencv.cpp:5)
==332118== 
==332118== 1,440,000 bytes in 10,000 blocks are possibly lost in loss record 2,679 of 2,681
==332118==    at 0x4838DEF: operator new(unsigned long) (vg_replace_malloc.c:342)
==332118==    by 0x55A195C: _M_clone (std_function.h:161)
==332118==    by 0x55A195C: _M_manager (std_function.h:196)
==332118==    by 0x55A195C: std::_Function_handler<void (clover::event&), std::function<void (clover::event&)> (anonymous namespace)::soft_copy_op<clover::buffer*, void const*>(clover::command_queue&, clover::buffer*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, void const*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&)::{lambda(clover::event&)#1}>::_M_manager(std::_Any_data&, std::_Function_handler<void (clover::event&), std::function<void (clover::event&)> (anonymous namespace)::soft_copy_op<clover::buffer*, void const*>(clover::command_queue&, clover::buffer*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, void const*, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&, std::array<unsigned long, 3ul> const&)::{lambda(clover::event&)#1}> const&, std::_Manager_operation) (std_function.h:283)
==332118==    by 0x55AE8AF: function (std_function.h:593)
==332118==    by 0x55AE8AF: clover::event::event(clover::context&, clover::ref_vector<clover::event> const&, std::function<void (clover::event&)>, std::function<void (clover::event&)>) (event.cpp:31)
==332118==    by 0x55AE9FF: clover::hard_event::hard_event(clover::command_queue&, unsigned int, clover::ref_vector<clover::event> const&, std::function<void (clover::event&)>) (event.cpp:130)
==332118==    by 0x55A3ED2: clover::intrusive_ref<clover::hard_event> clover::create<clover::hard_event, clover::command_queue&, int, clover::ref_vector<clover::event>&, std::function<void (clover::event&)> >(clover::command_queue&, int&&, clover::ref_vector<clover::event>&, std::function<void (clover::event&)>&&) [clone .isra.0] (pointer.hpp:241)
==332118==    by 0x55A4960: clEnqueueWriteBuffer (transfer.cpp:376)
==332118==    by 0x4A251D4: cv::ocl::OpenCLAllocator::unmap(cv::UMatData*) const (ocl.cpp:5777)
==332118==    by 0x49A3BC1: cv::Mat::release() (matrix.cpp:554)
==332118==    by 0x49A3BDC: cv::Mat::~Mat() (matrix.cpp:483)
==332118==    by 0x4A74A23: cv::UMat::setTo(cv::_InputArray const&, cv::_InputArray const&) (umatrix.cpp:1303)
==332118==    by 0x4A751DA: cv::UMat::operator=(cv::Scalar_<double> const&) (umatrix.cpp:1310)
==332118==    by 0x4A75FA6: cv::UMat::zeros(int, int, int) (umatrix.cpp:1405)
==332118== 
==332118== 2,960,000 bytes in 10,000 blocks are possibly lost in loss record 2,680 of 2,681
==332118==    at 0x4838DEF: operator new(unsigned long) (vg_replace_malloc.c:342)
==332118==    by 0x55A3EB8: clover::intrusive_ref<clover::hard_event> clover::create<clover::hard_event, clover::command_queue&, int, clover::ref_vector<clover::event>&, std::function<void (clover::event&)> >(clover::command_queue&, int&&, clover::ref_vector<clover::event>&, std::function<void (clover::event&)>&&) [clone .isra.0] (pointer.hpp:241)
==332118==    by 0x55A46AE: clEnqueueReadBuffer (transfer.cpp:343)
==332118==    by 0x4A256A2: cv::ocl::OpenCLAllocator::map(cv::UMatData*, cv::AccessFlag) const (ocl.cpp:5706)
==332118==    by 0x4A735A6: cv::UMat::getMat(cv::AccessFlag) const (umatrix.cpp:1032)
==332118==    by 0x4A74A0D: cv::UMat::setTo(cv::_InputArray const&, cv::_InputArray const&) (umatrix.cpp:1303)
==332118==    by 0x4A751DA: cv::UMat::operator=(cv::Scalar_<double> const&) (umatrix.cpp:1310)
==332118==    by 0x4A75FA6: cv::UMat::zeros(int, int, int) (umatrix.cpp:1405)
==332118==    by 0x1091CC: main (opencv.cpp:5)
==332118== 
==332118== 2,960,000 bytes in 10,000 blocks are possibly lost in loss record 2,681 of 2,681
==332118==    at 0x4838DEF: operator new(unsigned long) (vg_replace_malloc.c:342)
==332118==    by 0x55A3EB8: clover::intrusive_ref<clover::hard_event> clover::create<clover::hard_event, clover::command_queue&, int, clover::ref_vector<clover::event>&, std::function<void (clover::event&)> >(clover::command_queue&, int&&, clover::ref_vector<clover::event>&, std::function<void (clover::event&)>&&) [clone .isra.0] (pointer.hpp:241)
==332118==    by 0x55A4960: clEnqueueWriteBuffer (transfer.cpp:376)
==332118==    by 0x4A251D4: cv::ocl::OpenCLAllocator::unmap(cv::UMatData*) const (ocl.cpp:5777)
==332118==    by 0x49A3BC1: cv::Mat::release() (matrix.cpp:554)
==332118==    by 0x49A3BDC: cv::Mat::~Mat() (matrix.cpp:483)
==332118==    by 0x4A74A23: cv::UMat::setTo(cv::_InputArray const&, cv::_InputArray const&) (umatrix.cpp:1303)
==332118==    by 0x4A751DA: cv::UMat::operator=(cv::Scalar_<double> const&) (umatrix.cpp:1310)
==332118==    by 0x4A75FA6: cv::UMat::zeros(int, int, int) (umatrix.cpp:1405)
==332118==    by 0x1091CC: main (opencv.cpp:5)

```

### Steps to reproduce

This is my C++ test program:

```cpp
#include <opencv2/core/mat.hpp>
int main(int argc, char** argv) {
    cv::UMat input;
    for (int i = 0; i < 10000; i++){
      input.zeros(64, 64, CV_8U);
      input.release();
   }
    return 0;
}
```
I ran the above with valgrind to produce the valgrind backtrace


And here is my python test program:

```python
import cv2 as cv
import os, psutil
process = psutil.Process()
img = cv.UMat(64,64, cv.CV_8U)
out = cv.UMat()

for i in range(100000):
    out = cv.resize(img, (64,64))
    #out = img
    if i % 10000 == 0:
        print("Heap: " +  str(process.memory_info().rss/1024/1024) + "MiB")
```

The output of that is this:

```
OpenCL program build log: imgproc/resize
Status -11: CL_BUILD_PROGRAM_FAILURE
-D INTER_LINEAR -D depth=0 -D T=uchar -D T1=uchar -D WT=int -D convertToWT=convert_int -D convertToDT=convert_uchar_sat -D cn=1 -D INTER_RESIZE_COEF_BITS=11 -D AMD_DEVICE
fatal error: cannot open file '/usr/lib/clc/gfx909-amdgcn-mesa-mesa3d.bc': No such file or directory

Heap: 107.15625MiB
Heap: 144.52734375MiB
Heap: 181.13671875MiB
Heap: 217.48828125MiB
Heap: 253.83984375MiB
Heap: 290.19140625MiB
Heap: 326.54296875MiB
Heap: 362.89453125MiB
Heap: 399.24609375MiB
Heap: 435.59765625MiB
```

Some things I noticed about this, switching to CPU always fixes it (so setting the environmental variable OPENCV_OPENCL_DEVICE=disabled or OPENCV_OPENCL_DEVICE=:CPU: fixes it). It does not happen in my VM without a GPU, and it does not happen on my other system with a radeonsi GPU.

### Issue submission checklist

- [X] I report the issue, it's not a question
- [X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
- [X] I updated to the latest OpenCV version and the issue is still there
- [X] There is reproducer code and related data files (videos, images, onnx, etc)