Repository: opencv
Issue: OpenCL infrastructure fragile in multithreaded apps for default/common use cases
Labels: category: core,category: ocl,RFC

OpenCV's default of creating an independent new Queue (cl_command_queue) for every thread leads to a breakdown of OpenCL command execution ordering, command blocking, clFinish, etc. for multithreaded apps in default/common use cases.

It ..is.. possible for apps to not be affected by this OpenCV approach. However, it requires a high-level of knowledge (both OpenCV internals and OpenCL) and likely beyond the typical user of OpenCV. When OpenCV is easily used in multithreaded scenarios of Python or Java, it often leads to chaos and difficult to understand bugs.

I believe these bugs are specific examples of this issue
https://github.com/opencv/opencv/issues/13472
https://github.com/opencv/opencv/issues/13407

#### System information (version)

- OpenCV => 3.x and 4.x
- Operating System / Platform => all
- Compiler => all

#### Detailed description

There is no code in OpenCV which ensures the OpenCL device memory after critical operations like `unmap()` has completed its sync to the device *before* an OpenCL Kernel uses it. `umap()` is used often to support Mat<->UMat conversions. In general, OpenCV assumes there is a single OpenCL command queue and that since `clEnqueueUnmapMemObject()` was first placed in the command queue... that a later call to `clEnqueueNDRangeKernel()` will be placed later in that queue. This assumption is ok with single-threaded apps.

This is not true in multithreaded use.

Default OpenCV behavior is to create a new `ocl::Queue` for every new thread. This is caused by `OpenCLExecutionContext::getCurrent()` which is itself called by almost every `ocl` module command. This means, if there are five threads, there are five independent `cl_command_queue`. With no shared cl_command_queue, that means there is no shared execution order, no shared blocking commands, no shared clFinish, etc.

When thread1 calls clEnqueueUnmapMemObject() and thread2 calls clEnqueueNDRangeKernel(), there is no promise that the OpenCL runtime has completed the Unmap work in the cl_command_queue of thread1 *before* the OpenCL starts work on the clEnqueueNDRangeKernel() in thread2. This is because there was no dependency synchronization across queues (e.g. `clWaitForEvents()`), no single shared queue, etc. https://www.khronos.org/registry/OpenCL/sdk/2.0/docs/man/xhtml/clEnqueueUnmapMemObject.html#id-1.6.5


 . |   | single threaded app |   | multithreaded app | .
-- | -- | -- | -- | -- | --
thread |   | thread 1 |   | thread 1 | thread 2
work |   | …do something… |   | …do something… | …do something…
work |   | unmap() |   | unmap() | …do something…
work |   | …do something… |   | …do something… | kernel::run()
work |   | kernel::run() |   |   |  
work |   |   |   |   |  
----- |   |   |   |   |  
result |   | good, order is guaranteed |   | bad... |  random ordering and no dependency guarantees


This issue is likely the cause of the above two related issues. And likely a catalyst of this code hack...

https://github.com/opencv/opencv/blob/f787c49b539983be20b77f0193cdffac92b85b6f/modules/core/src/ocl.cpp#L5927-L5931

#### Recommendation

I recommend that OpenCV be reliable for typical use cases while also having optional and non-default support for advanced users. In technical details, I can see two options:

1. Use a single cl_command_queue by default.
2. Issue clFinish, clWaitForEvents, clEnqueueSVMFree, and other like APIs to strengthen consistency at important sync points

#### Issue submission checklist

 - [X] I report the issue, it's not a question
 - [X] I checked the problem with documentation, FAQ, open issues,
       forum.opencv.org, Stack Overflow, etc and have not found solution
 - [X] I updated to latest OpenCV version and the issue is still there
 - [X] There is reproducer code (in related bugs) and related data files: videos, images, onnx, etc