Repository: opencv
Issue: BFMatcher SigSegv (OpenCL implementation)
Labels: bug,priority: normal,affected: 3.4,category: ocl,category: t-api

My application very rarely segfaults around BFMatching.knnMatch (OpenCL via transparent API).
After investigation I created sample code that also segfaults:

``` cpp
#include <opencv2/features2d.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/opencv.hpp>
#include <opencv2/core/ocl.hpp>
#include <vector>
#include <iostream>
#include <stdio.h>
#include <tuple>

using namespace std;
using namespace cv;

void print_ocl_device_name() {
    vector<ocl::PlatformInfo> platforms;
    ocl::getPlatfomsInfo(platforms);

    for (size_t i = 0; i < platforms.size(); i++) {
        const cv::ocl::PlatformInfo* platform = &platforms[i];
        cout << "Platform Name: " << platform->name().c_str() << "\n";

        for (int j = 0; j < platform->deviceNumber(); j++) {
            cv::ocl::Device current_device;
            platform->getDevice(current_device, j);
            int deviceType = current_device.type();
            cout << "Device " << j << ": " << current_device.name() << "\n";
        }
    }

    ocl::Device default_device = ocl::Device::getDefault();
    cout << "Used device: " << default_device.name() << "\n";
}

int main(void) {
    print_ocl_device_name();

    RNG r(239);
    Mat desc1(5100, 64, CV_32F);
    Mat desc2(5046, 64, CV_32F);
    r.fill(desc1, RNG::UNIFORM, Scalar::all(0.0), Scalar::all(1.0));
    r.fill(desc2, RNG::UNIFORM, Scalar::all(0.0), Scalar::all(1.0));

    for (int i = 0; i < 1000 * 1000; i++) {
        BFMatcher matcher(NORM_L2);

        UMat udesc1, udesc2;
        desc1.copyTo(udesc1);
        desc2.copyTo(udesc2);
        vector< vector<DMatch> > nn_matches;

        matcher.knnMatch(udesc1, udesc2, nn_matches, 2);

        printf("%d\n", i);
    }

    return 0;
}
```

**Note**: this happens very rare. I see it after tens of thousands of iterations (tens of minutes on fast GPU).
## Environment

OpenCV version: **3.0.0**

I have segfaults on Nvidia GTX 680 (**OpenCL 1.1**):

```
Device name: GeForce GTX 680
Nvidia driver verison: 331.113
Platform Version: OpenCL 1.1 CUDA 6.0.1
```

The same behaivour on Nvidia GTX 580 with 352.30 driver (**OpenCL 1.1**).

On CPU device (not CPU implementation, here I mean CPU as OpenCL device) I was trying - but for now there is no segfaults. But I think this is only because of speed difference between CPU and GPU. (On GPU it takes tens of minutes or even more - CPU slower up to to one or two orders). Or these can be explained driver implementation details, or the reason, that CPU RAM is the same that the Host RAM.

**GDB backtraces**:

```
Thread 7 (Thread 0x7f64ad44a700 (LWP 2212)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x00007f64b1784a7e in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#2  0x00007f64b0ffe548 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#3  0x00007f64b17872d9 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#4  0x00007f64b2dad0a5 in start_thread (arg=0x7f64ad44a700) at pthread_create.c:309
#5  0x00007f64b4943cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 6 (Thread 0x7f64acc49700 (LWP 2213)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x00007f64b1784a7e in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#2  0x00007f64b0ffe548 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#3  0x00007f64b17872d9 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#4  0x00007f64b2dad0a5 in start_thread (arg=0x7f64acc49700) at pthread_create.c:309
#5  0x00007f64b4943cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 5 (Thread 0x7f64adc4b700 (LWP 2211)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x00007f64b1784a7e in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#2  0x00007f64b0ffe548 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#3  0x00007f64b17872d9 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#4  0x00007f64b2dad0a5 in start_thread (arg=0x7f64adc4b700) at pthread_create.c:309
#5  0x00007f64b4943cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 4 (Thread 0x7f64b6209780 (LWP 2208)):
#0  0x00007f64b48cc774 in __GI___libc_malloc (bytes=32) at malloc.c:2903
#1  0x00007f64b4e84688 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x00007f64b5d8b42e in allocate (this=0x26997e0, __n=<error reading variable: Cannot access memory at address 0xfffffffffffffcc8>)
    at /usr/include/c++/4.9/ext/new_allocator.h:104
#3  allocate (__a=..., __n=<error reading variable: Cannot access memory at address 0xfffffffffffffcc8>) at /usr/include/c++/4.9/ext/alloc_traits.h:182
#4  _M_allocate (this=0x26997e0, __n=<error reading variable: Cannot access memory at address 0xfffffffffffffcc8>)
    at /usr/include/c++/4.9/bits/stl_vector.h:170
#5  _M_allocate_and_copy<cv::DMatch*> (this=0x26997e0, __last=0x0, __first=0x0, 
    __n=<error reading variable: Cannot access memory at address 0xfffffffffffffcc8>) at /usr/include/c++/4.9/bits/stl_vector.h:1224
#6  std::vector<cv::DMatch, std::allocator<cv::DMatch> >::reserve (this=this@entry=0x26997e0, __n=__n@entry=2) at /usr/include/c++/4.9/bits/vector.tcc:75
#7  0x00007f64b5d913b0 in ocl_knnMatchConvert (compactResult=<optimized out>, matches=std::vector of length 4825, capacity 5100 = {...}, distance=..., 
    trainIdx=...) at /home/polarnick/libraries/opencv/opencv_3.0.0/modules/features2d/src/matchers.cpp:240
#8  ocl_knnMatchDownload (compactResult=<optimized out>, matches=std::vector of length 4825, capacity 5100 = {...}, distance=..., trainIdx=...)
    at /home/polarnick/libraries/opencv/opencv_3.0.0/modules/features2d/src/matchers.cpp:270
#9  cv::ocl_knnMatch (query=..., _train=..., matches=std::vector of length 4825, capacity 5100 = {...}, k=k@entry=2, dstType=<optimized out>, 
    compactResult=compactResult@entry=false) at /home/polarnick/libraries/opencv/opencv_3.0.0/modules/features2d/src/matchers.cpp:713
#10 0x00007f64b5d938ac in cv::BFMatcher::knnMatchImpl (this=0x262b720, _queryDescriptors=..., matches=std::vector of length 4825, capacity 5100 = {...}, 
    knn=2, _masks=..., compactResult=false) at /home/polarnick/libraries/opencv/opencv_3.0.0/modules/features2d/src/matchers.cpp:786
#11 0x00007f64b5d8bade in cv::DescriptorMatcher::knnMatch (this=0x262b720, queryDescriptors=..., matches=std::vector of length 4825, capacity 5100 = {...}, 
    knn=knn@entry=2, masks=..., compactResult=compactResult@entry=false)
    at /home/polarnick/libraries/opencv/opencv_3.0.0/modules/features2d/src/matchers.cpp:630
#12 0x00007f64b5d8c68c in cv::DescriptorMatcher::knnMatch (this=<optimized out>, queryDescriptors=..., trainDescriptors=..., 
    matches=std::vector of length 4825, capacity 5100 = {...}, knn=2, mask=..., compactResult=false)
    at /home/polarnick/libraries/opencv/opencv_3.0.0/modules/features2d/src/matchers.cpp:577
#13 0x0000000000401c13 in main () at bfmatcher_segfault.cpp:50

Thread 3 (Thread 0x7f64af9ee700 (LWP 2209)):
#0  0x00007f64b493984d in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007f64b1785373 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#2  0x00007f64b1135f15 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#3  0x00007f64b17872d9 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#4  0x00007f64b2dad0a5 in start_thread (arg=0x7f64af9ee700) at pthread_create.c:309
#5  0x00007f64b4943cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 2 (Thread 0x7f64ae44c700 (LWP 2210)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x00007f64b1784a7e in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#2  0x00007f64b0ffe548 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#3  0x00007f64b17872d9 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#4  0x00007f64b2dad0a5 in start_thread (arg=0x7f64ae44c700) at pthread_create.c:309
#5  0x00007f64b4943cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 1 (Thread 0x7f64a7fff700 (LWP 2214)):
#0  0x00007f64b0ffdae2 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#1  0x00007f64b0ffe514 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#2  0x00007f64b17872d9 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
#3  0x00007f64b2dad0a5 in start_thread (arg=0x7f64a7fff700) at pthread_create.c:309
#4  0x00007f64b4943cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
```

**Valgrind** can't help because of too big slow down.
**AddressSanitizer** just fails on startup with SigSegv, this seems to be bad combination of ASan and OpenCL.
## Workaround

It seems, that explicit sync needed after calculating and before result downloading.
If I change `return k.run(2, globalSize, localSize, false);` to `return k.run(2, globalSize, localSize, true);`, i.e. pass sync=true - I stops encounter SigSegv.
This fix must be applied in:
- ocl_matchSingle: https://github.com/Itseez/opencv/blob/3.0.0/modules/features2d/src/matchers.cpp#L114  
- ocl_knnMatchSingle: https://github.com/Itseez/opencv/blob/3.0.0/modules/features2d/src/matchers.cpp#L214
- ocl_radiusMatchSingle: https://github.com/Itseez/opencv/blob/3.0.0/modules/features2d/src/matchers.cpp#L327
## Results

clEnqueueMapBuffer fetches data in concurrent with unfinished kernel, and this is **undefined behaviour**.

May be `sync=false` is just randomly passed param? While line at https://github.com/Itseez/opencv/blob/3.0.0/modules/core/src/ocl.cpp#L3382 should change `sync` to `true`? But this is not so - I checked, this `if` branch does not executes in my scenario. The reason is in https://github.com/Itseez/opencv/blob/3.0.0/modules/core/src/ocl.cpp#L3119 - `m.u->tempUMat()` fails that if-branch. Because UMatData.flags=3 for all Kernel.addUMat(UMat) calls, while TEMP_UMAT=8.

What is TEMP_UMAT? Are those umats should be tempUMat? And even if so - all three Kernel.run in matchers.cpp should be called with sync=true, because there are no async behaviour.

It seems that such bug with OpenCL methods result fetching can be in a lot of places, can you check them please? It is quite dangerous bug. (I checked LKOptFlow - there Kernel.run called with sync=true)

Also, there are no bugfixes bug-porting to 3.0.0 version, because there is no 3.0.0 stable branch. Why so? 3.0.0 branch like 2.4 branch seems to be useful.