Repository: opencv
Issue: SIGSEGV before entering main() when module viz is compiled into OpenCV
Labels: bug,auto-transferred,priority: normal,affected: 3.4,category: viz

Transferred from http://code.opencv.org/issues/3492

```
|| David W.  on 2014-01-20 16:59
|| Priority: Normal
|| Affected: branch 'master' (3.0-dev)
|| Category: viz
|| Tracker: Bug
|| Difficulty: Medium
|| PR: 
|| Platform: x64 / Linux
```
## SIGSEGV before entering main() when module viz is compiled into OpenCV

```
I wanted to try out the new viz module, so i run cmake with '-D WITH_VTK=ON'.
The building process is completed properly, but if i link a program against OpenCV, it crashes with SIGSEGV before entering main().
My debugger stops somewhere in /usr/lib/libQtGui.so.4, like here: http://www.qtcentre.org/threads/54158-OpenCV-Seg-fault

Stacktrace:
@Thread 1 (Thread 0x7ffff7f827c0 (LWP 27951)):
#0  0x00007fffe550631c in ?? () from /usr/lib/libQtGui.so.4
No symbol table info available.
#1  0x00007ffff7deb8da in call_init.part.0 () from /lib64/ld-linux-x86-64.so.2
No symbol table info available.
#2  0x00007ffff7deb9c3 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
No symbol table info available.
#3  0x00007ffff7dde29a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
No symbol table info available.
#4  0x0000000000000002 in ?? ()
No symbol table info available.
#5  0x00007fffffffea42 in ?? ()
No symbol table info available.
#6  0x00007fffffffea6d in ?? ()
No symbol table info available.
#7  0x0000000000000000 in ?? ()
No symbol table info available.
@

I have Qt5 aswell as Qt4 installed on my machine. Cmake outputs:

@--   GUI: 
--     QT 5.x:                      YES (ver 5.2.0)
--     QT OpenGL support:           NO
--     OpenGL support:              NO
--     VTK support:                 YES (ver 5.10.1)
@

Without viz there are no problems. I hope this is enough information.

Looks like viz is linked against the wrong version of Qt?
```
## History
##### David W.  on 2014-01-20 17:06

```
-   Assignee set to Anatoly Baksheev
```
##### David W.  on 2014-01-20 19:16

```
I added '-D WITH_OPENGL=ON' to cmake, so now i get:

@--   GUI: 
--     QT 5.x:                      YES (ver 5.2.0)
--     QT OpenGL support:           YES (Qt5::OpenGL 5.2.0)
--     OpenGL support:              YES (/lib64/libGLU.so /lib64/libGL.so /lib64/libSM.so /lib64/libICE.so /lib64/libX11.so /lib64/libXext.so)
--     VTK support:                 YES (ver 5.10.1)
@

But the problem remains.
```
##### Anatoly Baksheev on 2014-01-20 21:00

```
Thank you for the reporting. The OpenGL flag has no effect there, it is related to another functionality. 

I don't have understanding why it happerns, need to investigate. Any help appreciated. 

Now I can only suggest you trying to donwload VTK 6.0 sources and compile it with Qt5. For this you need to switch coresponding Qt5 flag in VTK cmake, otherwise it will continue to search qt4.

-   Target version set to 3.0-alpha
-   Difficulty changed from Easy to Medium
-   Category set to viz
```
##### Anatoly Baksheev on 2014-01-20 21:01

```
Does the crash happen even for console applications that don't use qt?
```
##### David W.  on 2014-01-20 23:39

```
Anatoly Baksheev wrote:
> Does the crash happen even for console applications that don't use qt?

Yes. I commented out every use of highgui or viz in my project, which is build with cmake.
-   Target version deleted (3.0-alpha)
```
##### David W.  on 2014-01-20 23:49

```
Anatoly Baksheev wrote:
> Thank you for the reporting. The OpenGL flag has no effect there, it is related to another functionality. 
> 
> I don't have understanding why it happerns, need to investigate. Any help appreciated. 
> 
> Now I can only suggest you trying to donwload VTK 6.0 sources and compile it with Qt5. For this you need to switch coresponding Qt5 flag in VTK cmake, otherwise it will continue to search qt4.

I had a quick look on the CMakeLists of VTK and can't find any central QT5 switch. @grep -r QT5 VTK6.0.0@ gives no results.

If you can point me to, i will try that tomorrow.
```
##### Anatoly Baksheev on 2014-01-21 08:28

```
Here is howto:
https://coderwall.com/p/akk-og

Probably only solution for this is to prevent adding opnecv_viz.so to ${OpenCV_LIBS} in case OpenCV is compiled with Qt5.
```
##### Andrew Senin on 2014-01-21 10:14

```
-   Status changed from New to Open
```
##### David W.  on 2014-01-22 13:03

```
Anatoly Baksheev wrote:
> Here is howto:
> https://coderwall.com/p/akk-og
> 
> Probably only solution for this is to prevent adding opnecv_viz.so to ${OpenCV_LIBS} in case OpenCV is compiled with Qt5.

The original problem was solved by building viz against VTK 6.1, which required a change in OpenCVDetectVTK.cmake. My project compiles and runs properly, until I try to create a viz-window:

@ERROR: In vtk-git/src/VTK/Rendering/OpenGL/vtkOpenGLActor.cxx, line 94
vtkOpenGLActor (0x14679a0): failed after Render 1 OpenGL errors detected
  0 : (1284) Stack underflow@
```
##### Anatoly Baksheev on 2014-01-23 11:08

```
Thank very much. I know this occasional issue. I think this requires investigation and some work to add support of newer VTKs. It's TODO for me when I have time (I spend my personal time on it)

Now I will only push guard that prevents linking Viz if OpenCV is compiled with qt5.
```
##### Anatoly Baksheev on 2014-01-26 16:52

```
I can't reproduce the OpenGL error on my Ubuntu 12.04 64bit. I compiled both OpenCV and VTK from git (6.2 and 6.1) with/without Qt5.2. Everything just works for me. I tried debug and release configurations. BTW, if you run 'opencv_test_viz' executable for testing, you need to specify export OPENCV_TEST_DATA_PATH=<opencv_extra_repository_path/testdaa>.

In VTK 6.1 an OpenGL error reporting layer was added, it catches many errors which were just ignored before. That's why I have a lot of OpenGL errors on my OSX 10.9 with VTK6.2. I think VTK support of 10.9 system is immature, so plan to wait stable release before looking into this.

Regarding VTK 5.x, I learned that 'apt-get install'ed VTK depends on qt4. So, you need to recompile VTK5.10 from source without Qt4 to use it with OpenCV+qt5.
```
##### David W.  on 2014-02-01 19:56

```
I am working on a current arch linux x86_64 and did a clean rebuild of my project against OpenCV/master against VTK 6.1 against Qt 5.2 and finally got a working Viz window!

I think they patched something in VTK for the final 6.1 release, that made the OpenGL error disappear. I am looking forward to try out all the nice features!

Thanks for your investigations!
```
##### Vladislav Vinogradov on 2014-02-02 12:42

```
Hello David,

Since the issue seems to be resolved, I am closing the ticket.
Please reopen it, if the crash still appears.
-   Status changed from Open to Cancelled
```
##### David W.  on 2014-02-03 16:26

```
David W.  wrote:
> I am working on a current arch linux x86_64 and did a clean rebuild of my project against OpenCV/master against VTK 6.1 against Qt 5.2 and finally got a working Viz window!

Cheered to soon!

The crash happens if setCamera() is called. I called it like this:
@
viz::Camera vizCam(Vec2d(100,100), myWindow.getWindowSize());
myWindow.setCamera(vizCam);
myWindow.spin();
@


*Console output:*
<pre>
ERROR: In VTK/Rendering/OpenGL/vtkOpenGLActor.cxx, line 94
vtkOpenGLActor (0xc828f0): failed after Render 1 OpenGL errors detected
  0 : (1284) Stack underflow
</pre>

*Stack trace of main thread:*
<pre>
0   vtkRenderer::UpdateGeometry()                               /lib64/libvtkRenderingCore-6.1.so.1                                0x7fffd9ff257a   
1   vtkOpenGLRenderer::DeviceRender()                           /lib64/libvtkRenderingOpenGL-6.1.so.1                              0x7fffd972a44b   
2   vtkRenderer::Render()                                       /lib64/libvtkRenderingCore-6.1.so.1                                0x7fffd9ff4884   
3   cv::viz::Viz3d::VizImpl::setCamera(cv::viz::Camera const&)  /home/davidek/ba/opencv/src/opencv/build/lib/libopencv_viz.so.3.0  0x7ffff7baf547   
4   main                                                        main.cpp 91                                                        0x41da0b 
</pre>

Additionally I attach the report of the viz tests. But setCamera is no testet there.
-   File test_report.txt added
```
##### David W.  on 2014-02-03 16:27

```
-   Status changed from Cancelled to Open
```