Issue 1:
System information (version)

OpenCV => 4.1.0

Operating System / Platform => Windows 64 Bit

Compiler => Visual Studio 2017 (15.9)

Detailed description

The build of opencv_highgui fails with several errors of the form X already defined in window.obj. The other projects compile successfully.

Steps to reproduce

Clone the opencv and opencv_contrib repos, checkout 4.1.0 in both.

Run CMake, enable WITH_CUDA, OPENCV_ENABLE_NONFREE and set OPENCV_EXTRA_MODULES_PATH to the opencv_contrib\modules directory.

Generate the solution.

Open the solution in Visual Studio 2017 and launch the build.

The build runs smoothly and all projects except one succeed. The one that fails is opencv_highgui, with these errors:

60>opencv_highgui_main.cpp
60>c:\opencv\opencv\modules\highgui\src\window.cpp(250): warning C4702: unreachable code
60>c:\opencv\opencv\modules\highgui\src\window.cpp(252): warning C4702: unreachable code
60>c:\opencv\opencv\modules\highgui\src\window.cpp(254): warning C4702: unreachable code
60>c:\opencv\opencv\modules\highgui\src\window.cpp(255): warning C4702: unreachable code
60>c:\opencv\opencv\modules\highgui\src\window.cpp(257): warning C4702: unreachable code
60>window_w32.obj : error LNK2005: "void __cdecl cv::setWindowTitle(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?setWindowTitle@cv@@YAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@0@Z) already defined in window.obj
60>window_w32.obj : error LNK2005: cvCreateTrackbar already defined in window.obj
60>window_w32.obj : error LNK2005: cvCreateTrackbar2 already defined in window.obj
60>window_w32.obj : error LNK2005: cvDestroyAllWindows already defined in window.obj
60>window_w32.obj : error LNK2005: cvDestroyWindow already defined in window.obj
60>window_w32.obj : error LNK2005: cvGetTrackbarPos already defined in window.obj
60>window_w32.obj : error LNK2005: cvGetWindowHandle already defined in window.obj
60>window_w32.obj : error LNK2005: cvGetWindowName already defined in window.obj
60>window_w32.obj : error LNK2005: cvInitSystem already defined in window.obj
60>window_w32.obj : error LNK2005: cvMoveWindow already defined in window.obj
60>window_w32.obj : error LNK2005: cvNamedWindow already defined in window.obj
60>window_w32.obj : error LNK2005: cvResizeWindow already defined in window.obj
60>window_w32.obj : error LNK2005: cvSetMouseCallback already defined in window.obj
60>window_w32.obj : error LNK2005: cvSetTrackbarMax already defined in window.obj
60>window_w32.obj : error LNK2005: cvSetTrackbarMin already defined in window.obj
60>window_w32.obj : error LNK2005: cvSetTrackbarPos already defined in window.obj
60>window_w32.obj : error LNK2005: cvShowImage already defined in window.obj
60>window_w32.obj : error LNK2005: cvStartWindowThread already defined in window.obj
60>window_w32.obj : error LNK2005: cvWaitKey already defined in window.obj
60> Creating library C:/OpenCV/opencv/build/lib/Release/opencv_highgui410.lib and object C:/OpenCV/opencv/build/lib/Release/opencv_highgui410.exp
60>C:\OpenCV\opencv\build\bin\Release\opencv_highgui410.dll : fatal error LNK1169: one or more multiply defined symbols found
60>Done building project "opencv_highgui.vcxproj" -- FAILED.

Issue 2:
System information (version)

OpenCV => : 4.1.1.26

Operating System / Platform => Ubuntu 18.04

Compiler => ython3.5

Detailed description

Hello I am having the following issue. I am trying to freeze EfficientNet taken from this repo and use the protobuf file for using it with OpenCV dnn module.

The model is a simple classification network. Right after the the Feature Extractor specified in the link, I just try add an extra dense & and an extra classification layer of num_classes.

I have frozen several networks and used them with the dnn library before and I know all the issues that can arise, which I have already tried.

Example code in python (also seen in attached script below):

net = cv2.dnn.readNetFromTensorflow('frozen.pb')
x_cv = np.random.random((224, 224, 3)).astype(np.uint8)
blob = cv2.dnn.blobFromImage(x_cv, 1.0, (224, 224), (0, 0, 0))
net.setInput(blob)
# (1 x 200)
s = net.forward().shape
print(s)

However life is not that easy. This is the following error:

error: OpenCV(4.1.1) /io/opencv/modules/dnn/src/layers/eltwise_layer.cpp:116: error: (-215:Assertion failed) inputs[0] == inputs[i] in function 'getMemoryShapes'

Eventually I Found out that the problem is caused when using the Squeeze and excitation module (SE module for short). If I disable the flag & remove those modules from the network, the forward pass works.

In my opinion is the flow of the graph of the SE Module that is not acceptable from OpenCV from going to an average pooling to 1x1 convs and finally multiplying the features.

Some remarks:

I had this issue with SE modules in the past, with the difference that the model was using more basic operations such as:

reduce_mean across the features instead of average pooling

fully connected layers instead of 1x1 Convolutions, since they result in the same number of features.

Additionally:

I used the optimize_for_inference_lib tool with no success.

Using the following command: net.getUnconnectedOutLayersNames() gave me the following results:

['block1a_se_squeeze/Mean/flatten', 'block2a_se_squeeze/Mean/flatten', 'block2b_se_squeeze/Mean/flatten', 'block3a_se_squeeze/Mean/flatten', 'block3b_se_squeeze/Mean/flatten', 'block4a_se_squeeze/Mean/flatten', 'block4b_se_squeeze/Mean/flatten', 'block4c_se_squeeze/Mean/flatten', 'block5a_se_squeeze/Mean/flatten', 'block5b_se_squeeze/Mean/flatten', 'block5c_se_squeeze/Mean/flatten', 'block6a_se_squeeze/Mean/flatten', 'block6b_se_squeeze/Mean/flatten', 'block6c_se_squeeze/Mean/flatten', 'block6d_se_squeeze/Mean/flatten', 'block7a_se_squeeze/Mean/flatten', 'avg_pool/Mean/flatten', 'dense_1/Softmax']

Follow up

I would like to know if there is a possible way to add the SE Module either by

reshaping the network in a compatible manner

transforming the graph and using a .pbtxt file additionally

register the module as custom & override the getMemoryShapes()

Of course the easiest solution would be preferrable :)

Steps to reproduce

Running the attached .py (extension currently txt) file, will follow the pipeline:

Initialize EfficientnetB0 (smallest) with SE modules.

Create a dummy training set and run keras training fit() function.

Save the model as keras extension and reset graph

Load & freeze model with standard keras freezing method. You should see 344 variables frozen in the terminal.

Load model for OpenCV, create dummy blob & run net.forward(). It will crash with the aforementioned error.

opencv_efficientnet.txt

Making the model freezable.

If you jump to line 254 of the attached script, you can override the has_se=True to False. This will make the model compatible with OpenCV, proving the detailed description above that this is the factor that makes it show the assertion error.