Issue 1:
System information (version)

OpenCV => 3.4.1 pre-built release

Tensorflow => 1.8.0 latest master

Operating System / Platform => Windows 10 64bit

Compiler => Visual Studio 2015

I have retrained an image classifier based on TFHub Mobilenet V2, using the following command:

retrain.py --image_dir .\data --output_graph output.pb --output_labels labels.txt --tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/feature_vector/1 --saved_model_dir .\export

The resulting output.pb works fine with TF label image:

tensorflow\examples\label_image\label_image.py --graph=.\output.pb --labels=.\labels.txt --input_layer=Placeholder --output_layer=final_result --input_height=224 --input_width=224 --image=.\111.jpg

But when using OpenCV DNN module, it crashes at the following line:

dnn::Net net = readNetFromTensorflow(modelFile);

with error message:

OpenCV(3.4.1) Error: Unspecified error (Unknown layer type Sub in op module_apply_default/hub_input/Sub) in cv::dnn::experimental_dnn_v4::'anonymous-namespace'::TFImporter::populateNet, file C:\build\master_winpack-build-win64-vc14\opencv\modules\dnn\src\tensorflow\tf_importer.cpp, line 1582

Running optimize_for_inference.py and transform_graph does not help.

After some inspecting with tensorboard graph, it seems that one hub_input block caused this problem.

Looks like TF Hub models have this block between Placeholder(input node) and the MobilenetV2 block, and the Sub operation is not supported by DNN.

Am I missing something here?

Here is my retrained model, load it with readNetFromTensorflow() to reproduce the problem

output.zip

Any help is appreciated!

Issue 2:
System information (version)

OpenCV => 4.1

Operating System / Platform => OS X 10.13.6

Compiler => clang

Detailed description

I installed opencv using homebrew and it installed fine. I edited the rb file to include ffmpeg, which was the solution to this issue in ocv 3.1. I was unable to install with the --with-ffmpeg flag for ver 4.1. My code works on Ubuntu 18.04 but not OS X. This is the error I get:

2019-04-22 17:40:46.985 Python[29225:130069098] !!! BUG: The current event queue and the main event queue are not the same. Events will not be handled correctly. This is probably because _TSGetMainThread was called for the first time off the main thread.
2019-04-22 17:40:46.993 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.004 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.006 Python[29225:130069098] pid(29225)/euid(504) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
2019-04-22 17:40:47.135 Python[29225:130069179] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff5779623b __exceptionPreprocess + 171
1 libobjc.A.dylib 0x00007fff7ea28c76 objc_exception_throw + 48
2 AppKit 0x00007fff55441297 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 4167
3 libopencv_highgui.4.1.dylib 0x000000013088fb8e cvWaitKey + 352
4 libopencv_highgui.4.1.dylib 0x000000013088b598 _ZN2cv9waitKeyExEi + 34
5 libopencv_highgui.4.1.dylib 0x000000013088b600 _ZN2cv7waitKeyEi + 34
6 cv2.so 0x000000012eb96d9d _ZL19pyopencv_cv_waitKeyP7_objectS0_S0_ + 99
7 Python 0x0000000107938e32 PyEval_EvalFrameEx + 19200
8 Python 0x0000000107934118 PyEval_EvalCodeEx + 1540
9 Python 0x00000001078da838 function_call + 327
10 Python 0x00000001078bcc19 PyObject_Call + 97
11 Python 0x00000001079396f3 PyEval_EvalFrameEx + 21441
12 Python 0x000000010793d121 fast_function + 190
13 Python 0x0000000107938d1c PyEval_EvalFrameEx + 18922
14 Python 0x000000010793d121 fast_function + 190
15 Python 0x0000000107938d1c PyEval_EvalFrameEx + 18922
16 Python 0x0000000107934118 PyEval_EvalCodeEx + 1540
17 Python 0x00000001078da838 function_call + 327
18 Python 0x00000001078bcc19 PyObject_Call + 97
19 Python 0x00000001078c751b instancemethod_call + 163
20 Python 0x00000001078bcc19 PyObject_Call + 97
21 Python 0x000000010793cab4 PyEval_CallObjectWithKeywords + 159
22 Python 0x0000000107968db4 t_bootstrap + 70
23 Python 0x0000000107964a70 pythread_wrapper + 25
24 libsystem_pthread.dylib 0x00007fff7f95a661 _pthread_body + 340
25 libsystem_pthread.dylib 0x00007fff7f95a50d _pthread_body + 0
26 libsystem_pthread.dylib 0x00007fff7f959bf9 thread_start + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6

Steps to reproduce

import sys, cv2, time,cmapy
import numpy as np
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt

from threading import Thread, RLock
from Queue import Queue
import imutils
from cv2 import *

ip = '192.168.122.1'

class VideoGet:
def __init__(self,url):
self.stream=cv2.VideoCapture(url)
self.stream.set(cv2.CAP_PROP_BUFFERSIZE, 0)
(self.grabbed,self.frame)=self.stream.read()
self.frame = self.frame[:,:,0]
#self.frame = cv2.resize(self.frame, (1920,1080))
self.stopped = False
def start(self):
Thread(target=self.get, args=()).start()
return self
def get(self):
while not self.stopped:
if not self.grabbed:
self.stop()
else:
(self.grabbed, self.frame) = self.stream.read()
self.frame = self.frame[:,:,0]
def stop(self):
self.stopped = True

class VideoGetAll:
def __init__(self,url1,url2,url3,url4,url5,url6,url7):
self.cap1=cv2.VideoCapture(url1)
(self.grabbed1,self.frame1)=self.stream1.read()
self.cap2=cv2.VideoCapture(url2)
(self.grabbed2,self.frame2)=self.stream2.read()
self.cap3=cv2.VideoCapture(url3)
(self.grabbed3,self.frame3)=self.stream3.read()
self.cap4=cv2.VideoCapture(url4)
(self.grabbed4,self.frame4)=self.stream4.read()
self.cap5=cv2.VideoCapture(url5)
(self.grabbed5,self.frame5)=self.stream5.read()
self.cap6=cv2.VideoCapture(url6)
(self.grabbed6,self.frame6)=self.stream6.read()
self.cap7=cv2.VideoCapture(url7)
(self.grabbed7,self.frame7)=self.stream7.read()
self.grabbed = self.grabbed1&self.grabbed2&self.grabbed3&self.grabbed4&self.grabbed5&self.grabbed6&self.grabbed7
self.frame = concat(self.frame1[0:1080,:],self.frame2[0:1080,:],self.frame3[0:1080,:],self.frame4[0:1080,:],self.frame5[0:1080,:],self.frame6[0:1080,:])
#self.frame = cv2.resize(self.frame, (1920,1080))
self.stopped = False
def start(self):
Thread(target=self.get, args=()).start()
return self
def get(self):
while not self.stopped:
if not self.grabbed:
self.stop()
else:
(self.grabbed1, self.frame1) = self.stream1.read()
(self.grabbed2, self.frame2) = self.stream2.read()
(self.grabbed3, self.frame3) = self.stream3.read()
(self.grabbed4, self.frame4) = self.stream4.read()
(self.grabbed5, self.frame5) = self.stream5.read()
(self.grabbed6, self.frame6) = self.stream6.read()
(self.grabbed7, self.frame7) = self.stream7.read()
self.frame0 = concat(self.frame1[0:1080,:],self.frame2[0:1080,:],self.frame3[0:1080,:],self.frame4[0:1080,:],self.frame5[0:1080,:],self.frame5[0:1080,:],self.frame6[0:1080,:])
self.frame = cv2.resize(self.frame0, (1280,720))
def stop(self):
self.stopped = True

class VideoShow:
def __init__(self, frame1=None,frame2=None,frame3=None):
self.frame1 = frame1
self.frame2 = frame2
self.frame3 = frame3
self.stopped = False
self.xoff1 = 0
self.yoff1 = 0
self.angle1 = 0
self.xoff2 = 0
self.yoff2 = 0
self.angle2 = 0
self.rchan = 5
self.gchan = 4
self.bchan = 3
def start(self):
Thread(target=self.show, args=()).start()
return self
def show(self):
cv2.namedWindow('Frame',cv2.WINDOW_NORMAL)
cv2.resizeWindow('Frame',1280,720)
cv2.namedWindow("Transform")
cv2.createTrackbar("X1","Transform",91,300,nothing)
cv2.createTrackbar("Y1","Transform",-85,300,nothing)
cv2.createTrackbar("Angle1","Transform",0,180,nothing)
cv2.setTrackbarMin("X1","Transform",-300)
cv2.setTrackbarMin("Y1","Transform",-300)
cv2.setTrackbarMin("Angle1","Transform",-180)
cv2.createTrackbar("X2","Transform",-87,300,nothing)
cv2.createTrackbar("Y2","Transform",-1,300,nothing)
cv2.createTrackbar("Angle2","Transform",-2,180,nothing)
cv2.setTrackbarMin("X2","Transform",-300)
cv2.setTrackbarMin("Y2","Transform",-300)
cv2.setTrackbarMin("Angle2","Transform",-180)
cv2.createTrackbar('Red',"Transform",5,5,nothing)
cv2.createTrackbar('Green',"Transform",4,5,nothing)
cv2.createTrackbar('Blue',"Transform",3,5,nothing)
while not self.stopped:
cv2.imshow('Frame', cv2.merge((self.frame1,self.frame2,self.frame3)))
self.xoff1 = cv2.getTrackbarPos("X1","Transform")
self.yoff1 = cv2.getTrackbarPos("Y1","Transform")
self.angle1 = cv2.getTrackbarPos("Angle1","Transform")
self.xoff2 = cv2.getTrackbarPos("X2","Transform")
self.yoff2 = cv2.getTrackbarPos("Y2","Transform")
self.angle2 = cv2.getTrackbarPos("Angle2","Transform")
self.rchan = cv2.getTrackbarPos("Red","Transform")
self.gchan = cv2.getTrackbarPos("Green","Transform")
self.bchan = cv2.getTrackbarPos("Blue","Transform")
if cv2.waitKey(1) == 27:
self.stopped = True
def stop(self):
self.stopped = True
cv2.destroyAllWindows()

def concat(f1,f2,f3,f4,f5,f6):
return np.concatenate((np.concatenate((f1,f2,f3),axis=1),np.concatenate((f4,f5,f6),axis=1)),axis=0)

def shift(img,x,y,theta):
return imutils.rotate(cv2.warpAffine(img,np.float32([[1,0,x],[0,1,y]]), (640, 512)),theta)

get1 = VideoGet('http://'+ip+':8081/?action=stream').start()
get2 = VideoGet('http://'+ip+':8082/?action=stream').start()
get3 = VideoGet('http://'+ip+':8083/?action=stream').start()
get4 = VideoGet('http://'+ip+':8084/?action=stream').start()
get5 = VideoGet('http://'+ip+':8080/?action=stream').start()
get6 = VideoGet('http://'+ip+':8085/?action=stream').start()
get7 = VideoGet('http://'+ip+':8086/?action=stream').start()

show = VideoShow(cv2.merge((get3.frame,get4.frame, get5.frame))).start()

while True:
if show.stopped:
show.stop()
get1.stop()
get2.stop()
get3.stop()
get4.stop()
get5.stop()
get6.stop()
get7.stop()
break
if show.rchan>=0 and show.rchan<1:
show.frame3=shift(get1.frame,show.xoff2,show.yoff2,show.angle2)
elif show.rchan>=1 and show.rchan<2:
show.frame3=shift(get2.frame,show.xoff2,show.yoff2,show.angle2)
elif show.rchan>=2 and show.rchan<3:
show.frame3=shift(get3.frame,show.xoff2,show.yoff2,show.angle2)
elif show.rchan>=3 and show.rchan<4:
show.frame3=shift(get4.frame,show.xoff2,show.yoff2,show.angle2)
elif show.rchan>=4 and show.rchan<5:
show.frame3=shift(get5.frame,show.xoff2,show.yoff2,show.angle2)
elif show.rchan>=5 and show.rchan<=6:
show.frame3=shift(get6.frame,show.xoff2,show.yoff2,show.angle2)
if show.gchan>=0 and show.gchan<1:
show.frame2=shift(get1.frame,show.xoff1,show.yoff1,show.angle1)
elif show.gchan>=1 and show.gchan<2:
show.frame2=shift(get2.frame,show.xoff1,show.yoff1,show.angle1)
elif show.gchan>=2 and show.gchan<3:
show.frame2=shift(get3.frame,show.xoff1,show.yoff1,show.angle1)
elif show.gchan>=3 and show.gchan<4:
show.frame2=shift(get4.frame,show.xoff1,show.yoff1,show.angle1)
elif show.gchan>=4 and show.gchan<5:
show.frame2=shift(get5.frame,show.xoff1,show.yoff1,show.angle1)
elif show.gchan>=5 and show.gchan<=6:
show.frame2=shift(get6.frame,show.xoff1,show.yoff1,show.angle1)
if show.bchan>=0 and show.bchan<1:
show.frame1=get1.frame
elif show.bchan>=1 and show.bchan<2:
show.frame1=get2.frame
elif show.bchan>=2 and show.bchan<3:
show.frame1=get3.frame
elif show.bchan>=3 and show.bchan<4:
show.frame1=get4.frame
elif show.bchan>=4 and show.bchan<5:
show.frame1=get5.frame
elif show.bchan>=5 and show.bchan<=6:
show.frame1=get6.frame

if cv2.waitKey(1) == 27:
break