Issue 1:
I am using tranform_graph to quantize darkflow's tiny-yolo-v2 *.pb

https://github.com/thtrieu/darkflow

found several issues

when i use video to do inference (or images) the performance drops to very low with gpu enable

from 5.xx fps to 1.xx fps, and i can feel the images inference speed drops, by input with lots of pictures.

after use tranform_graph's quantization, the mAP drops from 68% to 13% (with my 500 pics selcted from pascal_voc2012)

the function i used list below

../tensorflow-src/tensorflow/bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=_151875_12_28_02_36_51_tiny-yolo-v2.pb --inputs='input' --outputs='output' --out_graph=_151875_12_28_02_36_51_tiny-yolo-v2.quantize_node.pb --transforms='remove_nodes(op=Identity, op=CheckNumerics)
fold_constants(ignore_errors=true)
fold_batch_norms
fold_old_batch_norms
quantize_weights
quantize_nodes
strip_unused_nodes
merge_duplicate_nodes
sort_by_execution_order'

after quantize mAP drops, i use "insert_logging" & "freeze_requantization_ranges"

tried to bring back mAP with that pascal_voc selected 500 pictures

but the mAP drops to 0%.

i wrote a script try to fix the logging file

I go thorough logging with single image, one by one

if the mAP goes up, i merge with the last highest mAP ...

until the last image

so comes out with a valid logging file, which bring back the mAP to 62%

now the questions are

is that possible to quantize yolo-tiny-v2?

because the mAP and performance drops so much (even thou the mAP can be adjust back)

are there any other neural network models that you suggest me to use?

my target is to create some quantized small models for running on ic chips

does that means logging and freeze_requantization_ranges might not suitable for tiny-yolo-v2?

Thanks for your time for reading this

== cat /etc/issue ===============================================

Linux 8fbb96ca119d 4.15.0-34-generic #37~16.04.1-Ubuntu SMP Tue Aug 28 10:44:06 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

VERSION="16.04.5 LTS (Xenial Xerus)"

VERSION_ID="16.04"

VERSION_CODENAME=xenial

== are we in docker =============================================

Yes

== compiler =====================================================

c++ (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609

Copyright (C) 2015 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

== uname -a =====================================================

Linux 8fbb96ca119d 4.15.0-34-generic #37~16.04.1-Ubuntu SMP Tue Aug 28 10:44:06 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

== check pips ===================================================

numpy 1.15.4

protobuf 3.6.1

tensorflow-gpu 1.12.0

== check for virtualenv =========================================

False

== tensorflow import ============================================

tf.VERSION = 1.12.0

tf.GIT_VERSION = v1.12.0-0-ga6d8ffae09

tf.COMPILER_VERSION = v1.12.0-0-ga6d8ffae09

Sanity check: array([1], dtype=int32)

== env ==========================================================

LD_LIBRARY_PATH /usr/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64

DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================

Sat Jan 26 05:56:00 2019

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 390.87 Driver Version: 390.87 |

|-------------------------------+----------------------+----------------------+

| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |

| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |

|===============================+======================+======================|

| 0 Quadro GV100 Off | 00000000:18:00.0 Off | Off |

| 40% 54C P0 50W / 250W | 1MiB / 32508MiB | 0% Default |

+-------------------------------+----------------------+----------------------+

| 1 Quadro GV100 Off | 00000000:3B:00.0 Off | Off |

| 40% 54C P0 50W / 250W | 0MiB / 32508MiB | 5% Default |

+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+

| Processes: GPU Memory |

| GPU PID Type Process name Usage |

|=============================================================================|

| No running processes found |

+-----------------------------------------------------------------------------+

== cuda libs ===================================================

/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart_static.a

/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176

Issue 2:
System information

Have I written custom code: yes

OS Platform and Distribution: Linux Fedora 16.04

TensorFlow installed from: binary

TensorFlow version: v1.4.0-19-ga52c8d9, 1.4.1

Python version: 2.7

Bazel version: N/A

CUDA/cuDNN version: cuda_8.0.61, cudnnv5

GPU model and memory: GeForce GTX TITAN X , 12207MiB

Exact command to reproduce:

We have a computer cluster that some of the machines have GPU and others don't. I have installed tensorflow-gpu-1.4 from wheel file in a virtualenv in a folder on the file-server which means that it is accessible on all the machines in the cluster.

My program is a distributed software which means that some of the tasks are done on all the cluster nodes. (data generation and configuration) and the machine learning part is only done on the machines with GPU. I activate the aforementioned virtualenv before running the servers on nodes of the cluster so all the nodes are running inside the same virtual environment.

On the machines that have a GPU when I import tensorflow everything works fine, but when I import the tensorflow on the machines that do not have the gpu (and Cuda is not installed on them) I get following error:

In [1]: import tensorflow as tf

ImportError Traceback (most recent call last)

in ()

----> 1 import tensorflow as tf

/virtualenv/lib/python2.7/site-packages/tensorflow/init.py in ()

22

23 # pylint: disable=wildcard-import

---> 24 from tensorflow.python import *

25 # pylint: enable=wildcard-import

26

/virtualenv/lib/python2.7/site-packages/tensorflow/python/init.py in ()

47 import numpy as np

48

---> 49 from tensorflow.python import pywrap_tensorflow

50

51 # Protocol buffers

/virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py in ()

70 for some common reasons and solutions. Include the entire stack trace

71 above this error message when asking for help.""" % traceback.format_exc()

---> 72 raise ImportError(msg)

73

74 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

ImportError: Traceback (most recent call last):

File "virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in

from tensorflow.python.pywrap_tensorflow_internal import *

File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in

_pywrap_tensorflow_internal = swig_import_helper()

File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper

_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)

ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions. Include the entire stack trace

above this error message when asking for help.

I am aware that tensorflow-gpu is statically linked to the Cuda libraries and I installed a local version of Cuda using the runfile in a folder on the file-server (accessible to all the nodes) and added its path to the LD_LIBRARY_PATH and PATH and now importing tf gives me the following error:

2018-02-15 01:41:58.519693: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU

supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

2018-02-15 01:41:58.519972: E tensorflow/stream_executor/cuda/cuda_driver.cc:406] failed call to cuInit: CUresult(-1)

2018-02-15 01:41:58.520007: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:158] retrieving CUDA diagnostic information for host: maserati

2018-02-15 01:41:58.520017: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: maserati

2018-02-15 01:41:58.520140: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: Not found: was unable to find libcuda.so DSO loaded into this program

2018-02-15 01:41:58.520167: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:369] driver version file contents: """NVRM version: NVIDIA UNIX x86_64 Kernel Module 384.111 Tue Dec 19 23:51:45 PST 2017

GCC version: gcc version 7.2.1 20170915 (Red Hat 7.2.1-2) (GCC)

"""

2018-02-15 01:41:58.520191: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: 384.111.0

And when i run a hello world script it gives me the following error (which clearly means it cant run anything on gpu because of previous error):

In [5]: with tf.device("/GPU:0"):

...: hello = tf.constant('Hello, TensorFlow!')

...: sess = tf.Session()

...: print(sess.run(hello))

...:

InvalidArgumentError Traceback (most recent call last)

in ()

2 hello = tf.constant('Hello, TensorFlow!')

3 sess = tf.Session()

----> 4 print(sess.run(hello))

5

/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in run(self, fetches, feed_dict, options, run_metadata)

887 try:

888 result = self._run(None, fetches, feed_dict, options_ptr,

--> 889 run_metadata_ptr)

890 if run_metadata:

891 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _run(self, handle, fetches, feed_dict, options, run_metadata)

1118 if final_fetches or final_targets or (handle and feed_dict_tensor):

1119 results = self._do_run(handle, final_targets, final_fetches,

-> 1120 feed_dict_tensor, options, run_metadata)

1121 else:

1122 results = []

/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)

1315 if handle is None:

1316 return self._do_call(_run_fn, self._session, feeds, fetches, targets,

-> 1317 options, run_metadata)

1318 else:

1319 return self._do_call(_prun_fn, self._session, handle, feeds, fetches)

/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _do_call(self, fn, *args)

1334 except KeyError:

1335 pass

-> 1336 raise type(e)(node_def, op, message)

1337

1338 def _extend_graph(self):

InvalidArgumentError: Cannot assign a device for operation 'Const_1': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.

[[Node: Const_1 = Constdtype=DT_STRING, value=Tensor<type: string shape: [] values: Hello, TensorFlow!>, _device="/device:GPU:0"]]

Caused by op u'Const_1', defined at:

File "/virtualenv/bin/ipython", line 11, in

sys.exit(start_ipython())

File "/virtualenv/lib/python2.7/site-packages/IPython/init.py", line 119, in start_ipython

return launch_new_instance(argv=argv, **kwargs)

File "/virtualenv/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance

app.start()

File "/virtualenv/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 355, in start

self.shell.mainloop()

File "/virtualenv/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 493, in mainloop

self.interact()

File "/virtualenv/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 484, in interact

self.run_cell(code, store_history=True)

File "/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2718, in run_cell

interactivity=interactivity, compiler=compiler, result=result)

File "/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes

if self.run_code(code, result):

File "/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2882, in run_code

exec(code_obj, self.user_global_ns, self.user_ns)

File "", line 2, in

hello = tf.constant('Hello, TensorFlow!')

File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant

name=name).outputs[0]

File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op

op_def=op_def)

File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in init

self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'Const_1': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.

[[Node: Const_1 = Constdtype=DT_STRING, value=Tensor<type: string shape: [] values: Hello, TensorFlow!>, _device="/device:GPU:0"]]

I dont wan't/ can't install cuda on non-gpu machines is there any work around for this issue?