Issue 1:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04

TensorFlow installed from (source or binary): Source

TensorFlow version (use command below): r1.5

Python version: 3.5.2

Bazel version (if compiling from source): 0.9.0

GCC/Compiler version (if compiling from source): 5.4.0

CUDA/cuDNN version: Cuda 8.0 and cuDNN 7.0

GPU model and memory: Jetson TX2

The problem

I created a fully convolutional float 16 (half precision) neural network in tensorflow. When I run this network with some inputs, the inference time is slower than when I run the same network in float 32 (full precision) mode.

I should also note that the following variables are set:

os.environ['TF_FP16_CONV_USE_FP32_COMPUTE'] = '0' os.environ['TF_FP16_MATMUL_USE_FP32_COMPUTE'] = '0'

As Nvidia Jetson TX2 support FP16 operations, I expected an inference time not worse than when I use FP32, but surprisingly it is about 1.5 times worse! (36 miliseconds vs 22 miliseconds). I guess it is becuase of the overhead of internal type conversion in the tensorflow core between float16 and float32!

Is it a problem with Tensorflow or TX2?

Issue 2:
#1300

I have got a nvidia p100 GPU which is support fp16, and I run the TF case 'cifar10_train.py'. Without option '--use_fp16', the performance is also 1600 examples/sec, and with the option '--use_fp16', the performance down to 500 examples/sec. Any ideas about this issue?

userid@ubuntu-WK-4xP100:/weike/tensorflow-r0.11/tensorflow/models/image/cifar10$ vi cifar10_train.py

userid@ubuntu-WK-4xP100:/weike/tensorflow-r0.11/tensorflow/models/image/cifar10$ python cifar10_train.py

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.8.0 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so.5 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.8.0 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.8.0 locally

Downloading cifar-10-binary.tar.gz 100.0%

...

2016-11-14 00:07:57.143739: step 0, loss = 4.67 (12.1 examples/sec; 10.549 sec/batch)

2016-11-14 00:07:58.395209: step 10, loss = 4.57 (1693.6 examples/sec; 0.076 sec/batch)

2016-11-14 00:07:59.177525: step 20, loss = 4.97 (1668.9 examples/sec; 0.077 sec/batch)

2016-11-14 00:07:59.957789: step 30, loss = 4.43 (1588.3 examples/sec; 0.081 sec/batch)

2016-11-14 00:08:00.738431: step 40, loss = 4.52 (1690.5 examples/sec; 0.076 sec/batch)

2016-11-14 00:08:01.501940: step 50, loss = 4.33 (1680.4 examples/sec; 0.076 sec/batch)

2016-11-14 00:08:02.241604: step 60, loss = 4.20 (1733.4 examples/sec; 0.074 sec/batch)

2016-11-14 00:08:03.001845: step 70, loss = 4.27 (1706.0 examples/sec; 0.075 sec/batch)

2016-11-14 00:08:03.765522: step 80, loss = 4.18 (1601.3 examples/sec; 0.080 sec/batch)

2016-11-14 00:08:04.516780: step 90, loss = 4.25 (1646.3 examples/sec; 0.078 sec/batch)

userid@ubuntu-WK-4xP100:~/weike/tensorflow-r0.11/tensorflow/models/image/cifar10$ python cifar10_train.py --use_fp16

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.8.0 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so.5 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.8.0 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.8.0 locally

Filling queue with 20000 CIFAR images before starting to train. This will take a few minutes.

..

2016-11-14 00:09:09.382854: step 0, loss = 4.67 (12.3 examples/sec; 10.411 sec/batch)

2016-11-14 00:09:11.923842: step 10, loss = 4.58 (659.6 examples/sec; 0.194 sec/batch)

2016-11-14 00:09:13.918448: step 20, loss = 6.62 (460.6 examples/sec; 0.278 sec/batch)

2016-11-14 00:09:16.211809: step 30, loss = 4.37 (583.7 examples/sec; 0.219 sec/batch)

2016-11-14 00:09:18.327690: step 40, loss = 4.30 (618.3 examples/sec; 0.207 sec/batch)

2016-11-14 00:09:20.395409: step 50, loss = 4.37 (643.8 examples/sec; 0.199 sec/batch)

2016-11-14 00:09:22.466230: step 60, loss = 4.32 (574.0 examples/sec; 0.223 sec/batch)

2016-11-14 00:09:24.533225: step 70, loss = 4.17 (646.2 examples/sec; 0.198 sec/batch)

2016-11-14 00:09:26.609277: step 80, loss = 2.59 (601.5 examples/sec; 0.213 sec/batch)

2016-11-14 00:09:28.703648: step 90, loss = 4.15 (625.5 examples/sec; 0.205 sec/batch)