Issue 1:
NOTE: Only file GitHub issues for bugs and feature requests. All other topics will be closed.

For general support from the community, see StackOverflow.

To make bugs and feature requests more easy to find and organize, we close issues that are deemed

out of scope for GitHub Issues and point people to StackOverflow.

For bugs or installation issues, please provide the following information.

The more information you provide, the more easily we will be able to offer

help and advice.

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

Only one, but it was not solved.

Environment info

Operating System:

Windows 10 (anaconda 4.3.8)

conda --version conda 4.3.8

Installed version of CUDA and cuDNN:

nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2016 NVIDIA Corporation Built on Sat_Sep__3_19:05:48_CDT_2016 Cuda compilation tools, release 8.0, V8.0.44

If installed from binary pip package, provide:

A link to the pip package you installed:

pip install --upgrade https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-0.12.0rc0-cp35-cp35m-win_amd64.whl

The output from python -c "import tensorflow; print(tensorflow.__version__)".

I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cublas64_80.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cudnn64_5.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cufft64_80.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library nvcuda.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library curand64_80.dll locally

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)

When I tried Single GPU computing example with tensorflow and get the following error:

Placeholder_1: (Placeholder): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] Placeholder_1: (Placeholder)/job:localhost/replica:0/task:0/gpu:0 MatMul_10: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_10: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_11: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_11: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_12: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_12: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_13: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_13: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_14: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_14: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_15: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_15: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_16: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_16: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_17: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_17: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_18: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_18: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_19: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_19: (MatMul)/job:localhost/replica:0/task:0/gpu:0 Placeholder: (Placeholder): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] Placeholder: (Placeholder)/job:localhost/replica:0/task:0/gpu:0 MatMul: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_1: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_1: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_2: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_2: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_3: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_3: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_4: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_4: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_5: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_5: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_6: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_6: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_7: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_7: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_8: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_8: (MatMul)/job:localhost/replica:0/task:0/gpu:0 MatMul_9: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul_9: (MatMul)/job:localhost/replica:0/task:0/gpu:0 AddN: (AddN): /job:localhost/replica:0/task:0/cpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] AddN: (AddN)/job:localhost/replica:0/task:0/cpu:0 E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_blas.cc:372] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED W c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\stream.cc:1390] attempting to perform BLAS operation using StreamExecutor without BLAS support E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_blas.cc:372] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED W c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\stream.cc:1390] attempting to perform BLAS operation using StreamExecutor without BLAS support

What other attempted solutions have you tried?

Then I tried a sample matrix multiplication:

sess = tf.Session(config=tf.ConfigProto(log_device_placement=True)) I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\gpu_device.cc:885] Found device 0 with properties: name: Quadro M2000M major: 5 minor: 0 memoryClockRate (GHz) 1.137 pciBusID 0000:01:00.0 Total memory: 4.00GiB Free memory: 3.35GiB I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\gpu_device.cc:906] DMA: 0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\gpu_device.cc:916] 0: Y I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Quadro M2000M, pci bus id: 0000:01:00.0) Device mapping: /job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Quadro M2000M, pci bus id: 0000:01:00.0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\direct_session.cc:255] Device mapping: /job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Quadro M2000M, pci bus id: 0000:01:00.0

and the following error:

a = tf.random_normal((100,100))

b = tf.random_normal((100,500))

c = tf.matmul(a,b)

sess.run(c)

random_normal_1/RandomStandardNormal: (RandomStandardNormal): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal_1/RandomStandardNormal: (RandomStandardNormal)/job:localhost/replica:0/task:0/gpu:0 random_normal_1/mul: (Mul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal_1/mul: (Mul)/job:localhost/replica:0/task:0/gpu:0 random_normal_1: (Add): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal_1: (Add)/job:localhost/replica:0/task:0/gpu:0 random_normal/RandomStandardNormal: (RandomStandardNormal): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal/RandomStandardNormal: (RandomStandardNormal)/job:localhost/replica:0/task:0/gpu:0 random_normal/mul: (Mul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal/mul: (Mul)/job:localhost/replica:0/task:0/gpu:0 random_normal: (Add): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal: (Add)/job:localhost/replica:0/task:0/gpu:0 MatMul: (MatMul): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] MatMul: (MatMul)/job:localhost/replica:0/task:0/gpu:0 random_normal_1/stddev: (Const): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal_1/stddev: (Const)/job:localhost/replica:0/task:0/gpu:0 random_normal_1/mean: (Const): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal_1/mean: (Const)/job:localhost/replica:0/task:0/gpu:0 random_normal_1/shape: (Const): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal_1/shape: (Const)/job:localhost/replica:0/task:0/gpu:0 random_normal/stddev: (Const): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal/stddev: (Const)/job:localhost/replica:0/task:0/gpu:0 random_normal/mean: (Const): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal/mean: (Const)/job:localhost/replica:0/task:0/gpu:0 random_normal/shape: (Const): /job:localhost/replica:0/task:0/gpu:0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\simple_placer.cc:827] random_normal/shape: (Const)/job:localhost/replica:0/task:0/gpu:0 E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_blas.cc:372] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED W c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\stream.cc:1390] attempting to perform BLAS operation using StreamExecutor without BLAS support

`

Like @kingtaurus and @menggangmark,

I then copied the cudnn64_5.dll (cuda\bin\cudnn64_5.dll) from that zip archive into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin;

cuda\include\cudnn.h to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\include;

and

cuda\lib\x64\cudnn.lib to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64\

cupti64_80.dll (C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64) to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin;

and cupti.lib(C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64) to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64

WHERE C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0 is my install PATH for the CUDA toolkit.

I had already added C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin\ to my PATH

Issue 2:
As many of us (#14126,#14691,#12052), I am trying to get TF1.4 build successfully on windows using the latest version of everything. As far as I can judge I could do it but with some hacks. As it is too long for me to complete, I would like to share what I did for help finalizing. It is too early for a PR.

I am using CMake 3.9.6 (though 3.10 came out). I have low cmake skill level.

I am not trying the python bindings.

VS2017 is the community edition.

Without GPU it is easy. The only issue is the heap overflow (C1002 or C1006 #11096). The trick is to reduce parallel build by msbuild /m:4 /p:CL_MPCount=2 ... such that 4*2 is approximately the number of core you really have (at least it worked for me). Using /Zm2000 did not work for me, despite a lot of available memory (32G).

With GPU it is more tricky: the tf_core_gpu_kernels.vcxproj does not compile at all. AFAIU, the CMake strategy changed from v3.6, to allow parallel computing. CUDA is now treated as another language. Without modifications nvcc simply returns with code error 1 (or nothing happen I am not sure). Here are my modifications (from v1.4).

From tensorflow/tensorflow/contrib/cmake/

1/ adapt cmakelists.txt a little:

Change CUDA 8.0 to CUDA 9.0 l.223.

Add enable_language("CUDA") l.224.

The set(CUDA_NVCC_FLAGS ...) directives do not work anymore. See below.

Add capabilities 6.0 and 6.1 in l.232, as well l.246. Might not be needed (it is only for performance).

Change 64_80 to 64_90 and 64_6 to 64_7 l.247 and 248, similarly in l.272-276.

2/ in tf_core_kernels.cmake:

Add set_source_files_properties(${tf_core_gpu_kernels_srcs} PROPERTIES LANGUAGE CUDA) to recognize '.cu.cc' extensions as cuda files in l.209.

Rename cuda_add_library(...) as add_library(...) l.210.

3/ edit (this is the trick) tf_core_gpu_kernels.vcxproj, in the release section:

Encompass cl.exe flags, ie /bigobj /nologo ... -Ob2 with the -Xcompiler="/bigobj ... -Ob2" directive l.147. These former flags are for the c++ compiler not for nvcc and result in the crash.

Add just before --expt-relaxed-constexpr, still in the AdditionalOptions.

Switch PerformDeviceLinkfrom false to true l.164.

Then everything compile (msbuild on tf_tutorials_example_trainer.vcxproj) (and this tuto works). The remaining point before PR is to avoid third step, i.e. give the right directives to nvcc, by understanding how the CUDA_NVCC_FLAGS works, and add the linking. Hope this solution will work without missing symbols (#6396).

Otherwise it is a nightmare: both CUDA 8 and CMake 3.6 are not aware of VS2017. CMake compilation is not incremental (#14194) and takes about 4-5H (could use precompiled headers especially in tf_core_kernels)...