Issue 1:
Hello,

I get the following errors when compiling:

____[110 / 2,641] Compiling external/png_archive/libpng-1.2.53/pngerror.c

external/protobuf/src/google/protobuf/util/internal/field_mask_utility.cc:47:14: warning: 'google::protobuf::util::Status google::protobuf::util::converter::{anonymous}::CreatePublicError(google::protobuf::util::error::Code, const string&)' defined but not used [-Wunused-function]

util::Status CreatePublicError(util::error::Code code,

external/protobuf/src/google/protobuf/util/internal/field_mask_utility.cc:47:14: warning: 'google::protobuf::util::Status google::protobuf::util::converter::{anonymous}::CreatePublicError(google::protobuf::util::error::Code, const string&)' defined but not used [-Wunused-function]

util::Status CreatePublicError(util::error::Code code,

external/protobuf/src/google/protobuf/util/internal/field_mask_utility.cc:47:14: warning: 'google::protobuf::util::Status google::protobuf::util::converter::{anonymous}::CreatePublicError(google::protobuf::util::error::Code, const string&)' defined but not used [-Wunused-function]

util::Status CreatePublicError(util::error::Code code,

== ) // Compilation error with CHECK_EQ(NULL, x)?

== ) // Compilation error with CHECK_EQ(NULL, x)?

ERROR: /home/user/Downloads/tensorflow/tensorflow/stream_executor/BUILD:5:1: C++ compilation of rule '//tensorflow/stream_executor:stream_executor' failed: crosstool_wrapper_driver_is_not_gcc failed: error executing command

tensorflow/stream_executor/cuda/cuda_dnn.cc:266:10: error: 'CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING' was not declared in this scope

tensorflow/stream_executor/cuda/cuda_dnn.cc:284:10: error: 'CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT_TILING' was not declared in this scope

tensorflow/stream_executor/cuda/cuda_dnn.cc:942:7: error: 'CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING' was not declared in this scope

tensorflow/stream_executor/cuda/cuda_dnn.cc:947:4: error: no matching function for call to 'std::vector::assign()'

tensorflow/stream_executor/cuda/cuda_dnn.cc:958:7: error: 'CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT_TILING' was not declared in this scope

tensorflow/stream_executor/cuda/cuda_dnn.cc:963:4: error: no matching function for call to 'std::vector::assign()'

tensorflow/stream_executor/cuda/cuda_dnn.cc:166:39: error: too many arguments to function

Complete log of build: http://pastebin.com/NzpiwmGm

The compilation command that I am using is:

bazel build --verbose_failures --genrule_strategy=standalone --spawn_strategy=standalone -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer >& build.log &

My systems information is as follows

Environment info

Operating System:

Distributor ID: Ubuntu

Description: Ubuntu 16.04.1 LTS

Release: 16.04

Codename: xenial

Installed version of CUDA and cuDNN:

/usr/lib/x86_64-linux-gnu/libcudart.so.7.5.18

/usr/lib/x86_64-linux-gnu/libcudnn.so.7.0

If installed from source, provide

The commit hash: bf31051

The output of bazel version

Build time: Thu Jan 01 00:00:00 1970 (0)

Build timestamp: Thu Jan 01 00:00:00 1970 (0)

Build timestamp as int: 0

I installed cudatoolkit, and cudann following this instructions:

sudoapt-get install nvidia-cuda-toolkit

sudo apt-get install nvidia-cuda-361-updates

sudo apt-get install nvidia-nsight

sudo apt-get install nvidia-profiler

sudo apt-get install libcupti-dev zlib1g-dev

Put symlinks in /usr/local/cuda

sudo mkdir /usr/local/cuda

cd /usr/local/cuda

sudo ln -s /usr/lib/x86_64-linux-gnu/ lib64

sudo ln -s /usr/include/ include

sudo ln -s /usr/bin/ bin

sudo ln -s /usr/lib/x86_64-linux-gnu/ nvvm

sudo mkdir -p extras/CUPTI

cd extras/CUPTI

sudo ln -s /usr/lib/x86_64-linux-gnu/ lib64

sudo ln -s /usr/include/ include

Install cudann

http://askubuntu.com/questions/767269/how-can-i-install-cudnn-on-ubuntu-16-04

Download cudann as detailed above and extract

cd ~/Downloads/cuda

sudo cp include/cudnn.h /usr/include

sudo cp lib64/libcudnn* /usr/lib/x86_64-linux-gnu/

sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*

... Install TensorFlow from source ...

instruction found at https://devtalk.nvidia.com/default/topic/936212/cuda-setup-and-installation/tensorflow-cannot-find-cudnn-ubuntu-16-04-cuda7-5-/post/4880549/#4880549

I ecounter the problems described in issue #1066. I solved it using the solution described by @chrisburr: @drufat I ran into a similar issue when compiling on Arch and found the solution in #1346. I'm unsure if -D__STRICT_ANSI__ is actually required but the following patch worked for me:

diff --git a/third_party/gpus/crosstool/CROSSTOOL b/third_party/gpus/crosstool/CROSSTOOL

index dfde7cd..15fa9fd 100644

--- a/third_party/gpus/crosstool/CROSSTOOL

+++ b/third_party/gpus/crosstool/CROSSTOOL

@@ -46,6 +46,9 @@ toolchain {

Use "-std=c++11" for nvcc. For consistency, force both the host compiler

and the device compiler to use "-std=c++11".

cxx_flag: "-std=c++11"

cxx_flag: "-D_MWAITXINTRIN_H_INCLUDED"

cxx_flag: "-D_FORCE_INLINES"

cxx_flag: "-D__STRICT_ANSI__"

linker_flag: "-lstdc++"

linker_flag: "-B/usr/bin/"

Next, I got the issue #698.

Solved it by compiling with --genrule_strategy=standalone --spawn_strategy=standalone, as suggested by @damienmg

Issue 2:
System information

TensorFlow version (you are using): 1.12

Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.

AttentionWrapper fuses the cell_output and the context vector using an array_ops.concat operation when specifying an attention layer: link

In my case, this behaviour is not flexible enough, I and would like to be able to define my own function to replace the concat operation.

Will this change the current api? How?

Similarly to cell_input_fn, I would like to have a attention_layer_fn in AttentionWrapper, which by default will perform a concat, preserving the old behaviour. This function would be passed to _compute_attention, which could be modified like this:

if attention_layer is not None:
attention_layer_input = attention_layer_fn(cell_output, context)
attention = attention_layer(attention_layer_input)
else:
attention = context

where

if attention_input_fn is None:
attention_input_fn = (
lambda cell_output, context: array_ops.concat([cell_output, context], 1))

Who will benefit with this feature?

TensorFlow users designing new attention architectures may find this feature useful.

The users have to implement their own function taking cell_output and context as inputs.

Any Other info.

I am not a super experienced developer, so I would also like to ask for advice.

I implemented the feature locally and successfully trained a network, but ran into a new problem.

The main goal would be to debug/visualise some internal ops defined by the user inside the new function, just like with alignment_history. And here is where things get complicated.

Since these operations are created inside the while_loop of dynamic_rnn, I'd have to use a TensorArray and write the variables at every step (otherwise it says that the operation has been marked as not fetchable), but this also requires state.time when calling write(). Then the object returned by write() should be added to the AttentionWrapperState, to show up in the final_state when running dynamic_rnn. Is there a simpler way to visualise all the necessary ops inside the envisaged attention_input_fn ?