Issue 1:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:No

TensorFlow installed from (source or binary):Binary with gpu supported

TensorFlow version (use command below):1.11.0

Python version:python 27

Bazel version (if compiling from source):

GCC/Compiler version (if compiling from source):

CUDA/cuDNN version:9.0

GPU model and memory: ec2 p2.16xlarge 16 NVIDIA K80 GPUs

Exact command to reproduce:

Describe the problem

Hi all, I already opened this issue in tensorflow-serving but I think this might be a better place to open this issue. Please let know if I need to close the previous one.

I used the code below to test whether tensor-serving works with tensorflow. Here are some combinations and results:

TF 1.10.0 TF-Serving 1.10.0 Success

TF 1.11.0 TF-Serving 1.10.0 Failed

TF 1.11.0 TF-Serving 1.11.0rc1 Failed

Source code / logs

python tensorflow_serving/example/mnist_saved_model.py /tmp/mnist_model || exit 1

tensorflow_model_server --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/ &
SERVER_PID=$!

python mnist_client.py --num_tests=1000 --server=localhost:9000 || exit 1

kill -9 $SERVER_PID

Log

Training model...
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting /tmp/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting /tmp/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting /tmp/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting /tmp/t10k-labels-idx1-ubyte.gz
training accuracy 0.9092
Done training!
Exporting trained model to /tmp/mnist_model/1
WARNING:tensorflow:From mnist_saved_model.py:139: calling add_meta_graph_and_variables (from tensorflow.python.saved_model.builder_impl) with legacy_init_op is deprecated and will be removed in a future version.
Instructions for updating:
Pass your op to the equivalent parameter main_op instead.
Done exporting!
Extracting /tmp/train-images-idx3-ubyte.gz
Extracting /tmp/train-labels-idx1-ubyte.gz
Extracting /tmp/t10k-images-idx3-ubyte.gz
Extracting /tmp/t10k-labels-idx1-ubyte.gz
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
2018-10-02 21:18:32.604303: E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range)]]
2018-10-02 21:18:32.604833: E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range)]]
2018-10-02 21:18:32.604980: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: mnist version: 1} failed: Invalid argument: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range)]]
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>

Issue 2:
System information

Hardware: AWS EC2 p2.16xlarge

OS Platform and Distribution: Linux Ubuntu 16.04

TensorFlow Serving installed from (source or binary):

pip install tensorflow-serving-api

pip install tensorflow-serving-api==1.11.0rc1

TensorFlow Serving version:

both 1.10.0 and 1.11.0rc1

Describe the problem

I used the code below to test whether tensor-serving works with tensorflow. Here are some combinations and results:

TF 1.10.0 TF-Serving 1.10.0 Success

TF 1.11.0 TF-Serving 1.10.0 Failed

TF 1.11.0 TF-Serving 1.11.0rc1 Failed

Exact Steps to Reproduce

Here is the example code that I run to reproduce the problem:

python mnist_saved_model.py /tmp/mnist_model || exit 1

tensorflow_model_server --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/ &
SERVER_PID=$!

python mnist_client.py --num_tests=1000 --server=localhost:9000 || exit 1

kill -9 $SERVER_PID

Source code / logs

For minimum code please see above reproduce section. Logs:

Training model...
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting /tmp/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting /tmp/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting /tmp/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting /tmp/t10k-labels-idx1-ubyte.gz
training accuracy 0.9092
Done training!
Exporting trained model to /tmp/mnist_model/1
WARNING:tensorflow:From mnist_saved_model.py:139: calling add_meta_graph_and_variables (from tensorflow.python.saved_model.builder_impl) with legacy_init_op is deprecated and will be removed in a future version.
Instructions for updating:
Pass your op to the equivalent parameter main_op instead.
Done exporting!
Extracting /tmp/train-images-idx3-ubyte.gz
Extracting /tmp/train-labels-idx1-ubyte.gz
Extracting /tmp/t10k-images-idx3-ubyte.gz
Extracting /tmp/t10k-labels-idx1-ubyte.gz
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
2018-10-02 21:18:32.604303: E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range)]]
2018-10-02 21:18:32.604833: E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range)]]
2018-10-02 21:18:32.604980: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: mnist version: 1} failed: Invalid argument: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: index_to_string/ToInt64 = Cast[DstT=DT_INT64, SrcT=DT_INT32, Truncate=false, _output_shapes=[[10]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](index_to_string/range)]]
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded)>
<_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>