Repository: OpenNMT/OpenNMT-tf

Issue:
Error while serving nmtsmall model 
Hi
I trained nmtsmall model by using opennmt 1.21.3. Then I upgradeed to 1.21.4 for exporting model.
When I used export model serving, I have met this error.

Error
Traceback (most recent call last):
  File "D:\GNMT\venv\lib\site-packages\grpc\beta\_client_adaptations.py", line 95, in result
    return self._future.result(timeout=timeout)
  File "D:\GNMT\venv\lib\site-packages\grpc\_channel.py", line 276, in result
    raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero
	 [[Node: seq2seq/Reshape_1 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _class=["loc:@seq2seq/cond/strided_slice/Switch"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](seq2seq/decoder_1/strided_slice_17, seq2seq/Reshape_1/shape)]]
	 [[Node: seq2seq/decoder_1/while/concat_5/_225 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1187_seq2seq/decoder_1/while/concat_5", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopseq2seq/decoder_1/while/decoder/decoder/attention_wrapper/assert_equal/Assert/Assert/data_0/_39)]]"
	debug_error_string = "{"created":"@1552276258.713000000","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1095,"grpc_message":"Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero\n\t [[Node: seq2seq/Reshape_1 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _class=["loc:@seq2seq/cond/strided_slice/Switch"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](seq2seq/decoder_1/strided_slice_17, seq2seq/Reshape_1/shape)]]\n\t [[Node: seq2seq/decoder_1/while/concat_5/_225 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1187_seq2seq/decoder_1/while/concat_5", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopseq2seq/decoder_1/while/decoder/decoder/attention_wrapper/assert_equal/Assert/Assert/data_0/_39)]]","grpc_status":3}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python35\Lib\unittest\case.py", line 59, in testPartExecutor
    yield
  File "C:\Python35\Lib\unittest\case.py", line 605, in run
    testMethod()
  File "D:\GNMT\tests\onmt_rpc_client_test.py", line 78, in test_request_from_file
    address, result = client.request(line.strip("\n").split(" "))
  File "D:\GNMT\address\client\onmt_rpc_client.py", line 11, in request
    result = self._parse_translation_result(future.result())
  File "D:\GNMT\venv\lib\site-packages\grpc\beta\_client_adaptations.py", line 97, in result
    raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero
	 [[Node: seq2seq/Reshape_1 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _class=["loc:@seq2seq/cond/strided_slice/Switch"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](seq2seq/decoder_1/strided_slice_17, seq2seq/Reshape_1/shape)]]
	 [[Node: seq2seq/decoder_1/while/concat_5/_225 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1187_seq2seq/decoder_1/while/concat_5", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopseq2seq/decoder_1/while/decoder/decoder/attention_wrapper/assert_equal/Assert/Assert/data_0/_39)]]")


===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: builtins.tuple: (<class 'grpc.framework.interfaces.face.face.AbortionError'>, AbortionError(), <traceback object at 0x000000001C31D748>)

onmt_rpc_client_test.TestOpenNMTPredictRPCClient.test_request_from_file
-------------------------------------------------------------------------------
Ran 1 tests in 45.879s

FAILED (errors=1)
Process finished with exit code 1



Discussion:
Hi,

Can you post the configuration file that was used when exporting the model?
Hi 
Configuration:

params:
  optimizer: GradientDescentOptimizer
  learning_rate: 1.0
  param_init: 0.1
  clip_gradients: 5.0
  decat_type: exponential_decay
  decay_rate: 0.7
  decay_steps: 50000
  start_decay_steps: 500000
  beam_width: 5
  maximum_iterations: 250
  replace_unknown_target: true

train:
  batch_size: 64
  bucket_width: 5
  save_checkpoints_steps: 10000
  save_summary_steps: 1000
  train_steps: 1000000
  maximum_features_length: 30
  maximum_labels_length: 30
  sample_buffer_size: -1

eval:
  batch_size: 32
  num_threads: 4
  eval_delay: 36000
  save_eval_predictions: true
  external_evaluators: BLEU
  exporters: last

infer:
  batch_size: 32
  num_threads: 1
  n_best: 1

Resolving patch:
diff --git a/opennmt/models/sequence_to_sequence.py b/opennmt/models/sequence_to_sequence.py
--- a/opennmt/models/sequence_to_sequence.py
+++ b/opennmt/models/sequence_to_sequence.py
@@ -10,7 +10,7 @@
 from opennmt.models.model import Model
 from opennmt.utils import compat
 from opennmt.utils.losses import cross_entropy_sequence_loss
-from opennmt.utils.misc import print_bytes, format_translation_output, merge_dict
+from opennmt.utils.misc import print_bytes, format_translation_output, merge_dict, shape_list
 from opennmt.decoders.decoder import get_sampling_probability
 
 
@@ -277,7 +277,9 @@ def _call(self, features, labels, params, mode):
         # Merge batch and beam dimensions.
         original_shape = tf.shape(target_tokens)
         target_tokens = tf.reshape(target_tokens, [-1, original_shape[-1]])
-        attention = tf.reshape(alignment, [-1, tf.shape(alignment)[2], tf.shape(alignment)[3]])
+        align_shape = shape_list(alignment)
+        attention = tf.reshape(
+            alignment, [align_shape[0] * align_shape[1], align_shape[2], align_shape[3]])
         # We don't have attention for 
 but ensure that the attention time dimension matches
         # the tokens time dimension.
         attention = reducer.align_in_time(attention, tf.shape(target_tokens)[1])


Test patch: