Repository: OpenNMT/OpenNMT-py

Issue:
RuntimeError: The size of tensor a (25) must match the size of tensor b (30) at non-singleton dimension 0
I used sentencepiece to encode my sentences, and get this error during translation.
_Command_ 
`python translate.py -model /workspace/snipgen/models/rnn_coverage_150d_4l_b10_h256_70000.pt -src /workspace/snipgen/data/content.test -tgt /workspace/snipgen/data/snippets.test  -output /workspace/snipgen/data/encoded/predictions/150d_4l_b10_h256.txt -verbose -gpu 0 -batch_size 6 -min_length 30  -beam_size 5 >> /workspace/snipgen/logs/trans_150d_4l_b10_h256.log 2>&1
`

_Output_
```
[2019-03-04 10:24:28,802 INFO] Translating shard 0.
/opt/conda/lib/python3.6/site-packages/torchtext/data/field.py:359: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  var = torch.tensor(arr, dtype=self.dtype, device=device)
Traceback (most recent call last):
  File "translate.py", line 48, in <module>
    main(opt)
  File "translate.py", line 32, in main
    attn_debug=opt.attn_debug
  File "/root/OpenNMT-py/onmt/translate/translator.py", line 322, in translate
    batch, data.src_vocabs, attn_debug
  File "/root/OpenNMT-py/onmt/translate/translator.py", line 511, in translate_batch
    return_attention=attn_debug or self.replace_unk)
  File "/root/OpenNMT-py/onmt/translate/translator.py", line 658, in _translate_batch
    batch_offset=beam._batch_offset)
  File "/root/OpenNMT-py/onmt/translate/translator.py", line 549, in _decode_and_generate
    decoder_in, memory_bank, memory_lengths=memory_lengths, step=step
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/OpenNMT-py/onmt/decoders/decoder.py", line 210, in forward
    tgt, memory_bank, memory_lengths=memory_lengths)
  File "/root/OpenNMT-py/onmt/decoders/decoder.py", line 405, in _run_forward_pass
    coverage = p_attn if coverage is None else p_attn + coverage
RuntimeError: The size of tensor a (25) must match the size of tensor b (30) at non-singleton dimension 0
```
(I am using Pytorch 1.0 and latest version of OpenNMT. I also checked for empty lines in both src and tgt and there are none.)



Discussion:
Can you please remove the min_length and run it again ?
I still get the same error with min_length removed.
FYI , my example looks like this
```

 ▁robert ▁falcon ▁scott ▁was ▁born ▁on ▁june ▁in ▁devon port ▁. ▁he ▁became ▁a ▁naval ▁cad et ▁at ▁the ▁age ▁of ▁and ▁served ▁on ▁a ▁number ▁of ▁royal ▁navy ▁ships ▁in ▁the ▁and ▁. ▁he ▁attracted ▁the ▁notice ▁of ▁the ▁royal ▁geographical ▁society ▁, ▁which ▁appointed ▁him ▁to ▁command ▁the ▁national ▁antarctic ▁expedition ▁of ▁. ▁the ▁expedition ▁which ▁included ▁ernest ▁shack leton ▁reached ▁further ▁south ▁than ▁anyone ▁before ▁them ▁and ▁scott ▁returned ▁to ▁britain ▁a ▁national ▁hero ▁. ▁he ▁had ▁caught ▁the ▁exploring ▁bug ▁and ▁began ▁to ▁plan ▁an ▁expedition ▁to ▁be ▁the ▁first ▁to ▁reach ▁the ▁south ▁pole ▁. ▁he ▁spent ▁years ▁raising ▁funds ▁for ▁the ▁trip ▁. ▁the ▁whaling ▁ship ▁terra ▁nova ▁left ▁cardiff ▁, ▁wales ▁in ▁june ▁and ▁the ▁expedition ▁set ▁off ▁from ▁base ▁the ▁following ▁october ▁, ▁with ▁mechanical ▁sled ges ▁, ▁pon ies ▁and ▁dogs ▁. ▁however ▁, ▁the ▁sled ges ▁and ▁pon ies ▁could ▁not ▁cope ▁with ▁the ▁conditions ▁and ▁the ▁expedition ▁carried ▁on ▁without ▁them ▁, ▁through ▁app alling ▁weather ▁and ▁increasingly ▁tough ▁terrain ▁. ▁in ▁mid ▁december ▁, ▁the ▁dog ▁teams ▁turned ▁back ▁, ▁leaving ▁the ▁rest ▁to ▁face ▁the ▁ascent ▁of ▁the ▁beard more ▁glacier ▁and ▁the ▁polar ▁plateau ▁. ▁by ▁january ▁, ▁only ▁five ▁remained ▁scott ▁, ▁wilson ▁, ▁o ates ▁, ▁bow ers ▁and ▁evans ▁. ▁on ▁january ▁, ▁they ▁reached ▁the ▁pole ▁, ▁only ▁to ▁find ▁that ▁a ▁norwegian ▁party ▁led ▁by ▁ro ald ▁am und sen ▁, ▁had ▁beaten ▁them ▁there ▁. ▁they ▁started ▁the ▁km ▁journey ▁back ▁. ▁evans ▁died ▁in ▁. ▁by ▁march ▁, ▁o ates ▁was ▁suffering ▁from ▁severe ▁frost b ite ▁and ▁, ▁knowing ▁he ▁was ▁holding ▁back ▁his ▁companions ▁, ▁walked ▁out ▁into ▁the ▁freezing ▁conditions ▁never ▁to ▁be ▁seen ▁again ▁. ▁the ▁remaining ▁three ▁men ▁died ▁of ▁starvation ▁and ▁exposure ▁in ▁their ▁tent ▁on ▁march ▁. ▁they ▁were ▁in ▁fact ▁only ▁km ▁from ▁a ▁supply ▁depot ▁. 


```
post your training command line so that we make sure we know all the options used (coverage atttn in particular)
@flauted if you get a chance to have a look. thanks.
Training command

```
python train.py -save_model /workspace/snipgen/models/rnn_coverage_150d_4l_b10_h256  -data /workspace/snipgen/data/snip_800  -encoder_type brnn  -decoder_type rnn -coverage_attn  -global_attention general  -word_vec_size 150 -share_embeddings -layers 4  -rnn_size 256  -train_steps 250000 -self_attn_type scaled-dot -seed 1234 -max_grad_norm 5  -batch_size 10  -valid_batch_size 6  -optim adam  -adam_beta1 0.9  -adam_beta2 0.999  -learning_rate 0.001  -seed 777 -learning_rate_decay 0.2 -world_size 1  -share_embeddings  -tensorboard  -tensorboard_log_dir /workspace/snipgen/logs/tblogs/rnn_coverage_150d_3l_b32  -gpu_ranks 0  -log_file /workspace/snipgen/logs/train_rnn_coverage_150d_3l_b32.log 2>&1

```
I'm able to reproduce this with the demo data on master (d6870f907d77ec90bab482b356dd4599b803b477) and on 8.2 (518a19d).
```
python preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo [-share_vocab]
```
```
python train.py -save_model testing  -data data/demo  -encoder_type brnn  -decoder_type rnn -coverage_attn  -global_attention general [-share_embeddings]  -save_checkpoint_steps 10 -max_grad_norm 5  -batch_size 10  -valid_batch_size 6  -optim adam  -adam_beta1 0.9  -adam_beta2 0.999  -learning_rate 0.001  -seed 777 -learning_rate_decay 0.2 -world_size 1  -gpu_ranks 0
```
```
python translate.py -model testing_step_110.pt -src data/src-val.txt -tgt data/tgt-val.txt -output testing.txt -verbose -gpu 0 -batch_size 6 -min_length 30 -beam_size 5
```
translates 36 sentences successfully, then
```
Traceback (most recent call last):
  File "translate.py", line 48, in <module>
    main(opt)
  File "translate.py", line 32, in main
    attn_debug=opt.attn_debug
  File "/home/dylan/code/OpenNMT-py/onmt/translate/translator.py", line 322, in translate
    batch, data.src_vocabs, attn_debug
  File "/home/dylan/code/OpenNMT-py/onmt/translate/translator.py", line 511, in translate_batch
    return_attention=attn_debug or self.replace_unk)
  File "/home/dylan/code/OpenNMT-py/onmt/translate/translator.py", line 658, in _translate_batch
    batch_offset=beam._batch_offset)
  File "/home/dylan/code/OpenNMT-py/onmt/translate/translator.py", line 549, in _decode_and_generate
    decoder_in, memory_bank, memory_lengths=memory_lengths, step=step
  File "/home/dylan/miniconda3/envs/opennmt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/dylan/code/OpenNMT-py/onmt/decoders/decoder.py", line 210, in forward
    tgt, memory_bank, memory_lengths=memory_lengths)
  File "/home/dylan/code/OpenNMT-py/onmt/decoders/decoder.py", line 405, in _run_forward_pass
    coverage = p_attn if coverage is None else p_attn + coverage
RuntimeError: The size of tensor a (5) must match the size of tensor b (30) at non-singleton dimension 0
```

You have to train a little longer (like, extra 30 seconds) for this to show up without Adam but it does.
```
python train.py -save_model testing  -data data/demo  -encoder_type brnn  -decoder_type rnn -coverage_attn  -global_attention general -save_checkpoint_steps 10 -max_grad_norm 5  -batch_size 10  -valid_batch_size 6  -seed 777 -world_size 1  -gpu_ranks 0
```
```
python translate.py -model testing_step_220.pt -src data/src-val.txt -tgt data/tgt-val.txt -output testing.txt -verbose -gpu 0 -batch_size 6 -min_length 30 -beam_size 5
```

I'm not _seeing_ it with a unidirectional encoder. Doesn't mean it doesn't happen. It is odd that this seems to depend on how trained your model is.
but what are the tensor sizes supposed to be (5) and (30) in your run [(25) and (30) in the original post] ?
I mean what dimensions are they supposed to represent ?
We're talking about [this line](https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/decoders/decoder.py#L405)

``tensor a`` is ``p_attn`` and ``tensor b`` is ``coverage``. Dimension 0 is batch.

Training: At step 0, ``coverage`` is None. ``p_attn`` has shape ``(batch_size, seq_len_in)``. At step >0 ``coverage`` and ``p_attn`` should be ``(batch_size, seq_len_in)``.

Translation: Same, except ``batch_size x beam_size``, with the beam search dropping finished batches.

Looks like the problem is that the finished batches don't get dropped from ``coverage``.

Adding this to [``map_state``](https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/decoders/decoder.py#L185) seems to fix the problem.
```
if self._coverage and self.state["coverage"] is not None:
    self.state["coverage"] = fn(self.state["coverage"], 1)
```

I'll send a PR shortly.

Resolving patch:
diff --git a/onmt/decoders/decoder.py b/onmt/decoders/decoder.py
--- a/onmt/decoders/decoder.py
+++ b/onmt/decoders/decoder.py
@@ -182,6 +182,8 @@ def _fix_enc_hidden(hidden):
     def map_state(self, fn):
         self.state["hidden"] = tuple(fn(h, 1) for h in self.state["hidden"])
         self.state["input_feed"] = fn(self.state["input_feed"], 1)
+        if self._coverage and self.state["coverage"] is not None:
+            self.state["coverage"] = fn(self.state["coverage"], 1)
 
     def detach_state(self):
         self.state["hidden"] = tuple(h.detach() for h in self.state["hidden"])


Test patch: