Repository: OpenNMT/OpenNMT-py

Issue:
Transformer model for summarization: copy_generator.py - IndexError: too many indices for tensor of dimension 0
When I try to train my model I get the following

```
Traceback (most recent call last):
  File "onmt_train.py", line 123, in <module>
    main(opt)
  File "onmt_train.py", line 54, in main
    single_main(opt, 0)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/onmt/train_single.py", line 154, in main
    opt.valid_steps)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/onmt/trainer.py", line 175, in train
    report_stats)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/onmt/trainer.py", line 288, in _gradient_accumulation
    trunc_size, self.shard_size, normalization)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/onmt/utils/loss.py", line 170, in sharded_compute_loss
    loss, stats = self._compute_loss(batch, **shard)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/onmt/modules/copy_generator.py", line 188, in _compute_loss
    loss = self.criterion(scores, align, target)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/onmt/modules/copy_generator.py", line 132, in forward
    copy_tok_probs[align == self.unk_index] = 0
IndexError: too many indices for tensor of dimension 0
```

I have no idea where it comes from, it comes in step 800, but after 337 steps the network should have seen all my data (I've set it to checkpoint and validate every 400 steps).

I've also tried to find whether this comes from an empty sequence, but I explicitly deleted examples that contain less than one token (after stripping whitespace).


The command I use for training:
```
train.py -data /home/ubuntu/efs/data/WikiHowSep \
    ...:     -save_model /home/ubuntu/efs/models/WikiHowSep \
    ...:     -layers 4 \
    ...:     -rnn_size 512 \
    ...:     -word_vec_size 512 \
    ...:     -max_grad_norm 0 \
    ...:     -optim adam \
    ...:     -encoder_type transformer \
    ...:     -decoder_type transformer \
    ...:     -position_encoding \
    ...:     -dropout 0\.2 \
    ...:     -param_init 0 \
    ...:     -warmup_steps 100 \
    ...:     -learning_rate 2 \
    ...:     -decay_method noam \
    ...:     -label_smoothing 0.1 \
    ...:     -adam_beta2 0.998 \
    ...:     -save_checkpoint_steps 400 \
    ...:     -batch_size $batch_size \
    ...:     -batch_type tokens \
    ...:     -normalization tokens \
    ...:     -max_generator_batches 2 \
    ...:     -train_steps $train_steps \
    ...:     -accum_count 4 \
    ...:     -share_embeddings \
    ...:     -copy_attn \
    ...:     -param_init_glorot \
    ...:     -gpu_ranks 0 \
    ...:     -valid_steps 400 \
    ...:     -tensorboard \
    ...:     -tensorboard_log_dir /home/ubuntu/efs/models/tensorboard/WikiHowSep
```

Command I used for preprocessing:
```
preprocess.py -train_src $DATA_PATH/wikihow_train_text.txt \
                     -train_tgt $DATA_PATH/wikihow_train_headline.txt \
                     -valid_src $DATA_PATH/wikihow_val_text.txt \
                     -valid_tgt $DATA_PATH/wikihow_val_headline.txt \
                     -save_data ~/efs/data/WikiHowSep \
                     -src_seq_length_trunc 200 \
                     -tgt_seq_length_trunc 20 \
                     -dynamic_dict \
                     -share_vocab \
                     -shard_size 100000
```


Discussion:
give more log, but if it happens at step 800, it must be during validation.
@bpopeters can you double check if it could be related to #1057 ?
thanks
I'll look into it.
It appears that the problem comes from the use of `squeeze` without a specified dimension in the `CopyGeneratorLoss` forward. If the batch dimension of the `scores` tensor is only of size 1, then squeeze will cause the `copy_tok_probs` tensor to be dimensionless, i.e. just a scalar, and that causes the indexing bug because indexing a scalar doesn't make any sense. Setting `-batch_type tokens` can cause the batch dimension of scores to be of size 1. The error is not specific to validation -- with the set I'm debugging with, I get it at train time as well.

Fix upcoming.

Resolving patch:
diff --git a/onmt/modules/copy_generator.py b/onmt/modules/copy_generator.py
--- a/onmt/modules/copy_generator.py
+++ b/onmt/modules/copy_generator.py
@@ -123,11 +123,11 @@ def forward(self, scores, align, target):
         target (LongTensor): (batch_size*tgt_len)
         """
         # probabilities assigned by the model to the gold targets
-        vocab_probs = scores.gather(1, target.unsqueeze(1)).squeeze()
+        vocab_probs = scores.gather(1, target.unsqueeze(1)).squeeze(1)
 
         # probability of tokens copied from source
         copy_ix = align.unsqueeze(1) + self.vocab_size
-        copy_tok_probs = scores.gather(1, copy_ix).squeeze()
+        copy_tok_probs = scores.gather(1, copy_ix).squeeze(1)
         # Set scores for unk to 0 and add eps
         copy_tok_probs[align == self.unk_index] = 0
         copy_tok_probs += self.eps  # to avoid -inf logs


Test patch: