Getting "TypeError: forward() got multiple values for argument 'attention_mask'" when replacing pytorch_transformers with transformers
Author: wailoktam
Pull request: no
Labels: name: wontfix

name: Migration

# 📚 Migration

## Information

<!-- Important information -->

Model I am using (Bert):

Language I am using the model on (Japanese):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [* ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [* ] my own task or dataset: (give details below)

## Details

<!-- A clear and concise description of the migration issue.
    If you have code snippets, please provide it here as well.
    Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
    Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
    -->

This is the complaint from python:

/content/train_extractive.py in train_ext(args, device_id)
    225         train_multi_ext(args)
    226     else:
--> 227         train_single_ext(args, device_id)
    228 
    229 

/content/train_extractive.py in train_single_ext(args, device_id)
    267 
    268     trainer = build_trainer(args, device_id, model, optim)
--> 269     trainer.train(train_iter_fct, args.train_steps)

/content/trainer_ext.py in train(self, train_iter_fct, train_steps, valid_iter_fct, valid_steps)
    150                         self._gradient_accumulation(
    151                             true_batchs, normalization, total_stats,
--> 152                             report_stats)
    153 
    154                         report_stats = self._maybe_report_training(

/content/trainer_ext.py in _gradient_accumulation(self, true_batchs, normalization, total_stats, report_stats)
    393             mask_cls = batch.mask_cls
    394 
--> 395             sent_scores, mask = self.model(src, segs, clss, mask, mask_cls)
    396 
    397             loss = self.loss(sent_scores, labels.float())

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/content/model_builder.py in forward(self, src, segs, clss, mask_src, mask_cls)
    176         print (type(mask_src))
    177         print (mask_src)
--> 178         top_vec = self.bert(src, segs, mask_src)
    179         sents_vec = top_vec[torch.arange(top_vec.size(0)).unsqueeze(1), clss]
    180         sents_vec = sents_vec * mask_cls[:, :, None].float()

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/content/model_builder.py in forward(self, x, segs, mask)
    126     def forward(self, x, segs, mask):
    127         if(self.finetune):
--> 128             top_vec, _ = self.model(x, segs, attention_mask=mask)
    129         else:
    130             self.eval()

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

TypeError: forward() got multiple values for argument 'attention_mask'

***

I get the above complaint after replacing pytorch-transfomers with transformers. 

from pytorch_transformers import BertModel
->
from transformers import BertForMaskedLM

I have to make this change because I am importing the Japanese model when the original code calling BertModel only caters to English model

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

python: can't open file 'transformers-cli': [Errno 2] No such file or directory 

- `transformers` version:
- Platform: ubuntu colab
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101
- Tensorflow version (GPU?): 2.3
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:

<!-- IMPORTANT: which version of the former library do you use? -->
* `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch):
pytorch-transformers

## Checklist

- [ *] I have read the migration guide in the readme.
 ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
  [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [ *] I checked if a related official extension example runs on my machine.


Comments:
Hi, I believe this is the cause of your issue: https://huggingface.co/transformers/migration.html#positional-order-of-some-models-keywords-inputs-attention-mask-token-type-ids-changed

Thanks. I agree. Can you suggest where I should fix in the codes given in the error log?

I can't really see where's your code, do you mind pasting a snippet that reproduces the error? (using backticks \`\`\` to format it)

Thanks. I highlight the codes given in the error log. In case, it is of no use. Please let me know how I should dig up the relevant portion. 

/content/train_extractive.py in train_ext(args, device_id)
```
225 train_multi_ext(args)
226 else:
--> 227 train_single_ext(args, device_id)
228
229
```

/content/train_extractive.py in train_single_ext(args, device_id)
```
267
268 trainer = build_trainer(args, device_id, model, optim)
--> 269 trainer.train(train_iter_fct, args.train_steps)
```

/content/trainer_ext.py in train(self, train_iter_fct, train_steps, valid_iter_fct, valid_steps)

```
150 self._gradient_accumulation(
151 true_batchs, normalization, total_stats,
--> 152 report_stats)
153
154 report_stats = self._maybe_report_training(
```

/content/trainer_ext.py in _gradient_accumulation(self, true_batchs, normalization, total_stats, report_stats)
```
393 mask_cls = batch.mask_cls
394
--> 395 sent_scores, mask = self.model(src, segs, clss, mask, mask_cls)
396
397 loss = self.loss(sent_scores, labels.float())
```

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
```
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
```

/content/model_builder.py in forward(self, src, segs, clss, mask_src, mask_cls)
```
176 print (type(mask_src))
177 print (mask_src)
--> 178 top_vec = self.bert(src, segs, mask_src)
179 sents_vec = top_vec[torch.arange(top_vec.size(0)).unsqueeze(1), clss]
180 sents_vec = sents_vec * mask_cls[:, :, None].float()
```

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
```
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
```

/content/model_builder.py in forward(self, x, segs, mask)
```
126 def forward(self, x, segs, mask):
127 if(self.finetune):
--> 128 top_vec, _ = self.model(x, segs, attention_mask=mask)
129 else:
130 self.eval()
```

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)

```
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
```


I try replacing all the parameters in 

```
--> 128 top_vec, _ = self.model(x, segs, attention_mask=mask)
```
of /content/model_builder.py in forward(self, x, segs, mask)
to
```
--> 128 top_vec, _ = self.model(input_ids=x, token_type_ids=segs, attention_mask=mask)
```
Now the changed line gives this error:

ValueError: not enough values to unpack (expected 2, got 1)

Now I am left with no clue. 


I remove the "_" from the returned value of model in 

```
--> 128 top_vec, _ = self.model(input_ids=x, token_type_ids=segs, attention_mask=mask)
```

And get away with the error: ValueError: not enough values to unpack (expected 2, got 1)

But now I receive from this line:

```
    178         print (mask_src)
    179         top_vec = self.bert(src, segs, mask_src)
--> 180         sents_vec = top_vec[torch.arange(top_vec.size(0)).unsqueeze(1), clss]
    181         sents_vec = sents_vec * mask_cls[:, :, None].float()
    182         sent_scores = self.ext_layer(sents_vec, mask_cls).squeeze(-1)
```

of /content/model_builder.py in forward(self, src, segs, clss, mask_src, mask_cls) the following complaint:

AttributeError: 'tuple' object has no attribute 'size'

So the returned value of bert has changed. bert is an instance of BertForMaskedLM.from_pretrained('cl-tohoku/bert-base-japanese-whole-word-masking'). How do I get back the size of the first returned value of a pretrained model in the old torch_transformers?


Instead of simply removing the `_` value, which will not unpack the tuple anymore, you can get the first value of the tuple (which has a single value in your case):

```py
top_vec = self.model(x, segs, attention_mask=mask)[0]
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.