text: 
  
  TITLE
  Getting "TypeError: forward() got multiple values for argument 'attention_mask'" when replacing pytorch_transformers with transformers
  
  COMMENTS
  8
  
  REACTIONS
  +1: 1
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 📚 Migration
  
  ## Information
  
  <!-- Important information -->
  
  Model I am using (Bert):
  
  Language I am using the model on (Japanese):
  
  The problem arises when using:
  * [ ] the official example scripts: (give details below)
  * [* ] my own modified scripts: (give details below)
  
  The tasks I am working on is:
  * [ ] an official GLUE/SQUaD task: (give the name)
  * [* ] my own task or dataset: (give details below)
  
  ## Details
  
  <!-- A clear and concise description of the migration issue.
      If you have code snippets, please provide it here as well.
      Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
      Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
      -->
  
  This is the complaint from python:
  
  /content/train_extractive.py in train_ext(args, device_id)
      225         train_multi_ext(args)
      226     else:
  --> 227         train_single_ext(args, device_id)
      228 
      229 
  
  /content/train_extractive.py in train_single_ext(args, device_id)
      267 
      268     trainer = build_trainer(args, device_id, model, optim)
  --> 269     trainer.train(train_iter_fct, args.train_steps)
  
  /content/trainer_ext.py in train(self, train_iter_fct, train_steps, valid_iter_fct, valid_steps)
      150                         self._gradient_accumulation(
      151                             true_batchs, normalization, total_stats,
  --> 152                             report_stats)
      153 
      154                         report_stats = self._maybe_report_training(
  
  /content/trainer_ext.py in _gradient_accumulation(self, true_batchs, normalization, total_stats, report_stats)
      393             mask_cls = batch.mask_cls
      394 
  --> 395             sent_scores, mask = self.model(src, segs, clss, mask, mask_cls)
      396 
      397             loss = self.loss(sent_scores, labels.float())
  
  /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      720             result = self._slow_forward(*input, **kwargs)
      721         else:
  --> 722             result = self.forward(*input, **kwargs)
      723         for hook in itertools.chain(
      724                 _global_forward_hooks.values(),
  
  /content/model_builder.py in forward(self, src, segs, clss, mask_src, mask_cls)
      176         print (type(mask_src))
      177         print (mask_src)
  --> 178         top_vec = self.bert(src, segs, mask_src)
      179         sents_vec = top_vec[torch.arange(top_vec.size(0)).unsqueeze(1), clss]
      180         sents_vec = sents_vec * mask_cls[:, :, None].float()
  
  /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      720             result = self._slow_forward(*input, **kwargs)
      721         else:
  --> 722             result = self.forward(*input, **kwargs)
      723         for hook in itertools.chain(
      724                 _global_forward_hooks.values(),
  
  /content/model_builder.py in forward(self, x, segs, mask)
      126     def forward(self, x, segs, mask):
      127         if(self.finetune):
  --> 128             top_vec, _ = self.model(x, segs, attention_mask=mask)
      129         else:
      130             self.eval()
  
  /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      720             result = self._slow_forward(*input, **kwargs)
      721         else:
  --> 722             result = self.forward(*input, **kwargs)
      723         for hook in itertools.chain(
      724                 _global_forward_hooks.values(),
  
  TypeError: forward() got multiple values for argument 'attention_mask'
  
  ***
  
  I get the above complaint after replacing pytorch-transfomers with transformers. 
  
  from pytorch_transformers import BertModel
  ->
  from transformers import BertForMaskedLM
  
  I have to make this change because I am importing the Japanese model when the original code calling BertModel only caters to English model
  
  ## Environment info
  <!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
  
  python: can't open file 'transformers-cli': [Errno 2] No such file or directory 
  
  - `transformers` version:
  - Platform: ubuntu colab
  - Python version: 3.6.9
  - PyTorch version (GPU?): 1.6.0+cu101
  - Tensorflow version (GPU?): 2.3
  - Using GPU in script?: yes
  - Using distributed or parallel set-up in script?:
  
  <!-- IMPORTANT: which version of the former library do you use? -->
  * `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch):
  pytorch-transformers
  
  ## Checklist
  
  - [ *] I have read the migration guide in the readme.
   ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
    [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
  - [ *] I checked if a related official extension example runs on my machine.
  