Repository: facebookresearch/ParlAI

Issue:
HuggingFace model only works with transformers <4.0.0
**Bug description**

With transfomers 4.0 or greater:
```
$ parlai dm -t convai2 -m hugging_face/gpt2 -bs 1
```
```python-traceback
Traceback (most recent call last):
  File "/private/home/roller/.conda/envs/chat202001/bin/parlai", line 33, in <module>
    sys.exit(load_entry_point('parlai', 'console_scripts', 'parlai')())
  File "/private/home/roller/working/parlai/parlai/__main__.py", line 14, in main
    superscript_main()
  File "/private/home/roller/working/parlai/parlai/core/script.py", line 307, in superscript_main
    return SCRIPT_REGISTRY[cmd].klass._run_from_parser_and_opt(opt, parser)
  File "/private/home/roller/working/parlai/parlai/core/script.py", line 90, in _run_from_parser_and_opt
    return script.run()
  File "/private/home/roller/working/parlai/parlai/scripts/display_model.py", line 91, in run
    display_model(self.opt)
  File "/private/home/roller/working/parlai/parlai/scripts/display_model.py", line 70, in display_model
    world.parley()
  File "/private/home/roller/working/parlai/parlai/core/worlds.py", line 346, in parley
    acts[1] = agents[1].act()
  File "/private/home/roller/working/parlai/parlai/core/torch_agent.py", line 1946, in act
    response = self.batch_act([self.observation])[0]
  File "/private/home/roller/working/parlai/parlai/core/torch_agent.py", line 2007, in batch_act
    output = self.eval_step(batch)
  File "/private/home/roller/working/parlai/parlai/core/torch_generator_agent.py", line 880, in eval_step
    loss, model_output = self.compute_loss(batch, return_output=True)
  File "/private/home/roller/working/parlai/parlai/core/torch_generator_agent.py", line 710, in compute_loss
    model_output = self.model(*self._model_input(batch), ys=batch.label_vec)
  File "/private/home/roller/.conda/envs/chat202001/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/private/home/roller/working/parlai/parlai/core/torch_generator_agent.py", line 328, in forward
    scores, preds = self.decode_forced(encoder_states, ys)
  File "/private/home/roller/working/parlai/parlai/agents/hugging_face/gpt2.py", line 197, in decode_forced
    latent, _ = self.decoder(inputs, encoder_states)
  File "/private/home/roller/.conda/envs/chat202001/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/private/home/roller/working/parlai/parlai/agents/hugging_face/gpt2.py", line 113, in forward
    transformer_outputs = self.transformer(
  File "/private/home/roller/.conda/envs/chat202001/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'past'
```

Upgrade path is unclear given fbcode dependency.


Discussion:
Hi, 
I have the same error with transformers 4.2.1.
Simply installing transformers 3.5.1. gives a more clear course. When back to <4.0.0 It's a full deprecation.
The warning in t@3.5.1 is: 
```/opt/conda/lib/python3.8/site-packages/transformers/modeling_gpt2.py:534: FutureWarning: The `past` argument is deprecated and will be removed in a future version, use `past_key_values` instead.```
In the ParlAI code, I found a call to the transformers library with the mentioned deprecated argument:
please find it here:
https://github.com/facebookresearch/ParlAI/blob/e4f31276bdd8090cd254a8aa319011a14ee54ee9/parlai/agents/hugging_face/gpt2.py#L115

Hope this helps solve the issue faster.
Thanks
FYI:
after updating that line of code into ```past_key_values=incr_state,``` error is solved. 
Tested in transformers 3.5.1 and 4.2.1, deprecation warning does not appear, code works.
Thanks for debugging! Would love a PR!
(We do have some complications with internal customers being possibly on ~3.2 but we can work around that)

Resolving patch:
diff --git a/parlai/agents/hugging_face/gpt2.py b/parlai/agents/hugging_face/gpt2.py
--- a/parlai/agents/hugging_face/gpt2.py
+++ b/parlai/agents/hugging_face/gpt2.py
@@ -115,7 +115,7 @@ def forward(self, input, encoder_state, incr_state=None):
         model_input = model_input.clamp_(min=0)
         transformer_outputs = self.transformer(
             model_input,
-            past=incr_state,
+            past_key_values=incr_state,
             attention_mask=attention_mask,
             position_ids=position_ids,
         )


Test patch: