Use reformer in down stream task meet problem
Author: LeopoldACC
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:4.2.2
- Platform:CentOS
- Python version:3.7
- PyTorch version (GPU?):1.5.1 cpu only
- Tensorflow version (GPU?):
- Using GPU in script?:no
- Using distributed or parallel set-up in script?:no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: 
### sequence classification task under glue
- bug
```
Traceback (most recent call last):
  File "examples/text-classification/run_glue.py", line 584, in <module>
    main()
  File "examples/text-classification/run_glue.py", line 410, in main
    datasets = datasets.map(preprocess_function, batched=True, load_from_cache_file=not data_args.overwrite_cache)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/dataset_dict.py", line 386, in map
    for k, dataset in self.items()
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/dataset_dict.py", line 386, in <dictcomp>
    for k, dataset in self.items()
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1120, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1091, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "examples/text-classification/run_glue.py", line 403, in preprocess_function
    result = tokenizer(*args, padding=padding, max_length=data_args.max_seq_length, truncation=True)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/tokenization_utils_base.py", line 2335, in __call__
    **kwargs,
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/tokenization_utils_base.py", line 2500, in batch_encode_plus
    **kwargs,
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/tokenization_utils_base.py", line 2217, in _get_padding_truncation_strategies
    "Asking to pad but the tokenizer does not have a padding token. "
ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as `pad_token` `(tokenizer.pad_token = tokenizer.eos_token e.g.)` or add a new pad token via `tokenizer.add_special_tokens({'pad_token': '[PAD]'})`.
```
- shell code
```python
python examples/text-classification/run_glue.py --model_type reformer --model_name_or_path google/reformer-crime-and-punishment --task_name $TASK_NAME --do_train --do_eval --max_seq_length 512 --per_gpu_eval_batch_size=32 --per_gpu_train_batch_size=32 --learning_rate 2e-5 --num_train_epochs 3.0 --output_dir /home2/zhenggo1/checkpoint/reformer_mrpc
```
### translation task under wmt_en_ro
- bug
```
Traceback (most recent call last):
  File "examples/seq2seq/finetune_trainer.py", line 451, in <module>
    main()
  File "examples/seq2seq/finetune_trainer.py", line 215, in main
    cache_dir=model_args.cache_dir,
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/auto/modeling_auto.py", line 1226, in from_pretrained
    ", ".join(c.__name__ for c in MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING.keys()),
ValueError: Unrecognized configuration class <class 'transformers.models.reformer.configuration_reformer.ReformerConfig'> for this kind of AutoModel: AutoModelForSeq2SeqLM.
Model type should be one of LEDConfig, BlenderbotSmallConfig, MT5Config, T5Config, PegasusConfig, MarianConfig, MBartConfig, BlenderbotConfig, BartConfig, FSMTConfig, EncoderDecoderConfig, XLMProphetNetConfig, ProphetNetConfig.
```
- shell code
```python
python examples/seq2seq/finetune_trainer.py --model_name_or_path google/reformer-crime-and-punishment --do_train --do_eval --task translation_en_to_ro --data_dir examples/seq2seq/test_data/wmt_en_ro/ --output_dir /home2/zhenggo1/checkpoint/reformer_translation --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir --predict_with_generate
```
### clm task under wikitext
- bug
```
Traceback (most recent call last):
  File "examples/language-modeling/run_clm.py", line 472, in <module>
    main()
  File "examples/language-modeling/run_clm.py", line 365, in main
    train_result = trainer.train(model_path=model_path)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/trainer.py", line 888, in train
    tr_loss += self.training_step(model, inputs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/trainer.py", line 1250, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/trainer.py", line 1277, in compute_loss
    outputs = model(**inputs)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 2244, in forward
    return_dict=return_dict,
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 2090, in forward
    start_idx_pos_encodings=start_idx_pos_encodings,
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 264, in forward
    position_embeddings = self.position_embeddings(position_ids)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 158, in forward
    self.axial_pos_shape, self.axial_pos_shape, sequence_length, reduce(mul, self.axial_pos_shape)
AssertionError: If training, make sure that config.axial_pos_shape factors: (512, 1024) multiply to sequence length. Got prod((512, 1024)) != sequence_length: 1024. You might want to consider padding your sequence length to 524288 or changing config.axial_pos_shape.
```
- shell code
```python
python examples/language-modeling/run_clm.py --model_name_or_path google/reformer-crime-and-punishment --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --do_train --do_eval --output_dir /home2/zhenggo1/checkpoint/reformer_clm
```
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. run shell code as shown as above(translation dataset may not use the local)


<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I just need a task to do evaluation and can compute a metrics.
Thks a lot if you can help me to give just a task that I can do evaluation!!!
<!-- A clear and concise description of what you would expect to happen. -->


Comments:
If I train the model on the crime-and-punishment
```shell
python examples/language-modeling/run_clm.py --model_name_or_path google/reformer-crime-and-punishment --dataset_name crime_and_punish --do_train --do_eval --output_dir /home2/zhenggo1/checkpoint/reformer_clm
```
the bug is below
```python
AssertionError: If training, make sure that config.axial_pos_shape factors: (512, 1024) multiply to sequence length. Got prod((512, 1024)) != sequence_length: 1024. You might want to consider padding your sequence length to 524288 or changing config.axial_pos_shape.
```

Hi,

I have been playing around with Reformer these few days so I hope I can give some insights. Axial positional encoding in Reformer requires that sequence length must be fixed to the product of `axial_pos_embds_dim`. See the documentation here

https://huggingface.co/transformers/model_doc/reformer.html#axial-positional-encodings

So you have to either pad the sequence length to that fixed size, or change the value for `axial_pos_embds_dim` to a smaller value. Due to this reason, I believe example scripts won't work with Reformer out of the box.

The Reformer examples from Google's Trax actually don't use axial positional encoding, just normal positional encoding (see [here](https://github.com/google/trax/blob/master/trax/examples/NER_using_Reformer.ipynb)). So I actually disable axial positional encoding (passing `axial_pos_embds=False` to Reformer config) and it works fine. By disabling this, I can also use dynamic padding (pad to max length within a batch) and saves even more memory.

I haven't tested the accuracy difference between with and without axial positional encoding. But axial positional encoding is so slow for a dataset with varying sequence lengths that I find it impractical.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.