Add Doc Tests for Reformer PyTorch
Author: hiromu166
Pull request: yes

# What does this PR do?

https://github.com/huggingface/transformers/issues/16292

Fixing doc tests in modeling_reformer.py.
- ReformerModelWithLMHead.forward
- ReformerForMaskedLM.forward
- ReformerForQuestionAnswering.forward
- ReformerForSequenceClassification.forward

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

 @patrickvonplaten
 @ydshieh
 @patil-suraj

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I found two errors in the ReformerForMaskedLM.forward example.

1. config.is_decoder = True in default.
```python
# before
model = ReformerForMaskedLM.from_pretrained("google/reformer-crime-and-punishment")
# AssertionError: If you want to use `ReformerForMaskedLM` make sure `config.is_decoder=False` for bi-directional self-attention.

# after
model = ReformerForMaskedLM.from_pretrained("google/reformer-crime-and-punishment", is_decoder=False)
# It's OK.
```

2. tokenizer.mask_token_id is None.
```python
mask_token_index = (inputs.input_ids == tokenizer.mask_token_id)[0].nonzero(as_tuple=True)[0]
# TypeError: 'bool' object is not subscriptable
# Because tokenizer.mask_token_id is None.
```

This issue reports same problems.
https://github.com/huggingface/transformers/issues/10813

Hi, @hiromu166 , could you remind me of the reasons why you need to overwrite the code sample in the model file, instead of just using `add_code_sample_docstrings` and provide the expected outputs and checkpoints?

(sorry if you already mentioned before!)

Hi @ydshieh, I commented about the reason for using `replace_return_docstrings`!
Please check them🙏

Hi, @hiromu166 

I think we can change back to 

```
_CHECKPOINT_FOR_DOC = "google/reformer-crime-and-punishment"
```
and use this (`_CHECKPOINT_FOR_DOC `) for `ReformerModelWithLMHead`. If we do so, it should be possible to use `add_code_sample_docstrings`. Let me know if you still have problem doing so.

For `ReformerForMaskedLM`, we use `https://huggingface.co/hf-internal-testing/tiny-random-reformer`, and we can use `replace_return_docstrings`, as you point out there is `mask token` issue.

Let's try to make these 2 working first :-)

Hi @ydshieh, thank you for the suggestion.

OK, I'll change `_CHECKPOINT_FOR_DOC` like below:
- `"google/reformer-crime-and-punishment"` for `ReformerModel`, `ReformerModelWithLMHead`.
- `"hf-internal-testing/tiny-random-reformer"` for  `ReformerForMaskedLM`, `ReformerForSequenceClassification`,  `ReformerForQuestionAnswering` to solve randomness and other problems.

Hi, @hiromu166 Could you try to resolve the conflicts `utils/documentation_tests.txt` 🙏(you can also pull the latest main branch, rebase this PR on main, fix the conflict, then force push). 

(This is not necessary for me to review, but is required to fix the conflict before merging)

I will review this PR tomorrow :-)

Hi @ydshieh, I tried to resolve the conflict like below. Is this OK?
```bash
git fetch upstream
git rebase upstream/main
-- fix conflict --
git rebase --continue
git push -f origin add_doctest_reformer_pt
```

> Hi @ydshieh, I tried to resolve the conflict like below. Is this OK?
> 
> ```shell
> git fetch upstream
> git rebase upstream/main
> -- fix conflict --
> git rebase --continue
> git push -f origin add_doctest_reformer_pt
> ```

Looks good! I will review this PR now

PR Merged. Thank you again, @hiromu166 ❤️ !

I'm glad it was merged smoothly. Thank you for your cooperation!!

Thanks a mille for your contribution @hiromu166 !