Add doc tests for Albert and Bigbird
Author: vumichien
Pull request: yes

# What does this PR do?

Add doc tests for Albert and Bigbird, a part of issue https://github.com/huggingface/transformers/issues/16292


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@patrickvonplaten, @ydshieh
Documentation: @sgugger


Comments:
@ydshieh Could you please take a look at it? I think we still have a problem with `AlbertTokenizer ` as we have discussed on the Discord channel `the AlbertTokenizer will add an extra "_" just after the "[MASK]" token` which will lead to the different shape between `input_text` and `target_text`. This is the code snippet for checking the output.
```python
from transformers import AlbertTokenizer, AlbertForMaskedLM
import torch

tokenizer = AlbertTokenizer.from_pretrained("albert-base-v2")
model = AlbertForMaskedLM.from_pretrained("albert-base-v2")

input_text = "The capital of France is [MASK]."
target_text = "The capital of France is Paris."

tokenizer.tokenize(input_text)
# ['▁the', '▁capital', '▁of', '▁france', '▁is', ' [MASK]', '▁', '.']
tokenizer.tokenize(target_text )
# ['▁the', '▁capital', '▁of', '▁france', '▁is', '▁paris', '.']
```

_The documentation is not available anymore as the PR was closed or merged._

Hi, @vumichien 

I won't be available for the next few days. Will check when I am back, or my colleague could check this PR :-) 

Regarding the Albert tokenizer, do you encounter any runtime error due to the shape issue? I understand that the shapes are different,  and had a short discussion with the team. But we thought it should still work. Sorry for not responding this part earlier, but if you see errors due to these shapes, could you post it here, please?

@ydshieh When I run the test for doc for **modeling_albert.py** in local, the error will show like the following (sorry for very long error)
```
======================================================================================= FAILURES =======================================================================================
____________________________________________________ [doctest] transformers.models.albert.modeling_albert.AlbertForMaskedLM.forward ____________________________________________________
1034     >>> predicted_token_id = logits[0, mask_token_index].argmax(axis=-1)
1035     >>> tokenizer.decode(predicted_token_id)
1036     'reims'
1037 
1038     ```
1039 
1040     ```python
1041     >>> labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]
1042     >>> # mask labels of non-[MASK] tokens
1043     >>> labels = torch.where(inputs.input_ids == tokenizer.mask_token_id, labels, -100)
UNEXPECTED EXCEPTION: RuntimeError('The size of tensor a (10) must match the size of tensor b (9) at non-singleton dimension 1')
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.albert.modeling_albert.AlbertForMaskedLM.forward[10]>", line 1, in <module>
RuntimeError: The size of tensor a (10) must match the size of tensor b (9) at non-singleton dimension 1
/home/vumichien/Detomo/transformers/src/transformers/models/albert/modeling_albert.py:1043: UnexpectedException
1036     'reims'
1037 
1038     ```
1039 
1040     ```python
1041     >>> labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]
1042     >>> # mask labels of non-[MASK] tokens
1043     >>> labels = torch.where(inputs.input_ids == tokenizer.mask_token_id, labels, -100)
1044 
1045     >>> outputs = model(**inputs, labels=labels)
UNEXPECTED EXCEPTION: ValueError('Expected input batch_size (10) to match target batch_size (9).')
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.albert.modeling_albert.AlbertForMaskedLM.forward[11]>", line 1, in <module>
  File "/home/vumichien/Detomo/transformers/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/vumichien/Detomo/transformers/src/transformers/models/albert/modeling_albert.py", line 964, in forward
    masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1))
  File "/home/vumichien/Detomo/transformers/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/vumichien/Detomo/transformers/venv/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 1163, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "/home/vumichien/Detomo/transformers/venv/lib/python3.8/site-packages/torch/nn/functional.py", line 2996, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
ValueError: Expected input batch_size (10) to match target batch_size (9).
/home/vumichien/Detomo/transformers/src/transformers/models/albert/modeling_albert.py:1045: UnexpectedException
1037 
1038     ```
1039 
1040     ```python
1041     >>> labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]
1042     >>> # mask labels of non-[MASK] tokens
1043     >>> labels = torch.where(inputs.input_ids == tokenizer.mask_token_id, labels, -100)
1044 
1045     >>> outputs = model(**inputs, labels=labels)
1046     >>> round(outputs.loss.item(), 2)
UNEXPECTED EXCEPTION: NameError("name 'outputs' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.albert.modeling_albert.AlbertForMaskedLM.forward[12]>", line 1, in <module>
NameError: name 'outputs' is not defined
/home/vumichien/Detomo/transformers/src/transformers/models/albert/modeling_albert.py:1046: UnexpectedException
=================================================================================== warnings summary ===================================================================================
venv/lib/python3.8/site-packages/flatbuffers/compat.py:19
  /home/vumichien/Detomo/transformers/venv/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================================================== short test summary info ================================================================================
FAILED src/transformers/models/albert/modeling_albert.py::transformers.models.albert.modeling_albert.AlbertForMaskedLM.forward
======================================================================= 1 failed, 6 passed, 1 warning in 52.18s ========================================================================
```
The error log is the same when I run test with doc for **modeling_tf_albert.py**

Maybe a quick easy way is just to overwrite the examples for AlbertForMaskedLM in the model files. Something similar to https://github.com/huggingface/transformers/pull/16565#discussion_r843972010

But that case is reversed: masked input has fewer tokens. So you need to have some different operations. 

Let's wait @patrickvonplaten to see if he has better suggestion.

@vumichien @ydshieh, I'd be in favor of overwriting both Albert (so that MLM is correct) as well as BigBird (to show that it's long-range). What do you think?

@ydshieh @patrickvonplaten I have overwritten both the doc-test examples of Albert and Bigbird. What do you think about them?

> @ydshieh @patrickvonplaten I have overwritten both the doc-test examples of Albert and Bigbird. What do you think about them?

That's great! The classification and QA example could be made even much longer for BigBird :-) The examples look already great though. Happy to merge as is as well :-) 

I have changed the longer examples for doctest. The examples are quite long, but in my opinion,  they are good to show that Bigbird is long-range model

Can we put that text in some dataset instead? The documentation will become a bit unreadable with such a long text, where as we could just load a dataset in one line and take the first sample.

@sgugger Thank you for your suggestion. I have changed to use the examples from squad datasets. How do you think about that?

Way better, and great that you're showing the shape! Good for me if @patrickvonplaten is okay.

@ydshieh I have revised as your suggestion. Please let me know if I need to revise something.

> @ydshieh I have revised as your suggestion. Please let me know if I need to revise something.

Love it! Thank you.
I will let @patrickvonplaten to have a final look (if any) & click the merge button 💯 


~~Running the last time -> will merge if all tests pass~~

Merged 🚀 Thanks again!