Albert pretrain datasets/ datacollator
Author: yl-to
Pull request: yes

partially fix #5984 
Add supports for Albert model pretraining:
- Add `AlbertTextDataset` class
  - Create `segment_ids` and `sentence_order_labels` attributes for sentence order prediction task

- Add `DataCollatorForAlbertPretrain` class
  - inherited from `DataCollatorForLanguageModeling` class
  - create `attention_mask` for both masked and padding tokens

@sgugger 

Comments:
@sgugger @LysandreJik thanks for the advice, will update soon!

Great, let us know when this PR is ready to review again!

@LysandreJik @sgugger Ready for reviewing again! Thanks for all the suggestions. 
I noticed that the check_code_quality test is failed however I can't see the actual failing part, please let me know if this matters.
And also please let me know if there are addition modification required, thanks guys!

@LysandreJik @sgugger tests added and style check modification were done. Please help me to review this if got time, thanks!

Besides, I did black reformat all the check_code_quality required files. However it could not pass the tests in CI, have no idea why this happen.

```
black --line-length 119 --target-version py35 src/transformers/data/datasets/language_modeling.py
All done! ✨ 🍰 ✨
1 file left unchanged.
```

This is because your black/isort versions aren't up to date. This is not a problem, I just pushed to your branch with the fix, but there's a remaining issue with flake8 that you will have to fix:

```
src/transformers/data/data_collator.py:253:13: F841 local variable 'attention_padding_mask' is assigned to but never used
src/transformers/data/datasets/language_modeling.py:153:21: F541 f-string is missing placeholders
```

Thanks for adding the test, it's great!

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=h1) Report
> Merging [#6168](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ed71c21d6afcbfa2d8e5bb03acbb88ae0e0ea56a?el=desc) will **decrease** coverage by `0.38%`.
> The diff coverage is `96.58%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6168/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6168      +/-   ##
==========================================
- Coverage   79.51%   79.13%   -0.39%     
==========================================
  Files         164      164              
  Lines       31022    31137     +115     
==========================================
- Hits        24668    24641      -27     
- Misses       6354     6496     +142     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.31% <ø> (ø)` | |
| [...rc/transformers/data/datasets/language\_modeling.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFzZXRzL2xhbmd1YWdlX21vZGVsaW5nLnB5) | `92.94% <95.23%> (+2.13%)` | :arrow_up: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `93.18% <100.00%> (+0.88%)` | :arrow_up: |
| [src/transformers/data/datasets/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFzZXRzL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9sb25nZm9ybWVyLnB5) | `16.37% <0.00%> (-82.31%)` | :arrow_down: |
| [src/transformers/modeling\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19sb25nZm9ybWVyLnB5) | `19.71% <0.00%> (-72.34%)` | :arrow_down: |
| [src/transformers/configuration\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2xvbmdmb3JtZXIucHk=) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [src/transformers/modeling\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190NS5weQ==) | `76.70% <0.00%> (-6.07%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `86.66% <0.00%> (-0.55%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.41% <0.00%> (-0.26%)` | :arrow_down: |
| ... and [7 more](https://codecov.io/gh/huggingface/transformers/pull/6168/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=footer). Last update [ed71c21...911b5b4](https://codecov.io/gh/huggingface/transformers/pull/6168?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


> This is because your black/isort versions aren't up to date. This is not a problem, I just pushed to your branch with the fix, but there's a remaining issue with flake8 that you will have to fix:
> 
> ```
> src/transformers/data/data_collator.py:253:13: F841 local variable 'attention_padding_mask' is assigned to but never used
> src/transformers/data/datasets/language_modeling.py:153:21: F541 f-string is missing placeholders
> ```

resolved. @LysandreJik 
Please help to review again, thanks!

Thanks for all your efforts on this!

> Thanks for all your efforts on this!

thanks for reviewing!

> Very cool, I updated the style again.
> 
> Thanks for iterating!

thanks for your help!

First of all I have to admit, I am new here, so still trying to understand how different modalities work in hugging face.
Going through the documents, it seems that the modifications proposed by @yl-to are only for PyTorch. Right ?
Pretraining of ALBERT with TF is still not supported ?  

That is correct @UmarSpa! However, models trained in PyTorch can easily be ported to TensorFlow if you're looking to serve a model using TensorFlow.