`group_texts` duplicates special tokens
Author: hwijeen
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.2
- Platform: Linux-5.4.0-66-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 --> @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run [run_mlm.py](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py#L381) with pdb break point at line 409.
2. see value for `tokenized_datasets['train'][0]`
```
ipdb> tokenized_datasets['train'][0]
{'attention_mask': ..., 'input_ids': [2, 11699, 6139, 23923, 6354, 6216, 3, 2, 124, 6149, 6228, 6164, 5125, 27479, 6228, 11699, 6139, 23923, 6354, 6216, 11961, 9121, 9804, 10602, 10293, 5328, 6721, 121, 6997, 15520, 16117, 10602, 11302, 5328, 6721, 121, 6997, 13014, 6177, 22111, 25147,
6189, 6106, 6315, 6110, 5084, 5158, 6291, 12836, 6108, 15512, 6726, 18139, 25596, 12701, 6291, 6106, 6315, 6616, 112, 5171, 113, 4363, 6380, 14946, 13769, 13928, 17518, 10216, 12299, 12571, 12850, 26355, 5315, 6457, 6117, 6303, 6213, 19358, 117, 122, 5201, 6361, 6211, 6377, 6312, 22259, 6631, 9268, 112, 10538, 113, 10728, 22278, 117, 14870, 13905, 142, 15214, 112, 10538, 113, 10728, 22278, 117, 14870, 14934, 7575, 10524, 186, 14921, 30912, 10758, 118, 10022, 680, 6275, 117, 181, 9860, 186, 14921, 30912, 10758, 136, 20107, 18973, 6358, 118, 3, 2, 10022, 7539, 25147, 6189, 116, 24690, 6915, 128, 116, 11134, 14216, 15650, 15373, 117, 13531, 20100, 117, 10028, 6132, 117, 127, 112, 124, 3866, 113, 15798, 15650, 15373, 117,
13531, 20100, 117, 9840, 6403, 117, 23999, 25006, 6131, 112, 14604, 113, 5084, 15466, 112, 5171, 113, 4363, 6380, 14946, 6213, 13579, 10393, 11023, 6187, 9218, 13014, 6236, 23534, 4587, 12827, 11069, 9422, 25686, 9112, 9220, 12112, 13538, 10112, 9427, 9215, 9260, 19036, 10393, 13514, 6187, 10112, 14882, 6130, 20150, 9279, 118, 3, 2, 14233, 15466, 18609, 16080, 118, 3, 2, 25147, 6189, 24864, 28007, 13581, 6149, 6228, 6164, 5125, 27479, 6228, 124, 5134, 16109, 28372, 3814, 6224, 20116, 6158, 12221, 6595, 105, 3, 2, 5466, 11794, 10393, 4700, 6224, 12819, 10694, 6187, 4671, 6628, 6119, 5502, 24468, 5743, 6125, 7111, 18452, 105, 3, 2, 14368, 6164, 5125, 27479, 6228, 6309, 6221, 6139, 4174, 6428, 6243, 167, 11699, 6139, 13295, 16589, 18619, 15924, 6131, 22573, 19515, 11914, 23850, 11914, 11512, 11346, 25763, 5134, 16109, 28372, 5134, 16109, 28372, 7031, 6114, 6114, 6626, 7020, 118, 3, 2, 5476, 6214, 116, 4121, 7788, 6107, 7788, 118, 4822, 6503, 6236, 15053, 4606, 9117, 118, 3, 2, 5676, 26156, 4973, 7088, 6114, 23122, 6114, 25444, 6422, 4218, 14246, 11920, 6147, 12097, 4011, 9117, 118, 3, 2, 4417, 25703, 9205, 9271, 9165, 19235, 4202, 6115, 14810, 6187, 19915, 6164, 4839, 6361, 11721, 4378, 7063, 15482, 9156, 11976, 30627, 9291, 3788, 19018, 20146, 4202, 9172, 118, 9868, 16712, 29634, 6115, 5206, 6203, 4469, 5294, 11019, 10250, 4973, 6284, 6203, 9691, 118, 3, 2, 9310, 10574, 5330, 9799, 11042, 13237, 6149, 9237, 118, 3, 2, 4378, 7063, 9126, 9271, 9242, 9822, 6236, 15472, 23041, 16135, 18119, 15314, 118, 3, 2, 5134, 6341, 6187, 9159, 14990, 5656, 4241, 14059, 6139, 4913, 12802, 9822, 9181, 9841, 4788, 18037, 116, 14059, 10825, 5087, 5178, 11699, 6213, 15171, 6333, 9242, 4645, 10212, 9691, 118, 3, 2, 9620, 10021, 11699, 6139, 17626, 6236, 12258, 4378, 7063, 6185, 16269, 26623, 30683, 12901, 118, 3, 2, 18276, 6130, 4378, 7063, 9126, 11699, 6187, 9159, 9242, 9319, 13793, 17451, 6260, 4184, 9242, 17561, 10724, 20756, 12126, 4789, 9172, 118, 9567, 4144, 12062, 10780, 5466, 6803, 4202, 4378, 7063, 9126, 4422, 12303, 6164, 9165, 10350, 571, 9467, 20853, 7177, 11947, 10441, 9270, 18480, 3795, 9207, 12098, 11725, 118], 'special_tokens_mask':  ... , 'token_type_ids': ...}
```
When `group_texts` is `map`ed to `tokenized_datasets`, whose examples already contain special tokens (e.g. [CLS] and [SEP]), the mapped results have the following format: `[CLS] ... [SEP][CLS] ... [SEP][CLS] ... [SEP]`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The output of `group_texts` applied `tokenized_datasets` has the following format `[CLS] ... [SEP]` or `[CLS] ... [SEP] ... [SEP]`.
<!-- A clear and concise description of what you would expect to happen. -->

The current input format is different from the original implementation, [ELECTRA](https://github.com/google-research/electra/blob/f93f3f81cdc13435dd3e85766852d00ff3e00ab5/build_pretraining_dataset.py#L100) for example. Is this a trivial issue? I think the downstream task performance of the model pretrained with the current script could tell if this is a serious bug or not. Could someone share the results?

Comments:
ELECTRA or BERT are not pretrained using this option, so you should use `--line_by_line` to mimick their pretraining objective.

Also note that this is a generic example, not an actual pretraining script (for instance the BERT next sentence prediction objective is not there). It's purpose is to expose all data processing so you can easily tweak it to your needs.

Thank you for the clarification. Could you name models that are trained with this option(not --line_by_line)?

GPT and GPT-2 is trained this way for instance.

#11840