pad error in BertTokenizer.batch_encode_plus 
Author: PosoSAgapo
Pull request: no
Labels: name: wontfix

As the document writes,the value that bert do not attend should have attention value of 0.However, I was using the BertTokenizer,the result is different.
This is my code:
```
input_wsrq1ids=tokenizer.batch_encode_plus(d[0],text_pair=d[1],add_special_tokens=True,return_tensors='pt')
>>> input_wsrq1ids['input_ids'][0]
tensor([  101,  2198,  2001,  3015,  4581,  2005,  2010,  2047,  2201,  1012,
         2002,  2318, 13417,  3213,  1005,  1055,  3796,  1012,  2002,  2699,
         2000,  2486,  2370,  2000,  4339,  2021,  2009,  2052,  1050,  1005,
         1056,  2079,  2505,  5369,  2699,  2000,  2486,  2370,  2000,  4339,
         2021,  2009,  2052,  1050,  1005,  1056,  2079,  2505,  1012,  2002,
         2165,  1037,  3328,  1010,  5112,  2041,  2007,  2070,  2814,  1010,
         1998,  2246,  2012,  3267,  5369,  2165,  1037,  3328,  1010,  5112,
         2041,  2007,  2070,  2814,  1010,  1998,  2246,  2012,  3267,  5369,
         2165,  1037,  3328,  1010,  5112,  2041,  2007,  2070,  2814,  1010,
         1998,  2246,  2012,  3267,   102,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0])
>>> input_wsrq1ids['attention_mask'][0]
tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
```
The attention_mask are all ones,however, the input_ids are padded with zero,which seems do not match the attention.


Comments:
This will greatly effect the output of BERT model,I do not quite know whether this is the problem of the code or the problem of the package.
d[0] and d[1] are lists of sentences,which is used to train the model.

When I was trying to decode the input_ids, the result shows that the text_pair is not encoded properly.
`tokenizer.encode(input_wsrq1ids['input_ids'][0])`
`"[CLS] john was writing lyrics for his new album. he started experiencing writer's block. he tried to force himself to write but it wouldn't do anythinghe tried to force himself to write but it wouldn't do anything. he took a walk, hung out with some friends, and looked at naturehe took a walk, hung out with some friends, and looked at naturehe took a walk, hung out with some friends, and looked at nature [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]"`
which I should expect the following answer:
`[CLS] john was writing lyrics for his new album. he started experiencing writer's block. he tried to force himself to write but it wouldn't do anythinghe tried to force himself to write but it wouldn't do anything. he took a walk, hung out with some friends, and looked at naturehe took a walk, hung out with some friends, and looked at naturehe took a walk, hung out with some friends, and looked at nature [SEP] He felt inspiration and then went back home to write [PAD]`
Which seems did not concatenate the sentences in d[0] and d[1]

The `encode_plus` methods creates the attention mask according to the length of the passed input and the max length you're asking it to encode to. It doesn't look into the list to see which tokens are padding tokens, as it expects to perform the padding itself.

Instead of padding the sequences yourself, you could use a combination of the `pad_to_max_length` and `max_length` flags for `encode_plus`/`batch_encode_pus`. The attention mask will be correct then.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.