Is encode_plus supposed to pad to max_length? 
Author: tkmaker
Pull request: no

## ❓ Questions & Help

I am using AutoTokenizer and AutoModelForSequenceClassification and `encode_plus` to encode text. I am calling it like this:

`
            
            tokenizer = AutoTokenizer.from_pretrained(self.model_name)
            encoded_inputs = tokenizer.encode_plus(text,add_special_tokens=True,max_length=max_seq_length)
            input_ids = encoded_inputs["input_ids"]
            special_tokens_mask = encoded_inputs["special_tokens_mask"]
            token_type_ids = encoded_inputs["token_type_ids"]

            loggerinfo(logger, "len of encoded vals {} {} {}".format(len(input_ids),len(special_tokens_mask),len(token_type_ids)))
`

The output indicates that the encoded values are of different length. I expected them to all be = max_length which is 100 in this case. 

Output:

> max seq len = 100
> len of encoded vals 39 39 39
> len of encoded vals 24 24 24
> len of encoded vals 11 11 11

Is that an incorrect expectation? 

Comments:
From what I remember (can't check now), padding up to max model seq length is not done and not necessary. The tokenizer will limit longer sequences to the max seq length, but otherwise you can just make sure the batch sizes are equal (so pad up to max _batch_ length, so you can actually create m-dimensional tensors (all rows in a matrix have to have the same length).

@BramVanroy Not sure I understand. The above code does the encoding for one row of a text column. The encodings are appended to a list to capture all the encoding for the text column. If I convert that list to a tensor, without any padding it errors out due to different vector lengths. 


Exactly. So the tokenizer limits the length (to a max seq length) but doesn't pad it. You'll have to do that manually. You can pad up to the largest sequence _in the batch_ (rather than the max seq length) so that all items in the batch are the same size, which you can then convert to a tensor. A general usage could look like this. The padding happens at the end. Here I pad up to the MAX_SEQ_LEN if available, and otherwise up to the largest sequence in the batch.

```python
def tokenize(text):
    all_input_ids = []
    all_input_mask = []

    for sentence in text:
        tokens = tokenizer.tokenize(sentence)

        # limit size to make room for special tokens
        if MAX_SEQ_LEN:
            tokens = tokens[0:(MAX_SEQ_LEN - 2)]

        # add special tokens
        tokens = [tokenizer.cls_token, *tokens, tokenizer.sep_token]
        
        # convert tokens to IDs
        input_ids = tokenizer.convert_tokens_to_ids(tokens)
        # create mask same size of input
        input_mask = [1] * len(input_ids)

        all_input_ids.append(input_ids)
        all_input_mask.append(input_mask)

    # pad up to max length
    # up to max_seq_len if provided, otherwise the max of current batch
    max_length = MAX_SEQ_LEN if MAX_SEQ_LEN else max([len(ids) for ids in all_input_ids])

    all_input_ids = torch.LongTensor([i + [tokenizer.pad_token_id] * (max_length - len(i))
                                      for i in all_input_ids])
    all_input_mask = torch.FloatTensor([m + [0] * (max_length - len(m)) for m in all_input_mask])

    return all_input_ids, all_input_mask
```

Thanks, that clarifies things. I will close this issue. 

> From what I remember (can't check now), padding up to max model seq length is not done and not necessary. The tokenizer will limit longer sequences to the max seq length, but otherwise you can just make sure the batch sizes are equal (so pad up to max _batch_ length, so you can actually create m-dimensional tensors (all rows in a matrix have to have the same length).

I am wondering if there are any disadvantages to just padding all inputs to 512. It would certainly cut down on batch processing time. But standard practice seems to be dynamically padding to the largest sequence length. 

I have wondered about this myself but I have no answer. Perhaps someone else can help. 

I think it comes down to loading CPU computation vs LA GPU computation. It might be worth doing some statistical analysis on the inputIDs length and checking if 1 standard limit is possible. For example if at least sample of 512 occurs in 90% of your batches,  it would be worth just setting to padding to 512 for all batches.