How to use padding with Mistral?
Author: TopCoder2K
Pull request: no

I'm trying to understand how to use Mistral with `batch_size` > 1. One aspect of this is setting `padding="longest"` in, e.g., `MistralCommonTokenizer.encode()`. But I'm getting `TypeError: 'set' object is not callable` when I try this. Example:
```python
import torch
from transformers import MistralForCausalLM, MistralCommonTokenizer

tokenizer = MistralCommonTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")
model = MistralForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
    device_map="auto",
)

messages = [
    "You are a pirate chatbot who always responds in pirate speak!",
    "Who are you?",
]

model_inputs = tokenizer.encode(messages, return_tensors="pt", padding="longest").to(
    model.device
)
```

Output:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 17
      5 model = MistralForCausalLM.from_pretrained(
      6     "mistralai/Mistral-7B-Instruct-v0.3",
      7     dtype=torch.bfloat16,
      8     attn_implementation="sdpa",
      9     device_map="auto",
     10 )
     12 messages = [
     13     "You are a pirate chatbot who always responds in pirate speak!",
     14     "Who are you?",
     15 ]
---> 17 model_inputs = tokenizer.encode(messages, return_tensors="pt", padding="longest").to(
     18     model.device
     19 )
     21 generated_ids = model.generate(model_inputs, max_new_tokens=100, do_sample=True)
     22 tokenizer.batch_decode(generated_ids)[0]

File ~/ad_hoc_analysis/src/asr_and_summarization/.venv/lib/python3.13/site-packages/transformers/tokenization_mistral_common.py:407, in MistralCommonTokenizer.encode(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, pad_to_multiple_of, padding_side, return_tensors, verbose, **kwargs)
    404 if text_pair:
    405     raise ValueError("`MistralCommonTokenizer.encode` does not support `text_pair`.")
--> 407 padding_strategy, truncation_strategy, max_length, _ = self._get_padding_truncation_strategies(
    408     padding=padding,
    409     truncation=truncation,
    410     max_length=max_length,
    411     pad_to_multiple_of=pad_to_multiple_of,
    412     verbose=verbose,
    413 )
    415 encoded_inputs = self._encode_plus(
    416     text,
    417     add_special_tokens=add_special_tokens,
   (...)    429     verbose=verbose,
    430 )
    432 return encoded_inputs["input_ids"]

File ~/ad_hoc_analysis/src/asr_and_summarization/.venv/lib/python3.13/site-packages/transformers/tokenization_mistral_common.py:1034, in MistralCommonTokenizer._get_padding_truncation_strategies(self, padding, truncation, max_length, pad_to_multiple_of, verbose, **kwargs)
   1031             max_length = self.model_max_length
   1033 # Test if we have a padding token
-> 1034 if padding_strategy != PaddingStrategy.DO_NOT_PAD and (self.pad_token is None or self.pad_token_id < 0):
   1035     raise ValueError(
   1036         "Asking to pad but the tokenizer does not have a padding token. "
   1037         "Please select a token to use as `pad_token` `(tokenizer.pad_token = tokenizer.eos_token e.g.)` "
   1038         "or add a new pad token via `tokenizer.add_special_tokens({'pad_token': '[PAD]'})`."
   1039     )
   1041 # Check that we will truncate to a multiple of pad_to_multiple_of if both are provided

File ~/ad_hoc_analysis/src/asr_and_summarization/.venv/lib/python3.13/site-packages/transformers/tokenization_mistral_common.py:334, in MistralCommonTokenizer.pad_token(self)
    329 @property
    330 def pad_token(self) -> str:
    331     """
    332     String associated to the padding token in the vocabulary.
    333     """
--> 334     return self.convert_ids_to_tokens(self.pad_token_id)

File ~/ad_hoc_analysis/src/asr_and_summarization/.venv/lib/python3.13/site-packages/transformers/tokenization_mistral_common.py:548, in MistralCommonTokenizer.convert_ids_to_tokens(self, ids, skip_special_tokens)
    546 tokens: list[str] = []
    547 for token_id in ids:
--> 548     if self._is_control_token(token_id) and skip_special_tokens:
    549         continue
    550     tokens.append(self.tokenizer.instruct_tokenizer.tokenizer.id_to_piece(token_id))

File ~/ad_hoc_analysis/src/asr_and_summarization/.venv/lib/python3.13/site-packages/transformers/tokenization_mistral_common.py:513, in MistralCommonTokenizer._is_control_token(self, token_id)
    511 def _is_control_token(self, token_id: int) -> bool:
    512     if self._tokenizer_type == MistralTokenizerType.spm:
--> 513         return token_id in self.tokenizer.instruct_tokenizer.tokenizer._control_tokens()
    514     elif self._tokenizer_type == MistralTokenizerType.tekken:
    515         return token_id < self.tokenizer.instruct_tokenizer.tokenizer.num_special_tokens

TypeError: 'set' object is not callable
```

Env:
```
- `transformers` version: 4.57.1
- Platform: Linux-5.15.0-112-generic-x86_64-with-glibc2.35
- Python version: 3.13.7
- Huggingface_hub version: 0.36.0
- Safetensors version: 0.6.2
- Accelerate version: 1.11.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0+cu128 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: Yes
- GPU type: NVIDIA RTX A5000
```

P.S.
I prefer specialized classes over Auto Classes in order to better control and understand what is going on inside. But there are also problems when using Auto Classes: strange warnings are emitted for this code taken from the [docs](https://huggingface.co/docs/transformers/model_doc/mistral?usage=AutoModel#mistral):
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3", dtype=torch.bfloat16, attn_implementation="sdpa", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")

messages = [
    {"role": "user", "content": "What is your favourite condiment?"},
    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
    {"role": "user", "content": "Do you have mayonnaise recipes?"}
]

model_inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)

generated_ids = model.generate(model_inputs, max_new_tokens=100, do_sample=True)
tokenizer.batch_decode(generated_ids)[0]
```
produces
```
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
```

Comments:
By the way, I've noticed that `MistralCommonTokenizer.encode()` requires a `str` or a `List[int]` as its first argument. Does this mean that Mistral doesn't support batching? I see `MistralCommonTokenizer.pad()` as well...

Hi @topcoder2k, I believe this bug was fixed in #41439!

@Rocketknight1, ah, thank you for the link. It should be fixed in 4.57.2 then. I'm closing the issue.