Repository: huggingface/transformers
Pull request: Fix false positive right-padding warning for decoder-only models in pipeline

## What does this PR do?

Fixes #43906 (related to #38071)

### Problem
When using `pipeline('text-generation')` with batched inference on Qwen3 (and other models where `pad_token_id == bos_token_id`), a spurious warning is emitted:

> A decoder-only architecture is being used, but right-padding was detected!

This happens for two reasons:
1. The `TextGenerationPipeline` doesn't set `padding_side='left'` for decoder-only models, so the default `'right'` padding is used during batch collation
2. The right-padding detection heuristic in `generate()` only checks if the last token equals `pad_token_id`, which can produce false positives when `pad_token_id` equals other special tokens

### Fix
1. **`TextGenerationPipeline.__init__`**: Automatically set `tokenizer.padding_side = 'left'` for decoder-only models (since `TextGenerationPipeline` is exclusively for causal LM)
2. **`GenerationMixin.generate`**: When an `attention_mask` is available, use it to detect right-padding (check if last position has mask=0) instead of relying solely on the token id heuristic. Falls back to the original check when no attention mask is provided.

### Root Cause Analysis
For Qwen3, `pad_token_id = bos_token_id = 151643` (`
`). The tokenizer's default `padding_side='right'` means shorter sequences in a batch get right-padded. The existing check `inputs_tensor[:, -1] == pad_token_tensor` then correctly detects this — but the real issue is that the pipeline should have been left-padding all along.

Even after fixing the pipeline, the attention-mask-based detection is more robust for cases where users call `model.generate()` directly with properly left-padded inputs whose content happens to end with the pad token.

## Who can review?
@gante @ArthurZucker

Changed files:
filename: src/transformers/generation/utils.py
patch: @@ -2535,16 +2535,22 @@ def generate(
         if not self.config.is_encoder_decoder:
             # If `input_ids` was given, check if the last id in any sequence is `pad_token_id`
             # Note: If using, `inputs_embeds` this check does not work, because we want to be more hands-off.
-            if (
-                generation_config._pad_token_tensor is not None
-                and batch_size > 1
-                and len(inputs_tensor.shape) == 2
-                and torch.sum(inputs_tensor[:, -1] == generation_config._pad_token_tensor) > 0
-            ):
-                logger.warning(
-                    "A decoder-only architecture is being used, but right-padding was detected! For correct "
-                    "generation results, please set `padding_side='left'` when initializing the tokenizer."
-                )
+            if generation_config._pad_token_tensor is not None and batch_size > 1 and len(inputs_tensor.shape) == 2:
+                # When an attention mask is provided, use it to detect right-padding (more reliable than
+                # checking token ids, which can produce false positives when pad_token_id == eos_token_id
+                # or pad_token_id == bos_token_id, as is the case for Qwen3 and other models).
+                attention_mask = model_kwargs.get("attention_mask", None)
+                if attention_mask is not None and attention_mask.shape == inputs_tensor.shape:
+                    # Right-padding means there are zeros (masked positions) at the end of some sequences
+                    has_right_padding = torch.any(attention_mask[:, -1] == 0).item()
+                else:
+                    # Fallback: check if the last token is a pad token (original heuristic)
+                    has_right_padding = torch.sum(inputs_tensor[:, -1] == generation_config._pad_token_tensor) > 0
+                if has_right_padding:
+                    logger.warning(
+                        "A decoder-only architecture is being used, but right-padding was detected! For correct "
+                        "generation results, please set `padding_side='left'` when initializing the tokenizer."
+                    )
 
         # 4. Define other model kwargs
         # decoder-only models with inputs_embeds forwarding must use caching (otherwise we can't detect whether we are

filename: src/transformers/pipelines/text_generation.py
patch: @@ -99,6 +99,12 @@ class TextGenerationPipeline(Pipeline):
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
         self.check_model_type(MODEL_FOR_CAUSAL_LM_MAPPING_NAMES)
+        # Decoder-only models require left-padding for correct batched generation.
+        # Only override when there is no feature_extractor, to avoid padding_side conflicts
+        # (e.g., WhisperForCausalLM has a feature_extractor that pads on the right).
+        if self.tokenizer is not None and self.tokenizer.padding_side == "right":
+            self.tokenizer.padding_side = "left"
+
         if "prefix" not in self._preprocess_params:
             # This is very specific. The logic is quite complex and needs to be done
             # as a "default".

filename: tests/pipelines/test_pipelines_text_generation.py
patch: @@ -297,9 +297,6 @@ def get_test_pipeline(
         text_generator = TextGenerationPipeline(
             model=model,
             tokenizer=tokenizer,
-            feature_extractor=feature_extractor,
-            image_processor=image_processor,
-            processor=processor,
             dtype=dtype,
             max_new_tokens=5,
         )