Inconsistant `input_feature` length and `attention_mask` length in `WhisperFeatureExtractor`
Author: BakerBunker
Pull request: no
Labels: name: bug

name: Whisper

### System Info

transformers `main`  branch

### Who can help?

@eustlb @zucchini-nlp 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoProcessor
import numpy as np

audios=[np.random.randn(16000*5)]
processor=AutoProcessor.from_pretrained("openai/whisper-large-v3")
print(processor(
    [audios[0][: 160 * 5 - 1]], return_attention_mask=True, sampling_rate=16000, padding=False
)["attention_mask"].shape)
print(processor(
    [audios[0][: 160 * 5]], return_attention_mask=True, sampling_rate=16000, padding=False
)["attention_mask"].shape)
print(processor(
    [audios[0][: 160 * 5 + 1]], return_attention_mask=True, sampling_rate=16000, padding=False
)["attention_mask"].shape)
```


### Expected behavior

The `input_feature` and `attention_mask` length should be `audio_length // hop_length`, but the code here:
https://github.com/huggingface/transformers/blob/e8e0c76162263840661fc0ca0da3952861754759/src/transformers/models/whisper/feature_extraction_whisper.py#L327-L329

makes the `attention_mask` length equal to `(audio_length+hop_length-1) // hop_length`, it should be changed to:

```diff
diff --git a/src/transformers/models/whisper/feature_extraction_whisper.py b/src/transformers/models/whisper/feature_extraction_whisper.py
index 68c52c6eb3..b9f3b4cb35 100644
--- a/src/transformers/models/whisper/feature_extraction_whisper.py
+++ b/src/transformers/models/whisper/feature_extraction_whisper.py
@@ -326,7 +326,9 @@ class WhisperFeatureExtractor(SequenceFeatureExtractor):
 
         if return_attention_mask:
             # rescale from sample (48000) to feature (3000)
-            padded_inputs["attention_mask"] = padded_inputs["attention_mask"][:, :: self.hop_length]
+            padded_inputs["attention_mask"] = padded_inputs["attention_mask"][
+                :, self.hop_length - 1 :: self.hop_length
+             ]
```

Comments:
Hey @BakerBunker, this looks like a nice catch! It's never been an issue since there is no reason to use Whisper feature extractor with `paddind=False` since Whisper only accepts 30sec inputs, unless you want to use it for another model of curse (which is likely your use case, right?)

Would you mind opening a fixing PR? happy to review 🤗

In some models using whisper encoders like qwen omni, even with `paddind=True`, the sum of `attention_mask` is different with `input_feature` length, and qwen omni uses `attention_mask` to calculate the length of `input_feature`

https://github.com/huggingface/transformers/blob/e6a8063ef1af16df964b644b07e1d17e96555d23/src/transformers/models/qwen2_5_omni/processing_qwen2_5_omni.py#L159-L169


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.