[Whisper] ASR Pipeline with "return_timestamps=True" gives IndexError: index -1 is out of bounds for axis 0 with size 0
Author: MohammedRakib
Pull request: no

### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-5.10.147+-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): 2.9.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker @Narsil  @sanchit-gandhi 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Here is the link to [Google Colab Notebook](https://colab.research.google.com/drive/1ZLQXzD1IW2D1fz0WZOSEghUpewd0N3Bn?usp=sharing)

```python
!pip install git+https://github.com/huggingface/transformers

from transformers import pipeline
pipe = pipeline(
    task="automatic-speech-recognition",
    model='openai/whisper-small.en',
    chunk_length_s=30, 
    stride_length_s=(5,5), 
    device=0,
    return_timestamps=True,
)

pipe.model.config.forced_decoder_ids = pipe.tokenizer.get_decoder_prompt_ids(
    language='en', task='transcribe'
)
res = pipe('trial.wav')
print(res)
```

Here is the stack trace for the error:
```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-8-3813911bc8bc>](https://localhost:8080/#) in <module>
      1 # run with transformers installed from latest commit: 00ba7cadd812437708b380ab078a3cfe8cfaff31 at the moment.
      2 # index out of bounds error with return_timestamps=True!!
----> 3 res = pipe('trial.wav')
      4 print(res)
      5 

[/usr/local/lib/python3.8/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in __call__(self, inputs, **kwargs)
    370                         `"".join(chunk["text"] for chunk in output["chunks"])`.
    371         """
--> 372         return super().__call__(inputs, **kwargs)
    373 
    374     def _sanitize_parameters(

[/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py](https://localhost:8080/#) in __call__(self, inputs, num_workers, batch_size, *args, **kwargs)
   1074             return self.iterate(inputs, preprocess_params, forward_params, postprocess_params)
   1075         elif self.framework == "pt" and isinstance(self, ChunkPipeline):
-> 1076             return next(
   1077                 iter(
   1078                     self.get_iterator(

[/usr/local/lib/python3.8/dist-packages/transformers/pipelines/pt_utils.py](https://localhost:8080/#) in __next__(self)
    123         # We're out of items within a batch
    124         item = next(self.iterator)
--> 125         processed = self.infer(item, **self.params)
    126         # We now have a batch of "inferred things".
    127         if self.loader_batch_size is not None:

[/usr/local/lib/python3.8/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in postprocess(self, model_outputs, decoder_kwargs, return_timestamps)
    620             items = _find_longest_common_sequence(final_items, self.tokenizer)
    621         elif stride and self.type == "seq2seq_whisper" and return_timestamps:
--> 622             items = _find_timestamp_sequence(
    623                 final_items, self.tokenizer, self.feature_extractor, self.model.config.max_source_positions
    624             )

[/usr/local/lib/python3.8/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in _find_timestamp_sequence(sequences, tokenizer, feature_extractor, max_source_positions)
    103         timestamp_tokens = sequence >= timestamp_begin
    104         consecutive = np.where(timestamp_tokens[:-1] & timestamp_tokens[1:])[0] + 1
--> 105         last_timestamp = np.where(timestamp_tokens)[0][-1]
    106         consecutive = np.append(consecutive, last_timestamp) if last_timestamp not in consecutive else consecutive
    107         if seq_idx != 0:

IndexError: index -1 is out of bounds for axis 0 with size 0
```

### Expected behavior

The problem occurs when using any Whisper models from the Hub with ```return_timestamps=True``` in the ASR Pipeline.  The error does NOT occur if timestamps is not forced. 

Comments:
Thanks, this is normal and is currently being fixed 😉 see #21252 

I have also the same problem, how you fixed this problem ? 

@avishai119 Are you running on `transformers@main`  ?  It should be fixed there.

what you mean ?
this is my code:

`
from transformers import AutoProcessor, pipeline
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
from transformers import WhisperProcessor, WhisperForConditionalGeneration
from optimum.onnxruntime import ORTModelForSpeechSeq2Seq
import librosa
import torchaudio
from pydub.silence import split_on_silence


processor = AutoProcessor.from_pretrained('.\saved_processor',local_files_only=True)
model = ORTModelForSpeechSeq2Seq.from_pretrained('.\saved_model',local_files_only=True)

model.config.forced_decoder_ids  = processor.tokenizer.get_decoder_prompt_ids(language="hebrew", task="transcribe")
speech_recognition_pipeline = pipeline(
    "automatic-speech-recognition",
     model=model,
     feature_extractor=processor.feature_extractor,
     tokenizer=processor.tokenizer,
  )

speech_recognition_pipeline.model.config.forced_decoder_ids = speech_recognition_pipeline.tokenizer.get_decoder_prompt_ids(language="hebrew", task="transcribe")
#testing
audio,sr = librosa.load("C:\\Users\\avishai\\Desktop\\whisper-interface\\3.mp3",sr=16000)


result = speech_recognition_pipeline(audio,max_new_tokens=440)

print(result)
`

when i trying to add this : 
 `speech_recognition_pipeline(audio,max_new_tokens=440,return_timestamps=True)`
it's don't work :( 

Do you mind outputting the output of `transformers-cli env` ?

Hey there. I still face this same issue. Am I the only one? is there any known workaround/solution? Why has this issue been closed?