whisper return_timestamp error
Author: pearl-yu
Pull request: no

### System Info

- `transformers` version: 4.27.1
- Platform: Linux-5.10.147+-x86_64-with-glibc2.31
- Python version: 3.9.16
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): 2.11.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <yes>
- Using distributed or parallel set-up in script?: <no>

### Who can help?

@ArthurZucker  @younesbelkada @sanchit-gandhi

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
device = "cuda:0" if torch.cuda.is_available() else "cpu"

pipe = pipeline(
  "automatic-speech-recognition",
  model="openai/whisper-tiny",
  chunk_length_s=30,
  device=device,
)

audio, _ = librosa.load(mypath+ filename, sr = 16000)

prediction = pipe(np.array(audio),return_timestamps=True)['chunks'] 
```

Below is the full error message.

```
IndexError                                Traceback (most recent call last)
[<ipython-input-20-17a93ca487ee>](https://localhost:8080/#) in <module>
----> 1 prediction = pipe(np.array(audio),return_timestamps=True,stride_length_s=(4, 2))['chunks']

4 frames
[/usr/local/lib/python3.9/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in __call__(self, inputs, **kwargs)
    376                 logger.warning(
    377                     "Using `chunk_length_s` is very experimental with seq2seq models. The results will not necessarily"
--> 378                     " be entirely accurate and will have caveats. More information:"
    379                     " https://github.com/huggingface/transformers/pull/20104. Ignore this warning with pipeline(...,"
    380                     " ignore_warning=True)"

[/usr/local/lib/python3.9/dist-packages/transformers/pipelines/base.py](https://localhost:8080/#) in __call__(self, inputs, num_workers, batch_size, *args, **kwargs)
   1074             )
   1075 
-> 1076         is_dataset = Dataset is not None and isinstance(inputs, Dataset)
   1077         is_generator = isinstance(inputs, types.GeneratorType)
   1078         is_list = isinstance(inputs, list)

[/usr/local/lib/python3.9/dist-packages/transformers/pipelines/pt_utils.py](https://localhost:8080/#) in __next__(self)
    123         # We're out of items within a batch
    124         item = next(self.iterator)
--> 125         processed = self.infer(item, **self.params)
    126         # We now have a batch of "inferred things".
    127         if self.loader_batch_size is not None:

[/usr/local/lib/python3.9/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in postprocess(self, model_outputs, decoder_kwargs, return_timestamps)
    625                     if previous_sequence[0] < (timestamp_begin + offset - overlap_time) and idx != 0:
    626                         break  # the previous sequence is too far in the past
--> 627                     if len(previous_tokens) > 0:
    628                         # find the longest common sequence between the overlapping parts
    629                         index_left, index_right, match_length = _fast_find_longest_common_sequence(

[/usr/local/lib/python3.9/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in _find_timestamp_sequence(sequences, tokenizer, feature_extractor, max_source_positions)
    174             <Tip>
    175 
--> 176             For more information on how to effectively use `stride_length_s`, please have a look at the [ASR chunking
    177             blog post](https://huggingface.co/blog/asr-chunking).
    178 

IndexError: list index out of range
```

### Expected behavior

The tiny.en model returns a 'list out of index' error for some files. 
It works for all files if not adding the return_timetamps = True argument. 

The tiny model also returns the same error for some different audio files when  return_timetamps = True.

The base.en, and the base model also returns the same error for some (but different) files.


Comments:
Hey! Thanks for reporting. Could you share the audio that you are using? We never stumbled upon something like this 😅 
Problem seems to come from `sequence[1:relevant_timestamp]` ,but the traceback is a bit messed up

cc @Narsil 

`prediction = pipe(np.array(audio), return_timestamps=True, stride_length_s=(4, 2))['chunks']`

The "stride_length_s" parameter determines the length of the audio chunks to be processed at each time, as well as the length of the gaps between them. This parameter is different from the "chunk_length_s" parameter and is set by default to half of the "chunk_length_s" parameter.

The recommended parameters are:
* `chunk_length_s=30.0`
* `stride_length_s=(6, 0)` (or `stride_length_s=None`, and the pipeline will set this to `(chunk_length_s / 5, 0)` for you)

See the following Colab for details: https://colab.research.google.com/drive/1rS1L4YSJqKUH_3YxIQHBI982zso23wor?usp=sharing#scrollTo=Mh_e6rV62QUM

Any luck here @ataturkiyebmka changing the hyper parameters?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.