[whisper] alternative fix for long-form timestamps
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Fixes #31942 and supersedes #32003 by implementing new logic in the tokenizer method `_compute_offsets`.

The advantages of this PR over #32003 are:
1. Works with the processor and tokenizer standalone
2. No code changes for the user when passing args to the processor

```python
from transformers import AutoProcessor, WhisperForConditionalGeneration
from datasets import load_dataset

processor = AutoProcessor.from_pretrained("openai/whisper-tiny.en")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-tiny.en")

dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]

inputs = processor(sample["array"], sampling_rate=sample["sampling_rate"], return_tensors="pt", truncation=False)

output = model.generate(**inputs, return_timestamps=True, return_segments=True)

# pass token ids to processor's decode method
result = processor.batch_decode(output["sequences"], skip_special_tokens=True, output_offsets=True)
print(result)

# we can also pass them directly to the tokenizer (not possible with #32003)
result = processor.tokenizer.batch_decode(output["sequences"], skip_special_tokens=True, output_offsets=True)
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32131). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sanchit-gandhi thanks for working on this, this is indeed a better solution that in PR https://github.com/huggingface/transformers/pull/32003! 

Could you make sure that the slow test `test_tiny_longform_timestamps_generation` implemented in PR #32003 is still passing ? 

Thank you! 

The test does indeed pass. I've updated it to use a different dataset that makes it a bit easier to see the effect of the correct timestamps (a 1 mins sample of LibriSpeech, rather than 10 concatenated samples of the same utterance).

> let's say we have a 45s audio sample with speech in the first 15s and in the last 15s, and no speech in-between

<details>

<summary> Here's a script for creating such an example and running inference: </summary>

```python
from datasets import load_dataset
from transformers import WhisperForConditionalGeneration, AutoProcessor
import numpy as np

# load model + processor
processor = AutoProcessor.from_pretrained("openai/whisper-small.en")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small.en")

# load dataset
dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]["array"]
sampling_rate = dataset[0]["audio"]["sampling_rate"]

# Contrived audio sample
# 1. First 15-seconds: standard speech
# 2. Next 15-seconds: silence (zeroes)
# 3. Last 15-seconds: speech

sample = [*sample[:15 * sampling_rate], *np.zeros(15 * sampling_rate).tolist(), *sample[15 * sampling_rate:]]
sample = np.array(sample)

# pre-process
inputs = processor(
    sample,
    sampling_rate=16_000,
    padding="longest",
    truncation=False,
    return_attention_mask=True,
    return_tensors="pt",
)

# inference
output = model.generate(**inputs, return_timestamps=True, return_segments=True)

# pass token ids to processor's decode method
result = processor.batch_decode(output["sequences"], skip_special_tokens=True, output_offsets=True)
print(result)
```
</details>

What we get for this Transformers PR (formatted in VTT style):

```
00:00.000 --> 00:06.480  Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.
00:06.480 --> 00:11.280  Nor is Mr. Quilter's manner less interesting than his matter.
00:11.280 --> 00:30.160  He tells us that at this festive season of the year, with…
00:30.160 --> 00:35.780  Christmas and roast beef looming before us, similes drawn from eating and its results
00:35.780 --> 00:38.800  occur most readily to the mind.
00:38.800 --> 00:44.560  His grave doubts whether Sir Frederick Leighton's work is really Greek after all and can
00:44.560 --> 00:45.560  discover.
```

What we get for original Whisper:
```
00:00.000 --> 00:06.480  Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.
00:06.480 --> 00:11.280  Nor is Mr. Quilter's manner less interesting than his matter.
00:11.280 --> 00:30.160  He tells us that at this festive season of the year, with...
00:30.160 --> 00:35.780  Christmas and roast beef looming before us, similes drawn from eating and its results
00:35.780 --> 00:38.760  occur most readily to the mind.
00:38.760 --> 00:44.560  He has grave doubts whether Sir Frederick Layton's work is really Greek after all, and can
```

=> the start/end timings vary slightly between implementations (as a result of numerical precision), but both handle the long period of silence in the same way: they lump it into the preceding segment, giving an end timestamp that encompasses all of the silence:
```
00:11.280 --> 00:30.160  He tells us that at this festive season of the year, with…
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I've clean-installed transformers today, and used whisper-large-v3. The issue still stands, when using the pipeline class.

Can you open a new issue please and ping @eustlb !