Timestamps in AutomaticSpeechRecognitionPipeline not aligned in sample space
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.17.0.dev0
- Platform: Linux-5.10.98-1-MANJARO-x86_64-with-glibc2.33
- Python version: 3.9.8
- PyTorch version (GPU?): 1.10.2+cpu (False)
- Tensorflow version (GPU?): 2.8.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.4.0 (cpu)
- Jax version: 0.3.1
- JaxLib version: 0.3.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@Narsil
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- ALBERT, BERT, XLM, DeBERTa, DeBERTa-v2, ELECTRA, MobileBert, SqueezeBert: @LysandreJik
- T5, BART, Marian, Pegasus, EncoderDecoder: @patrickvonplaten
- Blenderbot, MBART: @patil-suraj
- Longformer, Reformer, TransfoXL, XLNet, FNet, BigBird: @patrickvonplaten
- FSMT: @stas00
- Funnel: @sgugger
- GPT-2, GPT: @patrickvonplaten, @LysandreJik
- RAG, DPR: @patrickvonplaten, @lhoestq
- TensorFlow: @Rocketknight1
- JAX/Flax: @patil-suraj
- TAPAS, LayoutLM, LayoutLMv2, LUKE, ViT, BEiT, DEiT, DETR, CANINE: @NielsRogge
- GPT-Neo, GPT-J, CLIP: @patil-suraj
- Wav2Vec2, HuBERT, SpeechEncoderDecoder, UniSpeech, UniSpeechSAT, SEW, SEW-D, Speech2Text: @patrickvonplaten, @anton-l

If the model isn't in the list, ping @LysandreJik who will redirect you to the correct contributor.

Library:

- Benchmarks: @patrickvonplaten
- Deepspeed: @stas00
- Ray/raytune: @richardliaw, @amogkam
- Text generation: @patrickvonplaten @narsil
- Tokenizers: @SaulLu
- Trainer: @sgugger
- Pipelines: @Narsil
- Speech: @patrickvonplaten, @anton-l
- Vision: @NielsRogge, @sgugger

Documentation: @sgugger

Model hub:

- for issues with a model, report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

For research projetcs, please ping the contributor directly. For example, on the following projects:

- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Issue

Generating timestamps in the `AutomaticSpeechRecognitionPipeline` does not match the timestamps generated from `Wav2Vec2CTCTokenizer.decode()`. The timestamps from the pipeline are exceeding the duration of the audio signal because of the strides.

## Expected behavior

Generating timestamps using the pipeline gives the following prediction IDs and offsets:

```python
pred_ids=array([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 12, 14, 14,  0,  0,
        0, 22, 11,  0,  7,  4, 15, 16,  0,  0,  4, 17,  5,  7,  7,  4,  4,
       14, 14, 18, 18, 15,  4,  4,  0,  7,  5,  0,  0, 13,  0,  9,  0,  0,
        0,  8,  0, 11, 11,  0,  0,  0, 27,  4,  4,  0, 23,  0, 16,  0,  5,
        7,  7,  0, 25, 25,  0,  0, 22,  7,  7, 11,  0,  0,  0, 10, 10,  0,
        0,  8,  0,  5,  5,  0,  0, 19, 19,  0,  4,  4,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  5,  0,  6,  4,  4,  0, 25,  0,
       14, 14, 16,  0,  0,  0,  0, 10,  9,  9,  0,  0,  0,  0, 25,  0,  0,
        0,  0,  0, 26, 26, 16, 12, 12,  0,  0,  0, 19,  0,  5,  0,  8,  8,
        4,  4, 27,  0, 16,  0,  0,  4,  4,  0,  3,  3,  0,  0,  0,  0,  0,
        0,  0,  0,  4,  4, 17, 11, 11, 13,  0, 13, 11, 14, 16, 16,  0,  6,
        5,  6,  6,  4,  0,  5,  5, 16, 16,  0,  0,  7, 14,  0,  0,  4,  4,
       12,  5,  0,  0,  0, 26,  0, 13, 14,  0,  0,  0,  0, 23,  0,  0, 21,
       11, 11, 11,  0,  5,  5,  5,  7,  7,  0,  8,  0,  4,  4,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0, 12, 21, 11, 11, 11,  0,  4,  4,  0,  0, 12, 11, 11,  0,  0,  0,
        0,  0,  7,  7,  5,  5,  0,  0,  0,  0, 23,  0,  0,  8,  0,  0,  4,
        4,  0,  0,  0,  0, 12,  5,  0,  0,  4,  4, 10,  0,  0, 24, 14, 14,
        0,  7,  7,  0,  8,  8, 10, 10,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0, 26,  0,  5,  0,  0,  0, 20,  0,  5,  0,
        0,  5,  5,  0,  0, 19, 16, 16,  0,  6,  6, 19,  0,  5,  6,  6,  4,
        4,  0, 14, 18, 18, 15,  4,  4,  0,  0, 25, 25,  0,  5,  4,  4, 19,
       16,  8,  8,  0,  8,  8,  4,  4,  0, 23, 23,  0, 14, 17, 17,  0,  0,
       17,  0,  5,  6,  6,  4,  4])
       
decoded['word_offsets']=[{'word': 'dofir', 'start_offset': 12, 'end_offset': 22}, {'word': 'hu', 'start_offset': 23, 'end_offset': 25}, {'word': 'mer', 'start_offset': 28, 'end_offset': 32
}, {'word': 'och', 'start_offset': 34, 'end_offset': 39}, {'word': 'relativ', 'start_offset': 42, 'end_offset': 60}, {'word': 'kuerzfristeg', 'start_offset': 63, 'end_offset': 94}, {'word'
: 'en', 'start_offset': 128, 'end_offset': 131}, {'word': 'zousazbudget', 'start_offset': 134, 'end_offset': 170}, {'word': 'vu', 'start_offset': 172, 'end_offset': 175}, {'word': '<unk>',
 'start_offset': 180, 'end_offset': 182}, {'word': 'milliounen', 'start_offset': 192, 'end_offset': 207}, {'word': 'euro', 'start_offset': 209, 'end_offset': 217}, {'word': 'deblokéiert', 
'start_offset': 221, 'end_offset': 249}, {'word': 'déi', 'start_offset': 273, 'end_offset': 278}, {'word': 'direkt', 'start_offset': 283, 'end_offset': 303}, {'word': 'de', 'start_offset': 311, 'end_offset': 313}, 
{'word': 'sportsbeweegungen', 'start_offset': 317, 'end_offset': 492}, {'word': 'och', 'start_offset': 495, 'end_offset': 499}, {'word': 'ze', 'start_offset': 503, 'end_offset': 507}, {'word': 'gutt', 'start_offset': 509, 'end_offset': 516}, 
{'word': 'kommen', 'start_offset': 519, 'end_offset': 532}]
```

However, the following is computed using `Wav2Vec2CTCTokenizer.decode()` and these offsets are expected:

```python
pred_ids=tensor([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 12, 14, 14,  0,  0,  0,
        22, 11,  0,  7,  4, 15, 16,  0,  0,  4, 17,  5,  7,  7,  4,  0, 14, 14,
        18, 18, 15,  4,  4,  0,  7,  5,  0,  0, 13,  0,  9,  0,  0,  0,  8,  0,
        11, 11,  0,  0,  0, 27,  4,  4,  0, 23,  0, 16,  0,  5,  7,  7,  0, 25,
        25,  0,  0, 22,  7,  7, 11,  0,  0,  0, 10, 10,  0,  0,  8,  0,  5,  5,
         0,  0, 19, 19,  0,  4,  4,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  5,  0,  6,  4,  4,  0, 25,  0, 14, 14, 16,  0,  0,  0,  0, 10,
         9,  9,  0,  0,  0,  0, 25,  0,  0,  0,  0,  0, 26, 26, 16, 12, 12,  0,
         0,  0, 19,  0,  5,  0,  8,  8,  4,  4, 27,  0, 16,  0,  0,  4,  4,  0,
         3,  3,  0,  0,  0,  0,  0,  0,  0,  0,  4,  4, 17, 11, 11, 13,  0, 13,
        11, 14, 16, 16,  0,  6,  5,  6,  6,  4,  0,  5,  5, 16, 16,  0,  0,  7,
        14,  0,  0,  4,  4, 12,  5,  0,  0,  0, 26,  0, 13, 14,  0,  0,  0,  0,
        23,  0,  0, 21, 11, 11, 11,  0,  5,  5,  5,  7,  7,  0,  8,  0,  4,  4,
         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  0, 12, 21, 11, 11, 11,  0,  4,  4,  0,  0, 12, 11, 11,  0,  0,
         0,  0,  0,  7,  7,  5,  5,  0,  0,  0,  0, 23,  0,  0,  8,  0,  0,  4,
         4,  0,  0,  0,  0, 12,  5,  0,  0,  4,  4, 10,  0,  0, 24, 14, 14,  0,
         7,  7,  0,  8,  8, 10, 10,  0,  0,  0, 26,  5,  5,  0,  0,  0, 20,  5,
         0,  0,  0,  5,  0,  0, 19,  0, 16, 16,  6,  6, 19, 19,  5,  5,  6,  6,
         4,  0, 14,  0, 18, 15, 15,  4,  4,  0,  0, 25,  0, 16,  0,  4,  4, 19,
        16,  8,  0,  0,  8,  0,  4,  0,  0, 23,  0, 14,  0, 17,  0,  0,  0, 17,
         5,  5,  6,  6,  4,  4])
         
word_offsets=[{'word': 'dofir', 'start_offset': 12, 'end_offset': 22}, {'word': 'hu', 'start_offset': 23, 'end_offset': 25}, {'word': 'mer', 'start_offset': 28, 'end_offset': 32},  {'word': 'och', 'start_offset': 34, 'end_offset': 39}, {'word': 'relativ', 'start_offset': 42, 'end_offset': 60}, {'word': 'kuerzfristeg', 'start_offset': 63, 'end_offset': 94},  {'word': 'en', 'start_offset': 128, 'end_offset': 131}, {'word': 'zousazbudget', 'start_offset': 134, 'end_offset': 170},  {'word': 'vu', 'start_offset': 172, 'end_offset': 175}, {'word': '<unk>', 'start_offset': 180, 'end_offset': 182}, {'word': 'milliounen', 'start_offset': 192, 'end_offset': 207},  {'word': 'euro', 'start_offset': 209, 'end_offset': 217}, {'word': 'deblokéiert', 'start_offset': 221, 'end_offset': 249},  {'word': 'déi', 'start_offset': 273, 'end_offset': 278}, {'word': 'direkt', 'start_offset': 283, 'end_offset': 303},  {'word': 'de', 'start_offset': 311, 'end_offset': 313}, {'word': 'sportsbeweegungen', 'start_offset': 317, 'end_offset': 360}, {'word': 'och', 'start_offset': 362, 'end_offset': 367},  {'word': 'zu', 'start_offset': 371, 'end_offset': 374}, {'word': 'gutt', 'start_offset': 377, 'end_offset': 383},  {'word': 'kommen', 'start_offset': 387, 'end_offset': 400}]
```

## Potential Fix

A fix could be removing the strides instead of filling them with `first_letter` and `last_letter` in `apply_stride()`:

```python
def apply_stride(tokens, stride):

    input_n, left, right = rescale_stride(tokens, stride)[0]
    left_token = left
    right_token = input_n - right

    return tokens[:, left_token:right_token]
```

<!-- A clear and concise description of what you would expect to happen. -->