Fix Whisper Beam Search Output Tests #32373
Author: sambhavnoobcoder
Pull request: yes

## Problem
The Whisper beam search output tests were failing due to incorrect assumptions about beam indices handling in multi-batch scenarios. The tests expected local beam indices per batch, but Whisper's implementation uses global indices across batches.

## Approach
1. Analyzed BeamSearchTester implementation to understand correct beam index behavior
2. Identified that beam indices are global, with each batch's indices offset by batch_idx * num_beams
3. Updated tests to validate against correct index ranges while maintaining padding position checks

## Implementation
Modified the beam search output tests to:
1. Properly validate global beam indices across batches
2. Handle padding positions (-1) correctly
3. Add detailed debugging output for easier troubleshooting
4. Maintain separate validation logic for single-batch case

## Tests
Two test cases validate different scenarios:

1. `test_beam_search_generation_outputs`:
   - Tests multi-batch beam search
   - Validates global beam indices across batches
   - Checks proper batch offset handling

2. `test_beam_search_short_form`:
   - Tests single-batch beam search
   - Validates local beam indices
   - Confirms basic beam search functionality

## Test Results
<img width="1289" alt="Screenshot 2025-01-28 at 3 44 15 AM" src="https://github.com/user-attachments/assets/2211acc6-46aa-4c01-8167-3fbd3dff61ee" />

Both test cases now pass successfully, confirming proper beam index validation for both single and multi-batch scenarios.

Fixes #32373 

**cc:** @eustlb 


Comments:
Not sure who exactly to ping for this one, so you're all getting one! @eustlb for audio, @gante for beam search, @ydshieh for tests

This one is for me! 🤗

Hey @sambhavnoobcoder,

Few points here that make that this PR won't be merged:
1. it's missing the point: I'll give more details in the subsequent PR ([here](https://github.com/huggingface/transformers/pull/35939))
2. the way the test is written does not fit the way we test things. I quick glance to `tests/models` or `test_modeling_whisper` would have give you goods hints on what to do here

Overall, I won't risk hurting you (and apologize in advance if it is not the case), but all that looks a bit generated... Nevertheless thanks a lot for the work you've put into this, it triggered the solving of this issue anyway 🤗

Hi @eustlb,

I’m sorry to hear that the PR couldn’t be merged. This oversight is entirely on me—I misinterpreted the issue and should have aligned more closely with the existing codebase before making the changes. I’ll ensure to approach things more thoroughly moving forward.

I’d also love to review your PR to better understand what I missed and the areas I needed to address.

Thank you for taking the time to provide detailed feedback and for responding so quickly. I sincerely apologize if my mistake caused any inconvenience. I’ll make sure to be more mindful next time

Thanks, @sambhavnoobcoder, and of course, no worries! If it's all yours, it's well written and simply does not fit Transformers philosophy from a broad implementation POV (and that is why reviewers like us are there 😉). It has been a long time since this issue should have been solved anyway, but it is clearly a hard one: the main issue here is that `num_return_sequences` does not really make sense from Whisper's generate point of view. Indeed, this argument refers to the best n-generate sequences of the beam search. Yet, one call to Whisper's generate can trigger multiple calls to GenerationMixin's generate, which is where the beam search happens. What needs to be done is actually to handle `num_return_sequences` well in the underlying generate call and then in the `_need_fallback` method. Feel free to review my PR 🤗 I'm gonna do my best with what is also on my plate to have it ready ASAP