Repository: huggingface/transformers
Pull request: avoiding conditional indexing in positionalencoding to avoid possibil…


# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR avoids the use of conditional indexing in the SpeechT5 RelativePositionalEncoding forward pass. As explained in 
 https://github.com/huggingface/transformers/issues/42087 this type of advanced indexing can result in zero-shaped tensors which can be problematic for custom AI accelerators / torch backends.

This PR provides a simple and equivalent alternative. Instead of using:

```
        pos_seq[pos_seq < -self.max_length] = -self.max_length
        pos_seq[pos_seq >= self.max_length] = self.max_length - 1
```

These torch operations can be used:

```
        pos_seq = torch.where(pos_seq < -self.max_length, -self.max_length, pos_seq)
        pos_seq = torch.where(pos_seq >= self.max_length, self.max_length - 1, pos_seq)
```


Fixes https://github.com/huggingface/transformers/issues/42087


### Quick test of equivalence

This simple script can be used to verify that indeed these two expressions are same:

```
import torch

def method1(pos_seq, max_length):
    pos_seq = torch.where(pos_seq < -max_length, -max_length, pos_seq)
    pos_seq = torch.where(pos_seq >= max_length, max_length - 1, pos_seq)
    return pos_seq

def method2(pos_seq, max_length):
    pos_seq[pos_seq < -max_length] = -max_length
    pos_seq[pos_seq >= max_length] = max_length - 1
    return pos_seq

seq_len = 512
max_length = 160
device = 'cpu'
pos_seq = torch.arange(0, seq_len).to(device=device, dtype=torch.long)
pos_seq = pos_seq[:, None] - pos_seq[None, :]

print(torch.allclose(method1(pos_seq, max_length), method2(pos_seq, max_length)))
print(torch.max(torch.abs(method1(pos_seq, max_length) - method2(pos_seq, max_length))))
```
Output of the script above:
```
True
tensor(0)
```


## Who can review?
@eustlb @ebezzam 


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Changed files:
filename: src/transformers/models/speecht5/modeling_speecht5.py
patch: @@ -434,8 +434,8 @@ def forward(self, hidden_states):
         pos_seq = torch.arange(0, seq_len).to(device=hidden_states.device, dtype=torch.long)
         pos_seq = pos_seq[:, None] - pos_seq[None, :]
 
-        pos_seq[pos_seq < -self.max_length] = -self.max_length
-        pos_seq[pos_seq >= self.max_length] = self.max_length - 1
+        pos_seq = torch.where(pos_seq < -self.max_length, -self.max_length, pos_seq)
+        pos_seq = torch.where(pos_seq >= self.max_length, self.max_length - 1, pos_seq)
         pos_seq = pos_seq + self.max_length
 
         return self.pe_k(pos_seq)