internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: nv-legate/cunumeric

Issue:
cunumeric.mean with dtype=np.complex64 causes fatal python error
```
arr_num = num.random.randint(-5, 5, size=(1,))
dt = np.complex64
num.mean(arr_num, dtype=np.dtype(dt))
```
In Numpy, pass
In cuNumeric, although the case pass, the whole test aborted with fatal python error.
For dt=np.float32 or np.int32, pass for cuNumeric.

legion_python: /home/robinw/work_2021/porting/legate/build_20230309/cunumeric/src/cunumeric/binary/binary_op_template.inl:72: void cunumeric::BinaryOpImpl<KIND, OP_CODE>::operator()(cunumeric::BinaryOpArgs&) const [with legate_core_type_code_t CODE = (legate_core_type_code_t)13; int DIM = 1; std::enable_if_t<(! cunumeric::BinaryOp<OP_CODE, CODE>::valid)>* <anonymous> = 0; cunumeric::VariantKind KIND = (cunumeric::VariantKind)2; cunumeric::BinaryOpCode OP_CODE = (cunumeric::BinaryOpCode)10]: Assertion `false' failed.
   Fatal Python error: Aborted



Discussion:

Resolving patch:
diff --git a/cunumeric/array.py b/cunumeric/array.py
--- a/cunumeric/array.py
+++ b/cunumeric/array.py
@@ -3050,7 +3050,7 @@ def mean(
             divisor = self.shape[axis]
         # Divide by the number of things in the collapsed dimensions
         # Pick the right kinds of division based on the dtype
-        if dtype.kind == "f":
+        if dtype.kind == "f" or dtype.kind == "c":
             sum_array.__itruediv__(
                 np.array(divisor, dtype=sum_array.dtype),
             )


Test patch:
Repository: nv-legate/cunumeric

Issue:
"where" fails with IndexError
https://github.com/nv-legate/cunumeric/blob/2666c08bbdb2ad5f83876ac8ed5c61bbe01af712/cunumeric/eager.py#L1618

```
Welcome to Legion Python interactive console
>>> import cunumeric
>>> import numpy
>>> cunumeric.where(2,1,1)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/auser/mambaforge/envs/legate/lib/python3.1/site-packages/legate/core/runtime.py", line 2149, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/auser/mambaforge/envs/legate/lib/python3.1/site-packages/cunumeric/coverage.py", line 120, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/auser/mambaforge/envs/legate/lib/python3.1/site-packages/cunumeric/array.py", line 148, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/auser/mambaforge/envs/legate/lib/python3.1/site-packages/cunumeric/module.py", line 2995, in where
    return ndarray._perform_where(a, x, y)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/auser/mambaforge/envs/legate/lib/python3.1/site-packages/cunumeric/array.py", line 4386, in _perform_where
    out._thunk.where(mask._thunk, one._thunk, two._thunk)
  File "/home/auser/mambaforge/envs/legate/lib/python3.1/site-packages/cunumeric/eager.py", line 1618, in where
    self.array[:] = np.where(rhs1.array, rhs2.array, rhs3.array)
    ~~~~~~~~~~^^^
IndexError: too many indices for array: array is 0-dimensional, but 1 were indexed
>>> numpy.where(2,1,1)
array(1)
>>>
```


Discussion:

Resolving patch:
diff --git a/cunumeric/eager.py b/cunumeric/eager.py
--- a/cunumeric/eager.py
+++ b/cunumeric/eager.py
@@ -1615,7 +1615,7 @@ def where(self, rhs1: Any, rhs2: Any, rhs3: Any) -> None:
         if self.deferred is not None:
             self.deferred.where(rhs1, rhs2, rhs3)
         else:
-            self.array[:] = np.where(rhs1.array, rhs2.array, rhs3.array)
+            self.array[...] = np.where(rhs1.array, rhs2.array, rhs3.array)
 
     def argwhere(self) -> NumPyThunk:
         if self.deferred is not None:


Test patch:
Repository: nv-legate/cunumeric

Issue:
[BUG] myarr[np.newaxis] = 1 raises KeyError
### Software versions

Python      :  3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:40:35) [GCC 12.3.0]
Platform    :  Linux-5.4.0-155-generic-x86_64-with-glibc2.31
Legion      :  v23.11.00.dev-22-ga562200
Legate      :  23.11.00.dev+22.ga562200
Cunumeric   :  23.11.00.dev+10.g586a005a.dirty
Numpy       :  1.24.4
Scipy       :  1.11.3
Numba       :  0.57.1
CTK package :  cuda-version-11.7-h67201e3_2 (conda-forge)
GPU driver  :  535.54.03
GPU devices :
  GPU 0: Tesla P100-SXM2-16GB
  GPU 1: Tesla P100-SXM2-16GB
  GPU 2: Tesla P100-SXM2-16GB
  GPU 3: Tesla P100-SXM2-16GB

### Jupyter notebook / Jupyter Lab version

_No response_

### Expected behavior

```
>>> import numpy as np
>>> works = np.array((4,))
>>> works[np.newaxis] = 1
>>> works
array([1])
>>> works.shape
(1,)
```

### Observed behavior

```
>>> import cunumeric as cnp
>>> breaks = cnp.array((4,))
>>> breaks[cnp.newaxis] = 1
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../mambaforge/envs/legate/lib/python3.1/site-packages/legate/core/runtime.py", line 2149, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/legate/cunumeric/cunumeric/coverage.py", line 120, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/legate/cunumeric/cunumeric/array.py", line 148, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../projects/legate/cunumeric/cunumeric/array.py", line 1677, in __setitem__
    raise KeyError("invalid key passed to cunumeric.ndarray")
KeyError: 'invalid key passed to cunumeric.ndarray'
```

### Example code or instructions

```Python
import cunumeric as cnp
breaks = cnp.array((4,))
breaks[cnp.newaxis] = 1
```


### Stack traceback or browser console output

[Numpy newaxis docs](https://numpy.org/doc/stable/reference/constants.html#numpy.newaxis). 


Discussion:
~~@seberg is this expected NumPy behavior?~~ Nevermind, I see it
```
x[newaxis, :] is equivalent to x[newaxis] and x[None]:
```

Resolving patch:
diff --git a/cunumeric/array.py b/cunumeric/array.py
--- a/cunumeric/array.py
+++ b/cunumeric/array.py
@@ -1675,8 +1675,6 @@ def __setitem__(self, key: Any, value: ndarray) -> None:
 
         """
         check_writeable(self)
-        if key is None:
-            raise KeyError("invalid key passed to cunumeric.ndarray")
         if value.dtype != self.dtype:
             temp = ndarray(value.shape, dtype=self.dtype, inputs=(value,))
             temp._thunk.convert(value._thunk)


Test patch:
diff --git a/tests/integration/test_set_item.py b/tests/integration/test_set_item.py
--- a/tests/integration/test_set_item.py
+++ b/tests/integration/test_set_item.py
@@ -29,6 +29,14 @@ def test_basic():
     assert x[2] == 3
 
 
+def test_newaxis():
+    arr = num.ones((4,))
+    arr[None] = 1
+    assert np.array_equal(arr, [1, 1, 1, 1])
+    arr[None, :] = 2
+    assert np.array_equal(arr, [2, 2, 2, 2])
+
+
 ARRAYS_4_3_2_1_0 = [
     4 - num.arange(5),
     4 - np.arange(5),
Repository: huggingface/optimum

Issue:
Add Support for DeBERTaV2
I would like to use DeBERTaV2 for sequence classification as a quantized model. Please let me know what needs to be done to open a PR to add this support!


Discussion:
Hi @sam-h-bean,

Before quantizing a `DeBERTaV2` model, you need to be able to export it to the ONNX format, which is [not supported yet](https://github.com/huggingface/transformers/issues/16308). To do so, you could implement a custom OnnxConfig in transformers and add "sequence-classification" [feature](https://github.com/huggingface/transformers/blob/main/src/transformers/onnx/features.py#L181)(if it is the only one that you are looking for) with [following steps](https://huggingface.co/docs/transformers/serialization#exporting-a-model-for-an-unsupported-architecture).

Feel free to reach out if you have any other question!




@JingyaHuang I opened a PR [here](https://github.com/huggingface/transformers/pull/17617)! Excited to get to the quantizing once this makes its way into main. Or is there a way I can begin that work using my local copy of transformers?
@sam-h-bean Sure, feel free to test `ORTQuantizer` with your copy of transformers, I can also add the support of DeBERTa-V2 for `ORTOptimizer` in case you want to do some graph optimizations.

Resolving patch:
diff --git a/optimum/onnxruntime/utils.py b/optimum/onnxruntime/utils.py
--- a/optimum/onnxruntime/utils.py
+++ b/optimum/onnxruntime/utils.py
@@ -57,6 +57,8 @@ class ORTConfigManager:
         "albert": ("num_attention_heads", "hidden_size", "bert"),
         "camembert": ("num_attention_heads", "hidden_size", "bert"),
         "distilbert": ("n_heads", "dim", "bert"),
+        "deberta": ("num_attention_heads", "hidden_size", "bert"),
+        "deberta-v2": ("num_attention_heads", "hidden_size", "bert"),
         "electra": ("num_attention_heads", "hidden_size", "bert"),
         "roberta": ("num_attention_heads", "hidden_size", "bert"),
         "bart": ("encoder_attention_heads", "d_model", "bart"),


Test patch:
Repository: huggingface/optimum

Issue:
AutoConfig is not imported in optimization.py
### System Info

```shell
optimium master : fb7e303d9254fcee194aa76f4a0b7fa9d9b140d0
```


### Who can help?

@echarlaix 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Try to optimize a model and get `NameError: name 'AutoConfig' is not defined` as it's not imported 

### Expected behavior

No runtime error 


I made a PR here to fix that : https://github.com/huggingface/optimum/pull/360


Discussion:

Resolving patch:
diff --git a/optimum/onnxruntime/optimization.py b/optimum/onnxruntime/optimization.py
--- a/optimum/onnxruntime/optimization.py
+++ b/optimum/onnxruntime/optimization.py
@@ -17,6 +17,7 @@
 from typing import Callable, Dict, List, Optional, Tuple, Union
 
 import transformers
+from transformers.models.auto.configuration_auto import AutoConfig
 
 from onnx import load_model
 from onnxruntime.transformers.fusion_options import FusionOptions


Test patch:
Repository: huggingface/optimum

Issue:
Empty file suffix when optimizing or quantizing is not handled
### System Info

```shell
latest master
```


### Who can help?

@echarlaix 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Optimize or quantize a model but user do not want a suffix so use : 
`file_suffix=None` => RuntimeError
`file_suffix=""` => file_name like `encoder_.onnx`


### Expected behavior

Have the same name as the original onnx file like `encoder.onnx`

#363 fixes that


Discussion:

Resolving patch:
diff --git a/optimum/onnxruntime/optimization.py b/optimum/onnxruntime/optimization.py
--- a/optimum/onnxruntime/optimization.py
+++ b/optimum/onnxruntime/optimization.py
@@ -93,7 +93,7 @@ def optimize(
         self,
         optimization_config: OptimizationConfig,
         save_dir: Union[str, os.PathLike],
-        file_suffix: str = "optimized",
+        file_suffix: Optional[str] = "optimized",
         use_external_data_format: bool = False,
     ):
         """
@@ -144,7 +144,8 @@ def optimize(
                 # keep_io_types to keep inputs/outputs as float32
                 optimizer.convert_float_to_float16(keep_io_types=True)
 
-            output_path = save_dir.joinpath(f"{model_path.stem}_{file_suffix}").with_suffix(model_path.suffix)
+            suffix = f"_{file_suffix}" if file_suffix else ""
+            output_path = save_dir.joinpath(f"{model_path.stem}{suffix}").with_suffix(model_path.suffix)
             optimizer.save_model_to_file(output_path.as_posix(), use_external_data_format)
 
         LOGGER.info(f"Optimized model saved at: {save_dir} (external data format: " f"{use_external_data_format})")
diff --git a/optimum/onnxruntime/quantization.py b/optimum/onnxruntime/quantization.py
--- a/optimum/onnxruntime/quantization.py
+++ b/optimum/onnxruntime/quantization.py
@@ -286,8 +286,8 @@ def quantize(
             The path of the resulting quantized model.
         """
         use_qdq = quantization_config.is_static and quantization_config.format == QuantFormat.QDQ
-        os.makedirs(save_dir, exist_ok=True)
-        quantized_model_path = Path(save_dir).joinpath(f"{self.onnx_model_path.stem}_{file_suffix}.onnx")
+        save_dir = Path(save_dir)
+        save_dir.mkdir(parents=True, exist_ok=True)
 
         if not quantization_config.is_static:
             if quantization_config.mode != QuantizationMode.IntegerOps:
@@ -343,6 +343,8 @@ def quantize(
         LOGGER.info("Quantizing model...")
         quantizer.quantize_model()
 
+        suffix = f"_{file_suffix}" if file_suffix else ""
+        quantized_model_path = save_dir.joinpath(f"{self.onnx_model_path.stem}{suffix}").with_suffix(".onnx")
         LOGGER.info(f"Saving quantized model at: {save_dir} (external data format: " f"{use_external_data_format})")
         quantizer.model.save_model_to_file(quantized_model_path, use_external_data_format)
 


Test patch:
Repository: huggingface/optimum

Issue:
AttributeError: type object 'ORTModelForCustomTasks' has no attribute 'export_feature'
### System Info

```shell
Mac OS X
Python 3.9.10

transformers	4.22.2
onnxruntime	1.12.1
onnx	1.12.0
torch	1.12.1
```


### Who can help?

@lewtun, @michaelbenayoun
@JingyaHuang, @echarlaix

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from optimum.onnxruntime import ORTModelForCustomTasks

ort_model = ORTModelForCustomTasks.from_pretrained('microsoft/mdeberta-v3-base', from_transformers=True)
```


### Expected behavior

### Possible solution 
It works if add `export_feature`:

```python
from optimum.onnxruntime import ORTModelForCustomTasks


class ORTModelFixed(ORTModelForCustomTasks): 
    export_feature = 'default'
    def __init__(self, model=None, config=None, **kwargs):
        super().__init__(model=model, config=config, **kwargs)

ort_model = ORTModelFixed.from_pretrained('microsoft/mdeberta-v3-base', from_transformers=True)
```
It will work w/o upper tittle-mentioned AttributeError



Discussion:
the same problem for  `ORTModel` class. This problem for `ORTModelForCustomTasks` class just is inherited from  `ORTModel`

Resolving patch:
diff --git a/optimum/onnxruntime/modeling_ort.py b/optimum/onnxruntime/modeling_ort.py
--- a/optimum/onnxruntime/modeling_ort.py
+++ b/optimum/onnxruntime/modeling_ort.py
@@ -1051,6 +1051,7 @@ class ORTModelForCustomTasks(ORTModel):
     Onnx Model for any custom tasks.
     """
 
+    export_feature = "default"
     auto_model_class = AutoModel
 
     def __init__(self, model=None, config=None, **kwargs):


Test patch:
Repository: huggingface/optimum

Issue:
Whisper model export failure
### System Info

```shell
Optimum - e8d987766c5ff5080c8b1452041427a4b675e701
Transformers - 227f8a708a7f8cf000770bdc3ecc8971d6012b1a
```


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The following command fails with error
```bash
python -m optimum.exporters.onnx --model="openai/whisper-tiny.en" --task=speech2seq-lm-with-past  output_whisper
```
**ERROR**
```bash
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:past_key_values.3.encoder.value
```

### Expected behavior

Should not fail


Discussion:

Resolving patch:
diff --git a/optimum/exporters/onnx/model_configs.py b/optimum/exporters/onnx/model_configs.py
--- a/optimum/exporters/onnx/model_configs.py
+++ b/optimum/exporters/onnx/model_configs.py
@@ -751,6 +751,23 @@ class WhisperOnnxConfig(AudioToTextOnnxConfig):
     NORMALIZED_CONFIG_CLASS = NormalizedSeq2SeqConfig
     ATOL_FOR_VALIDATION = 1e-3
 
+    def _create_dummy_input_generator_classes(self, **kwargs) -> List["DummyInputGenerator"]:
+        dummy_inputs_generators = super()._create_dummy_input_generator_classes(**kwargs)
+        # The generated encoder_hidden_states for Whisper has dimensions
+        # (batch_size, encoder_sequence_length / 2, hidden_size). Therefore,
+        # the sequence length is updated to generate the proper cross attention
+        # KVS in monolith case.
+        if self._behavior is ConfigBehavior.MONOLITH:
+            dummy_seq2seq_past_key_values_generator = self.DUMMY_INPUT_GENERATOR_CLASSES[2](
+                self.task,
+                self._normalized_config,
+                encoder_sequence_length=dummy_inputs_generators[0].nb_max_frames // 2,
+                **kwargs,
+            )
+            dummy_inputs_generators[2] = dummy_seq2seq_past_key_values_generator
+
+        return dummy_inputs_generators
+
 
 class MobileNetV1OnnxConfig(VisionOnnxConfig):
     NORMALIZED_CONFIG_CLASS = NormalizedVisionConfig


Test patch:
Repository: huggingface/optimum

Issue:
A weird input in onnx graph from ORTModel pretrained FlauBERT
### System Info

```shell
Python 3.8.10
"optimum[onnxruntime]==1.5.0"
"transformers==4.25.1"
```


### Who can help?

Probably export bug so -> @lewtun, @michaelbenayoun

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


```
from optimum.onnxruntime import ORTModelForFeatureExtraction
from transformers import AutoTokenizer

model = ORTModelForFeatureExtraction.from_pretrained('flaubert/flaubert_base_uncased', from_transformers=True)
tokenizer = AutoTokenizer.from_pretrained("flaubert/flaubert_base_uncased")
inputs = tokenizer("Le chat mange une pomme.", return_tensors="pt")
pred = model(**inputs)
```


### Expected behavior

I should get a `BaseModelOutPut` with `last_hidden_state` tensor as `pred` variable (or something close to that). Actually, when I replace the FlauBERT model `flaubert/flaubert_base_uncased` by another french model CamemBERT `camembert/camembert-base` the above snippet works as expected. (I tried this on my cpu, no gpus involved)
   
But I get an `InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:token_type_ids`.
So I investigate the input of the onnx model graph and find a weird third input with static dimensions:   
```
 name: "input.5"
type {
  tensor_type {
    elem_type: 7
    shape {
      dim {
        dim_value: 2
      }
      dim {
        dim_value: 8
      }
    }
  }
}
```
This input should probably not be there and I suspect that the export is broken in some way for FlauBert model.
(I also get  `TraceWarning` when exporting caused by asserts located in the `forward` method of the transformer implementation of FlauBERT but I am not sure if it is related to my issue)  



Discussion:

Resolving patch:
diff --git a/optimum/exporters/onnx/model_configs.py b/optimum/exporters/onnx/model_configs.py
--- a/optimum/exporters/onnx/model_configs.py
+++ b/optimum/exporters/onnx/model_configs.py
@@ -119,7 +119,7 @@ class CamembertOnnxConfig(DistilBertOnnxConfig):
     pass
 
 
-class FlaubertOnnxConfig(DistilBertOnnxConfig):
+class FlaubertOnnxConfig(BertOnnxConfig):
     pass
 
 


Test patch:
Repository: huggingface/optimum

Issue:
BetterTransformer support for Marian
`The model type marian is not yet supported supported to be used with BetterTransformer.`

Wondering if there is a way to get this working with BetterTransformer. I am trying to get this model translating faster. https://huggingface.co/Helsinki-NLP/opus-mt-zh-en?


Discussion:
@younesbelkada 
> `The model type marian is not yet supported supported to be used with BetterTransformer.`
> 
> Wondering if there is a way to get this working with BetterTransformer. I am trying to get this model translating faster. https://huggingface.co/Helsinki-NLP/opus-mt-zh-en?

I don't think that's possible for now. One reason is the fact that this architecture uses `SiLUActivation` which is not currently supported in the pytorch fastpath (only supports relu and gelu).
Thank you @IlyasMoutawwakil , would you like to submit a PR to update https://github.com/huggingface/optimum/blob/463eced7e565546e9b1aa7a1d45939b949b20cc5/optimum/bettertransformer/models/__init__.py#L71 ?
@fxmarty yes I'll submit a PR
@fxmarty support for marian is possible (same as Bart), it's actually the opus-mt variant that specifically uses `SiLUActivation`.

Resolving patch:
diff --git a/optimum/bettertransformer/models/__init__.py b/optimum/bettertransformer/models/__init__.py
--- a/optimum/bettertransformer/models/__init__.py
+++ b/optimum/bettertransformer/models/__init__.py
@@ -45,6 +45,7 @@ class BetterTransformerManager:
         "hubert": ("HubertEncoderLayer", Wav2Vec2EncoderLayerBetterTransformer),
         "layoutlm": ("LayoutLMLayer", BertLayerBetterTransformer),
         "m2m_100": ("M2M100EncoderLayer", MBartEncoderLayerBetterTransformer),
+        "marian": ("MarianEncoderLayer", BartEncoderLayerBetterTransformer),
         "markuplm": ("MarkupLMLayer", BertLayerBetterTransformer),
         "mbart": ("MBartEncoderLayer", MBartEncoderLayerBetterTransformer),
         "rembert": ("RemBertLayer", BertLayerBetterTransformer),


Test patch:
diff --git a/tests/bettertransformer/test_bettertransformer_encoder.py b/tests/bettertransformer/test_bettertransformer_encoder.py
--- a/tests/bettertransformer/test_bettertransformer_encoder.py
+++ b/tests/bettertransformer/test_bettertransformer_encoder.py
@@ -49,6 +49,7 @@
 ALL_ENCODER_DECODER_MODELS_TO_TEST = [
     "hf-internal-testing/tiny-random-bart",
     "hf-internal-testing/tiny-random-FSMTModel",
+    "hf-internal-testing/tiny-random-marian",
     "hf-internal-testing/tiny-random-mbart",
     "hf-internal-testing/tiny-random-nllb",
 ]
Repository: huggingface/optimum

Issue:
optional `attention_mask` in `ORTModelForxxx`
### Feature request

The behavior of some `ORTModelForxxx` differs from that of `AutoModelForxxx`.
For example, `ORTModelForSequenceClassification` doesn't accept `None` as `attention_mask` even though it's an optional argument.
There's also some missing masks (`attention_mask`, `pixel_mask` or `head_mask` for example in audio and vision models) but I think those are intentionally omitted for now since they're not generated in exporters.

### Motivation

Drop in replacement when using `ORTModelForxxx` instead of `AutoModelForxxx`.

### Your contribution

I will submit a PR.


Discussion:

Resolving patch:
diff --git a/optimum/onnxruntime/modeling_ort.py b/optimum/onnxruntime/modeling_ort.py
--- a/optimum/onnxruntime/modeling_ort.py
+++ b/optimum/onnxruntime/modeling_ort.py
@@ -888,6 +888,9 @@ def forward(
         self.raise_on_numpy_input_io_binding(use_torch)
 
         if self.device.type == "cuda" and self.use_io_binding:
+            if attention_mask is None:
+                attention_mask = torch.ones_like(input_ids)
+
             io_binding, output_shapes, output_buffers = self.prepare_io_binding(
                 input_ids,
                 attention_mask,
@@ -907,7 +910,10 @@ def forward(
         else:
             if use_torch:
                 input_ids = input_ids.cpu().detach().numpy()
-                attention_mask = attention_mask.cpu().detach().numpy()
+                if attention_mask is None:
+                    attention_mask = np.ones_like(input_ids)
+                else:
+                    attention_mask = attention_mask.cpu().detach().numpy()
                 if token_type_ids is not None:
                     token_type_ids = token_type_ids.cpu().detach().numpy()
 


Test patch: