Repository: huggingface/optimum

Issue:
"disable_shape_inference" in ORTOptimizer does not work
### System Info

```shell
optimum==1.6.1
```


### Who can help?

@echarlaix @philschmid 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Using following opt. config:
```
opt_conf = OptimizationConfig(optimization_level=0)
opt_conf.fp16 = True
opt_conf.enable_transformers_specific_optimizations = False
opt_conf.disable_embed_layer_norm_fusion = False
opt_conf.disable_shape_inference = True
```
The error keeps appearing (GPT-J model): `ZeroDivisionError: integer division or modulo by zero`

Error trace contains this:
```
File ~/miniconda3/envs/pytorch/lib/python3.10/site-packages/onnxruntime/transformers/models/gpt2/../../onnx_model.py:594, in OnnxModel.convert_float_to_float16(self, use_symbolic_shape_infer, **kwargs)
    591 if use_symbolic_shape_infer:
    592     # Use symbolic shape inference since custom operators (like Gelu, SkipLayerNormalization etc) are not recognized by onnx shape inference.
    593     shape_infer_helper = SymbolicShapeInferenceHelper(model)
--> 594     model = shape_infer_helper.infer_shapes(model, auto_merge=True, guess_output_rank=False)
    596 parameters = {"disable_shape_infer": use_symbolic_shape_infer}
    597 parameters.update(
    598     {
    599         key: kwargs[key]
   (...)
    609     }
    610 )
```

And if you check the code in optimum/onnxruntime/optimization.py on line number 159 you will find that
`optimizer.convert_float_to_float16(keep_io_types=True)` doesn't get a parameter that disables symbolic shape inference.

However, the signature of this method is:
`def convert_float_to_float16(self, use_symbolic_shape_infer=True, **kwargs):`

### Expected behavior

I would like the errors with shape inference to be better explained and to be able to turn it off for real.


Discussion:
Thanks for raising this issue @hivaze! I'm going to take care of it.

Resolving patch:
diff --git a/optimum/onnxruntime/optimization.py b/optimum/onnxruntime/optimization.py
--- a/optimum/onnxruntime/optimization.py
+++ b/optimum/onnxruntime/optimization.py
@@ -143,20 +143,30 @@ def optimize(
         LOGGER.info("Optimizing model...")
 
         for model_path in self.onnx_model_path:
-            optimizer = optimize_model(
-                model_path.as_posix(),
-                model_type,
-                self.normalized_config.num_attention_heads,
-                self.normalized_config.hidden_size,
-                opt_level=optimization_config.optimization_level,
-                optimization_options=optimization_options,
-                use_gpu=optimization_config.optimize_for_gpu,
-                only_onnxruntime=not optimization_config.enable_transformers_specific_optimizations,
-            )
-
-            if optimization_config.fp16:
-                # keep_io_types to keep inputs/outputs as float32
-                optimizer.convert_float_to_float16(keep_io_types=True)
+            try:
+                optimizer = optimize_model(
+                    model_path.as_posix(),
+                    model_type,
+                    self.normalized_config.num_attention_heads,
+                    self.normalized_config.hidden_size,
+                    opt_level=optimization_config.optimization_level,
+                    optimization_options=optimization_options,
+                    use_gpu=optimization_config.optimize_for_gpu,
+                    only_onnxruntime=not optimization_config.enable_transformers_specific_optimizations,
+                )
+
+                if optimization_config.fp16:
+                    # keep_io_types to keep inputs/outputs as float32
+                    optimizer.convert_float_to_float16(
+                        use_symbolic_shape_infer=not optimization_config.disable_shape_inference, keep_io_types=True
+                    )
+            except Exception as e:
+                if "Incomplete symbolic shape inference" in str(e):
+                    err = RuntimeError(
+                        f"{str(e)}. Try to set `disable_shape_inference=True` in your optimization configuration."
+                    )
+                    raise err from e
+                raise
 
             suffix = f"_{file_suffix}" if file_suffix else ""
             output_path = save_dir.joinpath(f"{model_path.stem}{suffix}").with_suffix(model_path.suffix)


Test patch: