Inference
#1
by
loretoparisi
- opened
How to run inference with ONNX runtime?
you can try our experimental code here: https://github.com/bes-dev/stable_diffusion.openvino/tree/sb/onnxruntime
But we don't recommend to use it
Hi, I am trying to convert the ONNX model to a Tensorflow model, to try to use it in tensorflow lite. But I get the following error with both vae_encoder and unet:
File "/usr/local/lib/python3.7/dist-packages/onnx_tf/backend_tf_module.py", line 99, in __call__ *
output_ops = self.backend._onnx_node_to_tensorflow_op(onnx_node,
File "/usr/local/lib/python3.7/dist-packages/onnx_tf/backend.py", line 347, in _onnx_node_to_tensorflow_op *
return handler.handle(node, tensor_dict=tensor_dict, strict=strict)
File "/usr/local/lib/python3.7/dist-packages/onnx_tf/handlers/handler.py", line 59, in handle *
return ver_handle(node, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/onnx_tf/handlers/backend/conv.py", line 15, in version_11 *
return cls.conv(node, kwargs["tensor_dict"])
File "/usr/local/lib/python3.7/dist-packages/onnx_tf/handlers/backend/conv_mixin.py", line 30, in conv *
x_rank = len(x.get_shape())
ValueError: Cannot take the length of shape with unknown rank.
From what I have been seeing debugging the code in the case of the encoder, the input "input.240" of the convolution "Conv_465" is not an array.Tensor("onnx_tf_prefix_Mul_464:0", dtype=float32)
I attach the code to reproduce it:
import onnx
from onnx_tf.backend import prepare
onnx_model = onnx.load("stable-diffusion-v1-4-onnx/vae_encoder.onnx") # load onnx model
print(onnx.helper.printable_graph(onnx_model.graph))
tf_rep = prepare(onnx_model) # prepare tf representation
tf_rep.export_graph("models/vae_encoder") # export the model
Do you have any idea why this might be or how it could be solved? Thanks!