Converting to Onnx Raises CUDA out of memory error

username: justinyue1643
cooked: <p>I currently have a basic jupyter notebook that follows this <a href="https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101/" rel="noopener nofollow ugc">article</a>. My hope is to later convert the onnx file into a tflite file in the future.</p>
<p>However, after running the code linked in the article, I receive a CUDA out of memory error by running the code below. I also ran this on a Google Colab and produced the same error, so I can assume that hardware is not the issue here.</p>
<pre><code class="lang-auto">torch.onnx.export(model, input_batch, '../model/deeplab_model_pytorch.onnx', verbose=True)
</code></pre>
<p>Does anybody know why this is causing an error?</p>

username: ptrblck
cooked: <p>Are you able to run the forward pass using the current <code>input_batch</code>?<br>
If I’m not mistaken, the <code>onnx.export</code> method would trace the model, so needs to pass the input to it and execute a forward pass to trace all operations.<br>
If it’s working before calling the <code>export</code> operation, could you try to export this model in a new script with an empty GPU, as your script might already be using some memory.</p>

username: justinyue1643
cooked: <p>I was able to run the current input_batch before calling the export operation. I ended up resolving this issue by making a dummy input a smaller, random tensor (calling <code>torch.randn(1,3,224,224)</code>)and passing that into the <code>export</code> function in place of input_batch in my first post here.</p>
<p>However, I’m running into a different issue regarding the following 2 cells.</p>
<p>Cell <span class="hashtag-raw">#1:</span></p>
<pre><code class="lang-auto"># https://discuss.pytorch.org/t/when-i-export-deeplab-v3-using-torch-onnx-export-i-find-that-the-onnx-model-has-two-outputs-why/92401
dummy_input = torch.randn(1,3,224,224).to('cuda')
dummy_output = model(dummy_input)
input_names = [f"input {i}" for i in range(21)]

torch.onnx.export(model, dummy_input, '../models/deeplab_model_pytorch.onnx', verbose=True, input_names=['input_img'], output_names=['output_img'], opset_version=11)
</code></pre>
<p>Cell <span class="hashtag-raw">#2</span></p>
<pre><code class="lang-auto">import onnx
from onnx_tf.backend import prepare

model_onnx = onnx.load('../models/deeplab_model_pytorch.onnx')

tf_rep = prepare(model_onnx)
tf_rep.export_graph('../models/deeplab_model_tf.pb')
</code></pre>
<p>When <code>tf_rep.export_graph</code> runs in the 2nd cell, I get an error that reads</p>
<blockquote>
<pre><code>RuntimeError: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow.
</code></pre>
</blockquote>
<p>I have read that changing my opset version to 10 in Cell <span class="hashtag-raw">#1</span> would remove the runtime error, but I’m unfortunately running into a different error where model is exported incorrectly between onnx and tensorflow. Would you have a suggestion on how to fix this?</p>

username: ptrblck
cooked: <p>No, I’m unfortunately completely unfamiliar with <code>onnx_tf</code> and its support for PyTorch.<br>
I’m also unsure what <code>pytorch_half_pixel</code> refers to, i.e. it doesn’t sound like the data type (<code>float16</code>), so I assume it’s related to the transformation.</p>