Tacotron pytorch model to ONNX not exporting?

MuruganR96: <p><strong>RuntimeError: Expected object of backend CPU but got backend CUDA for argument <span class="hashtag">#3</span> 'index’</strong></p>
<p><strong>RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED</strong></p>
<p><strong>RuntimeError: $ Torch: not enough memory: you tried to allocate 2442GB. Buy new RAM! at /pytorch/aten/src/TH/THGeneral.cpp:201</strong></p>
<p>i try to export pytorch tacotron pretrained model to onnx model. it is throwing this <strong>three kind of issues.</strong></p>
<h2>To Reproduce</h2>
<pre><code class="lang-auto">
Traceback (most recent call last):
  File "export_model.py", line 48, in &lt;module&gt;
    torch.onnx.export(model, y, "tts.onnx")
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/__init__.py", line 27, in export
    return utils.export(*args, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 104, in export
    operator_export_type=operator_export_type)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 281, in _export
    example_outputs, propagate)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 224, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 192, in _trace_and_get_graph_from_model
    trace, torch_out = torch.jit.get_trace_graph(model, args, _force_outplace=True)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/jit/__init__.py", line 197, in get_trace_graph
    return LegacyTracedModule(f, _force_outplace)(*args, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/jit/__init__.py", line 252, in forward
    out = self.inner(*trace_inputs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/TTS/models/tacotron.py", line 35, in forward
    inputs = self.embedding(characters)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 118, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/functional.py", line 1454, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of backend CPU but got backend CUDA for argument #3 'index'
</code></pre>
<pre><code class="lang-auto">
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 260, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
    module._apply(fn)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
    module._apply(fn)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
    module._apply(fn)
  [Previous line repeated 1 more time]
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 117, in _apply
    self.flatten_parameters()
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 113, in flatten_parameters
    self.batch_first, bool(self.bidirectional))
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
</code></pre>
<pre><code class="lang-auto">
Traceback (most recent call last):
  File "export_model.py", line 48, in &lt;module&gt;
    torch.onnx.export(model, y, "tts.onnx")
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/__init__.py", line 27, in export
    return utils.export(*args, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 104, in export
    operator_export_type=operator_export_type)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 281, in _export
    example_outputs, propagate)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 224, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 192, in _trace_and_get_graph_from_model
    trace, torch_out = torch.jit.get_trace_graph(model, args, _force_outplace=True)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/jit/__init__.py", line 197, in get_trace_graph
    return LegacyTracedModule(f, _force_outplace)(*args, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/jit/__init__.py", line 252, in forward
    out = self.inner(*trace_inputs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/TTS/models/tacotron.py", line 35, in forward
    inputs = self.embedding(characters)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 118, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/home/admin1/speech_recognition/text2speech/new_one/env/lib/python3.6/site-packages/torch/nn/functional.py", line 1454, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: $ Torch: not enough memory: you tried to allocate 2442GB. Buy new RAM! at /pytorch/aten/src/TH/THGeneral.cpp:201
</code></pre>
<h2>Expected behavior</h2>
<p>from torch.autograd import Variable<br>
import torch.onnx</p>
<p>from TTS.models.tacotron import Tacotron<br>
from TTS.layers import *<br>
from TTS.utils.data import *<br>
from TTS.utils.audio import AudioProcessor<br>
from TTS.utils.generic_utils import load_config<br>
from TTS.utils.text import text_to_sequence<br>
from TTS.utils.synthesis import synthesis<br>
from TTS.utils.visual import visualize<br>
from utils.text.symbols import symbols, phonemes</p>
<p><strong>os.environ[“CUDA_VISIBLE_DEVICES”]="0, 1"</strong></p>
<p>MODEL_PATH = ‘best_model.pth.tar’<br>
CONFIG_PATH = ‘/config.json’<br>
OUT_FOLDER =  ‘/test’<br>
CONFIG = load_config(CONFIG_PATH)<br>
use_cuda = True</p>
<p>CONFIG.audio[“preemphasis”] = 0.97<br>
ap = AudioProcessor(**CONFIG.audio)</p>
<p>cp = torch.load(MODEL_PATH)</p>
<p>print(cp[‘step’])<br>
num_chars = len(phonemes) if CONFIG.use_phonemes else len(symbols)<br>
model = Tacotron(num_chars, CONFIG.embedding_size, CONFIG.audio[‘num_freq’], CONFIG.audio[‘num_mels’], CONFIG.r, attn_windowing=False)<br>
model.load_state_dict(cp[‘model’])<br>
model.eval()<br>
model = Tacotron(num_chars, CONFIG.embedding_size, CONFIG.audio[‘num_freq’], CONFIG.audio[‘num_mels’], CONFIG.r, attn_windowing=False)<br>
model.load_state_dict(cp[‘model’])<br>
model.eval()</p>
<p><strong>x = Variable(torch.randn(61, 256, 1025, 80, 2))<br>
y = x.long()<br>
y = y.cuda() ##<span class="hashtag">#dummy</span> input###<br>
torch.onnx.export(model, y, “tts.onnx”)</strong></p>
<p>=========================================================<br>
<strong>i was followed pytorch instructions for pytorch model to onnx export.</strong></p>
<p>sir how to resolve this issue.</p>
<ol>
<li>this dummy input - shape problem (or)</li>
<li>is it wrong way for exporting model from pytorch to onnx? (or)</li>
<li>memory allocating issue?</li>
</ol>
<p>if memory allocation issue means,<br>
in system configuration is,</p>
<p>4 GPUs - each as 24 GB.<br>
190 GB RAM - 80 cores.</p>
<p>sir please help me how to resolve? how can export ONNX model by using pytorch.?</p>
<h2>Environment</h2>
<pre><code class="lang-auto">Collecting environment information...
PyTorch version: 1.0.1.post2
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.2 LTS
GCC version: (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CMake version: version 3.12.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: TITAN RTX
GPU 1: TITAN RTX
GPU 2: TITAN RTX
GPU 3: TITAN RTX

Nvidia driver version: 415.27
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.4.2

Versions of relevant libraries:
[pip3] numpy==1.14.3
[pip3] torch==1.0.1.post2
[pip3] torchvision==0.2.2.post3
[pip3] torchviz==0.0.1
[conda] Could not collect

</code></pre>
<p>i am beginner for pytorch. give me some suggestions sir. how to resolve?</p>
<p>Thank you so much.<img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>