Unable to export parakeet-ctc-0.6b.nemo to ONNX file
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

When exporting the parakeet-ctc-0.6b.nemo to an ONNX file via the export() method, it creates a 2.7MB ONNX file, and dumps the rest of the ONNX files into the same folder, instead of in the ONNX file.

**Steps/Code to reproduce bug**

`import nemo.collections.asr as nemo_asr
asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="nvidia/parakeet-ctc-0.6b")
asr_model.export("parakeet.onnx")
`


**Expected behavior**

A single ONNX file ~ 2.3GB

**Environment overview (please complete the following information)**

 - Environment location: Ubuntu 24.04 WSL2
 - Method of NeMo install: pip install nemo_toolkit
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- Nemo 1.23.0 and 2.0.0rc1 (neither works)
- Ubuntu 24.04
- PyTorch version 2.4.1
- Python version 3.10.12

**Additional context**

Add any other context about the problem here.
Example: GPU model


Same error for us:


```
model = EncDecCTCModelBPE.restore_from("model_lt_ctc.nemo")
ie = model.input_module.input_example(max_batch=1, max_dim=256)
model.export(output="model_lt_ctc_exp.onnx", input_example=ie, verbose=True)
```


Error:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/dist-packages/nemo/core/classes/exportable.py", line 117, in export
    out, descr, out_example = model._export(
  File "/usr/local/lib/python3.10/dist-packages/nemo/core/classes/exportable.py", line 197, in _export
    output_example = self.forward(*input_list, **input_dict)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/models/asr_model.py", line 288, in forward_for_export
    encoder_output = enc_fun(audio_signal=audio_signal, length=length)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/modules/conformer_encoder.py", line 461, in forward_for_export
    rets = self.forward_internal(
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/modules/conformer_encoder.py", line 583, in forward_internal
    audio_signal = layer(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1510, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1519, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/parts/submodules/conformer_modules.py", line 171, in forward
    x = self.self_attn(query=x, key=x, value=x, mask=att_mask, pos_emb=pos_emb, cache=cache_last_channel)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1510, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1519, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/parts/submodules/multi_head_attention.py", line 252, in forward
    scores = (matrix_ac + matrix_bd) / self.s_d_k  # (batch, head, time1, time2)
RuntimeError: The size of tensor a (12500) must match the size of tensor b (7500) at non-singleton dimension 3
```


I found out after I raised the initial issue that this is normal behaviour for models > 2GB. See https://onnxruntime.ai/docs/tutorials/web/large-models.html