Complex CNN on complex images gives RuntimeError

_zero: <p>Hi,</p>
<p>I am trying to train a UNet model on complex images (MRI data), and, while I can convert the model and get the complex images (both cfloat, same happens with cdouble) just fine, I get the following error during the forward pass:</p>
<pre><code class="lang-auto">RuntimeError: getCudnnDataTypeFromScalarType() not supported for ComplexDouble
</code></pre>
<p>And the full stack trace:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "&lt;MY FOLDER&gt;/train.py", line 105, in &lt;module&gt;
    main()
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main
    _run_hydra(
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 344, in _run_hydra
    run_and_report(
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 199, in run_and_report
    raise ex
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 196, in run_and_report
    return func()
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 345, in &lt;lambda&gt;
    lambda: hydra.run(
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 107, in run
    return run_job(
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/hydra/core/utils.py", line 126, in run_job
    ret.return_value = task_function(task_cfg)
  File "&lt;MY FOLDER&gt;/train.py", line 98, in main
    start_single_process(0, n_gpu, working_dir, config)
  File "&lt;MY FOLDER&gt;/train.py", line 83, in start_single_process
    train_worker(config)
  File "&lt;MY FOLDER&gt;/train.py", line 46, in train_worker
    trainer.train()
  File "&lt;MY FOLDER&gt;/srcs/trainer/base.py", line 81, in train
    result = self._train_epoch(epoch)
  File "&lt;MY FOLDER&gt;/srcs/trainer/trainer.py", line 52, in _train_epoch
    output = self.model(data)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "&lt;MY FOLDER&gt;/srcs/model/unet/unet.py", line 24, in forward
    x1 = self.inc(x)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "&lt;MY FOLDER&gt;/srcs/model/unet/unet_parts.py", line 25, in forward
    return self.double_conv(x)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 423, in forward
    return self._conv_forward(input, self.weight)
  File "&lt;PYTHON FOLDER&gt;/miniconda3/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 419, in _conv_forward
    return F.conv2d(input, weight, self.bias, self.stride,
RuntimeError: getCudnnDataTypeFromScalarType() not supported for ComplexDouble
</code></pre>
<p>I wasn’t although able to pinpoint the cause. Has anyone come across the same issue before?</p>

ptrblck: <p>Cudnn doesn’t support the <code>ComplexDouble</code> data type, so you would have to disable it via <code>torch.backends.cudnn.enabled = False</code> and rerun the code.</p>

_zero: <p>Thank you very much for the reply. Doing that causes another similar error</p>
<pre><code class="lang-auto">RuntimeError: _thnn_conv2d_forward not supported on CUDAType for ComplexFloat
</code></pre>
<p>For the same operation (F.conv2d)</p>

ptrblck: <p>Thanks for the update. In that case neither the cudnn nor the native conv kernels support complex inputs at the moment. As a workaround you could use <code>tensor.unfold</code> (same as the im2col operation) and apply a matrix multiplication using the complex inputs/fiters.</p>

sikhar: <p>I am getting this error while running a downloaded program which i am using for my understanding. Input values are in the form of complex number but i am not be able to initialize the weights for convolutional autoencoder in complex number form. so i am getting this error anyone please help me how to initialize weights in complex number form.<br>
Error message:<br>
‘’‘Input type (CUDAComplexDoubleType) and weight type (torch.cuda.FloatTensor) should be the same’’’</p>

J_Johnson: <p>Hey Patrick, Does this command work in torch 1.9.0?</p>
<p>Says: Cannot find reference ‘cudnn’ in ‘<strong>init</strong>.py’</p>

ptrblck: <p>I’m not sure which command you are referring to, but if it’s <code>torch.backends.cudnn.enabled</code>, then yes it’s still working in 1.9.0.</p>