Pytorch: Multiplation of tensors with different shape and dimensions

username: vin
cooked: <p>Hi,<br>
I have two different tensors with shape of one dimension with shape of <code>torch.Size([2])</code> and 2 dimensions of <code>torch.Size([2, 10, 5000])</code>. The multiplication of both tensors plot this error <code>RuntimeError: mat1 and mat2 shapes cannot be multiplied (10000x10 and 2x1)</code>. Please, need help. Thanks</p>
<pre><code class="lang-auto">exponentiated_signal = torch.matmul(exp, x)
</code></pre>
<p>When normaly multiplication, this error raise: <code>RuntimeError: The size of tensor a (2) must match the size of tensor b (5000) at non-singleton dimension 2</code></p>
<pre><code class="lang-auto">exponentiated_signal = exp * x
</code></pre>
<p>Instead, this operation <code>exponentiated_signal = exp * x.T</code> is computing well. But when I use the bellow code, one issue rise <code>  RuntimeError: rfft expects a real input tensor, but got ComplexDouble</code>. How can I solve it?</p>
<pre><code class="lang-auto">def transformation(x, real_sigma_interval=torch.arange(-1, 1 + 0.001, 0.001)):
 
    x = torch.tensor(x)
    d = []
    for sigma in real_sigma_interval:
        exp = torch.exp(sigma * torch.tensor(range(len(x))))
        exp /= torch.sum(exp)
        exponentiated_signal = exp * x.T
        d.append(exponentiated_signal) 

    return torch.tensor([torch.fft.rfft(k) for k in d])
</code></pre>

username: Matias_Vasquez
cooked: <p>There are a couple of ways you can do this multiplication</p>
<pre data-code-wrap="python"><code class="lang-python">a = torch.rand([2])
b = torch.rand([2, 10, 5000])

# Option 1
out_1 = a.view(2, 1, 1) * b

# Option 2
out_2 = torch.einsum('i, ijk-&gt;ijk', [a, b])

# Same result
print(torch.all(out_1==out_2))
# -&gt; True
</code></pre>
<hr>
<p>For this other error I would need to know what your input <code>x</code> is.</p>
<p>Trying like this I get no error (maybe you need to update pytorch, can you also post which version you have?)<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/b/2bdb719e44a24acab288b502b4b7e9e50da5db6f.png" alt="image" data-base62-sha1="6fYGCy5ZaIntSx5vmYhjDo61nz1" width="548" height="96"></p>

username: vin
cooked: <p>Here my pytorch version: <code>1.11.0+cpu</code>.<br>
I have used your method and it  giving <code>tensor(True)</code> as answer, which is true. But still I can run <code>transformation(x, real_sigma_interval=torch.arange(-1, 1 + 0.001, 0.001))</code>, this error raised <code>RuntimeError: rfft expects a real input tensor, but got ComplexDouble</code>. If i understand well, the issue focus in this ligne  <code> return torch.tensor([torch.fft.rfft(k) for k in d])</code>. How to figure out this issue?</p>

username: Matias_Vasquez
cooked: <p>Can you post what <code>x</code> is?</p>
<p>I cannot reproduce the error.</p>

username: vin
cooked: <p>The entire  code</p>
<pre><code class="lang-auto">import torch
import numpy as np
import matplotlib.pyplot as plt
import os
from scipy import signal
import math
import scipy
import sympy as sym

os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"

f_s = 48000
f_c = 16000
f_l = 19000
f_r = 17000
b = f_l - f_r
n_order = 5
cut_off = 10

print(torch.__version__)


def high_pass_filter():
    # generate a random signal
    array_list = np.random.randn(1, 100, 1000)

    tensor_array = torch.from_numpy(array_list)

    reshape_tensor = tensor_array.reshape(2, 10, -1)

    reshape_img = reshape_tensor.reshape(-1)

    # compute the fft
    fft_signal = torch.fft.fft(reshape_tensor)
    print(fft_signal)
    reshape_fft_signal = fft_signal.reshape(-1)
    normalize_cutoff = (((f_c - b)/cut_off) / 2) / (f_s/2)  # normalize cut_off
    print("normalize : ", normalize_cutoff)
    down = math.sqrt(1 + ((f_c / normalize_cutoff)**(2*n_order)))
    print("down reponse : ", down)

    butterworth = torch.tensor(1 / down)

    G = torch.mul(butterworth, fft_signal)
    G_reshape = G.reshape(-1)
    output_filter = torch.fft.ifft2(G)
    # print(output_filter)
    output_filter_reshape = output_filter.reshape(-1)

    # plt.figure()
    # plt.plot(reshape_img)
    # plt.figure()
    # plt.plot(reshape_fft_signal)
    # plt.figure()
    # plt.plot(G_reshape)
    # plt.figure()
    # plt.plot(output_filter_reshape)
    # plt.show()
    return output_filter


def transformation(x, real_sigma_interval=torch.arange(-1, 1 + 0.001, 0.001)):

    x = x
    d = []
    for sigma in real_sigma_interval:
        exp = torch.exp(sigma * torch.tensor(range(len(x))))
        exp /= torch.sum(exp)

        # Option 1
        out_1 = exp.view(2, 1, 1) * x

        # Option 2
        out_2 = torch.einsum('i, ijk-&gt;ijk', [exp, x])

        # Same result
        # print(torch.all(out_1 == out_2))

        # exponentiated_signal = exp * x.T
         d.append(out_1) 

    return torch.tensor([torch.fft.rfft(k) for k in d])


if __name__ == '__main__':

    signal = high_pass_filter()
    print("X signal : ", signal)
    print(signal.shape)

    l = transformation(signal)

</code></pre>
<p>Here is my X dataset :</p>
<pre><code class="lang-auto">tensor([[[ 3.4059e-30+1.0762e-46j,  4.8306e-30-1.9032e-46j,
           3.6210e-30-9.2822e-46j,  ...,
          -3.9725e-30-1.3303e-46j,  3.0239e-30-6.3144e-46j,
          -3.3859e-30+5.0432e-46j],
         [ 1.2419e-29+5.6312e-30j,  1.7369e-31+3.3395e-30j,
          -5.4256e-30-1.0258e-30j,  ...,
           1.9680e-30+8.6273e-30j,  1.7434e-30-2.6541e-30j,
           7.2166e-30+2.3827e-30j],
         [ 1.1664e-30+3.6048e-30j,  5.4630e-30+2.5606e-30j,
          -1.0161e-30-1.3846e-30j,  ...,
          -3.3588e-30+2.0733e-30j, -1.9508e-30-9.3269e-31j,
           1.3305e-32-7.5297e-30j],
         ...,
         [ 2.3808e-30+1.4374e-30j, -4.0271e-30+3.6610e-30j,
          -7.3671e-30+2.4534e-30j,  ...,
           2.5958e-30-3.9952e-30j,  4.3275e-30-1.1570e-30j,
          -2.1836e-30-4.2820e-30j],
         [ 1.1664e-30-3.6048e-30j,  5.4630e-30-2.5606e-30j,
          -1.0161e-30+1.3846e-30j,  ...,
          -3.3588e-30-2.0733e-30j, -1.9508e-30+9.3269e-31j,
           1.3305e-32+7.5297e-30j],
         [ 1.2419e-29-5.6312e-30j,  1.7369e-31-3.3395e-30j,
          -5.4256e-30+1.0258e-30j,  ...,
           1.9680e-30-8.6273e-30j,  1.7434e-30+2.6541e-30j,
           7.2166e-30-2.3827e-30j]],

        [[-1.0756e-30+1.9013e-46j,  1.2508e-29-3.5579e-46j,
           4.4410e-30-3.9580e-46j,  ...,
           5.8015e-30-7.1195e-46j, -4.0934e-30+2.1252e-45j,
           1.0737e-29+2.8130e-45j],
         [-6.4574e-30+8.5894e-30j, -2.3886e-31+9.6275e-30j,
          -1.6730e-30-6.5154e-30j,  ...,
          -2.2550e-30-1.3787e-30j,  2.2739e-30-2.2057e-30j,
          -7.7044e-31+1.0369e-30j],
         [ 7.7674e-31+1.1331e-31j, -2.0076e-30+4.8946e-31j,
          -1.1086e-30-3.7737e-30j,  ...,
           5.4034e-30+3.3697e-30j, -2.9261e-31-3.7502e-30j,
          -6.2307e-31+2.5731e-30j],
         ...,
         [-4.2135e-30-1.2409e-31j, -5.0822e-30-1.0209e-30j,
          -4.2199e-30+2.1307e-30j,  ...,
          -8.1933e-30+3.8817e-30j, -1.7231e-30+8.7054e-31j,
          -5.1156e-30+7.2661e-31j],
         [ 7.7674e-31-1.1331e-31j, -2.0076e-30-4.8946e-31j,
          -1.1086e-30+3.7737e-30j,  ...,
           5.4034e-30-3.3697e-30j, -2.9261e-31+3.7502e-30j,
          -6.2307e-31-2.5731e-30j],
         [-6.4574e-30-8.5894e-30j, -2.3886e-31-9.6275e-30j,
          -1.6730e-30+6.5154e-30j,  ...,
          -2.2550e-30+1.3787e-30j,  2.2739e-30+2.2057e-30j,
          -7.7044e-31-1.0369e-30j]]], dtype=torch.complex128)
</code></pre>
<p>shape : <code>torch.Size([2, 10, 5000])</code></p>

username: Matias_Vasquez
cooked: <p>I think this error will be solved if you use <code>fft</code> instead of <code>rfft</code>.</p>
<p><code>rfft</code> supports multiple dimensions, whereas <code>rfft</code> only the real part.</p>
<p>However, you will still get other errors.</p>

username: vin
cooked: <p>Still the same error while using the fft, as <code>ValueError: only one element tensors can be converted to Python scalars</code></p>

username: Matias_Vasquez
cooked: <p>This is a new error, not the same.</p>
<p><code>fft</code> is now working.</p>
<p>The problem is that you have a list of tensors as a result of doing <code>torch.fft.fft(k)</code> (you get one for each batch) So you have tensors inside a list and you are trying to make that into a tensor and that is why you get this new error.</p>
<p>if you do something like this you won’t get the error. <strong>BUT</strong> I have no idea what you are doing and if what I did actually makes sense for the data. So you can do this, but I cannot say for sure if that is what you want.</p>
<pre><code class="lang-python">def transformation(x, real_sigma_interval=torch.arange(-1, 1 + 0.001, 0.001)):
    x = x
    d = []
    for sigma in real_sigma_interval:
        exp = torch.exp(sigma * torch.tensor(range(len(x))))
        exp /= torch.sum(exp)

        out_1 = exp.view(2, 1, 1) * x
        d.append(out_1) 

    return torch.tensor([torch.fft.fft(k).numpy() for k in d])
</code></pre>

username: vin
cooked: <p>Thank you Sir, this working for me. Thank you for your time</p>

username: vin
cooked: <p>Sir, one more help please. Can you have any ideal on how can I resize my current tensor shape of <code>torch.Size([2001, 2, 10, 5000])</code>  by reshaping to  a 2 or 3 D tensor as : <code>torch.tensor([[[3, 100, 5000]]]</code> or any 2 or 3 dimensional shape. Thank you.</p>

username: Matias_Vasquez
cooked: <p>Sure, but first you need to define HOW you want your new tensor to look.</p>
<p>This new view has to have the same number of elements in the tensor</p>
<p>So for example <code>3 x 100 x 5000</code> will not work because it does not have the same number of elements as <code>2001 x 2 x 10 x 5000</code></p>
<p>This is the original</p>
<ul>
<li><code>2001 x 2 x 10 x 5000 = 200,100,000</code></li>
</ul>
<p>This will NOT work</p>
<ul>
<li><code>3 x 100 x 5000 = 1,500,000</code></li>
</ul>
<p>These are some possible solutions</p>
<ul>
<li><code>2001 x 20 x 5000 = 200,100,000</code></li>
<li><code>2001 x 2 x 50000 = 200,100,000</code></li>
<li><code>2 x 10 x 10,005,000 = 200,100,000</code></li>
</ul>
<p>There are many more combinations of how this might look, but it depends on what you want.</p>
<p>For this you can use <code>view</code>, <code>reshape</code>, <code>permute</code> and other operations.</p>

username: vin
cooked: <p>Both  <code>signal.reshape(2, 10, 10005000)</code> and  <code>signal.reshape(2001, 2, 50000)</code> are working. Thanks</p>