RuntimeError: mat2 must be a matrix, got 1-D tensor

username: insung3511
cooked: <pre><code class="lang-auto">new_RBM.py in v_to_h(self, v)
     37 
     38         p_h = F.sigmoid(
---&gt; 39             F.linear((v), (w).squeeze(), bias=h_bias)
     40         ).cuda()
     41 
RuntimeError: mat2 must be a matrix, got 1-D tensor
</code></pre>
<p>trouble with linear input tensor. when I remove <code>.squeeze()</code> code then got error like this.</p>
<pre><code class="lang-auto">     37 
     38         p_h = F.sigmoid(
---&gt; 39             F.linear((v), (w), bias=h_bias)
     40         ).cuda()
     41 

RuntimeError: The expanded size of the tensor (1) must match the existing size (13000000) at non-singleton dimension 1.  Target sizes: [1, 1].  Tensor sizes: [13000000]
</code></pre>
<p>same part, other error. I don’t know how solve this problem.</p>

username: AlphaBetaGamma96
cooked: <p>Can you share a minimal reproducible example of this error? (Or at least some of the source code?)</p>
<p>More info on the size of <code>v</code> and <code>w</code> would help solve this issue as it’s mostly likely due to the input Tensor not be compatible with the <code>F.linear</code> operation.</p>

username: insung3511
cooked: <p>Im not sure does it helpful here is it.</p>
<pre><code class="lang-python">    def v_to_h(self, v):
        v = (v.clone().detach()).reshape(-1, 13000000)
        h_bias = (self.h_bias.clone())        
        w = (self.W.clone())

        size_msg = '''
        {} {} {}
        '''.format(torch.flatten(v).size(), torch.flatten(w).size(), h_bias.size())
        print(size_msg)
        
        p_h = F.sigmoid(
            F.linear(torch.flatten(v), torch.flatten(w), bias=h_bias)
        ).cuda()

        sample_h = self.sample_from_p(p_h)
        return p_h, sample_h
</code></pre>

username: AlphaBetaGamma96
cooked: <p>Yeah, so the issue is that you’re using <code>torch.flatten()</code> on <code>v</code> and <code>w</code>, these two tensors need to be matrices and not vectors, the sizes are defined in the documentation <a href="https://pytorch.org/docs/stable/generated/torch.nn.functional.linear.html" rel="noopener nofollow ugc">here</a></p>

username: insung3511
cooked: <p>hmm, okay.<br>
I wondering why does not output size can’t be 13000000. Because <code>torch.flatten(v)</code> and bias size is 13000000, and original weight size <code>1, 13000000</code>. Thats reason why I changed the size.</p>
<p>Set it up all output feature size got error.</p>

username: AlphaBetaGamma96
cooked: <p>So the issue is the size of the bias Tensor,</p>
<pre><code class="lang-auto">v.shape #returns torch.Size([1300])
weight.shape #returns torch.Size([1, 1300])
bias.shape #returns torch.Size([1300])
out = F.linear(v.unsqueeze(0), weight) #linear without bias
out.shape #returns torch.Size([1, 1])
</code></pre>
<p>The bias needs to be same shape as <code>out</code> (so just a scalar). This is the issue</p>

username: insung3511
cooked: <aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="6" data-topic="151404">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p>The bias needs to be same shape as <code>out</code> (so just a scalar). This is the issue</p>
</blockquote>
</aside>
<p>really thank you… i got a other issue that overflow… i will try it to figure out. thanks a lot.</p>