How do pytorch know, if it is single input/multiple input

Ayush_Singhal: <p>I have a simple question</p>
<p>Lets assume we have a Linear Layer like</p>
<pre><code class="lang-auto">layer = nn.Linear(12 , 12)
</code></pre>
<p>If we pass something like</p>
<pre><code class="lang-auto">inp = torch.rand(12)

layer(inp)
</code></pre>
<p>We get a tensor of size <code>torch.Size([12])</code></p>
<p>And if we pass input like</p>
<pre><code class="lang-auto">inp = torch.rand(13 , 12)

layer(inp)
</code></pre>
<p>We get a tensor of size <code>torch.Size([13 , 12])</code><br>
So how do Pytorch actually knows if it is</p>
<ul>
<li>Single Input<br>
or</li>
<li>Multi Input</li>
</ul>
<p>Is it a simple <code>if-else condition</code> followed by <code>iterating for loop</code>  or <code>broadcasting</code>, or anything <code>totaly different</code> …?</p>

Mah_Neh: <p>That is how matrix multiplication works:</p>
<pre><code class="lang-auto">1  4 
          w1
2  5                =    [w1*1 + w2*4, 2*w1 + 5*w2,...]
          w2
3  6
</code></pre>
<p>Here the rectangle of numbers is your input tensor, where each row is a sample. It is only one row for a single sample (1,12) in your example.</p>
<p>The linear layer is a column vector <code>w = &lt; w1, w2 &gt;</code> where w1 and w2 are the weights.</p>
<p>Each row in the numeric matrix is a sample (1, 4), (2, 5)…</p>
<p>So all you need to do is iterating over whichever tensor a user gives as input, and this includes (1,4)</p>
<p>And carry out the multiplication between each row and the tensor, in this case you get:<br>
<code>Results = [w1*1 + w2*4, 2*w1 + 5*w2,...]</code></p>
<p>As to the exact code implementation I am not sure, but conceptually that is how you do it, and does not matter the number of samples in the input.</p>

Ayush_Singhal: <p>So is it  <code>if-else condition</code> first followed by <code>for loop</code> like</p>
<pre><code class="lang-auto">if multi_input : # 1 Extra Dimension then expected number of dimensions, here (13 , 12) ---&gt; 13 is extra
    # iterate over the list and caluclate the matmul for every value and append in the output list
else : # simply calculate the matmul
</code></pre>

Mah_Neh: <p>From my understanding, in the case of 1D <code>matmul</code> computes the dot product because those are now vectors.</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.matmul.html" rel="noopener nofollow ugc">See the docs here.</a>, first bullet point.</p>
<p>Also, the remaining explanation should help, I include it for easy of access:</p>
<blockquote>
<ul>
<li>If both tensors are 1-dimensional, the dot product (scalar) is returned.</li>
<li>If both arguments are 2-dimensional, the matrix-matrix product is returned.</li>
<li>If the first argument is 1-dimensional and the second argument is 2-dimensional, a 1 is prepended to its dimension for the purpose of the matrix multiply. After the matrix multiply, the prepended dimension is removed.</li>
</ul>
</blockquote>

Ayush_Singhal: <p>Okay got it, was a little confused. Thanks mate <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Mah_Neh: <p>you are welcome, feel free to message me to chat, i’m learning as well</p>