Why can't I have out_features = 1 in a Linear layer?

username: michaelw
cooked: <p>I cannot create and use a <code>torch.nn.Linear</code> layer with a single output (i.e., <code>out_features = 1</code>).</p>
<p>The simplest example I can reproduce is</p>
<pre><code class="lang-python">t = torch.tensor([
    [1.0, 2.0],
    [3.0, 4.0],
    [5.0, 6.0],
])
lin = torch.nn.Linear(2, 1)
lin(t)
</code></pre>
<p>which gives</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/workspaces/scratch/python/tsp.ipynb Cell 9 line 7
      1 t = torch.tensor([
      2     [1.0, 2.0],
      3     [3.0, 4.0],
      4     [5.0, 6.0],
      5 ])
      6 lin = torch.nn.Linear(2, 1)
----&gt; 7 lin(t)

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-&gt; 1518     return self._call_impl(*args, **kwargs)

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py:114, in Linear.forward(self, input)
    113 def forward(self, input: Tensor) -&gt; Tensor:
--&gt; 114     return F.linear(input, self.weight, self.bias)

RuntimeError: could not create a primitive descriptor for a matmul primitive
</code></pre>
<p>This is unexpected since it seems like all other values for <code>out_features</code> work fine. For example</p>
<pre><code class="lang-auto">t = torch.tensor([
    [1.0, 2.0],
    [3.0, 4.0],
    [5.0, 6.0],
])
lin = torch.nn.Linear(2, 3)
lin(t)
</code></pre>
<p>gives</p>
<pre><code class="lang-auto">tensor([[-1.7300, -1.4911, -0.7885],
        [-2.9954, -2.1098, -2.1714],
        [-4.2608, -2.7285, -3.5543]], grad_fn=&lt;AddmmBackward0&gt;)
</code></pre>
<p>How do I get a 1D output (for each example) from a <code>Linear</code> layer?</p>

username: J_Johnson
cooked: <aside class="quote no-group" data-username="michaelw" data-post="1" data-topic="191593">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/michaelw/48/63041_2.png" class="avatar"> michaelw:</div>
<blockquote>
<pre><code class="lang-auto">t = torch.tensor([
    [1.0, 2.0],
    [3.0, 4.0],
    [5.0, 6.0],
])
lin = torch.nn.Linear(2, 1)
lin(t)
</code></pre>
</blockquote>
</aside>
<p>Your code runs fine for me. No errors. Perhaps you could report an issue on the PyTorch github and include your version, OS, etc.</p>

username: vdw
cooked: <p>Same here, the example code runs fine, both on CPU or GPU (PyTorch version 2.0.1+cu117 on Ubuntu 22.04)</p>

username: michaelw
cooked: <p>Thanks for testing this!</p>
<p>It seems like this is a bug that affects ARM and was already handled in <a href="https://github.com/pytorch/pytorch/issues/110149" class="inline-onebox" rel="noopener nofollow ugc">nn.Linear forward error on AArch64 if the out_features equals to 1 · Issue #110149 · pytorch/pytorch · GitHub</a>. Apologies for not checking there first – I didn’t realize it was a bug and assumed I was misusing something.</p>
<p>Thanks again.</p>