Sum a tensor over a range of axes

f3ba: <p>I am trying to sum a tensor over its first <code>n</code> axes, where <code>n</code> is a parameter I don’t know in advance. Why something like the following doesn’t work?</p>
<pre><code class="lang-auto">v = torch.randn(100, 20, 10)
torch.sum(v, range(2))  # I would write range(n) here.
</code></pre>
<p>This gives an error:</p>
<pre><code class="lang-auto">TypeError: sum() received an invalid combination of arguments - got (Tensor, range), but expected one of:
 * (Tensor input, torch.dtype dtype)
      didn't match because some of the arguments have invalid types: (Tensor, range)
 * (Tensor input, tuple of names dim, bool keepdim, torch.dtype dtype, Tensor out)
 * (Tensor input, tuple of ints dim, bool keepdim, torch.dtype dtype, Tensor out)
</code></pre>
<p>However if I instantiate the <code>range</code>:</p>
<pre><code class="lang-auto">v = torch.randn(100, 20, 10)
torch.sum(v, list(range(2)))
</code></pre>
<p>Then it works. But this seems inefficient for large <code>n</code>.</p>

albanD: <p>Hi,</p>
<p>In general Tensors with a very large number of dimensions are not as efficient. Cuda will limit you to 25 dimensions if I recall correctly. And on cpu, you might hit slowdowns.<br>
But to answer your question, sum does expect a list/tuple and does not accept a generator (what range returns).</p>

f3ba: <p>Thanks. But there is no fundamental reason why it could not accept a range, right?</p>
<p>I opened a feature request issue:</p>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/32288" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/32288" target="_blank" rel="nofollow noopener">Allow range in dim argument of reducing operations such as sum</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-01-16" data-time="13:41:34" data-timezone="UTC">01:41PM - 16 Jan 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/cossio" target="_blank" rel="nofollow noopener">
          <img alt="cossio" src="https://avatars2.githubusercontent.com/u/10637482?v=4" class="onebox-avatar-inline" width="20" height="20">
          cossio
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">I am trying to sum a tensor over its first n axes, where n is a parameter I don’t know in...</p>
</div>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


albanD: <p>I don’t think there is any reason for it to accept the range because it will check all the values before doing work.</p>

f3ba: <aside class="quote no-group" data-username="albanD" data-post="4" data-topic="66856">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>I don’t think there is any reason for it to accept the range</p>
</blockquote>
</aside>
<p>Convenience? <code>range(n)</code> is better to type and read than <code>list(range(n))</code>.</p>

albanD: <p>Right. But that is a different argument than the one you mentionned above which was " But this seems inefficient for large <code>n</code> .".</p>

f3ba: <p>Yes, it’s a different argument, sorry. The efficiency seemed more important to me at first but as you explained that’s not so important. But even if that doesn’t matter, convenience is also a plus <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:">. I have updated the Github issue.</p>