Mean for empty tensor

jaehyung.ca: <p>I found that when I x.mean() for an empty tensor x, I got nan. Is it intended or not?<br>
I think it would confuse people.</p>

Nikronic: <p>Hi,</p>
<p>I think this behavior is intentional as it makes more sense than anything to me, at least.<br>
This is the prefered implementation in numpy too.</p>
<p>Personally, I think mean of empty tensor can be defined in this way: <code>nothing/len(nothing)</code> or clearer version <code>[nan] / len([nan])</code> which only can produce two kind of output, 1. crash in runtime! 2. nan as the output<br>
(I did not read the source code so I might be wrong about how it works exactly)</p>
<p>Also, if you look more intuitively, if we do not want to raise an exception, the best scenario is <code>nan</code> as there are many numerical operation that might produce <code>nan</code> as an unexpected bahvior.</p>
<aside class="quote no-group" data-username="jaehyung.ca" data-post="1" data-topic="90348">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jaehyung.ca/48/24625_2.png" class="avatar"> jaehyung.ca:</div>
<blockquote>
<p>I think it would confuse people</p>
</blockquote>
</aside>
<p>Can you explain what is your idea and prefered bahvior? And why it would confuse people?</p>
<aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/29688168/mean-nanmean-and-warning-mean-of-empty-slice">
  <header class="source">

      <a href="https://stackoverflow.com/questions/29688168/mean-nanmean-and-warning-mean-of-empty-slice" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/1832942/michael-currie" target="_blank" rel="noopener nofollow ugc">
    <img alt="Michael Currie" src="https://i.stack.imgur.com/slY05.jpg?s=256&amp;g=1" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/29688168/mean-nanmean-and-warning-mean-of-empty-slice" target="_blank" rel="noopener nofollow ugc">mean, nanmean and warning: Mean of empty slice</a>
</h4>

<div class="tags">
  <strong>python, numpy, warnings</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/1832942/michael-currie" target="_blank" rel="noopener nofollow ugc">
    Michael Currie
  </a>
  on <a href="https://stackoverflow.com/questions/29688168/mean-nanmean-and-warning-mean-of-empty-slice" target="_blank" rel="noopener nofollow ugc">12:22AM - 17 Apr 15 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/xla/issues/888">
  <header class="source">

      <a href="https://github.com/pytorch/xla/issues/888" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/xla</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/xla/issues/888" target="_blank" rel="noopener nofollow ugc">torch.mean with empty dim should nan</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-08-01" data-time="23:41:47" data-timezone="UTC">11:41PM - 01 Aug 19 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-09-05" data-time="15:39:50" data-timezone="UTC">03:39PM - 05 Sep 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/ailzhang" target="_blank" rel="noopener nofollow ugc">
          <img alt="ailzhang" src="https://avatars.githubusercontent.com/u/5248122?v=4" class="onebox-avatar-inline" width="20" height="20">
          ailzhang
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          pytorch api
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">```
import torch
import torch_xla

device = 'xla:0'
# device='cpu'

shape<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden"> = (2, 0, 4)
x = torch.randn(shape, device=device)
y = torch.mean(x)
print(y)
```
Expected behavior: cpu produce nan while xla produce 0.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Bests</p>

jaehyung.ca: <aside class="quote no-group" data-username="Nikronic" data-post="2" data-topic="90348">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<p>look more intuitively, if we do not want to raise an exception, the best scenario is <code>nan</code> as there are many numerical operation that might produce <code>nan</code> as an unexpected bahvior.</p>
</blockquote>
</aside>
<p>Thanks for the reply.<br>
Well, I think <code>mean() of an empty tensor</code> should be<code> an empty tensor</code>.</p>

Nikronic: <p>How can we consider <code>empty</code> when we for sure know that <code>len(an empty array)=0</code> ?<br>
Actually, this happens:<br>
<code>torch.sum(a) / len(a)</code>, and obviously <code>len(a)=0</code>, so if <code>torch.sum(a)</code> becomes anything rather than <code>0</code>, then division by zero exception will occur.<br>
Also, <code>torch.sum(torch.tensor([np.nan])) = tensor[nan]</code> so <code>[nan] / 0 = nan</code>.</p>
<p>Although there might be some other reasons due to stability which I do not know about.</p>
<p>The main question that I have too is that why the sum of empty array is zero? Although in the case of mean, it won’t make a difference as <code>nan/nan = nan = zero/nan </code>.<br>
After googling, I found that in numpy, fortran intrinsics follow this behavior:<br>
</p><aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/48479750/passing-an-empty-array-to-sum-and-product" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/3501546/roygvib" target="_blank" rel="nofollow noopener">
    <img alt="roygvib" src="https://i.stack.imgur.com/zzZH7.jpg?s=128&amp;g=1" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/48479750/passing-an-empty-array-to-sum-and-product" target="_blank" rel="nofollow noopener">Passing an empty array to sum() and product()</a>
</h4>

<div class="tags">
  <strong>python, arrays, fortran</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/3501546/roygvib" target="_blank" rel="nofollow noopener">
    roygvib
  </a>
  on <a href="https://stackoverflow.com/questions/48479750/passing-an-empty-array-to-sum-and-product" target="_blank" rel="nofollow noopener">07:32PM - 27 Jan 18 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>So as torch tries to implement similar behavior to numpy, same idea is happening here.</p>
<p>I hope someone can explain more accurately.</p>