Libtorch index API perform poorly under repetitive usage

Lin_Jia: <p>Hey folks:</p>
<p>I have a Libtorch project that has some execution speed issues. I have spent half a day to debug it. It turns out that using index API repetitively is much less efficient compared to using an accessor.</p>
<p>Here is the doc on index API:<br>
<a href="https://pytorch.org/cppdocs/notes/tensor_indexing.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/cppdocs/notes/tensor_indexing.html</a></p>
<p>Basically, the job is to read lots of tensors and copy their value into a data object in memory:</p>
<pre><code>  for (int i = 0; i &lt; tensor.size(0); i++)
    for (int j = 0; j &lt; tensor.size(1); j++)
      for (int k = 0; k &lt; tensor.size(2); k++)
      {
        data-&gt;setData(i, j, k, tensor.index({i, j, k}).item&lt;float&gt;())
      }
</code></pre>
<p>I benchmarked the above operation, it takes 3600ms.</p>
<p>Then I modified above using accessor API:</p>
<p>auto tensorAccessor = tensor.accessor&lt;float, 3&gt;();<br>
for (int i = 0; i &lt; tensor.size(0); i++)<br>
for (int j = 0; j &lt; tensor.size(1); j++)<br>
for (int k = 0; k &lt; tensor.size(2); k++)<br>
{<br>
data-&gt;setData(i, j, k, tensorAccessor[0][i][j][k])<br>
}</p>
<p>The same operation takes only 900ms.</p>
<p>BTW, tensor is on the CPU device side.</p>
<p>So accessor API is way more efficient compared to index (4X performance gain).</p>

Lin_Jia: <p>Filed a bug report:<br>
</p><aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/45139" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/45139" target="_blank" rel="nofollow noopener">Libtorch index API perform poorly under repetitive usage</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-09-22" data-time="17:04:32" data-timezone="UTC">05:04PM - 22 Sep 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/linjiapro" target="_blank" rel="nofollow noopener">
          <img alt="linjiapro" src="https://avatars0.githubusercontent.com/u/4970879?v=4" class="onebox-avatar-inline" width="20" height="20">
          linjiapro
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">🐛 Bug
I have a Libtorch project that has some execution speed issues. I have spent half a day to debug it....</p>
</div>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>