Accessors versus TensorIterators

kazem: <p>What is the difference between Accessors and TensorIterators? And what is the use case of each?<br>
I have read about TensorItreators in these:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/kurtamohler/pytorch-TensorIterator-examples/blob/main/examples.cpp">
  <header class="source">

      <a href="https://github.com/kurtamohler/pytorch-TensorIterator-examples/blob/main/examples.cpp" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/kurtamohler/pytorch-TensorIterator-examples/blob/main/examples.cpp" target="_blank" rel="noopener nofollow ugc">kurtamohler/pytorch-TensorIterator-examples/blob/main/examples.cpp</a></h4>


      <pre><code class="lang-cpp">#include &lt;iostream&gt;
#include &lt;cassert&gt;
#include &lt;ATen/ATen.h&gt;
#include &lt;ATen/native/TensorIterator.h&gt;
#include &lt;ATen/native/cpu/Loops.h&gt;
#include &lt;ATen/native/ReduceOpsUtils.h&gt;
#include &lt;ATen/AccumulateType.h&gt;

void example1() {
  at::Tensor a = at::ones({10});
  at::Tensor b = at::ones({10});
  at::Tensor out = at::zeros({0});
  std::cout
    &lt;&lt; "\n==========\n"
    &lt;&lt; "example1:"
    &lt;&lt; std::endl;

  //======== Start blog post code =========
  at::TensorIteratorConfig iter_config;
  iter_config
</code></pre>



  This file has been truncated. <a href="https://github.com/kurtamohler/pytorch-TensorIterator-examples/blob/main/examples.cpp" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://labs.quansight.org/blog/2021/04/pytorch-tensoriterator-internals-update">
  <header class="source">
      <img src="https://labs.quansight.org/favicon.ico" class="site-icon" width="16" height="16">

      <a href="https://labs.quansight.org/blog/2021/04/pytorch-tensoriterator-internals-update" target="_blank" rel="noopener nofollow ugc" title="08:00AM - 09 April 2021">Quansight Labs – 9 Apr 21</a>
  </header>

  <article class="onebox-body">
    <img src="https://labs.quansight.org/images/QuansightLabs_logo_V2_rect.png" class="thumbnail onebox-avatar" width="500" height="500">

<h3><a href="https://labs.quansight.org/blog/2021/04/pytorch-tensoriterator-internals-update" target="_blank" rel="noopener nofollow ugc">PyTorch TensorIterator Internals - 2021 Update</a></h3>

  <p>For contributors to the PyTorch codebase, one of the most commonly encountered
C++ classes is
TensorIterator.
TensorIterator offers a standardized way to iterate over elements of
a tensor, automatical</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/TensorIterator.h">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/TensorIterator.h" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/TensorIterator.h" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/main/aten/src/ATen/TensorIterator.h</a></h4>


      <pre><code class="lang-h">#pragma once

#include &lt;ATen/TensorMeta.h&gt;
#include &lt;ATen/core/Dimname.h&gt;
#include &lt;ATen/core/Range.h&gt;
#include &lt;ATen/core/TensorBase.h&gt;
#include &lt;c10/core/DynamicCast.h&gt;
#include &lt;c10/util/FunctionRef.h&gt;
#include &lt;c10/util/MaybeOwned.h&gt;
#include &lt;c10/util/SmallVector.h&gt;
#include &lt;c10/util/TypeCast.h&gt;
#include &lt;c10/util/irange.h&gt;

#include &lt;array&gt;
#include &lt;bitset&gt;

C10_CLANG_DIAGNOSTIC_PUSH()
#if C10_CLANG_HAS_WARNING("-Wshorten-64-to-32")
C10_CLANG_DIAGNOSTIC_IGNORE("-Wshorten-64-to-32")
#endif
</code></pre>



  This file has been truncated. <a href="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/TensorIterator.h" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>And about accessors and packed accessors towards the end here:<br>
<a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/advanced/cpp_extension.html</a></p>
<p>Basically, if I want to loop over a tensor elements and carry out a simple computation which one should I use? does the use case of (Accessors(CPU Tensors)/PackAccesors (CUDA Tensors) VS TensorIterators) depend on the device of the tensor?</p>

ptrblck: <p>The <code>TensorIterator</code> can be used to execute e.g. elementwise kernels as seen in the first link while accessors are used to index data inside a kernel from the tensor instead of using the pointer with strides and indices.</p>

kazem: <p>Is there any preference over which one better or faster?! Is the speed of using an accessor the same? Also When one should use TensorIterator over a cuda tensor instead of defining a cuda kernel from scratch?</p>

ptrblck: <p>These objects are used for different use cases as already mentioned.<br>
If you want to write an elementwise kernel you could use the <code>TensorIterator</code> and allow it to iterate all elements of your tensor. On the other hand, if you want to index a tensor manually and apply any operation the accessor can be used.<br>
I don’t think you would see a huge difference between using the accessor vs. manually indexing the pointer.</p>

kazem: <p>specifically, I would like to reimplement scipy.ndimage.find_objects using pytorch, only on the cpu at the moment, since it is the basic block for cellular analysis in biomedical imaging: Extract the cells from the image using an already built mask and then extract measurements from them.</p>
<p>They use a Numpy Iterator object in the base C code to iterate over all entries of the numpy array and record the smallest and largest index of each object in each direction/dimension in a numpy array pointer called regions, while knowing how many objects there is. The base C code can be found here:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c">
  <header class="source">

      <a href="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c" target="_blank" rel="noopener nofollow ugc">scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c</a></h4>


      <pre><code class="lang-c">/* Copyright (C) 2003-2005 Peter J. Verveer
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above
 *    copyright notice, this list of conditions and the following
 *    disclaimer in the documentation and/or other materials provided
 *    with the distribution.
 *
 * 3. The name of the author may not be used to endorse or promote
 *    products derived from this software without specific prior
 *    written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
 * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
</code></pre>



  This file has been truncated. <a href="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
And its numpy iterator struct is found here:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h">
  <header class="source">

      <a href="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h" target="_blank" rel="noopener nofollow ugc">scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h</a></h4>


      <pre><code class="lang-h">/* Copyright (C) 2003-2005 Peter J. Verveer
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above
 *    copyright notice, this list of conditions and the following
 *    disclaimer in the documentation and/or other materials provided
 *    with the distribution.
 *
 * 3. The name of the author may not be used to endorse or promote
 *    products derived from this software without specific prior
 *    written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
 * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
</code></pre>



  This file has been truncated. <a href="https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Do you recommend implementing it using an Accessor or a TensorIterator?<br>
Also if I write the code in torch C++ API the best I can, Can you please help me correct/debug/improve it, and add it as a new function to torch C++ API?<br>
I would really appreciate it since it will help me build a new torch api for biomedical imaging applications.</p>

kazem: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  I already started a thread that is slowly coming to life:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/102201">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/102201" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/102201" target="_blank" rel="noopener nofollow ugc">scipy.ndimage.find_objects</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2023-05-24" data-time="20:32:54" data-timezone="UTC">08:32PM - 24 May 23 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/kazemSafari" target="_blank" rel="noopener nofollow ugc">
          <img alt="kazemSafari" src="https://avatars.githubusercontent.com/u/32110820?v=4" class="onebox-avatar-inline" width="20" height="20">
          kazemSafari
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          feature
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: nn
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          needs research
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 🚀 The feature, motivation and pitch

This function a basic building block <span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">of any biomedical image analysis application.
It gives a list of tuple of slices of coordinates of labelled objects/cells within a mask image of dtype Uint16 or Uint8, assuming the image background is 0, and the labelled objects go from 1, 2, ..., max_label.

I was wondering it is possible to implement it in torch C++ using a simple TensorIterator.

Basically the simplest case would be it takes a 2D tensor of size (H, W) as input and
outputs a tensor of slices of size (N, 2) where N is the number of objects, and each row
is [slice(start,end,step), slice(start,end,step)]. 

### Alternatives

The implementation in C numpy can be found here:
https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c

which uses Iterators defined here:
https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h

### Additional context

Can it also be extended to allow extract objects from a tensor of dimension (B, C, W, H) where B is the batch size, C the number of
channels and W is the width and H is the height.

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>