N dimensional histogram function in pytorch CPU/GPU

username: Marian_Ivanov
cooked: <p>We are  working on interactive visualization of multi-dimensional data<br>
Usually O(2-7) dimensions (still dense) Currently we are using numpy implementation  for multidimensional histogram filling , slicing,  summing:</p>
<ul>
<li>creating NDimensional histogram  once:</li>
</ul>
<pre><code class="lang-auto">H, axis = np.histogramdd(inputArray.values, bins=bins, range=hRange)
</code></pre>
<ul>
<li>querying interactively - many times</li>
</ul>
<pre><code class="lang-auto">y = np.sum(H[hSliceLocal], axis=axis)
</code></pre>
<p>I would like to use PyTorch (or Numba) to get faster GPU implementation</p>
<p>My questions are following (my current Python experience is limitted):</p>
<ul>
<li>Before investing time, which speed-up factor do  you expect in histogramm filling resp. slicing ?</li>
<li>Did PyTorch team consider providing implementation such as  in numpy? We would be happy to contribute, but our experience is limited</li>
</ul>
<p>I’m aware of the discussion (from 2017) in</p><aside class="quote" data-post="1" data-topic="5350">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wasiahmad/48/248_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/histogram-function-in-pytorch/5350">Histogram function in pytorch</a> 
  </div>
  <blockquote>
    Is there any function in pytorch like<a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html" rel="noopener nofollow ugc"> numpy.histogram</a>?
  </blockquote>
</aside>
<p>
related to histogramming in 1D  resp. hack for 3D.</p>
<p>I’m looking for equivalent of the numpy.histogramdd.html (CPU/GPU)<br>
<a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogramdd.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogramdd.html</a></p>

username: albanD
cooked: <p>Hi,</p>
<p>Does <a href="https://pytorch.org/docs/stable/torch.html#torch.histc" rel="nofollow noopener">histc</a> work for you? Or you would need a multidimensional version of it?</p>

username: Marian_Ivanov
cooked: <p>I personally prefer multi-dimensional version of it with interface as in  the numpy.histogramdd<br>
(see my numpy examples in original post)</p>

username: albanD
cooked: <p>I am not familiar with this particular numpy function <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"><br>
I don’t think we have such function in pytorch at the moment, but I’m sure we will be happy to accept a PR to add this new feature.</p>
<p>Do you know how it is implemented? Is it only a wrapper around the 1D version, or does it need substantially different algorithms?</p>

username: Marian_Ivanov
cooked: <p>I checked numpy implementation in <a href="https://github.com/numpy/numpy/blob/v1.17.0/numpy/lib/histograms.py#L935-L1113" rel="nofollow noopener">https://github.com/numpy/numpy/blob/v1.17.0/numpy/lib/histograms.py#L935-L1113</a><br>
At the end algorithm in (step 3) is using 1D histogramming  - <code> np.bincount</code><br>
Hovewer, numpy implementation assume non uniform binning - to find bins they use internally <code>np.searchsorted</code>.</p>
<p>In case searchsorted will be difficult to implement (I did not find it in pytorch= only external <a href="https://github.com/aliutkus/torchsearchsorted" rel="nofollow noopener">https://github.com/aliutkus/torchsearchsorted</a>), uniform binning will be sufficient.</p>
<p>Algorithm:</p>
<ol start="0">
<li>Calculate bin edges if not provided</li>
<li>Compute the bin number each sample falls into - assuming non uniform binning<br>
<code>np.searchsorted</code>
</li>
<li>Compute the sample indices in the flattened histogram matrix<br>
<code>np.ravel_multi_index</code>
</li>
<li>Compute the number of repetitions in xy and assign it to the flattened histmat.<br>
<code>np.bincount</code>
</li>
<li>Shape into a proper matrix<br>
<code>hist.reshape(nbin)</code>
</li>
</ol>

username: albanD
cooked: <p>That would be a useful addition I think.<br>
Could you open an issue on github for a new feature with these informations?</p>

username: Marian_Ivanov
cooked: <p>New issue feature request opened: <a href="https://github.com/pytorch/pytorch/issues/29209" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/29209</a></p>