Use torch.combinations with large parameters

username: dofasol
cooked: <p>Hi,<br>
I would like to generate all combinations of elements in a list.I know I can use torch.combinations for that, but I need to use large parameters, which produces huge number of combinations. So the problem is that it does not fit into memory. For example :</p>
<pre><code class="lang-auto">torch.combinations(torch.arange(-10, 10, dtype=torch.int8).cuda(), 8)
</code></pre>
<p>yields</p>
<blockquote>
<p>RuntimeError: CUDA out of memory. Tried to allocate 23.84 GiB (GPU 0; 4.00 GiB total capacity; 1024 bytes already allocated; 2.62 GiB free; 2.00 MiB reserved in total by PyTorch)</p>
</blockquote>
<p>Is there a way to make kind of a pagination with this function ?</p>

username: KFrank
cooked: <p>Hi Dofasol!</p>
<aside class="quote no-group" data-username="dofasol" data-post="1" data-topic="135949" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dofasol/48/30982_2.png" class="avatar"> dofasol:</div>
<blockquote>
<p>So the problem is that it does not fit into memory.</p>
</blockquote>
</aside>
<p>The root of the problem is that something is wrong with how<br>
<code>torch.combinations()</code> is implemented.  See this new thread<br>
of mine and the github issue it references:</p>
<aside class="quote quote-modified" data-post="1" data-topic="136200">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/egregious-inefficiency-in-torch-combinations/136200">Egregious inefficiency in torch.combinations()</a> 
  </div>
  <blockquote>
    Hi All! 
This post adds context and details to the existing pytorch github 
issue 41325 and a recent forum thread (see below). 
torch.combinations() breaks on problems of moderate size and 
is vastly slower than other implementations, such as python’s 
itertools.combinations(). 
One oddity (as noted in the github issue) is that 
torch.combinations (elements, 1) can  work, where 
the essentially equivalent computation, 
torch.combinations (elements, len (elements) - 1), fails. 
Consider this scri…
  </blockquote>
</aside>

<aside class="quote no-group">
<blockquote>
<p>Is there a way to make kind of a pagination with this function ?</p>
</blockquote>
</aside>
<p>As an aside, generating all such combinations, either one at a time<br>
or in “pages,” is straightforward enough just using python loops.</p>
<p>However, if <code>torch.combinations()</code> <em>did</em> work correctly and you<br>
were working with a truly large problem, you could split your set<br>
of elements into multiple subsets, apply <code>torch.combinations()</code><br>
on the subsets, and recombine the sub-combinations back into<br>
combinations of elements of the original set.  You could do this a<br>
piece at a time (“pages”) so that you would never have to materialize<br>
the entire (potentially very large) set of combinations all at once.</p>
<p>Best.</p>
<p>K. Frank</p>