Egregious inefficiency in torch.combinations()

username: KFrank
cooked: <p>Hi All!</p>
<p>This post adds context and details to the existing pytorch github<br>
issue 41325 and a recent forum thread (see below).</p>
<p><code>torch.combinations()</code> breaks on problems of moderate size and<br>
is vastly slower than other implementations, such as python’s<br>
<code>itertools.combinations()</code>.</p>
<p>One oddity (as noted in the github issue) is that<br>
<code>torch.combinations (elements, 1)</code> can  work, where<br>
the essentially equivalent computation,<br>
<code>torch.combinations (elements, len (elements) - 1)</code>, fails.</p>
<p>Consider this script:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

import itertools

print ('pytorch:')
for  k in range (1, 8):
    print ('k, C(2k, k):', k, len (torch.combinations (torch.arange (2 * k), k)))

print ('python:')
for  k in range (1, 11):
    print ('k, C(2k, k):', k, len (list (itertools.combinations (range (2 * k), k))))

print ('specific cases:')

try:
    print ('trying torch.combinations (torch.arange (16), 8) ...')
    print (len (torch.combinations (torch.arange (16), 8)))
except (Exception) as e:
    print ('e:', e)

try:
    print ('trying torch.combinations (torch.arange (16), 1) ...')
    print (len (torch.combinations (torch.arange (16), 1)))
except (Exception) as e:
    print ('e:', e)

try:
    print ('trying itertools.combinations (range (16), 15) ...')
    print (len (list (itertools.combinations (range (16), 15))))
    print ('trying torch.combinations (torch.arange (16), 15) ...')
    print (len (torch.combinations (torch.arange (16), 15)))
except (Exception) as e:
    print ('e:', e)

print ('combinations should accept combinations of zero elements:')

try:
    print ('trying itertools.combinations (range (5), 0) ...')
    print (list (itertools.combinations (range (5), 0)))
    print ('trying torch.combinations (torch.arange (5), 0) ...')
    print (torch.combinations (torch.arange (5), 0))
except (Exception) as e:
    print ('e:', e)

</code></pre>
<p>And its output:</p>
<pre><code class="lang-plaintext">1.10.0
pytorch:
&lt;string&gt;:12: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1634272128894/work/aten/src/ATen/native/TensorShape.cpp:2157.)
k, C(2k, k): 1 2
k, C(2k, k): 2 6
k, C(2k, k): 3 20
k, C(2k, k): 4 70
k, C(2k, k): 5 252
k, C(2k, k): 6 924
k, C(2k, k): 7 3432
python:
k, C(2k, k): 1 2
k, C(2k, k): 2 6
k, C(2k, k): 3 20
k, C(2k, k): 4 70
k, C(2k, k): 5 252
k, C(2k, k): 6 924
k, C(2k, k): 7 3432
k, C(2k, k): 8 12870
k, C(2k, k): 9 48620
k, C(2k, k): 10 184756
specific cases:
trying torch.combinations (torch.arange (16), 8) ...
e: [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 34359738368 bytes. Error code 12 (Cannot allocate memory)
trying torch.combinations (torch.arange (16), 1) ...
16
trying itertools.combinations (range (16), 15) ...
16
trying torch.combinations (torch.arange (16), 15) ...
e: [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 1152921504606846976 bytes. Error code 12 (Cannot allocate memory)
combinations should accept combinations of zero elements:
trying itertools.combinations (range (5), 0) ...
[()]
trying torch.combinations (torch.arange (5), 0) ...
e: Expect a positive number, but got 0
</code></pre>
<p>I didn’t put timings into the script, but you can see by the wall clock<br>
that <code>torch.combinations()</code> is ridiculously slow.</p>
<p>(I get essentially the same results on 1.9.0 and today’s nightly build,<br>
1.11.0.dev20211106.)</p>
<p>Here is the github issue:</p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/41325">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/41325" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/41325" target="_blank" rel="noopener nofollow ugc">torch.combinations() - Tried to allocate 7869836414.81 GiB</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-07-12" data-time="22:44:50" data-timezone="UTC">10:44PM - 12 Jul 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/bennymi" target="_blank" rel="noopener nofollow ugc">
          <img alt="bennymi" src="https://avatars.githubusercontent.com/u/20195536?v=4" class="onebox-avatar-inline" width="20" height="20">
          bennymi
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: cuda
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: memory usage
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🐛 Bug



When using `torch.combinations(list, r=len(list))` with the `r` <span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">value set to the length of the list, I get a `CUDA out of memory error` for values where the length of the list is higher than `9`, even though `torch.combinations(list, r=len(list))` should just return the list itself. Link to the PyTorch Forum Issue: https://discuss.pytorch.org/t/tried-to-allocate-7869836414-81-gib/88880

## To Reproduce

Steps to reproduce the behavior:

```
for i in range(1, 20):
    values = torch.arange(i)
    combinations = torch.combinations(values, r=i)
    print(combinations)
```


For a list of length 23:
`CUDA out of memory error. Tried to allocate 7869836414.81 GiB (...)` 

## Expected behavior



`torch.combinations()` should return the list itself, when the `r` value is set to the length of the list.

## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

 - PyTorch Version (e.g., 1.0): 1.5.0
 - OS (e.g., Linux): Ubuntu 20.04
 - How you installed PyTorch (`conda`, `pip`, source): pip
 - Build command you used (if compiling from source):
 - Python version: 3.6.2
 - CUDA/cuDNN version: Cua compilation tools, release 10.1, V10.1.243
 - GPU models and configuration: RTX 2080


cc @ngimel</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>And here is the forum thread:</p>
<aside class="quote quote-modified" data-post="1" data-topic="135949">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dofasol/48/30982_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/use-torch-combinations-with-large-parameters/135949">Use torch.combinations with large parameters</a> 
  </div>
  <blockquote>
    Hi, 
I would like to generate all combinations of elements in a list.I know I can use torch.combinations for that, but I need to use large parameters, which produces huge number of combinations. So the problem is that it does not fit into memory. For example : 
torch.combinations(torch.arange(-10, 10, dtype=torch.int8).cuda(), 8)

yields 

RuntimeError: CUDA out of memory. Tried to allocate 23.84 GiB (GPU 0; 4.00 GiB total capacity; 1024 bytes already allocated; 2.62 GiB free; 2.00 MiB reserved …
  </blockquote>
</aside>

<p>Best.</p>
<p>K. Frank</p>