Batch balanced with some classes

RicLaGra: <p>Hello everyone,<br>
In PyTorch does exist a function to balance a batch using only N classes? I really interested to balance each batch using only some classes in a cyclic way of course, for instance:<br>
Batch 0 [5,5,5,0,0,0] (“5 instances of class 0,1,2, and 0 instances somewhere else”)<br>
Batch 1 [0,0,0,5,5,5]<br>
Epoch finished…</p>
<p>I would like to use this approach because a need to have many instances per class and in the sometime balanced. The reason which i would to have 0 instances in some classes is due to Cuda out of memory…<br>
Can you help me please?</p>

ptrblck: <p>I think the best approach would be to write a custom sampler by passing the targets to it and apply your sampling logic to return the right indices.<br>
You could use one of the already implemented samplers from <a href="https://github.com/pytorch/pytorch/blob/9d0c6e9469028cd7e7f44172e30aa4ce21b64be3/torch/utils/data/sampler.py">here</a> as the base class and add your filtering to it.</p>

RicLaGra: <p>thanks, <a class="mention" href="/u/ptrblck">@ptrblck</a>, I just done it. I’m testing the sampling logic in this moment.<br>
I would like to publish the code on Pytorch repo. Any suggestions?</p>

ptrblck: <p>Do you mean as an additional sampler into the PyTorch code base?<br>
If so, please create a feature request with the description of the use case and your changes on <a href="https://github.com/pytorch/pytorch/issues">GitHub</a>, where it can be discussed further. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

RicLaGra: <p>yes, thanks a lot <a class="mention" href="/u/ptrblck">@ptrblck</a>. I will create a feature request as you said.</p>

RicLaGra: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,<br>
I tried to follow the guide in order to submit our dataloader approach. However, I have the following error if I start run_test.py:</p>
<pre><code class="lang-auto">ImportError: cannot import name 'cpp_extensionp'
</code></pre>
<p>If I try to run only a class test I obtain:</p>
<pre><code class="lang-auto">ERROR collecting test/test_dataloader.py ___________________
ImportError while importing test module '/home/nataraja/Desktop/pytorch/test/test_dataloader.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.6/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_dataloader.py:16: in &lt;module&gt;
    from torch.utils.data import (_utils, Dataset, IterableDataset, TensorDataset, DataLoader, ConcatDataset,
E   ImportError: cannot import name 'BufferedShuffleDataset'
</code></pre>
<p>I read the CONTRIBUTING.md guide and i followed all steps,<br>
Can you help me?<br>
Thanks a lot.</p>

ptrblck: <p>This issue should have been fixed in the latest master version as described <a href="https://discuss.pytorch.org/t/torch-utils-data-issue-with-v1-7/101944/2">here</a>. Are you using the stable <code>1.7.0</code> code base? If so, could you switch to master?</p>

RicLaGra: <p>yes, I was in the master branch and not in the stable 1.7.0 code base. I forked yesterday, so I think i’ve the latest version of the branch master. Any suggestions?</p>

ptrblck: <p>Could you post the <code>test_dataloader.py</code> so that I could reproduce it with the current master?</p>

RicLaGra: <aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/pulsar2468/pytorch" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/711258?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/pulsar2468/pytorch" target="_blank" rel="noopener nofollow ugc">pulsar2468/pytorch</a></h3>

<p>Tensors and Dynamic neural networks in Python with strong GPU acceleration - pulsar2468/pytorch</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>This is the my fork (i’ve not modified anything)</p>

ptrblck: <p>So you are seeing these errors while only building from source and running the <code>run_test.py</code>?</p>

RicLaGra: <p>if i run run_test.py the error is:</p>
<pre><code class="lang-auto">    from torch.utils import cpp_extensionp
ImportError: cannot import name 'cpp_extensionp'
</code></pre>

ptrblck: <p>There seems to be a typo (<code>extensionp</code> instead of <code>extension</code>), so let me <code>grep</code> for it and check, what’s going on.</p>

RicLaGra: <p>yes, there was a typo error about extension<strong>p</strong>…<br>
I submitted the request and the pull code is:</p>
<pre><code class="lang-auto">https://github.com/pytorch/pytorch/pull/48180
</code></pre>
<p>Thanks a lot <a class="mention" href="/u/ptrblck">@ptrblck</a> <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>