Compatibility of subset dataset with disabled batch sampling?

dvirginz: <p>I think there is a competability issue with disabled batch sampling and subset dataset<br>
The use-case - define custom batch sampling, and split the dataset using pytorch split utility function<br>
Here’s a minimal working example</p>
<pre><code class="lang-auto">self.train_dataset, self.val_dataset, self.test_dataset = torch.utils.data.random_split(
            self.dataset, [100, 100, 100])

loader = DataLoader(
                dataset=self.train_dataset,
                batch_size=None,
                batch_sampler=None,
                sampler=BatchSampler(
                    SequentialSampler(dataset), batch_size=self.hparams.batch_size, drop_last=False),
                num_workers=self.hparams.num_data_workers,
            )
</code></pre>
<p>And when iterating the subset datasets this is the error</p>
<pre><code class="lang-auto">Exception has occurred: TypeError
list indices must be integers or slices, not list
  File "/path/utils/data/dataset.py", line 257, in __getitem__
    return self.dataset[self.indices[idx]]
  File "/path/utils/data/_utils/fetch.py", line 46, in fetch
    data = self.dataset[possibly_batched_index]
  File "/path/utils/data/dataloader.py", line 385, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/path/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/path/lib/python3.7/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 251, in _evaluate
    for batch_idx, batch in enumerate(dataloader):
  File "/path/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 843, in run_pretrain_routine
    False)
  File "/path/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 477, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/path/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 704, in fit
    self.single_gpu_train(model)
  File "/path/train.py", line 152, in main_train
    trainer.fit(model)
  File "/path/train.py", line 66, in main
    main_train(model_class_pointer, hyperparams, logger)
  File "/path/train.py", line 161, in &lt;module&gt;
    main()
  File "/path/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/path/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/path/lib/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/path/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/path/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
</code></pre>
<p>As the self.indices of the <code>subset</code> object is a simple python list.</p>

ptrblck: <p>I can reproduce it using the code sample.<br>
Could you please create an <a href="https://github.com/pytorch/pytorch/issues">issue</a> to track this bug?</p>
<p>As a workaround you could use a <code>SubsetRandomSampler</code> and pass the shuffled indices to it.<br>
Inside the <code>Dataset.__getitem__</code> you might need to create a single index tensor via:</p>
<pre><code class="lang-python">index = torch.stack(index)
x = self.data[index]
</code></pre>
<p>since the <code>SubsetRandomSampler</code> will pass a list of tensors to the <code>Dataset</code> for the <code>BatchSampler</code> approach.</p>

dvirginz: <p>I ended creating a sequential subset sampler, as my dataset is already randomized, and it is crucial (caching) to fetch samples sequentially.<br>
Thanks!</p>