Repository: pytorch
Issue: [Request] Batched Dataset->DataLoader interface
Labels: feature,module: dataloader,triaged

## 🚀 Feature

A batch-capable `Dataset`->`DataLoader`-like interface which, beyond just *collating* Dataset iterables into DataLoader batches, supports also *distributing*: So that e.g. an `IterableDataset` can yield **macro-batches** of data which the loader can split or aggregate to the desired size for loading.

## Motivation

Many data loading or transformation tasks are accelerated by batching & vectorized processing: For example loading CSV lines from disk and transforming certain columns, maybe with numpy.

For big/streaming datasets that don't fit in memory, we may be interested to select some "stream batch" size (different from and typically bigger than the model training batch size) with which to chunk this data: As in [pandas.read_csv() chunksize](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) for example.

Of course an example `Dataset` implementation may transparently perform these batched reads & transforms in the background and surface single-record `__getitem__` and/or `__iter__` APIs... But it seems like the PyTorch API could be imposing a bottleneck here by enforcing a record-level Python API between the two constructs?

## Pitch

`DataLoader` could be extended with some optional boolean parameter e.g. `bulk_dataset: bool=False` through which a user can indicate that the leading dimension of the `Dataset` **already is the batch dimension**.

In this case, `DataLoader` would provide an efficient implementation for aggregating and distributing batches of `batch_size` records, from whatever input batch sizes the `Dataset` yields.

For example:

- Given an `IterableDataset` implementation `train`, yielding arrays of (120, nCols) on each `next()` call
- Create a `DataLoader(train, batch_size=80, bulk_dataset=True)`
- Consuming 3 batches of 80 records from the DataLoader calls `next(train)` twice.

Benefits of this scenario include:

- The implementation of `train` doesn't have to concern itself with offering a per-record API over its internally batch-oriented operations
- Depending on the implementation of `DataLoader`, maybe this could reduce some Python overhead and so improve loading throughput?

## Alternatives

Providing some kind of formal API support for a concept like a `Dataset` which yields batches instead of records seems like a big change, which is why I proposed above to essentially introduce this as just an extended `DataLoader` functionality... But maybe it would still have some other implications?

- Could consider e.g. creating separate `Dataset` base/sub-classes for sets which provide only bulk access
- Maybe I'm just missing some core pattern by which this is already handled today, or making a false assumption that there's any scope to improve performance by batchifying this interface?

## Additional context

I'm no expert in PyTorch data loading, just observed that this could be a possible opportunity, from prior experience optimizing Python code by calling as little Python as possible 😂 My specific use-case is to performantly load bigger-than-memory CSV files with a couple of (currently numpy-based) column transformations.


cc @SsnL @VitalyFedyunin @ejguan