`ConcatMapDataset` workers duplicate data
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

`ConcatMapDataset` doesn't check it's `worker_id` and all workers are seeded with the same RNG, then return the same data.

```python
import torch
from pytorch_lightning import seed_everything

from nemo.collections.common.data import ConcatMapDataset

seed_everything(42)


class IntegerDataset(torch.utils.data.Dataset):
    def __init__(self, n: int, dataset_id: int):
        self._ints = list(range(n))
        self._id = torch.tensor(dataset_id)

    def __len__(self):
        return len(self._ints)

    def __getitem__(self, idx):
        worker_info = torch.utils.data.get_worker_info()
        worker_id = worker_info.id
        return self._id, worker_id, self._ints[idx]


concat_dataset = ConcatMapDataset(
    datasets=[
        IntegerDataset(5, 1),
        IntegerDataset(10, 2)
    ]
)

dataloader = torch.utils.data.DataLoader(
    dataset=concat_dataset,
    batch_size=1,
    num_workers=3
)

for batch in dataloader:
    batch_dataset_id, batch_worker_id, batch_value = batch
    print(
        f"Dataset {batch_dataset_id.item()} worker {batch_worker_id.item()} fetched {batch_value.item()}"
    )

```

Output:
```
Global seed set to 42
Dataset 2 worker 0 fetched 6
Dataset 2 worker 1 fetched 6
Dataset 2 worker 2 fetched 6
Dataset 1 worker 0 fetched 2
Dataset 1 worker 1 fetched 2
Dataset 1 worker 2 fetched 2
Dataset 2 worker 0 fetched 5
Dataset 2 worker 1 fetched 5
Dataset 2 worker 2 fetched 5
Dataset 2 worker 0 fetched 4
Dataset 2 worker 1 fetched 4
Dataset 2 worker 2 fetched 4
Dataset 2 worker 0 fetched 0
Dataset 2 worker 1 fetched 0
Dataset 2 worker 2 fetched 0
```

**Expected behavior**
`worker_id` should be utilized. See what happens with `ConcatDataset`:

```python
import torch
from pytorch_lightning import seed_everything

from nemo.collections.common.data import ConcatDataset

seed_everything(42)


class IntegerDataset(torch.utils.data.Dataset):
    def __init__(self, n: int, dataset_id: int):
        self._ints = list(range(n))
        self._id = torch.tensor(dataset_id)

    def __len__(self):
        return len(self._ints)

    def __getitem__(self, idx):
        worker_info = torch.utils.data.get_worker_info()
        worker_id = worker_info.id
        return self._id, worker_id, self._ints[idx]


concat_dataset = ConcatDataset(
    datasets=[
        IntegerDataset(5, 1),
        IntegerDataset(10, 2)
    ]
)

dataloader = torch.utils.data.DataLoader(
    dataset=concat_dataset,
    batch_size=1,
    num_workers=3
)

for batch in dataloader:
    batch_dataset_id, batch_worker_id, batch_value = batch
    print(
        f"Dataset {batch_dataset_id.item()} worker {batch_worker_id.item()} fetched {batch_value.item()}"
    )

```

Output:

```
Global seed set to 42
Dataset 1 worker 0 fetched 0
Dataset 1 worker 1 fetched 4
Dataset 1 worker 2 fetched 2
Dataset 2 worker 0 fetched 6
Dataset 1 worker 1 fetched 1
Dataset 2 worker 2 fetched 8
Dataset 2 worker 0 fetched 0
Dataset 2 worker 1 fetched 7
Dataset 2 worker 2 fetched 5
Dataset 2 worker 0 fetched 9
Dataset 2 worker 1 fetched 4
Dataset 1 worker 2 fetched 2
Dataset 2 worker 0 fetched 3
Dataset 2 worker 1 fetched 1
Dataset 2 worker 2 fetched 2
```

Also, shuffling seems hard-coded.