Weird Behavior of SubsetRandomSampler

Filos92: <p>Good afternoon everyone,<br>
my neural network shows a behavior I can’t understand. I have already found out the source of the error, it is the SubsetRandomSample. Maybe you can help me to understand <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>When is create my Dataloader like this:</p>
<pre><code class="lang-auto">Dataset_val   = Dataset(transform=Augmentation.transformation['val'])
Dataset_train   = Dataset(transform=Augmentation.transformation['train'])

dataset_size = len(self.Dataset_train)
indices = list(range(dataset_size))
split = int(np.floor(test_size * dataset_size))
shuffle_dataset =True
        if shuffle_dataset :
            np.random.seed(random_seed)
            np.random.shuffle(indices)
        
train_indices, val_indices = indices[split:], indices[:split]
train_sampler = SequentialSampler(train_indices)
valid_sampler = SequentialSampler(val_indices)
       
self.train_loader = torch.utils.data.DataLoader(Dataset_train,
                                                        batch_size= 64, 
                                                        sampler=train_sampler,
                                                        )
        
self.validation_loader = torch.utils.data.DataLoader(self.Dataset_val, 
                                                             batch_size=64,
                                                             sampler=valid_sampler,
                                                             )

</code></pre>
<p>Everything works fine.!!<br>
<a href="/uploads/short-url/7vXJEXMBSyiy9mKWJwafEGFufDr.png">Figure_2|668x500</a><br>
But if I do:</p>
<pre><code class="lang-auto">Dataset_val   = Dataset(transform=Augmentation.transformation['val'])
Dataset_train   = Dataset(transform=Augmentation.transformation['train'])

dataset_size = len(self.Dataset_train)
indices = list(range(dataset_size))
split = int(np.floor(test_size * dataset_size))

        
train_indices, val_indices = indices[split:], indices[:split]
train_sampler = SubsetRandomSampler(train_indices)
valid_sampler = SubsetRandomSampler(val_indices)
       
self.train_loader = torch.utils.data.DataLoader(Dataset_train,
                                                        batch_size= 64, 
                                                        sampler=train_sampler,
                                                        )
        
self.validation_loader = torch.utils.data.DataLoader(self.Dataset_val, 
                                                             batch_size=64,
                                                             sampler=valid_sampler,
                                                             )
</code></pre>
<p><a href="/uploads/short-url/51Rvfw2KHjIc6XbdTxl0tCEGM7R.png">Figure_1|668x500</a><br>
the training loss acts exactly like before, but the validation los stops decreasing almost immediately</p>
<p>I’ve tried to find any differences between the data, by looking at small Datasets of 5 to 10 datapoints, passed to the training or validation process, but in my opinion the same data, with the same type and size is passed <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"></p>
<p>is there something wrong with my code, or do i missunderstand something completely wrong?<br>
Best Greetings,<br>
Filos92</p>

ptrblck: <p>It seems the difference between both codes is the manual shuffling of the indices?</p>
<p>How did you define your <code>Dataset</code>? Could it be that the classes are sorted internally?<br>
If so, the second approach would pass only the first classes to the <code>train_sampler</code> and the last classes to the <code>valid_sampler</code>, which could explain your observation.</p>

Filos92: <p>Yes, the difference is the shuffling.<br>
My Dataset looks like this:</p>
<pre><code class="lang-auto">class Dataset():

    def __init__(self, Path):
        
        input_values = pd.read_csv(Path, header = 0 , sep=';')
        data = input_values[['position_x','position_y','position_z']].values.tolist()
        target =input_values['req_pos'].values.tolist()

        scaler = StandardScaler()

        self.data = scaler.fit_transform(data)
        self.target =  target
        
    def __len__(self):
        return len(self.data)

    def __getitem__(self, index):
        data=self.data[index]
        target=self.target[index]
        
        values = data, target
        
        return values
</code></pre>
<p>I don’t use classes, because I look at a regression based problem. In general I would say all entries are equal, so I don’t think I pass the wrong classes</p>

Filos92: <p>I do not use the RandomSampler any more, but my question is still open <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>