Split dataset (advanced way)

username: Alphonsito25
cooked: <p>Hi!</p>
<p>I would like to randomly split my dataset between training and test, but also I want to make it balanced in my 2 classes, and save this split to future trainings.</p>
<p>This is because I want to perform several trainings with different pretrained models under the same conditions (test images always the same in each training), but the split has to be created randomly only one time in the first place.</p>
<p>Thank you very much!</p>

username: ptrblck
cooked: <p>You could use e.g. <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html"><code>sklearn.model_selection.train_test_split</code></a> with the <code>stratify</code> argument and save the split indices once to recreate the dataset splits in future runs (seeding might also work, but saving the indices directly might be the better option).</p>

username: Alphonsito25
cooked: <p>Thank you! I have been researching the function but I don’t know what to put in each input parameter of it.<br>
If I have a dataset class like:</p>
<pre><code>class mi_dataset(Dataset):
    def __init__(self, csv_file, root_dir, transform):
    	self.annotations= pd.read_csv(csv_file)
    	self.root_dir= root_dir
    	self.transform= transform
    
    def __len__(self):
    	return len(self.annotations)
    def __getitem__(self,index):
    	img_path= os.path.join(self.root_dir, self.annotations.iloc[index, 0])
    	image= io.imread(img_path)
    	y_label= torch.tensor(int(self.annotations.iloc[index, 1])) 
    	
    	if self.transform:
    	    image= self.transform(image)
    	return(image, y_label)

transform= transforms.Compose([transforms.ToPILImage(),transforms.Resize([480, 480]),transforms.ToTensor(),transforms.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225])])

dataset= mi_dataset(csv_file= './Dataset/labels.csv', root_dir= './Dataset/images', transform= transform)
train_set, test_set = train_test_split(*arrays=, test_size=0.1, train_size=0.9, random_state=None, shuffle=True, stratify=)
</code></pre>
<p>Do you know where I should put in the arguments *arrays and stratify? Also, I don’t know where to save the split indices (adding two more outputs to train_set and test_set??)</p>

username: ptrblck
cooked: <p>You can use the indices in <code>range(len(dataset))</code> as the input array to split and provide the targets of your dataset to the <code>stratify</code> argument.<br>
The returned indices can then be used to create separate <code>torch.utils.data.Subset</code>s using your <code>dataset</code> and the corresponding split indices.</p>

username: Alphonsito25
cooked: <p>Like this?</p>
<pre><code>range_train, range_test = train_test_split(range(len(dataset)), test_size=0.1, train_size=0.9, random_state=None, shuffle=True, stratify=range(len(dataset)))

train_set = torch.utils.data.Subset(dataset, range_train)
test_set = torch.utils.data.Subset(dataset, range_test)
</code></pre>
<p>If I put this it shows me an error:</p>
<pre><code>The least populated class in y has only 1 member, which is too few. The minimum number of groups for any class cannot be less than 2.
</code></pre>

username: ptrblck
cooked: <p>No, stratify expects the targets, not a list of indices.</p>

username: Alphonsito25
cooked: <p>Then, how I put it?<br>
Taking into account that my csv file has 2 columns, the first one has the name of the images, and the second one if it is 1 of the 2 classes (with a 1 or a 0), and it is contained into the dataset object.<br>
Thank you very much.</p>

username: ptrblck
cooked: <p>You could try to extract the targets directly from the CSV file based on your description or alternatively you could also iterate your dataset once and store the target in a new numpy array.</p>

username: Alphonsito25
cooked: <p>Done it. It works. Thank you!</p>

username: ado_sar
cooked: <p>Random split can also be performed with <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.random_split" rel="noopener nofollow ugc">torch.utils.data.random_split</a> without the need of scikit-learn:</p>
<pre data-code-wrap="python"><code class="lang-python">indices = ['img_0', 'img_1', ...]  # A sequence with all indices.
train, val, test = random_split(indices, (0.8, 0.1, 0.1))
train_idx = train.indices  # Similar for val and test.
</code></pre>

username: ptrblck
cooked: <p>No, the built-in <code>torch.utils.random_split</code> cannot be used to create balanced class distributions as requested in this topic and the <code>stratify</code> argument mentioned in my post for <code>sklearn.model_selection.train_test_split</code> can be used instead.</p>

username: ado_sar
cooked: <p>Oops! Apologize, I didn’t realize OP was about stratified splits.</p>