Problems iterating through loader when using sampler

Laura_Montalvo: <p>**Hi there! **</p>
<p><strong>I have a custom dataset in which (even though I have more or less the same number of samples for each class) it missclassified some of the data for some specific classes. So what I’m trying to do is oversample by using “sampler” in the dataloader and WeightedRandomSampler as follows:</strong></p>
<p>from sklearn.utils.class_weight import compute_class_weight</p>
<p>def load_data(root_path, batch_size):<br>
transform1 = transforms.Compose(<br>
[transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485, 0.456, 0.406],<br>
std=[0.229, 0.224, 0.225]),<br>
])<br>
transform2 = transforms.ToTensor();<br>
data = datasets.ImageFolder(root=root_path, transform= transform1)</p>
<pre><code>class_weights = compute_class_weight('balanced', np.unique(data.targets), data.targets)
class_weights = []
for root, subdir, files in os.walk(root_path):
    if len(files) &gt; 0:
        class_weights.append(1/len(files))
sample_weights = [0]*len(data)

for idx, (data,label) in enumerate(data):
    class_weight = class_weights[label]
    sample_weights[idx] = class_weight
sampler = WeightedRandomSampler(sample_weights, num_samples=len(sample_weights), replacement=True)    
data_loader = torch.utils.data.DataLoader(data, batch_size=batch_size, sampler=sampler)#, drop_last=False, num_workers=4)
return data_loader 
</code></pre>
<p><em>trainloader = load_data(root_path_train, batch_size)</em><br>
<em>testloader = load_data(root_path_test, batch_size)</em><br>
<em>validloader = load_data(root_path_valid, batch_size)</em></p>
<p>for x, y in trainloader:<br>
print(y)<br>
break</p>
<p><strong>However, when running the code in the <em>for x, y in trainloader:</em> (where x = data and y = targets) I get an error:</strong></p>
<p>/home/laura/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py:70: FutureWarning: <em>targets</em> as keyword args. From version 0.25 passing these as positional arguments will result in an error<br>
FutureWarning)<br>
Traceback (most recent call last):</p>
<p>File “/home/laura/Documentos/DatasetAppLGG3v6/Sin título0.py”, line 104, in <br>
for x,y in trainloader:</p>
<p>ValueError: too many values to unpack (expected 2)</p>
<p><strong>or when i write instead: <em>for i, (x,y) in trainloader:</em> I get:</strong></p>
<p>Traceback (most recent call last):</p>
<p>File “/home/laura/Documentos/DatasetAppLGG3v6/Sin título0.py”, line 104, in <br>
for x,y in trainloader:</p>
<p>File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py”, line 345, in <strong>next</strong><br>
data = self._next_data()</p>
<p>File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py”, line 385, in _next_data<br>
data = self._dataset_fetcher.fetch(index)  # may raise StopIteration</p>
<p>File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in fetch<br>
data = [self.dataset[idx] for idx in possibly_batched_index]</p>
<p>File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in <br>
data = [self.dataset[idx] for idx in possibly_batched_index]</p>
<p>IndexError: index 2312 is out of bounds for dimension 0 with size 3</p>
<p><strong>Can anybody help me please???</strong></p>

Dwight_Foster: <p>In the for loop do you say</p>
<pre><code class="lang-auto">for i, (x,y) in trainloader: 
or 
for i (x,y) in enumerate(trainloader):
</code></pre>
<p>the second one should work.</p>

Laura_Montalvo: <p>Hi! Thanks, but with both options I still get:</p>
<p>Traceback (most recent call last):<br>
File “/home/laura/Documentos/DatasetAppLGG3v6/Sin título0.py”, line 104, in<br>
for x,y in trainloader:<br>
File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py”, line 345, in <strong>next</strong><br>
data = self._next_data()<br>
File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py”, line 385, in _next_data<br>
data = self._dataset_fetcher.fetch(index) # may raise StopIteration<br>
File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in fetch<br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “/home/laura/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in<br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
IndexError: index 2312 is out of bounds for dimension 0 with size 3</p>

Dwight_Foster: <p>Ok I think I see the problem in your for loop that defines the data here:</p>
<pre><code class="lang-auto">for idx, (data,label) in enumerate(data):
</code></pre>
<p>you define one of the outputs as data and your input is data, so when you put data as the dataset for the dataloader it is just using the last output of the for loop which doesn’t work. You just need to change the name of your outputs like (inputs, labels) or something.</p>

Laura_Montalvo: <p>That was it!! Thanks so much! Could I ask you something else just in case you know the problem?</p>
<p>When loading validation set into the loader I created above (the same code of the main question of this topic) I get this warning and I don’t know if it may affect something:</p>
<p><strong>/home/laura/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py:70: FutureWarning: Pass classes=[</strong> 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23<br>
24], y=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24] <strong>as keyword args. From version 0.25 passing these as positional arguments will result in an error</strong><br>
<strong>FutureWarning)</strong></p>
<p>Thanks so much!!!</p>

Dwight_Foster: <p>Sorry I have no idea what this future warning is saying. It probably won’t affect your training but you might want to start a new forum post about it because it might be important in the future.</p>

ptrblck: <p>The warning is raised by scikit-learn and asks you to use the <code>keyword arg</code> for the passed classes instead of a positional argument.<br>
I don’t know which method you are calling, but try to use it via: <code>sklearn.some_method(classes=classes)</code>.</p>

Laura_Montalvo: <p>Hi there! And thank you for answering. I was using “compute_class_weight”  from scikit-learn like this:<br>
class_weights = compute_class_weight(‘balanced’, np.unique(data.targets), data.targets)<br>
then I added as you told me (classes = classes) as follows:</p>
<p>NUM_POSITIONS = 25<br>
EPOCH =  17 <span class="hashtag">#config</span>.epochs<br>
optimizer = 2 <span class="hashtag">#config</span>.optimizer<br>
input_channels = 3</p>
<p>from sklearn.utils.class_weight import compute_class_weight</p>
<p>def load_data(root_path, batch_size):<br>
transform1 = transforms.Compose( [transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485, 0.456, 0.406],<br>
std=[0.229, 0.224, 0.225]),<br>
])<br>
transform2 = transforms.ToTensor();<br>
data = datasets.ImageFolder(root=root_path, transform= transform1)</p>
<pre><code>class_weights = compute_class_weight('balanced', np.unique(data.targets), data.targets,classes=np.arange(NUM_POSITIONS))
class_weights = []
</code></pre>
<p>However I’m still getting the warning… Thanks!</p>

ptrblck: <p>Which <code>sklearn</code> version are you using, as I’m getting an error running your code with random <code>target</code> data:</p>
<pre><code class="lang-python">targets = np.random.randint(0, 100, (100,))
class_weights = compute_class_weight('balanced', np.unique(targets), targets,classes=np.arange(NUM_POSITIONS))
&gt; TypeError: compute_class_weight() got multiple values for argument 'classes'
</code></pre>
<p>This work for me without a warning:</p>
<pre><code class="lang-python">targets = np.random.randint(0, 100, (100,))
class_weights = compute_class_weight('balanced', classes=np.unique(targets), y=targets)
</code></pre>

Laura_Montalvo: <p>I’m using sklearn.<strong>version</strong> 0.23.2.</p>
<pre><code class="lang-auto">targets = np.random.randint(0, 100, (100,))
class_weights = compute_class_weight('balanced', classes=np.unique(targets), y=targets)
</code></pre>
<p>This way works for me either but I still get the warning… <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"> <img src="https://discuss.pytorch.org/images/emoji/apple/pensive.png?v=9" title=":pensive:" class="emoji" alt=":pensive:"></p>

ptrblck: <p>Are you sure this line of code raises the warning or are you using it somewhere else in your script as well (without the changes)?<br>
I’m using a newer version without any warning:</p>
<pre><code class="lang-python">&gt;&gt;&gt; sklearn.__version__
'0.24.1'
&gt;&gt;&gt; from sklearn.utils.class_weight import compute_class_weight
&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; targets = np.random.randint(0, 100, (100,))
&gt;&gt;&gt; class_weights = compute_class_weight('balanced', classes=np.unique(targets), y=targets)
&gt;&gt;&gt;
</code></pre>
<p>and even downgrading to <code>0.23.2</code> gives no warnings:</p>
<pre><code class="lang-python">&gt;&gt;&gt; sklearn.__version__
'0.23.2'
&gt;&gt;&gt; from sklearn.utils.class_weight import compute_class_weight
&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; targets = np.random.randint(0, 100, (100,))
&gt;&gt;&gt; class_weights = compute_class_weight('balanced', classes=np.unique(targets), y=targets)
&gt;&gt;&gt;
</code></pre>
<p>If I don’t use the <code>classes=...</code> argument, I get your warning.</p>

Laura_Montalvo: <p>Thanks so much! I fixed it, I forgot introducing ‘y=’. <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>

raharth: <p>Just a question out of curiosity: Why did you change the sample probability instead of weighting the loss? I would have expected it to be more efficient in terms of runtime?</p>

Laura_Montalvo: <p>Actually I’m trying both, I just wanted to see the effect of both: changing sample probability and weighting the loss. That is because I do have a balanced dataset, but for some of the data that has less information, the network classifies worse (like for example it classifies classes 1,2,3,6 and 7 with accuracy 98% but classes 4 and 5 are classified with an accuracy of 20-30%).</p>

raharth: <p>Ah ok, got it! So did you find a difference between resampling and weighting in terms of accuracy and computational efficiency?</p>
<p>I usually use loss weighting, but once I had an extremely unbalanced data set, so that most batches would have not seen any samples from one class - that’s where I used the sampling, but I would be really interested in what the overall differences are! Maybe you could share your findings <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Laura_Montalvo: <p>Using sampling with my dataset didn’t enhanced the results, they were more or less the same and the error in the miss-classified classes was the same. However, giving higher weight to the loss of the classes that were miss-classified achieved better results, however they were hardly noticeable. So I think none of this techniques are the solution to my problem of miss-classified classes because my dataset is balanced</p>

raharth: <p>Interesting! I would have expected it to have impact - at least by theory it should have… <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=9" title=":thinking:" class="emoji" alt=":thinking:"></p>
<p>Thanks for sharing!</p>