WeightedRandomSampler Image Sequences list index out of range

Jam: <p>I have a sequence of images, so with batch size 1, my <strong>getitem</strong> method output would be a sequence of images and a sequence of labels of size [9] with sequence length 9. I am trying to use the WeightedRandomSampler. However, I am running into an error in the <strong>getitem</strong> method, which I didn’t when not using the WeightedRandomSampler.</p>
<p>I first used this approach:</p>
<pre><code class="lang-auto">class_sample_count = [6935, 805] 
weights = 1 / torch.Tensor(class_sample_count)

# load labels once to later get weights
targets = []
for _, labels in train_set:
    targets.append(np.array([weights[t] for t in labels]))
    
# transform to numpy array
targets = np.asarray(targets)

samples_weight = torch.from_numpy(targets)
samples_weight = samples_weight.double()
sampler = WeightedRandomSampler(samples_weight, len(samples_weight ))
</code></pre>
<p>samples_weigth had shape torch.Size([860, 9]) and I got the following error:</p>
<blockquote>
<p>list indices must be integers or slices, not list</p>
</blockquote>
<p>I then did the same but first flattened the 2D list to a 1D list before transforming it into a numpary array:</p>
<pre><code class="lang-auto">class_sample_count = [6935, 805] 
weights = 1 / torch.Tensor(class_sample_count)

# load labels once to later get weights
targets = []
for _, labels in train_set:
    targets.append(np.array([weights[t] for t in labels]))
    
# flatten list (2D to 1D)
targets = list(chain.from_iterable(targets))

# transform to numpy array
targets = np.asarray(targets)

samples_weight = torch.from_numpy(targets)
samples_weight = samples_weight.double()
sampler = WeightedRandomSampler(samples_weight, len(samples_weight))
</code></pre>
<p>but then got the following error:</p>
<blockquote>
<p>list index out of range</p>
</blockquote>
<p>In my <strong>getitem</strong> method, I am getting a sequence of images with seq = self.data[idx].<br>
How can I handle sequences with the WeightedRandomSampler?</p>

ptrblck: <p>If I understand your use case correctly, each <code>index</code> in <code>__getitem__</code> would load 9 samples and the corresponding 9 targets (9 would be the sequence length).<br>
If that’s the case, you would need to create a weight value for each sample, not each value in the sequence.<br>
The first approach seems to create <code>sample_weight</code> in the shape <code>[nb_samples, seq_len]</code>, while the latter would flatten it.<br>
Note that the sampler yields the <code>index</code> values, which are passed to the <code>__getitem__</code>, so it would have <code>nb_samples</code> weight values.</p>

Jam: <p>Thanks for your reply! Yes, exactly that’s how my <strong>getitem</strong> method works.<br>
That means the sampler should have 860 (number of sequences) weight values instead of 860*9 which would be the case right now when I would use the flattening approach?<br>
I am not quite sure how to realize that since this code snip</p>
<pre><code class="lang-auto">targets.append(np.array([weights[t] for t in labels]))
</code></pre>
<p>gets a weight for each sample in the sequence and a sequence can have targets from both classes (0 or 1). So how can I get a weight for the whole sequence?</p>

ptrblck: <p>It’s not straightforward to create weights for samples containing multiple targets and you could take a look at <a href="https://discuss.pytorch.org/t/unbalanced-data-in-multiple-object-detection/88104/2">this post</a>, which explains some potentially useful methods.</p>

Jam: <p>Thanks, I’ll take a look!</p>