WeightedRandomSampler on sequential data

username: Hiqmat_Nisa
cooked: <p>Hi,<br>
My dataset contains handwritten text line images. You can see the example below:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/c/ec04ce6024568eef152d8e1647dfa372318aaec4.png" data-download-href="https://discuss.pytorch.org/uploads/default/ec04ce6024568eef152d8e1647dfa372318aaec4" title="a06-090-02"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec04ce6024568eef152d8e1647dfa372318aaec4_2_690x55.png" alt="a06-090-02" data-base62-sha1="xFUYLKEkMM7LkfrBKnDmZdXk7ZO" width="690" height="55" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec04ce6024568eef152d8e1647dfa372318aaec4_2_690x55.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec04ce6024568eef152d8e1647dfa372318aaec4_2_1035x82.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec04ce6024568eef152d8e1647dfa372318aaec4_2_1380x110.png 2x" data-dominant-color="EDEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">a06-090-02</span><span class="informations">1804×144 51.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I am giving these images to CNN for feature extraction and then these features to BLSTM encoder and then LSTM decoder to decode single character per time step. We have 6161 line of images in train set. But the dataset having long tail. This can be seen in below histogram.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/0/1037e25203072974039372c9845f771f39cef8c9.png" data-download-href="https://discuss.pytorch.org/uploads/default/1037e25203072974039372c9845f771f39cef8c9" title="Trainset"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1037e25203072974039372c9845f771f39cef8c9_2_690x460.png" alt="Trainset" data-base62-sha1="2jtmYw2F15Y0UKiqi5etYHQVW9r" width="690" height="460" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1037e25203072974039372c9845f771f39cef8c9_2_690x460.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1037e25203072974039372c9845f771f39cef8c9_2_1035x690.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1037e25203072974039372c9845f771f39cef8c9_2_1380x920.png 2x" data-dominant-color="F2F7F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Trainset</span><span class="informations">1500×1000 46.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The dataset contain 79 classes.  Samples doesn’t have a single target, rather it consists of sequence of classes.  The WeightedRandomSampler gets weights of different classes and try to create batches which contains equal number of sample based on the given weights.  Now the question is, this WeightedRandomSampler  can work on sequential data?</p>

username: Hiqmat_Nisa
cooked: <p>Hi, <a class="mention" href="/u/ptrblck">@ptrblck</a> I am following your answers in some of the threads related to WeightedRandomSampler. Can you guide me regarding any benefit on the usage of WeightedRandomSampler on above mention situation?</p>

username: ptrblck
cooked: <p>Based on the description of your use case, it seems you are working on a multi-label classification use case. In that case I don’t think that the <code>WeightedRandomSampler</code> can be used easily to balance the batches. <a href="https://discuss.pytorch.org/t/unbalanced-data-in-multiple-object-detection/88104/2">This post</a> explains other methods how these use cases can potentially be balanced.</p>