Handle different feature input dimensions in the data loader

bryan123: <p>Hi, I already have the extracted features with dimensions [M, 192] where M can be any number (ex: [5, 192], [20, 192], [25, 192], etc). Would it be possible to include all of these features in the data loader with a batch size greater than 1? I have tried converting all of them to torch.Tensor and torch.stack and none of them works.</p>
<p>Since my goal is to put the input features into the encoder (VIT) in Masked Autoencoder, would it be better to convert them to the same dimension? Do you have any good ideas on how to convert?</p>

ptrblck: <p>Could you explain how you would like to use this data?<br>
From the technical point of view you could pad the data before trying to stack it, but I think the more important part would be what the data represents and how the model should use it for its training.</p>

bryan123: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> , my goal is actually to do a fully unsupervised gigapixel Whole Slide Image (which can be ~100k x 100k pixels) classification based on hierarchical pretraining in Hierarchical Image Pyramid Transformer (HIPT) (<a href="https://arxiv.org/abs/2206.02647" class="inline-onebox" rel="noopener nofollow ugc">[2206.02647] Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning</a>).</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/2/72e23c1f6ff00c73384e0895d6c9a1e34ee46b1d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/72e23c1f6ff00c73384e0895d6c9a1e34ee46b1d" title="Screenshot 2023-02-26 at 7.30.23 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72e23c1f6ff00c73384e0895d6c9a1e34ee46b1d_2_689x369.jpeg" alt="Screenshot 2023-02-26 at 7.30.23 AM" data-base62-sha1="goj8wPf8wc4SYkYOb8l8sHEyYgZ" width="689" height="369" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72e23c1f6ff00c73384e0895d6c9a1e34ee46b1d_2_689x369.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/7/2/72e23c1f6ff00c73384e0895d6c9a1e34ee46b1d_2_1033x553.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/2/72e23c1f6ff00c73384e0895d6c9a1e34ee46b1d.jpeg 2x" data-dominant-color="D9D4DA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-02-26 at 7.30.23 AM</span><span class="informations">1290×690 116 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The region-level representations are already available from the output of the CLS token in the second stage, and since a Whole Slide Image can have any number of regions [4096, 4096], the representations of whole slide images now look like [5, 192], [20. 192], [100. 192]. Note that in this case, I just concat all of the regions that belong to one slide together.</p>
<p>Now, I want to train a masked autoencoder (<a href="https://arxiv.org/abs/2111.06377" class="inline-onebox" rel="noopener nofollow ugc">[2111.06377] Masked Autoencoders Are Scalable Vision Learners</a>) in the third stage for making a Whole Slide Image pertaining model and the inputs are based on the extracted regions that I got before. Is it possible to do that?</p>