Sequential MNIST

conner: <p>How can I make a transformation that results in a sequential MNIST dataset? I cannot seem to figure this out. <code>.view(-1, 784)</code> seems like it would work, but does seem to be correct and I get the error:<br>
<code>RuntimeError: size mismatch, m1: [16 x 784], m2: [28 x 400]</code>. If I do <code>.view(-1, 784).t()</code> the program doesn’t have the error but the network doesn’t learn anything. This is with <code>batch_first</code> being set to <code>True</code>.</p>

tom: <p>Hi <a class="mention" href="/u/conner">@conner</a></p>
<p>To feed an rnn model with batch_first, would need to feed tensors of size batch * sequence * dimension, even if the dimension is one.</p>
<p>You could do this on the dataloader level (where you don’t have the batch dimension, so the output tensor is sequence * dimension) with a transform like</p>
<pre data-code-wrap="python"><code class="lang-python">transform = torchvision.transforms.Compose(
             [torchvision.transforms.ToTensor(),
              torchvision.transforms.Lambda(lambda x: x.view(-1,1))
             ])
</code></pre>
<p>Then you can use that in the dataset instantiation and use that in the usual  DataLoader.</p>
<p>If you want permuted sequential MNIST, you could take</p>
<pre data-code-wrap="python"><code class="lang-python">pixel_permutation = torch.randperm(28*28)
transform = torchvision.transforms.Compose(
             [torchvision.transforms.ToTensor(),
              torchvision.transforms.Lambda(lambda x: x.view(-1,1)[pixel_permutation])
             ])
</code></pre>
<p>(It is desired that the permutation is fixed.)</p>
<p>If you are looking at implementing this</p><aside class="quote" data-post="1" data-topic="1401">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ajaytalati/48/176_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/tunable-efficient-unitary-neural-networks-eunn/1401">Tunable Efficient Unitary Neural Networks (EUNN)</a> 
  </div>
  <blockquote>
    Hi, I wonder if there’s been a PyTorch implementation of, 
<a href="https://arxiv.org/pdf/1612.05231.pdf" rel="noopener nofollow ugc">Tunable Efficient Unitary Neural Networks (EUNN)</a> 
It’s something that definitely seems to be a solid piece of work ! 
<a class="mention" href="/u/smth">@smth</a> this seems like something FAIR must have in house already? You, Yann LeCun and Martin Arjovsky have been working on this for a quite a while if I remember correctly?
  </blockquote>
</aside>

<p>you’d instantly have a fan-club if you post a link to your implementation. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
<p>Best regards</p>
<p>Thomas</p>

AjayTalati: <p>Hi,</p>
<p>following on from what Thomas, <a class="mention" href="/u/tom">@tom</a>, said there seem to be two classes of AI “researchers”,</p>
<ol>
<li>
<p>Those that take a simple <code>vanilla</code> model, and then i) scale it up to more parameters/layers, ii) get it to run faster on the GPU, and distribute/parallelize it, iii) and then run it on massive datesets.</p>
</li>
<li>
<p>The complement of class 1</p>
</li>
</ol>
<p>If you’re working on sequential MNIST, and looking to do smart things like <code>EUNN</code> or <code>DNC</code>, then you’re in the second class <img src="//discuss.pytorch.org/images/emoji/apple/smile.png?v=5" title=":smile:" class="emoji" alt=":smile:">, and in the long run I’m sure you’ll learn a heck of a lot more <img src="//discuss.pytorch.org/images/emoji/apple/smile.png?v=5" title=":smile:" class="emoji" alt=":smile:"></p>
<p>Best of luck with your research!</p>
<p>Ajay</p>

conner: <p>Thank you for your responses <a class="mention" href="/u/tom">@tom</a> and <a class="mention" href="/u/ajaytalati">@AjayTalati</a>! Currently, I am starting with making a <code>LSTM</code> (and later a <code>GRU</code>) baseline for sequential and permutated MNIST. The current problem is that the loss refuses to go down and the accuracy remains fixed always at <code>11.35%</code>. My code can be found here: <a href="https://gist.github.com/c0nn3r/f1d6a11d18ba700cf6e0fcc446f84377" rel="nofollow noopener">https://gist.github.com/c0nn3r/f1d6a11d18ba700cf6e0fcc446f84377</a>. If anyone would be willing to take a look, I would really appreciate it.</p>
<p>Best,</p>
<p>Conner</p>

NaMo: <p>Hi, I am new to this Pytorch. I want to know if my images containing numbers like 1 in one image, 2 in the next image, 3 in 3rd image, 4 in 4th image…like this in sequence one after the other the how can my predicted image by RNN model will be number 5 in an image using Pytorch?</p>

kinwai_cheuk: <p>I am also facing the same problem. Have you figured out why?</p>

NaMo: <p>No,Not yet. But i am going with the values rather than images. Because my pixel size is more which takes lot of time for training.<br>
Have you solved the problem?<br>
If so, can you share in this discussion.</p>

jramapuram: <p>I do some hacks to get this working, see <a href="https://github.com/jramapuram/datasets/blob/master/loader.py#L178" rel="nofollow noopener">https://github.com/jramapuram/datasets/blob/master/loader.py#L178</a> and an older example using a different submodule commit: <a href="https://github.com/jramapuram/LifelongVAE_pytorch/blob/master/main.py#L317" rel="nofollow noopener">https://github.com/jramapuram/LifelongVAE_pytorch/blob/master/main.py#L317</a></p>
<p>The high level idea is you want to implement a sampler that filters your labels.</p>