How to iterate through nn.ModuleDict without ordering keys

username: nullgeppetto
cooked: <p>I want to define an <code>nn.ModuleDict()</code> and iterate through it preserving the order of keys as defined. That is, I would like to define the dict as follows:</p>
<pre><code class="lang-auto">D = nn.ModuleDict( 
    { 
        'b': nn.Linear(2,4), 
        'a': nn.Linear(4,8) 
    } 
) 
</code></pre>
<p>so as</p>
<pre><code class="lang-auto">for k, v in D.items(): 
    print(k, v)
                                                                                                                                       
b Linear(in_features=2, out_features=4, bias=True)
a Linear(in_features=4, out_features=8, bias=True)
</code></pre>
<p>However, <strong>by default</strong>, PyTorch will iterate through the dict after sorting alphanumerically its keys:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

D = nn.ModuleDict( 
    { 
        'b': nn.Linear(2,4), 
        'a': nn.Linear(4,8) 
    } 
) 

D
ModuleDict(
  (a): Linear(in_features=4, out_features=8, bias=True)
  (b): Linear(in_features=2, out_features=4, bias=True)
)

for k, v in D.items(): 
    print(k, v)
                                                                                                                                       
a Linear(in_features=4, out_features=8, bias=True)
b Linear(in_features=2, out_features=4, bias=True)
</code></pre>
<p>On the contrary, if we use <code>update()</code>, it works as intended. That is</p>
<pre><code class="lang-auto">D = nn.ModuleDict()
D.update({'b': nn.Linear(2, 4)})                                                                                                            
D.update({'a': nn.Linear(4, 8)})

D
ModuleDict(
    (b): Linear(in_features=2, out_features=4, bias=True)
    (a): Linear(in_features=4, out_features=8, bias=True)
)

for k, v in D.items(): 
    print(k, v)
                                                                                                                                       
b Linear(in_features=2, out_features=4, bias=True)
a Linear(in_features=4, out_features=8, bias=True)

</code></pre>
<p>Is there any way of iterating through the dict using the order of keys given during dict’s definition? One solution would be using keys that are ordered in the first place, but I would like to know if this is possible in the general case.</p>

username: tom
cooked: <p>If you pass in an ordered dict, the ordering will be preserved:</p>
<pre><code class="lang-auto">nn.ModuleDict(OrderedDict({ 
   'b': nn.Linear(2,4), 
   'a': nn.Linear(4,8) 
} ))
</code></pre>
<p>The crux is that Python 2 does not preserve order in <code>dict</code> (and for early Python 3.x it’s an implementation detail), so in order to have a deterministic dict-&gt;OrderedDict conversion, the key are sorted.</p>
<p>Alternatively, you can also pass in an iterator <code>{...}.items()</code> - looks funny, probably deserves an explanatory comment in the code, but works.</p>
<p>Best regards</p>
<p>Thomas</p>

username: nullgeppetto
cooked: <p>Hi <a class="mention" href="/u/tom">@tom</a>, many thanks for your quick response!</p>
<p>I see. That’s not bad solution of course, but especially in the case of a “Module Dictionary”, I cannot see why  this isn’t the default behavior. I mean, when someone defines a modules dictionary, like I did above, then they would also expect to iterate it in the order it’s been defined.</p>
<p>Many thanks again!</p>

username: dhpollack
cooked: <p>Also if you use python 3.7, it should preserve the order of the keys for a regular dict.</p>

username: nullgeppetto
cooked: <p>Hi <a class="mention" href="/u/dhpollack">@dhpollack</a>, I’m using python 3.7.3, but it doesn’t.</p>