Orthogonal Initialization Error

username: sabyUWO
cooked: <p>Hi,</p>
<p>I am trying to use orthogonal initialization, I have tried many ways but I keep getting an error, the code and stack trace is below.</p>
<p>Code:<br>
torch.nn.init.orthogonal(m.weight) or   torch.nn.init.orthogonal(m.weight.data)</p>
<p>Traceback (most recent call last):<br>
File “/home/saby2k13/projects/ctb-ilie/saby2k13/pyDPPI/python/optimise.py”, line 620, in <br>
main()<br>
File “/home/saby2k13/projects/ctb-ilie/saby2k13/pyDPPI/python/optimise.py”, line 465, in main<br>
torch.nn.init.orthogonal(m.weight)<br>
File “/project/6000341/saby2k13/DPPI2/venv/tensorflow/lib/python3.5/site-packages/torch/nn/init.py”, line 445, in deprecated_init<br>
return meth(*args, **kwargs)<br>
File “/project/6000341/saby2k13/DPPI2/venv/tensorflow/lib/python3.5/site-packages/torch/nn/init.py”, line 381, in orthogonal_<br>
raise ValueError(“Only tensors with 2 or more dimensions are supported”)<br>
ValueError: Only tensors with 2 or more dimensions are supported</p>

username: ptrblck
cooked: <p>From the docs:</p>
<blockquote>
<p>The input tensor must have<br>
at least 2 dimensions, and for tensors with more than 2 dimensions the<br>
trailing dimensions are flattened.</p>
</blockquote>
<p>While it seems your tensor only has a single dimension?<br>
If so, could you explain your use case a bit, and why <code>m.weight</code> is used (or which module it comes from)?</p>

username: sabyUWO
cooked: <p>Thanks for the response!<br>
This is how the code looks like</p>
<blockquote>
<p>for m in model.modules():<br>
if isinstance(m, nn.Conv2d):<br>
print(‘m.weight.shape’,m.weight.shape,',dimension = ',m.weight.ndimension())<br>
torch.nn.init.orthogonal(m.weight)</p>
</blockquote>
<p>The output of the print statement :</p>
<blockquote>
<p>m.weight.shape torch.Size([64, 20, 1, 5]) ,dimension =  4</p>
</blockquote>
<p>So definitely the tensor dimension &gt; 2.</p>

username: ptrblck
cooked: <p>Thanks for the code!<br>
It should work, as shown with this dummy code snippet:</p>
<pre><code class="lang-python">model = models.resnet50()

for m in model.modules():
    if isinstance(m, nn.Conv2d):
        print('m.weight.shape',m.weight.shape,',dimension = ',m.weight.ndimension())
        torch.nn.init.orthogonal_(m.weight)
</code></pre>
<p>Minor issue: you should get a <code>UserWarning</code>:</p>
<pre><code class="lang-python">UserWarning: nn.init.orthogonal is now deprecated in favor of nn.init.orthogonal_.
</code></pre>
<p>which shouldn’t be related to the error you are seeing.</p>

username: sabyUWO
cooked: <p>Thanks !<br>
Yes I had tried that with the underscore already once and then tried again after your suggestion but unfortunately the error is still there.</p>
<blockquote>
<p>m.weight.shape torch.Size([64, 20, 1, 5]) ,dimension =  4<br>
m.bias.shape torch.Size([64])<br>
Traceback (most recent call last):<br>
File “/home/saby2k13/projects/ctb-ilie/saby2k13/pyDPPI/python/optimise.py”, line 620, in <br>
main()<br>
File “/home/saby2k13/projects/ctb-ilie/saby2k13/pyDPPI/python/optimise.py”, line 465, in main<br>
torch.nn.init.orthogonal_(m.weight)<br>
File “/project/6000341/saby2k13/DPPI2/venv/tensorflow/lib/python3.5/site-packages/torch/nn/init.py”, line 381, in orthogonal_<br>
raise ValueError(“Only tensors with 2 or more dimensions are supported”)<br>
ValueError: Only tensors with 2 or more dimensions are supported</p>
</blockquote>

username: ptrblck
cooked: <p>Could you post the model definition, so that we can have a look at this issue?</p>

username: sabyUWO
cooked: <p>This was a code problem, Sorry about that. Thanks!</p>