PyTorch tutorial for Neural transfert of artistic style

alexis-jacq: <p>Hi,</p>
<p>If someones are interested, I’ve realized this PyTorch tutorial to implement the neural transfer of artistic style developed by Leon Gatys and AL:</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">
      <a href="https://github.com/alexis-jacq/Pytorch-Tutorials" target="_blank">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/9195965?s=400&amp;v=4" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://github.com/alexis-jacq/Pytorch-Tutorials" target="_blank">alexis-jacq/Pytorch-Tutorials</a></h3>

<p>Pytorch tutorials for Neural Style transfert. Contribute to alexis-jacq/Pytorch-Tutorials development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Any feedback is welcome!</p>

apaszke: <p>That looks great! But does the code work? You have that <code># -- WRONG CODE --</code> comment, and it indeed looks incorrect to me. You can’t construct an optimizer with a single Variable <code>input</code>. It needs a list of Variables, so <code>optim.Adam([input], lr = 0.01)</code> should work.</p>

alexis-jacq: <p>Yes, the code works and I give the whole script in the “.py”. I show the wrong code to explain the global idea, then I give the correct version just below (maybe not a good pedagogy…). But I didn’t know that just a list of Variable works, this is much simpler that what I did (I constructed a module with the variable as a parameter). Thanks !</p>

phenixcx: <p>When using a pretrained model ‘resnet18’ to train images with torch.optim.Adam, the system said:<br>
Traceback (most recent call last):<br>
File “”, line 1, in <br>
File “/root/anaconda2/lib/python2.7/site-packages/torch/optim/adam.py”, line 54, in step<br>
beta1, beta2 = group[‘betas’]<br>
TypeError: ‘float’ object is not iterable</p>
<p>The test codes were written as follows:<br>
lr=0.9<br>
momentum = 1e-4<br>
weight_decay= 0.1<br>
model = models.resnet18(pretrained=True)<br>
optimizer = torch.optim.Adam(model.parameters(),lr,momentum,weight_decay)<br>
output =model(torch.autograd.Variable(torch.ones(1,3,224,224)))<br>
yt = torch.autograd.Variable((torch.ones(1)).long())<br>
criterion = torch.nn.CrossEntropyLoss ()<br>
loss = criterion(output, yt)<br>
loss.backward()<br>
optimizer.step()</p>
<p>If using torch.optim.SGD instead of Adam, the codes work. However, I guessed Adam would give better performance in some cases, and thus tried to use Adam algorithm.</p>
<p>I have read the original codes and searched some related webpages like <a href="https://github.com/pytorch/pytorch/blob/master/docs/source/notes/extending.rst" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/docs/source/notes/extending.rst</a>. Unfortunately,<br>
It is still confusing for me to correctly use the input parameter group of Adam optimizer in this situation.</p>
<p>Would you mind give an example of using the Adam optimizer? Thank you Adam O(∩_∩)O</p>

smth: <p><a class="mention" href="/u/phenixcx">@phenixcx</a> you can look at <a href="https://github.com/pytorch/examples/tree/master/dcgan">https://github.com/pytorch/examples/tree/master/dcgan</a></p>

apaszke: <p>Third argument to constructor of <code>optim.Adam</code> is a tuple called <code>betas</code>. Optimizers have different constructors, you can find them in <a href="http://pytorch.org/docs/optim.html#torch.optim.Adam">the docs</a>.</p>

phenixcx: <p>Got it, thanks a lot for the reminders from <a class="mention" href="/u/apaszke">@apaszke</a> and <a class="mention" href="/u/smth">@smth</a>  !<br>
I missed the right constructor, though reading through available codes and specifications <img src="//discuss.pytorch.org/images/emoji/apple/cold_sweat.png?v=5" title=":cold_sweat:" class="emoji" alt=":cold_sweat:">.</p>

ecolss: <p><a class="mention" href="/u/apaszke">@apaszke</a>, I was playing with this impl, however doesn’t seem to work at all using Python3.6 + Pytorch 0.1, with the following errors:</p>
<pre><code> Traceback (most recent call last):
  File "Neural_Style.py", line 205, in &lt;module&gt;
    style_score += sl.backward()
  File "Neural_Style.py", line 105, in backward
    self.loss.backward(retain_variables=retain_variables)
  File "/usr/local/lib/python3.6/site-packages/torch/autograd/variable.py", line 146, in backward
    self._execution_engine.run_backward((self,), (gradient,), retain_variables)
  File "/usr/local/lib/python3.6/site-packages/torch/nn/_functions/conv.py", line 48, in backward
    if self.needs_input_grad[0] else None)
  File "/usr/local/lib/python3.6/site-packages/torch/nn/_functions/conv.py", line 119, in _grad_input
    return self._thnn('grad_input', input, weight, grad_output)
  File "/usr/local/lib/python3.6/site-packages/torch/nn/_functions/conv.py", line 161, in _thnn
    return impl[fn_name](self, self._bufs[0], input, weight, *args)
  File "/usr/local/lib/python3.6/site-packages/torch/nn/_functions/conv.py", line 251, in call_grad_input
    grad_input, weight, *args)
RuntimeError: Need gradOutput of dimension 4 and gradOutput.size[1] == 64 but got gradOutput to be of shape: [64 x 2401] at /Users/soumith/code/pytorch-builder/wheel/pytorch-src/torch/lib/THNN/generic/SpatialConvolutionMM.c:50</code></pre>

apaszke: <p>Did you update to 0.1.10?</p>

alexis-jacq: <p><a class="mention" href="/u/ecolss">@ecolss</a>, also I did the code using python2, I don’t know if it would work with python3.6.</p>

ecolss: <p>Yes, I did.</p>
<p><a class="mention" href="/u/alexis-jacq">@alexis-jacq</a> mentioned, it was a python2 implementation, however I don’t think it is the problem here.</p>
<p>I thought the problem was that, the input is cloned and resized in GramMatrix module, and the style loss is then computed on it, and as the error occurs at the stage of style loss backward(), so would it be that, the grad of style loss over GramMatrix is a 2 dim tensor, and further the grad over the cloned and resized input is not properly computed?</p>

ecolss: <p><a class="mention" href="/u/apaszke">@apaszke</a> any suggestions to debug this?</p>

ecolss: <p><a class="mention" href="/u/apaszke">@apaszke</a>  <a class="mention" href="/u/alexis-jacq">@alexis-jacq</a></p>
<p>Debug for a while, found the root cause of the error:</p>
<p>Variable.data.resize_() -&gt; Variable.resize().</p>
<p>replaced this line <a href="https://github.com/alexis-jacq/Pytorch-Tutorials/blob/master/Neural_Style.py#L82" rel="nofollow noopener">https://github.com/alexis-jacq/Pytorch-Tutorials/blob/master/Neural_Style.py#L82</a></p>

alexis-jacq: <p>Thanks for having reported this issue.</p>
<p>The code is working on my computer but anyway, I did it quickly when I discovered Pytorch, so I am not surprised if it causes bugs on another system. It is fool of hacks and the implementation is not clean (as you can see here: <a href="https://discuss.pytorch.org/t/how-to-extract-features-of-an-image-from-a-trained-model/119/13">How to extract features of an image from a trained model</a>). I have to re-write it, I will do it as soon as I have time for myself.</p>

mehdi-shiba: <p><a class="mention" href="/u/ecolss">@ecolss</a> Wouldn’t <code>data.view</code> be more appropriate than <code>data.resize</code> in this case? The output tensor has the same size, just a different shape. I think PyTorch’s <code>view</code> is very similar to numpy’s <code>reshape</code> method.</p>

apaszke: <p>Yes, it’s better to use .view</p>

ecolss: <p><code>.view</code> is cool, I just wasn’t aware of it before.<br>
However, <code>.resize</code> also returns a view, doesn’t it? I mean any particular difference between the two?</p>

apaszke: <p><code>.view</code> is way way safer than <code>.resize</code> and there are hardly any cases when <code>.resize</code> should be used in user scripts. It will raise an error if you try to get a tensor with a different number of elements, or if it’s not contiguous (<code>.resize</code> can give you a tensor that views on a data that wasn’t used before).</p>

ecolss: <p><a class="mention" href="/u/apaszke">@apaszke</a>  Noted, thanks</p>

leongatys: <p>Hi Alexis, cool work!<br>
I think you can get better results by using LBFGS though. You can check here for an implementation: <a href="https://github.com/leongatys/PytorchNeuralStyleTransfer" rel="nofollow noopener">https://github.com/leongatys/PytorchNeuralStyleTransfer</a></p>
<p>Best<br>
Leon</p>