Cannot insert a Tensor that requires grad as a constant

username: mloft
cooked: <p>I implemented my own version of CaffeNet in PyTorch and I am trying to save the model in an onnx file. But I have the error described in the title when I run this code:</p>
<p>"<br>
import torch.nn as nn<br>
import torch</p>
<p>class Flatten(nn.Module):<br>
def forward(self,x):<br>
return x.view(x.size(0),-1)</p>
<p>class CaffeNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(CaffeNet,self).<strong>init</strong>()</p>
<pre><code>    self.layers=[]

    self.layers.append(nn.Conv2d(in_channels=3,out_channels=3,kernel_size=11))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(3,96,55))
    self.layers.append(nn.MaxPool2d(kernel_size=2,stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(96,192,27))
    self.layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(192, 288, 13))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(288, 288, 13))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(288, 256, 11)) #12-th layer, normally kernel_size=13
    self.layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(256, 256, 3)) #15-th layer, normally kernel_size=13
    self.layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(Flatten())

    self.layers.append(nn.Linear(2304,2304)) #19-th layer, normally 4096 neurons
    self.layers.append(nn.ReLU())
    self.layers.append(nn.Dropout())

    self.layers.append(nn.Linear(2304,2304))
    self.layers.append(nn.ReLU())
    self.layers.append(nn.Dropout())

    self.layers.append(nn.Linear(2304,1000))
    self.layers.append(nn.Softmax())


def forward(self, x):
    for layer in self.layers:
        x=layer(x)
    return x
</code></pre>
<p>net=CaffeNet()<br>
input = torch.randn(20, 3, 227, 227,requires_grad=False)<br>
input_names = [ “input” ]<br>
output_names = [ “output” ]<br>
torch.onnx.export(net,input,“caffenet.onnx”,verbose=True, input_names=input_names, output_names=output_names)<br>
"</p>
<p>I am using python2.7. The last line of the stack trace is:<br>
“RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient”</p>

username: driazati
cooked: <p>The modules you’re adding have some Parameters, so they need to be submodules of <code>CaffeNet</code>. It should work if you use a <code>torch.nn.Sequential</code> instead of a regular Python list for <code>self.layers</code>.</p>

username: mloft
cooked: <p>Ok Thanks you. I’m gonna try this on monday when I’m going back to work. I hope it will work <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: mloft
cooked: <p>Thanks you very much driazati, it works <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: 111199
cooked: <p>hi，I have the same problem as you. How did you solve it，Look forward to your reply<br>
thanks</p>

username: mloft
cooked: <p>hi, the subLayers of your module should be its submodules. So:<br>
1/ I made my CaffeNet module derived from nn.sequential<br>
2/Then I defined a <strong>add</strong> method which adds layer to the subModule dictionary (self._modules) of a nn.sequential:<br>
def <strong>add</strong>(self, layer):<br>
self._modules[str(self.index)]=layer<br>
self.index=self.index+1<br>
3/Then I call this method with each layer of CaffeNet</p>

username: 111199
cooked: <p>Thank you for your reply, you are a good man, but I don’t know the specific operation, can you provide the code to solve it?</p>

username: mycpuorg
cooked: <p>Unfortunately, I seem to have the problem (and <a href="https://discuss.pytorch.org/t/runtimeerror-cannot-insert-a-tensor-that-requires-grad-as-a-constant/113767">replied in another similar question</a>). My model has a VGG (as encoder) and FCN as a decoder it’s not analogous. Any ideas?</p>

username: zylo117
cooked: <p>Just replace self.layers =[] with self.layers =nn.ModuleList() or something else</p>
<p>Otherwise, model.requires_grad() won’t affect those layers within python list.</p>

username: wangyang_zuo
cooked: <p>I have given my solution in <a href="https://discuss.pytorch.org/t/runtimeerror-cannot-insert-a-tensor-that-requires-grad-as-a-constant/113767/3" class="inline-onebox">RuntimeError: Cannot insert a Tensor that requires grad as a constant - #3 by smkoop</a>. We just need manually set all parameters <code>required_grad = False</code> .</p>