Loading pytorch model without a code

username: scitator
cooked: <p>Hello, everyone!</p>
<p>I have a question about PyTorch load mechanics, when we are using torch.save and torch.load. Let’s look at examples:</p>
<p>Suppose, I have a network:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F

from collections import OrderedDict


class ReallySimpleModel(nn.Module):
    def __init__(self, **params):
        super().__init__()
        
        self.net = nn.Sequential(OrderedDict([
            ("linear1", nn.Linear(20, 10)),
            ("bn1", nn.BatchNorm1d(10)),
            ("relu1", nn.ReLU()),
            ("linear2", nn.Linear(10, 5)),
            ("bn2", nn.BatchNorm1d(5)),
            ("relu2", nn.ReLU()),
            ("linear3", nn.Linear(5, 1)),
            ("bn3", nn.BatchNorm1d(1)),
            ("relu3", nn.Sigmoid()),
        ]))
    
    def forward(self, x):
        x = self.net.forward(x)
        return x
</code></pre>
<p>After that, I create the instance of it with:</p>
<pre><code class="lang-auto">from modules.model import ReallySimpleModel
net = ReallySimpleModel()
pprint(net)
</code></pre>
<p>So, our net:</p>
<pre><code class="lang-auto">ReallySimpleModel(
  (net): Sequential(
    (linear1): Linear(in_features=20, out_features=10)
    (bn1): BatchNorm1d(10, eps=1e-05, momentum=0.1, affine=True)
    (relu1): ReLU()
    (linear2): Linear(in_features=10, out_features=5)
    (bn2): BatchNorm1d(5, eps=1e-05, momentum=0.1, affine=True)
    (relu2): ReLU()
    (linear3): Linear(in_features=5, out_features=1)
    (bn3): BatchNorm1d(1, eps=1e-05, momentum=0.1, affine=True)
    (relu3): Sigmoid()
  )
)
</code></pre>
<p>Now, I save it with:</p>
<pre><code class="lang-auto">torch.save(dict(
    model=net, 
    model_state=net.state_dict()), 
           "net.checkpoint.pth.tar")
</code></pre>
<p>Okay, great, we save the model. We can even load it with:</p>
<pre><code class="lang-auto">checkpoint = torch.load("net.checkpoint.pth.tar")
net = checkpoint["model"]
pprint(net)
</code></pre>
<p>and the model structure would be correct. It works even without manual import of  <code>ReallySimpleModel</code> - very cool.</p>
<p>And the interesting part starts here. As we now, when we call <code>torch.save</code> PyTorch use pickle to serialize the model and <strong>it’s source code</strong>. So, even if we change our <a href="http://model.py" rel="nofollow noopener">model.py</a>:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F

from collections import OrderedDict


class ReallySimpleModel(nn.Module):
    pass
</code></pre>
<p>Load works! And it’s really great! With the warning, that original source code have changed, but it works!<br>
But, if we delete <code>model.py</code> or delete <code>ReallySimpleModel</code> from it - all goes wrong. ImportErrors or AttributeErrors will appear.</p>
<p>As you can see, PyTorch loading process doesn’t need any code of the model, I think it doesn’t need any code at all. But it needs the same projects structure to solve some “dict-keys” problem.</p>
<p>So, my question: is there any solution for this problem? Maybe we can somehow modulate project structure automatically? Personally, I cannot understand why do we use any project structure, when we load previous model?<br>
When I load previous model, I don’t want to use current source code, I want previous to come back <img src="//discuss.pytorch.org/images/emoji/apple/smile.png?v=5" title=":smile:" class="emoji" alt=":smile:">.</p>

username: smth
cooked: <p>There are limitations to loading pytorch model without code.</p>
<p><strong>First limitation:</strong><br>
We only save the source code of the class definition. We do not save beyond that (like the package sources that the class is referring to).</p>
<p>For example:</p>
<pre><code class="lang-auto">import foo

class MyModel(...):
    def forward(input):
        foo.bar(input)
</code></pre>
<p>Here the package <code>foo</code> is not saved in the model checkpoint.</p>
<p><strong>Second limitation:</strong><br>
There are limitations on robustly serializing python constructs. For example the default picklers cannot serialize lambdas. There are helper packages that can serialize more python constructs than the standard, but they still have limitations. <a href="https://github.com/uqfoundation/dill">Dill</a> is one such package.</p>
<p>Given these limitations, there is no robust way to have torch.load work without having the original source files.</p>

username: scitator
cooked: <p>Thank you for your answer.</p>
<p>Nevertheless, few more questions about the limitations. I know, the most correct model serialisation is really challenging problem, but what if we use some tricks?</p>
<p>For example, large variety of models can be described by <code>nn.Sequential</code> with the forward pass looking like this:</p>
<pre><code class="lang-auto">def forward(self, x):
    x = self.net.forward(x)
    return x
</code></pre>
<p>For these “types” of models all needed source code will be saved. And with other hack like saving the project structure, the model should loads without many problems, is’t it?</p>

username: smth
cooked: <p>in this limited case, it might work if your Sequential itself is not containing any layers from other files.</p>

username: tobegit3hub
cooked: <p>Thanks for replying.</p>
<p>It has limitations without the code but I think PyTorch models can store all the needed computation graph by itself. Just like TensorFlow, MXNet and other frameworks.</p>
<p>This is really needed for the general serving service. We try to implement the service to load PyTorch models with user’s model files but not source files.</p>

username: cometyang
cooked: <p>Found this topic by googling, is there anyway to load model without including source code for pytorch?</p>

username: dashesy
cooked: <p>If you export to TorchScript you can load it without code.<br>
I wish PyTorch also had a runtime (not just library but set of utilities to make it easy to read exported models and run them in a session), then there would be really no need to source code.</p>

username: cometyang
cooked: <p>Dig into the jit capability, but found roadblock.<br>
Got errors like:<br>
RuntimeError: forward() Expected a value of type ‘Tensor’ for argument ‘ctx’ but instead found type ‘FunctionBackward’.<br>
Inferred ‘ctx’ to be of type ‘Tensor’ because it was not annotated with an explicit type.<br>
Position: 0</p>
<p>Seems autograd script is still WIP [<a href="https://github.com/pytorch/pytorch/pull/22582" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/22582</a>]<br>
Would be appreciate if anyone can help.</p>

username: f10w
cooked: <blockquote>
<p>We only save the source code of the class definition. We do not save beyond that (like the package sources that the class is referring to).</p>
</blockquote>
<p>I guess this has been changed recently?</p>
<p>After changing my source code folder name from <code>src</code> to <code>&lt;package-name&gt;</code> (e.g. <code>abc</code>), I got the following error:</p>
<blockquote>
<p>File “/home/user/.env/python3/lib/python3.5/site-packages/torch/serialization.py”, line 613, in _load<br>
result = unpickler.load()<br>
ImportError: No module named ‘src’</p>
</blockquote>
<p>Could you please tell me if there is a simple fix for the <code>.pth</code> files that I saved? I would like to keep <code>&lt;package-name&gt;</code> instead of <code>src</code> for the code.</p>
<p>Thank you very much in advance!</p>

username: Rizhao_Cai
cooked: <p>Thank you very much. Your finding is interesting and help solve my problem (some lib dependencies when deserializing a model)</p>

username: Daniel_Kurniadi
cooked: <p>The <code>.pth</code> file checkpoint has the computation graph? I see now. The whole time I thought you need the class module that defines the <code>forward(x)</code> to define the computation graph. Then my question is:</p>
<ol>
<li>In some case I need a output hook for output from certain layer instead of end/ final layer in neural network. How do I create this hook when loading in C++? I’m looking at loading C++ model using <code>libtorch</code>
</li>
</ol>

username: Nedir-bu
cooked: <p>Thank you very much.</p>