Advantages & Disadvantages of using pickle module to save models vs torch.save

pinocchio: <p>I was reading <a href="https://discuss.pytorch.org/t/save-and-load-model/6206/27" class="inline-onebox">Save and Load model</a> but it wasn’t clear why I’d use <code>torch.save</code> over <code>pickle.dump</code>.</p>
<p>What worries me is that my Neural Net modules/object have many more things inside of them besides only parameters. Since pytorch saves things using the <code>state_dict</code> that worried me that something might be missing (<a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/saving_loading_models.html</a>).</p>
<p>Isn’t for my use case easier to simply use the <code>pickle.dump</code> function? Why would I use <code>torch.save</code> in general?</p>
<p>Note I am using gpu’s. Not sure if that matters.</p>
<p>The easiest thing for me is if I could just save the whole python program state (similar to how matlab used to allow you to do it <a href="https://www.mathworks.com/help/matlab/ref/save.html" rel="nofollow noopener">https://www.mathworks.com/help/matlab/ref/save.html</a>).</p>

ptrblck: <aside class="quote no-group" data-username="pinocchio" data-post="1" data-topic="79016">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/48/22093_2.png" class="avatar"> pinocchio:</div>
<blockquote>
<p>What worries me is that my Neural Net modules/object have many more things inside of them besides only parameters.</p>
</blockquote>
</aside>
<p>The <code>state_dict</code> will store all registered parameters and buffers.<br>
If you need to serialize some tensors, you should thus create an <code>nn.Parameter</code>, if it’s trainable, or a buffer via <code>self.register_buffer(name, tensor)</code>, if it’s not trainable.</p>
<aside class="quote no-group" data-username="pinocchio" data-post="1" data-topic="79016">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/48/22093_2.png" class="avatar"> pinocchio:</div>
<blockquote>
<p>Isn’t for my use case easier to simply use the <code>pickle.dump</code> function? Why would I use <code>torch.save</code> in general?</p>
</blockquote>
</aside>
<p>Answered in the <a href="https://discuss.pytorch.org/t/save-and-load-model/6206/28">other thread</a>.</p>
<aside class="quote no-group" data-username="pinocchio" data-post="1" data-topic="79016">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/48/22093_2.png" class="avatar"> pinocchio:</div>
<blockquote>
<p>The easiest thing for me is if I could just save the whole python program state</p>
</blockquote>
</aside>
<p>That’s generally not recommended.<br>
Instead you should store the <code>state_dicts</code> and the source files separately.<br>
Storing e.g. the complete model could force you to recreate exactly the same file and folder structure.</p>

pinocchio: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> thanks for your responses! Greatly appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>I am curious, why is it not recommended to store the whole python program state? (or as close to that as possible).</p>
<p>I am currently using <code>dill</code> to simulate this as much as possible because my neural network module classes contain pointers to many things, including lambda functions I’d like to restore properly.</p>
<p>reference:</p>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/25348532/can-python-pickle-lambda-functions" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/1023470/lars" target="_blank" rel="nofollow noopener">
    <img alt="Lars" src="https://www.gravatar.com/avatar/1d68ceedad9a00addf67556bd2175a5e?s=128&amp;d=identicon&amp;r=PG" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/25348532/can-python-pickle-lambda-functions" target="_blank" rel="nofollow noopener">Can Python pickle lambda functions?</a>
</h4>

<div class="tags">
  <strong>python, python-2.7, lambda, pickle</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/1023470/lars" target="_blank" rel="nofollow noopener">
    Lars
  </a>
  on <a href="https://stackoverflow.com/questions/25348532/can-python-pickle-lambda-functions" target="_blank" rel="nofollow noopener">10:52AM - 17 Aug 14 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


ptrblck: <p>As mentioned in my last post, you could be forced to recreate the exact find and folder structure, if you save the complete model via <code>torch.save</code>.<br>
I don’t know how <code>dill</code> is working in this case.</p>

tothedistance: <p>yes, the same file structure needs to maintain.<br>
moreover, the tensor function will still be the one used when saving.<br>
i loaded a yolov5 model which is version 4.0 and uses silu as act. but i accidentally used local version of yolov5 version1.0 which uses leaky relu. however, the model still uses silu. i guess the function is stored in tensor.</p>

ado_sar: <p>Is the <code>with</code> statement necessary in case we use <code>torch.save</code>? That is, should we write something like:</p>
<p><code>with open('foo.pt', 'wb') as fhand:      torch.save(obj_to_save, fhand)</code></p>
<p>?</p>

ptrblck: <p>No, since <code>torch.save</code> will create and open the file for you if it doesn’t exist.</p>

heradsinn: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I have a follow-up question to this. In our production, we have frequently encountered incompatibility issues due to saving/loading and changed source files. For example, let’s say we trained and deployed a model in January which would be loaded like so:</p>
<pre><code class="lang-auto">from models.ExampleNet import ExampleNet

model = ExampleNet()

loaded = torch.load("example_saved_file.pt")
loaded_state_dict = loaded["state_dict"]
model.load_state_dict(loaded_state_dict)
</code></pre>
<p>This seems fine. However, in February some changes to the model definition were made as part of the R&amp;D process, perhaps additional norm layers were added or the model was made deeper. These changes were subsequently pushed to dev, staging, prod and deployed to the live system. The above loading procedure now fails since the <code>model</code> architecture doesn’t match that of the <code>loaded_state_dict</code>. Though saving the model directly is not generally recommended, would doing so here with something like <code>torch.save(trained_model)</code> (instead of its state dict) be a good approach or at least work around? I’m curious to know if there are additional factors that would impact this that still would make it a bad idea.</p>

ptrblck: <aside class="quote no-group" data-username="heradsinn" data-post="8" data-topic="79016">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/heradsinn/48/73999_2.png" class="avatar"> heradsinn:</div>
<blockquote>
<p>Though saving the model directly is not generally recommended, would doing so here with something like <code>torch.save(trained_model)</code> (instead of its state dict) be a good approach or at least work around?</p>
</blockquote>
</aside>
<p>I don’t believe so as source code changes could easily fail and the error messages might be quite confusing if the model definition changed.<br>
Especially if your model uses and depends on a lot of imports from other scripts, which could be changed, the debugging could become quite challenging.</p>