Initializing a model instance before loading

adaber: <p>Hi,</p>
<p>A model instance/object has to be initialized before we can load the model weight values to the object from a file if I understand correctly. Let me know if I am mistaken.</p>
<p>If I used a UNet, for example, where the only variable initialization parameters (used to initialize a UNet instance) are the number of feature planes/channels for each “depth” level, <strong>would there be a way to pull that information from a saved file prior to loading so I could “dynamically” initialize a UNet object that will have a valid structure for storing the data from the saved file ?</strong></p>
<p>One workaround would be to create a custom file format that will contain both the model and initialization parameter values but I’d rather extract that info from the Pytorch’s file format directly if possible.</p>
<p>Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <aside class="quote no-group" data-username="adaber" data-post="1" data-topic="72945">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/c2a13f/48.png" class="avatar"> adaber:</div>
<blockquote>
<p>A model instance/object has to be initialized before we can load the model weight values to the object from a file if I understand correctly. Let me know if I am mistaken.</p>
</blockquote>
</aside>
<p>That’s the recommended way. You would initialize the same model and just load the <code>state_dict</code> into it.</p>
<aside class="quote no-group" data-username="adaber" data-post="1" data-topic="72945">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/c2a13f/48.png" class="avatar"> adaber:</div>
<blockquote>
<p>would there be a way to pull that information from a saved file prior to loading so I could “dynamically” initialize a UNet object that will have a valid structure for storing the data from the saved file ?</p>
</blockquote>
</aside>
<p>The usual approach is to create a <code>dict</code> and store everything inside it.<br>
E.g. you could store the <code>model.state_dict()</code>, <code>optimizer.state_dict()</code> as well as your depth level.<br>
Once you’ve created this <code>dict</code>, you could store and load it directly using <code>torch.save</code> and <code>torch.load</code>, respectively.</p>

adaber: <p>Hi ptrblck,</p>
<p>Thanks for the reply.</p>
<p>Does your answer apply to C++ (libtorch) ? I cannot seem to find a way to execute what you suggested in C++.</p>
<p>Thanks!</p>

ptrblck: <p>You could use some utility funcitions from <a href="https://discuss.pytorch.org/t/how-to-copy-network-parameters-in-libtorch-c-api/32221/7">this topic</a> or load your jitted model directly (which seems to be the recommended way).</p>

adaber: <p>“load your jitted model directly (which seems to be the recommended way)”</p>
<p>I am using C++ for both training and evaluation.</p>
<p>I’ll take a look at the discussion you linked. Thanks.</p>
<p>P.S. Pytorch is a very powerful yet very easy-to-use Deep Learning library. The Pytorch team has done a great job so far. Keep up the good work <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>