Restoring a saved model created by architecture search

superunification: <p>Hello,</p>
<p>I have a model whose architecture is found by a search method. Many such models are saved as checkpoints, but it’s challenging to restore and run them again because the exact architecture was programmatically determined. Each time I would want to load one of the checkpoints, it would require re-writing by hand the model architecture to exactly fit the particular one of the checkpoint file.</p>
<p>It would be wonderful if there were a way to do this:</p>
<pre><code class="lang-auto">module = SomeModule()
checkpoint = torch.load(checkpoint_file)
module.restoreArchitecture(checkpoint["architecture"])
module.load_state_dict(checkpoint["state"])
</code></pre>
<p>In principle this could be done by saving the architecture configuration and building a parser, though it would get hairy for custom objects and expressions, where markup isn’t as expressive as code.</p>
<p>Is there a canonical way of doing this?</p>
<p>It seems a reasonably popular request:</p>
<aside class="quote" data-post="1" data-topic="81578">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/a4c791/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/load-pretrained-model-without-creating-architecture-manually/81578">Load pretrained model without creating architecture manually</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    How to load a base network such as Resnet152 or efficient net model. Do I need to recreate the architecture and copy the pretrained weights to my architecture?. Is there a way to use the model without creating the architecture and copying the parameters?
  </blockquote>
</aside>
<aside class="quote" data-post="10" data-topic="718">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rajarsheem/48/227_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-save-load-torch-models/718/10">How to save/load Torch models?</a> 
  </div>
  <blockquote>
    Is there any way to load a trained model without declaring the class definition before ? 
I want the model architecture as well as parameters to be loaded.
  </blockquote>
</aside>
<aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/59596075/how-to-save-model-architecture-in-pytorch">
  <header class="source">

      <a href="https://stackoverflow.com/questions/59596075/how-to-save-model-architecture-in-pytorch" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/11491606/m-z" target="_blank" rel="noopener nofollow ugc">
    <img alt="M.Z." src="https://www.gravatar.com/avatar/df681ab0a3133ea1a9acf8e192a654e3?s=256&amp;d=identicon&amp;r=PG&amp;f=y&amp;so-version=2" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/59596075/how-to-save-model-architecture-in-pytorch" target="_blank" rel="noopener nofollow ugc">How to save model architecture in PyTorch?</a>
</h4>

<div class="tags">
  <strong>pytorch</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/11491606/m-z" target="_blank" rel="noopener nofollow ugc">
    M.Z.
  </a>
  on <a href="https://stackoverflow.com/questions/59596075/how-to-save-model-architecture-in-pytorch" target="_blank" rel="noopener nofollow ugc">12:33AM - 05 Jan 20 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>