Editing ScriptModule parameters

NegatioN: <p>Hi.</p>
<p>I know this seems like an anti-pattern question to ask, but I was wondering if there’s a way to edit a ScriptModule’s parameters and delete references to the old object at the same time?</p>
<p>Currently it is not allowed to delete children of <code>ScriptModule._parameters</code>, which I assume is because the management of this happens in the lower C++ levels of the code.</p>
<p>Are there any thoughts on supporting this in the future?</p>
<p>For now it is at least possible to replace a value in the same <code>ScriptModule._parameters</code> as long as it’s of lesser size than the current value.</p>
<p>Example of something that will still execute (after the model is traced):</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self, num_emb):
        super(Net, self).__init__()
        self.emb = nn.Embedding(num_emb, 100)
        
    def forward(self, x):
        return self.emb(x)
m = Net(10000)
*trace model*
subset = torch.randn_like(m.emb.weight[:80])
m.emb.weight.data = subset
</code></pre>
<p>Are there any huge drawbacks to doing this, other than being anti-pattern?<br>
My usecase is training a model, shipping it to production, then trimming it down to a subset of our most recent items before accepting queries.</p>
<p>Bonus question: Are there any good ways to convert a traced model back to a “normal” model after doing <code>torch.jit.load()</code>?</p>

NegatioN: <p>Modifying a traced model, then doing a <code>copy()</code> forces a “proper” re-allocate at least. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Any assistance with some of the other questions would still be appreciated! But my immediate needs are at least met.</p>
<pre><code class="lang-auto">lm = torch.jit.load('m.pt')
subset = torch.randn_like(lm.emb.weight[:80])
lm.emb.weight.data = subset
tmp = lm.copy()
</code></pre>

sepid_kh: <aside class="quote no-group" data-username="NegatioN" data-post="1" data-topic="47744">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/negation/48/13031_2.png" class="avatar"> NegatioN:</div>
<blockquote>
<p>convert a traced model back to a “normal” model after doing <code>torch.jit.load()</code></p>
</blockquote>
</aside>
<p>Hi NegatioN,<br>
I know it’s a long time but just wondering have you found any answer to your question? I have been struggling with this question for a while and would be very grateful if someone could help me. How can we modify a traced model? How can we finetune them? And what’s the point if we cannot finetune these models?</p>

sepid_kh: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,<br>
Sorry to also mention you in this post. I appreciate it if you could help me.</p>

ptrblck: <p>I don’t think you can modify a scripted model, so would need to create the model architecture before scripting it. Afterwards you can fine-tune it as a plain eager model.</p>

sepid_kh: <p>Thanks so much, <a class="mention" href="/u/ptrblck">@ptrblck</a>, for your prompt answer. I am not familiar with the eager model. I just guess you mean I should apply pytorch_to_keras for fine-tuning?<br>
I work with big data, and it takes more than one week to train the model! so if I just want to change the size of a Linear layer like ( self.fc = nn.Linear(64,1)) to ( self.fc = nn.Linear(4096,1)) it is not possible, and I should modify the model and train that from the beginning?!</p>

ptrblck: <aside class="quote no-group" data-username="sepid_kh" data-post="6" data-topic="47744">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sepid_kh/48/32173_2.png" class="avatar"> sepid_kh:</div>
<blockquote>
<p>I am not familiar with the eager model. I just guess you mean I should apply pytorch_to_keras for fine-tuning?</p>
</blockquote>
</aside>
<p>No, by “eager mode” I meant the “normal” PyTorch model without scripting.</p>
<aside class="quote no-group" data-username="sepid_kh" data-post="6" data-topic="47744">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sepid_kh/48/32173_2.png" class="avatar"> sepid_kh:</div>
<blockquote>
<p>I work with big data, and it takes more than one week to train the model! so if I just want to change the size of a Linear layer like ( self.fc = nn.Linear(64,1)) to ( self.fc = nn.Linear(4096,1)) it is not possible, and I should modify the model and train that from the beginning?!</p>
</blockquote>
</aside>
<p>You can directly change the layer in the model before scripting it. E.g. something like this would work:</p>
<pre data-code-wrap="python"><code class="lang-python">model = MyModel()
model.fc = nn.Linear(...)
model_scripted = torch.jit.script(model) # script afterwards

# manipulating the scripted model might fail
model_scripted.fc = nn.Linear(...)
</code></pre>
<p>Would this work for your use case or do you need to load an already scripted model (and somehow cannot manipulate it beforehand)?</p>

sepid_kh: <p>Yes <a class="mention" href="/u/ptrblck">@ptrblck</a>, I need to load an already scripted model and then modify that.<br>
I actually didn’t know after loading a scripted model it cannot be fine_tuned or modified! So I guess I have to train my model from scratch and save that as a normal model? Isn’t there any way to convert an already saved script model (after loading) to a normal model?</p>

ptrblck: <p>Small correction: the scripted model can be fine-tuned (i.e. trained), but I don’t believe modified (JIT experts might correct me here).<br>
I’m not sure what restrictions you are working with and why loading a scripted model is necessary, i.e. I guess you might not have access to the model definition?</p>

sepid_kh: <p>for fine_tune I mean, if we want to change last layer size(e.g. number of class), is it still possible?</p>
<p>Let’s say my model is something like that:</p>
<p>class Mymodel(nn.Module):<br>
def <strong>init</strong>(self, num_classes):<br>
super(Mymodel, self).<strong>init</strong>()<br>
self.model1 = models.resnet50(pretrained=True)<br>
self._dropout = nn.Dropout(.4)<br>
self.sig = nn.Sigmoid()<br>
self.fc =nn.Linear(2048, num_classes)<br>
…<br>
then what I did is:</p>
<ul>
<li>train the model</li>
<li>evaluate model</li>
<li>save model using torch.jit.save(net_trace,‘model.pt’)</li>
</ul>

ptrblck: <p>Yes, your workload is possible assuming you are not trying to manipulate the model architecture of the scripted model.<br>
The potential issue I’ve mentioned before would arise if you are using this workflow:</p>
<ul>
<li>save model using torch.jit.save(net_trace,‘model.pt’)</li>
<li>load the model</li>
<li>change the model architecture #!!!</li>
<li>train the model</li>
</ul>

sepid_kh: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> for your time and your help <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=10" title=":blush:" class="emoji" alt=":blush:"></p>