Is it supported to use `ScriptModule` for training?

username: jiong-zhang
cooked: <p>Can a model serialized by <code>torch.jit.trace</code> be used for training? In the documentation it seems the <code>ScriptModule</code> is intented for inference and only training example is for <code>torch::nn::Module</code>. I wonder if training with <code>ScriptModule</code> is officially supported and is there documentation on that.</p>
<p>Related:</p><aside class="quote" data-post="1" data-topic="35932">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/ea666f/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/can-scriptmodule-be-used-for-training/35932">Can ScriptModule be used for training?</a> 
  </div>
  <blockquote>
    Can ScriptModule be used for training? The reason we ask this question is that we found some source code related back propagation in torch.jit. <a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/autodiff.cpp#L1151" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/autodiff.cpp#L1151</a>
  </blockquote>
</aside>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/17614">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/17614" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/17614" target="_blank" rel="noopener nofollow ugc">Document whether it is possible to train TorchScript modules</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-03-01" data-time="13:02:44" data-timezone="UTC">01:02PM - 01 Mar 19 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/martinruenz" target="_blank" rel="noopener nofollow ugc">
          <img alt="martinruenz" src="https://avatars.githubusercontent.com/u/303783?v=4" class="onebox-avatar-inline" width="20" height="20">
          martinruenz
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          oncall: jit
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: docs
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: cpp
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 📚 Documentation

I am interested in the following workflow, but my impressi<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">on is that this use case is not supported (yet?):

1. Prototype a model in **Python**
2. Export model to **C++** via TorchScript (`torch.jit.trace`)
3. Train model in **C++**

Based on the documentation, it is unclear whether this workflow is supported or not. The very first sentence of https://pytorch.org/docs/master/jit.html states

&gt; TorchScript is a way to create serializable and **optimizable models** from PyTorch code

However, it is ambiguous if "**optimizable**" refers to training or the jit compilation process here. 

It seems that `torch::jit::script::Module` is treated as a special case which does not share commonality / a base class with `torch::nn::Module`. Is there also a way to access the parameters of a `torch::jit::script::Module` in order to use it for training? At first I thought "**optimizable models**" was referring to this but I am unsure now.

This issue is related to the post: https://discuss.pytorch.org/t/can-scriptmodule-be-used-for-training/35932


cc @suo @yf225</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>