Optimize_for_mobile exports are double the size of normal torchscript

erksch: <p>I tried exporting a model for PyTorch Lite to use it on mobile.</p>
<p>Before I just used</p>
<pre><code class="lang-auto">script = torch.jit.trace(model, dummy_input)
script.save("model.pt")
</code></pre>
<p>which resulted in a 46 MB file.</p>
<p>But with</p>
<pre><code class="lang-auto">script = torch.jit.trace(model, dummy_input)
script_opt = optimize_for_mobile(script)
script_opt._save_for_lite_interpreter("model.ptl")
</code></pre>
<p>I get a resulting size of 92 MB.</p>
<p>I discovered that <code>optimize_for_mobile</code> is the problem.<br>
How does it make the export so much bigger?<br>
Does it just optimize for speed not for size?</p>

xta0: <p>Seems like the constants get doubled. cc <a class="mention" href="/u/cccclai">@cccclai</a> could you take a look?</p>

erksch: <p>Any update on this? Would be great to get the intended optimizations.</p>

guangy10: <p><a class="mention" href="/u/martin_yuan">@Martin_Yuan</a> <a class="mention" href="/u/cccclai">@cccclai</a> Can you follow up on this?</p>

iseeyuan: <p>It should have been resolved by <a class="mention" href="/u/cccclai">@cccclai</a> in <a href="https://github.com/pytorch/pytorch/pull/59722" class="inline-onebox" rel="noopener nofollow ugc">[After fix] Reuse constant and bump bytecode to v5 by cccclai · Pull Request #59722 · pytorch/pytorch · GitHub</a></p>

erksch: <p>I can confirm that with current nightly build it does not double in size anymore.</p>
<p>But I wonder a little bit because the release notes state:<br>
“Major improvements in on-device binary size with Mobile Interpreter” but neither model exports are smaller nor library binaries on Android, which is still around 30-40MB (haven’t checked iOS yet). What exactly do you mean with binary size?</p>
<p>Do you have any concrete numbers that show the improvement?</p>

erksch: <p><a class="mention" href="/u/cccclai">@cccclai</a> Could you elaborate on this?</p>

Linbin: <p>Hi Erik, the binary size optimization is for pytorch library (runtime), not model file. You need to use custom build to only include ops that are needed by your model. Please check this tutorial:</p>
<p><a href="https://pytorch.org/tutorials/recipes/mobile_interpreter.html#how-to-use-mobile-interpreter-custom-build" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/recipes/mobile_interpreter.html#how-to-use-mobile-interpreter-custom-build</a></p>

erksch: <p>Alright! Thanks for the response!</p>

dunky11: <p>I found out that the problem lies in the “Conv packed params hoisting” and “INSERT_FOLD_PREPACK_OPS” steps, whatever that means…<br>
The following code optimizes the model for mobile without doubling the model’s size by blacklisting those optimization steps, this works without the nightly build:</p>
<pre><code class="lang-auto">import torch
from torch.utils.mobile_optimizer import optimize_for_mobile
from torch._C import MobileOptimizerType

torchscript_model = torch.jit.script(model)
torchscript_model = optimize_for_mobile(
    torchscript_model,
    optimization_blocklist={
            MobileOptimizerType.HOIST_CONV_PACKED_PARAMS,
            MobileOptimizerType.INSERT_FOLD_PREPACK_OPS
    },
)
if use_lite:
    torchscript_model._save_for_lite_interpreter("model.ptl")
else:
    torch.jit.save(torchscript_model, "model.pt")
</code></pre>

kimishpatel: <p>Is this still a problem?</p>