Making torch load faster

Bjorn_Lindqvist: <p>Is there any way of making “import torch” run faster? I get that PyTorch is huge so loading all its code must take a few seconds. However, many times I’m just testing things out and only need a fraction of what PyTorch has to offer. Is it possible to load PyTorch incrementally or something to reduce the very annoying 5-10 second “lag” import torch causes?</p>

ptrblck: <p>5-10 seconds sounds way too long as a quick check shows my env imports torch in &lt;1 sec:</p>
<pre data-code-wrap="python"><code class="lang-python">%time import torch
CPU times: user 759 ms, sys: 70.2 ms, total: 829 ms
Wall time: 850 ms
</code></pre>
<p>(unsure how exact this way of timing is but it also corresponds to a naive manual timing)</p>

Bjorn_Lindqvist: <p>Here are my stats:</p>
<pre><code class="lang-auto">In [1]: %time import torch
CPU times: user 2.74 s, sys: 139 ms, total: 2.88 s
Wall time: 2.09 s
</code></pre>
<p>But the real culprit is actually another import which for some stupid reason loads TensorFlow:</p>
<pre><code class="lang-auto">In [1]: %time from torch.utils.tensorboard import SummaryWriter
2024-10-01 08:59:52.196854: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical r
esults due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_
ONEDNN_OPTS=0`.
2024-10-01 08:59:52.208956: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to
register factory for plugin cuFFT when one has already been registered
2024-10-01 08:59:52.222618: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to
register factory for plugin cuDNN when one has already been registered
2024-10-01 08:59:52.226873: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting
to register factory for plugin cuBLAS when one has already been registered
2024-10-01 08:59:52.236922: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU i
nstructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-01 08:59:52.933249: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
CPU times: user 3.25 s, sys: 652 ms, total: 3.91 s
Wall time: 3.36 s
</code></pre>
<p>I’m running PyTorch why would I need TensorFlow? <img src="https://discuss.pytorch.org/images/emoji/apple/stuck_out_tongue.png?v=12" title=":stuck_out_tongue:" class="emoji" alt=":stuck_out_tongue:" loading="lazy" width="20" height="20"></p>

AlexP4499: <p>Totally different approach - but would using Jupyter Notebook for dev be an approach that works for you?</p>
<p>I use VS code for my main development environment, but I switch to Jupyter for when I want to rapidly iterate.</p>
<p>I find the import/module caching to significantly help in this respect, as I have all my base classes and imports in a top tell which I don’t need to recalculate, then I am free to play with model params or new classes in new cells.</p>
<p>I know VS code supports notebooks… but I prefer the browser environment for some reason… I find it a bit more intuitive having the two completely separated.</p>

ptrblck: <aside class="quote no-group" data-username="Bjorn_Lindqvist" data-post="3" data-topic="210991">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bjorn_lindqvist/48/25326_2.png" class="avatar"> Bjorn_Lindqvist:</div>
<blockquote>
<p>I’m running PyTorch why would I need TensorFlow? <img src="https://discuss.pytorch.org/images/emoji/apple/stuck_out_tongue.png?v=12" title=":stuck_out_tongue:" class="emoji" alt=":stuck_out_tongue:" loading="lazy" width="20" height="20"></p>
</blockquote>
</aside>
<p>You don’t and I also don’t even have it installed:</p>
<pre data-code-wrap="python"><code class="lang-python">%time import torch
CPU times: user 813 ms, sys: 56.9 ms, total: 870 ms
Wall time: 896 ms

%time from torch.utils.tensorboard import SummaryWriter
CPU times: user 87.7 ms, sys: 4.02 ms, total: 91.7 ms
Wall time: 91.4 ms

import tensorflow
Traceback (most recent call last):

  Cell In[3], line 1
    import tensorflow

ModuleNotFoundError: No module named 'tensorflow'
</code></pre>

Bjorn_Lindqvist: <p>I removed TensorFlow but importing PyTorch is still slow:</p>
<pre><code class="lang-auto">In [1]: %time import torch
CPU times: user 2.66 s, sys: 765 ms, total: 3.42 s
Wall time: 2.92 s
In [2]:              
</code></pre>

Maxim_Egorushkin: <p>One design pattern I have been using for efficient parallelization, is import required libraries and load/construct datasets once, and then fork child processes in the middle of Python functions, which proceed with computations without a delay with PyTorch or TensorFlow. The child processes must terminate with os._exit to prevent destruction of process shared resources.</p>

roach: <p>I remember a couple of months back that i installed torch and then installed tensorboard (which also installed tensorflow as a dependency). Whenever i would load torch from that point on it would subsequently also load tensorflow. Maybe one of the modules within your installation of torch got crossovered with a tensorflow import?</p>

Bjorn_Lindqvist: <p>No, now I only have torch installed. The import lag really annoying when creating command-line utilities based on torch. E.g. “some-script.py --help” just shows a help message, but you still have the two seconds lag because of an “import torch” statement.</p>
<p>Here is a treemap created using “-X importtime”:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/4/441bd3e385e54e5ca738018f564a2dd36c92a342.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/441bd3e385e54e5ca738018f564a2dd36c92a342" title="importtime"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/4/441bd3e385e54e5ca738018f564a2dd36c92a342_2_690x460.jpeg" alt="importtime" data-base62-sha1="9Iw5um8rMxkK1NFdyumFnlVGLQe" width="690" height="460" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/4/441bd3e385e54e5ca738018f564a2dd36c92a342_2_690x460.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/4/441bd3e385e54e5ca738018f564a2dd36c92a342_2_1035x690.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/4/441bd3e385e54e5ca738018f564a2dd36c92a342_2_1380x920.jpeg 2x" data-dominant-color="79A261"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">importtime</span><span class="informations">1920×1280 200 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also found this: <a href="https://dev-discuss.pytorch.org/t/delving-into-what-happens-when-you-import-torch/1589" class="inline-onebox" rel="noopener nofollow ugc">Delving into what happens when you `import torch` - PyTorch Developer Mailing List</a></p>