Why does a `.pth` file take so much larger space? (I am missing something)

username: iArunava
cooked: <p>So,</p>
<pre><code class="lang-auto">&gt;&gt;&gt; net = nn.Conv2d(3, 3, 1, 1, 0)
</code></pre>
<p>takes</p>
<pre><code class="lang-auto">&gt;&gt;&gt; sys.getsizeof(net.weight.data)
72
</code></pre>
<p>which is in bytes.</p>
<p>But,</p>
<pre><code class="lang-auto">&gt;&gt;&gt; torch.save(net.state_dict(), './net.pth')
&gt;&gt;&gt; !du -h ./net.pth
4.0K    ./net.pth
</code></pre>
<p>which if converted to bytes</p>
<pre><code class="lang-auto">&gt;&gt;&gt; 4 * 1000
4000
</code></pre>
<p>is 4000 bytes!!!</p>
<p>Please, tell me what I am missing and why is there is a massive increase in size when it is stored in a file?</p>
<p>P.S. Its quite obvious I am missing something obvious, just point to some article or something, or maybe in the PyTorch Codebase where this is implemented.</p>

username: iArunava
cooked: <p>Okay, My second line is wrong.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; sys.getsizeof(net.weight.data)
72
</code></pre>
<p><code>getsizeof</code> can’t handle <code>torch.Tensor</code><br>
it returns <code>72</code> in case of empty tensors also!</p>
<p>So, I did this</p>
<pre><code class="lang-auto">&gt;&gt;&gt; t1 = torch.Tensor([5.])
&gt;&gt;&gt; sys.getsizeof(t1.item())
24
</code></pre>
<p>Okay, looks good.<br>
But in that case the model will contain</p>
<pre><code class="lang-auto">&gt;&gt;&gt; 24 * torch.numel(net.weight.data)
216
</code></pre>
<p>I am closer to 4000 bytes! But still far away!<br>
My guess is that each parameter uses far more bytes than its showing up.<br>
I am doing something wrong most likely.</p>

username: albanD
cooked: <p>Hi,</p>
<p>Few things I think:</p>
<ul>
<li>du -h  actually counts the size of the folder as well. Running “ls -lha” will show you that the empty folder takes 4K, not the .pth file which is only 515bytes.</li>
<li>sys.getsizeof() measure the size of the Python object. So it is very unreliable for most pytorch elements.</li>
<li>The storage format is not really optimized for space. You can actually “cat” the file and you’ll see that it contains more strings than actual data.</li>
<li>Using different pickle backend might improve that but I have never tried (<code>pickle_module</code> argument to the save function)</li>
</ul>

username: iArunava
cooked: <p>Okay, I didn’t knew that about <code>du -h</code>, now that you say so, I checked it though <code>ls -lha</code> and GUI and it shows to be of 5xx bytes, which is goood <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:">  Thanks!<br>
Maybe I will try changing the <code>pickle_module</code> once and play around with it a bit.</p>
<p>My main aim was to calculate what the size of a model is and so, getting the amount of memory it will take when I move it to GPU.<br>
It should take around the same memory, right? (the size of the model as calculated based on the datatype of the model and the memory it consumes in the GPU when I move it to GPU? <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=9" title=":thinking:" class="emoji" alt=":thinking:">) Should be, intuitively.</p>
<p>Please feel free to reply when free, I have no hurries.<br>
Thanks a ton for getting back to me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Really Appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>

username: albanD
cooked: <p>Hi,</p>
<p>The size it will take on gpu is the size of all the tensors. So the following will give you the memory in Bytes:</p>
<pre><code class="lang-auto">size = 0
for p in model.parameters():
  size += p.nelement() * p.element_size()
</code></pre>