Best way to set all tensor elements to zero

kovanostra: <p>Hello</p>
<p>I am creating a zero tensor before a loop (lets call it “test_tensor”) and at each time step I want to reset the elements to zero. Currently I am doing it as in the code snippet below:</p>
<p>auto test_tensor = torch::zeros_like({tensor_initializer});<br>
auto test_tensor_2 = torch::zeros_like({tensor_initializer});</p>
<pre><code>for (int time_step = 0; time_step&lt;time_steps; time_step++) {
   test_tensor_2 = test_tensor;
   test_tensor = torch::zeros_like({tensor_initializer});
   for (int i = 0; i&lt;limit; i++) {
     test_tensor[i] = function(*args);
   } 
</code></pre>
<p>If I try to set the elements of the “test_tensor” to zero using test_tensor.zero_() I obtain different results and my tests fail.</p>
<p>Questions:</p>
<ul>
<li>What could be the reason that these two ways work differently?</li>
<li>Is it really beneficial in terms of memory to use test_tensor.zero_() or I could stick to the current way without an issue?</li>
</ul>

albanD: <p>Hi,</p>
<p>Do you actually track gradients across these operations? If so that the two will have different behaviors yes.</p>
<p>For speed, the two will be very close. The one that creates a new Tensor will obviously use a bit more memory but that’s it.</p>

kovanostra: <p>Hello <a class="mention" href="/u/alband">@albanD</a></p>
<p>Thanks for your reply!</p>
<p>No I am not tracking any gradients. I am just calling the function that contains the above code in my tests and the output is different than the expected when I am using the .zero_().</p>

albanD: <p>The code you shared should work the same in both cases.<br>
I guess something else in the code keeps a reference to test_tensor and so gets its reference changed when you do the update inplace after zeroing it out.</p>

kovanostra: <p>Actually, it really confuses me because the tensor is not used in any other way than in the code above and I don’t pass is as reference to any other tensor. Anyway, I am going to investigate further.</p>
<p>Thanks a lot for the answers! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

pascal.soveaux: <aside class="quote no-group" data-username="kovanostra" data-post="1" data-topic="89618">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kovanostra/48/25272_2.png" class="avatar"> kovanostra:</div>
<blockquote>
<p><code>test_tensor_2 = test_tensor;</code></p>
</blockquote>
</aside>
<p>If I understood correctly that is suspicious because now you only have 1 tensor. If you zero_() you are losing your previous results. It works because you are creating a new tensor on the next line (and the first initialization is useless).</p>
<p>Maybe I did not understand.</p>

kovanostra: <p>Hello <a class="mention" href="/u/pascal.soveaux">@pascal.soveaux</a></p>
<p>I thought since I don’t pass test_tensor to test_tensor_2 as reference that it would do a copy so any changes in test_tensor would not affect test_tensor_2. If it is not really a copy then it should be that the issue.</p>
<p>Thanks for the answer!</p>

kovanostra: <p>I found the solution so I write it here for the record.</p>
<p>As correctly mentioned by <a class="mention" href="/u/pascal.soveaux">@pascal.soveaux</a> the test_tensor_2 = test_tensor; operation caused the problem. I changed it into</p>
<pre><code>test_tensor_2.copy_(test_tensor);
</code></pre>
<p>and it now works when using just after</p>
<pre><code>test_tensor.zero_();
</code></pre>
<p>Thanks everyone for the help!</p>

pascal.soveaux: <p>Hi,</p>
<p>I am terribly sorry but your solution is very wrong. You are not showing all your code, from what I see you neither need copy_ nor zero_.</p>

kovanostra: <p>Hello <a class="mention" href="/u/pascal.soveaux">@pascal.soveaux</a></p>
<p>I am actually very interested to understand why my solution is wrong.</p>
<p>Briefly, my goal is to move the values of test_sensor to test_tensor_2 and then turn test_tensor to a zero tensor <strong>without</strong> allocating new memory anywhere in the process (if possible). I can sacrifice a bit of computation time for this but not memory. What would you think is a better implementation?</p>
<p>I don’t think the rest of the code is very relevant, but in case you want to see more of it you could see the function here (it’s called “compose_messages”): <a href="https://github.com/kovanostra/message-passing-neural-network/blob/v1.6.0/message_passing_nn/utils/messages.cpp" rel="nofollow noopener">https://github.com/kovanostra/message-passing-neural-network/blob/v1.6.0/message_passing_nn/utils/messages.cpp</a></p>
<p>Thanks a lot for your time and please tell me if you need any more context! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

pascal.soveaux: <p>Hi,</p>
<p>I hope I do not sound pedantic, I can be wrong.</p>
<p>Given your piece of code there are 2 reasons :</p>
<ol>
<li>Zero_<br>
There are only assignments. Take this piece of code</li>
</ol>
<pre><code class="lang-auto">byte result[255];
memset(result,0, sizeof(result));

for(int i = 0; i &lt; limits; i++) 
 result[i] += fct(i); 
 // There is no reasons to use "+=" because result[i] is 0 (+= is from your git)
 // So result is only assigned
 // So there is no reason to initialize to 0 IIF the loop is entered at least once and limits == sizeof(results)
 // see empty_like
</code></pre>
<ol start="2">
<li>Copy_<br>
Copying is useless because you can just swap tensors instead: one is assigned (say current), the other holds the prev step (say prev).</li>
</ol>
<pre><code class="lang-auto">auto prev = test_tensor_2; // reference semantics (just like a shared_ptr if you prefer)
auto current = test_tensor; // 

for(int step=...
{
    for(int i= ...)
        current[i] = fct(i);

    swap(prev, current); // No copy
}
</code></pre>
<p>I can take a close look at your code if you like but I hope you get my point.</p>

kovanostra: <p>I think the swap saved me both from the copy and the zero! Coming from the python world I didn’t know I could do such operation. Thanks a lot!</p>
<p>I would be very interested to hear any other comments you may have about my code if you have the time to check <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

pascal.soveaux: <p>I’m glad it helped you. I will take a closer look at your code as soon as I have time to do so (pr).<br>
Bonne journée <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>