How to use condition flow?

t3476: <p>I want to do condition control similiar to <a href="https://www.tensorflow.org/api_docs/python/tf/cond" rel="nofollow noopener">tf.cond</a>.<br>
I got a ByteTensor after applying x &gt; 0 but I don’t know what to do then</p>

alexis-jacq: <p>You probably get this error when you try to use the ByteTensor as a boolean:</p>
<p><code>RuntimeError: bool value of non-empty torch.ByteTensor objects is ambiguous</code></p>
<p>One possibility is to use <code>(x&gt;0).numpy()</code> if your tensor only contains one single element.</p>

t3476: <blockquote>
<p>x if (x &gt; 0).numpy() else x-1</p>
</blockquote>
<p>Does that numpy bool create proper computing graph? If so, why that’s the case? I’m not sure why pytorch can create computing graph with mixing numpy arrays and pytorch variables.</p>

smth: <p>if <code>x</code> is a <code>Tensor</code>, then <code>x &gt; 0</code> will return a <code>ByteTensor</code> with value 1 where x[i] &gt; 0 and value 0 where x[i] &lt;= 0.</p>
<blockquote>
<p>x if (x &gt; 0).numpy() else x-1</p>
</blockquote>
<p>In this case, is x a <code>Tensor</code>? If x is part of the graph, it has to be a <code>Variable</code>.</p>
<p>Can you elaborate a bit more on what exactly you are trying to do?</p>

apaszke: <p>You could do that something like that in the <code>forward</code> method. It will be a correct graph:</p>
<pre><code class="lang-python">def forward(self, x):
    x = self.module1(x)
    if (x.data &gt; 0).all():
        return self.module2(x)
    else:
        return self.module3(x)
</code></pre>
<p>I think we don’t support <code>all()</code> on Variables yet, but we should add that. In this case unpacking the data is safe. You can also use <code>any()</code>.</p>

t3476: <p>Problem sovled. Thanks for help.</p>

rahul: <p>Hi, does the condition work on a sample by sample basis? or does the condition apply across all the samples in batch? I am confused about this. Can you please explain it?</p>

madmaxiwnl: <p>I don’t think it works on a per sample basis. Can someone please provide a proper equivalent to tf.cond().</p>

ptrblck: <p>Could you post your use case so that we could have a look at it?<br>
Since you can use Python conditions in PyTorch, it’s a bit hard to provide an example other than what was already posted.</p>

madmaxiwnl: <p>Well, it’s something like this:</p>
<pre><code class="lang-auto">out_tensor1, out_tensor2, out_tensor3 = tf.cond(some_condition,
                         lambda: some_tensor1, some_tensor2, some_tensor3,
                         lambda: some_tensor4, some_tensor5, some_tensor6)
</code></pre>
<p>Currently, I am using a workaround I found on another thread:</p>
<pre><code class="lang-auto">def where(cond, f1, f2):
    return c * f1() + (1-c) * f2()
</code></pre>
<p>and finding all 3 out_tensors seperately. Though having an inbuilt operation in pytorch would be useful.</p>

ptrblck: <p>I’m not sure, how the TF code works exactly, but if it’s just an assignment, wouldn’t this work:</p>
<pre><code class="lang-python">if some_condition:
    out_tensor1, out_tensor2, out_tensor3 = some_tensor1, some_tensor2, some_tensor3
else:
    out_tensor1, out_tensor2, out_tensor3 = some_tensor4, some_tensor5, some_tensor6
</code></pre>

madmaxiwnl: <p>Sorry, in my previous post, the variable some_condition is actually a tensor of conditions, like (<code>tensorA &gt; tensorB</code>).<br>
And, the code I used is actually:</p>
<pre><code class="lang-auto">some_condition = some_condition.type(torch.LongTensor)
out_tensor1 = some_condition * some_tensor1 + (1-some_condition) * some_tensor2
</code></pre>
<p>and, similarly for out_tensor 2 and 3. I couldn’t find a function which would perform the operation for all 3 out_tensors at once. Anyways it’s working now, thanks.</p>

abediee: <p>Use</p>
<blockquote>
<p><a href="https://pytorch.org/docs/stable/generated/torch.where.html" rel="noopener nofollow ugc">torch.where</a></p>
</blockquote>

wiiiktor: <p>Could the tensor.unbind() function be used on .data, to work per sample basis?</p>
<p>EDIT: For everyone interested, after some more research, I am going to use batch_size = 1, use any condition that I only dream of <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=10" title=":wink:" class="emoji" alt=":wink:"> and later use this approach: <a href="https://medium.com/@davidlmorton/increasing-mini-batch-size-without-increasing-memory-6794e10db672" class="inline-onebox" rel="noopener nofollow ugc">Increasing Mini-batch Size without Increasing Memory | by David Morton | Medium</a> to achieve a larger batch for gradient backward passing</p>