How to return a boolean tensor depending on condition?

username: Aldebaran
cooked: <p>Let the cartesian product between a set of <code>query_idx</code> and another set of <code>doc_idx</code> as shown below:</p>
<pre><code class="lang-python">torch.cartesian_prod(preds["query_idx"], preds["doc_idx"])
# tensor([
#     [0, 9], [0, 5], [0, 7], [1, 9], [1, 5], [1, 7], [2, 9], [2, 5], [2, 7]
# ])
</code></pre>
<p>Also consider the relevance map, stating which documents are relevant in relation to a given query:</p>
<pre><code class="lang-python">relevance_map = {
    0: [5,9], # the docs 5 and 9 are relevant results concerning the query 0.
    1: [9],
    2: [5,7]
}
</code></pre>
<p>Then, how to generate a new tensor classifying each pair of the cartesian product as <code>TRUE</code> (relevant) or <code>False</code> (non-relevant) according to the relevance map?</p>
<pre><code class="lang-python">torch.tensor([
    True, True, False, True, False, False, False, True, True
])
</code></pre>

username: Andrei_Cristea
cooked: <p>Hi Aldebaran, you could try this:</p>
<pre><code class="lang-auto">torch.Tensor([x[1].item() in relevance_map.get(x[0].item(), []) for x in cartesian_prod]).bool()
Output:
tensor([ True,  True, False,  True, False, False, False,  True,  True])
</code></pre>

username: Aldebaran
cooked: <p>It is a solution. However, it’s not differentiable because it uses the <code>.item()</code> operation.</p>

username: Andrei_Cristea
cooked: <p>I’m not sure that this type of result is differentiable, since it’s so discontinuous. I think you’d have to define a continuous function that somehow captures how close a pair in your <code>cartesian_prod</code> is from being relevant (say, with 1 for totally irrelevant and 0 if in the relevant map, <em>and somehow smooth in between</em>) and run .sum().backward() on that. Basically I think you need to extend this discrete math problem to a continuous case, in some way, and then differentiate that.</p>
<p>Personally I’m not sure how to do that.</p>
<p>Hopefully I’m wrong and there’s a far simpler and more direct way to do this, and someone more knowledgeable here can help you out.</p>
<p>EDIT: Thinking about this more, one way to convert this to a differentiable problem would be to make your output a matrix shaped like (number_of_docs x number_of_queries). Its entries would be floats in the (0, 1) range. Your target would be a similarly shaped matrix whose values are either 0 or 1 depending on whether that pair of (document, query) is in the relevance map or not. You can run some type of binary cross entropy loss between these two matrices. Of course, you could run into memory problems if you have lots of docs and queries but your relevance map is very sparse, etc.</p>

username: Matias_Vasquez
cooked: <p>Hi,</p>
<p>I am assuming that by looking for a differentiable solution, you mean that the resulting tensor has <code>require_grad = True</code> in the end. If so, then we have a problem because you want a boolean tensor.</p>
<p>If we look at the <a href="https://pytorch.org/docs/stable/autograd.html" rel="noopener nofollow ugc">documentation for autograd</a> we can see that only floating point tensors are supported.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/7/47940c3e67216901c58fb2062c3c913612363694.png" data-download-href="https://discuss.pytorch.org/uploads/default/47940c3e67216901c58fb2062c3c913612363694" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/7/47940c3e67216901c58fb2062c3c913612363694.png" alt="image" data-base62-sha1="add5FngJtzRvRhniPPHySjUWh24" width="690" height="164" data-dominant-color="DFE8F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">850×203 23.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>If this is the case, then I don´t think it is going to be possible to get what you want (a differentiable boolean tensor).</p>
<p>However, let me try and fail with extra steps:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/9/19a606621be9e1eb05586c7a3ee8cbb618e22a03.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/19a606621be9e1eb05586c7a3ee8cbb618e22a03" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19a606621be9e1eb05586c7a3ee8cbb618e22a03_2_333x250.jpeg" alt="image" data-base62-sha1="3ETDTMiM4mA5pjUIX82gVdi47SP" width="333" height="250" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19a606621be9e1eb05586c7a3ee8cbb618e22a03_2_333x250.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19a606621be9e1eb05586c7a3ee8cbb618e22a03_2_499x375.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19a606621be9e1eb05586c7a3ee8cbb618e22a03_2_666x500.jpeg 2x" data-dominant-color="A69FA1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">720×540 92.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<h2><a name="assuming-1" class="anchor" href="#assuming-1"></a>Assuming</h2>
<ul>
<li>You don´t care that <code>relevance_map</code> is a <code>dict</code></li>
</ul>
<p>We can change it into a tensor, such that all end nodes have the same size. For this, I added <code>-inf</code> where the vectors are too short.</p>
<pre data-code-wrap="python"><code class="lang-python">cartesian_prod = torch.tensor([[0., 9], [0, 5], [0, 7], [1, 9], [1, 5], [1, 7], [2, 9], [2, 5], [2, 7]], requires_grad=True)
relevance_map = torch.tensor([[5,9],[-float('inf'), 9],[5,7]])
</code></pre>
<h2><a name="solution-1-no-grad-2" class="anchor" href="#solution-1-no-grad-2"></a>Solution 1 - no grad</h2>
<p>We get a boolean tensor, however, <code>grad</code> is lost when we do the equal (<code>==</code>) operation. If this was not the case, <code>any</code> will also remove it.</p>
<pre data-code-wrap="python"><code class="lang-python">tmp = torch.any(relevance_map[cartesian_prod[:, 0].long()] == cartesian_prod[:, 1].unsqueeze(0).T, dim=1)
print(tmp)
print(tmp.requires_grad)
</code></pre>
<pre data-code-wrap="python"><code class="lang-python"># tensor([ True,  True, False,  True, False, False, False,  True,  True])
# False
</code></pre>
<h2><a name="solution-2-not-boolean-and-weird-format-3" class="anchor" href="#solution-2-not-boolean-and-weird-format-3"></a>Solution 2 - not boolean (and weird format)</h2>
<p>Here all opperations should be differentiable. The problem is that the output is a float tensor, where <code>0</code> means <code>True</code> and anything other than <code>0</code> is <code>False</code>. (as I said, weird format)</p>
<pre data-code-wrap="python"><code class="lang-python">tmp2, _ = torch.abs(relevance_map[cartesian_prod[:, 0].long()] - cartesian_prod[:, 1].unsqueeze(0).T).min(dim=1)
print(tmp2)
print(tmp2.requires_grad)
</code></pre>
<pre data-code-wrap="python"><code class="lang-python"># tensor([0., 0., 2., 0., 4., 2., 2., 0., 0.], grad_fn=&lt;MinBackward0&gt;)
# True
</code></pre>
<h2><a name="solution-3-even-more-unnecessary-steps-still-no-boolean-4" class="anchor" href="#solution-3-even-more-unnecessary-steps-still-no-boolean-4"></a>Solution 3 - even more unnecessary steps - still no boolean</h2>
<p>Using <code>tmp2</code> from the last solution.<br>
Here <code>1</code> means <code>True</code> and <code>0</code> is <code>False</code>. (that´s better)</p>
<pre data-code-wrap="python"><code class="lang-python">div = tmp2.clone()
div[div==0] = 1

tmp3 = -torch.div(tmp2, div) + 1
print(tmp3)
print(tmp3.requires_grad)
</code></pre>
<pre data-code-wrap="python"><code class="lang-python">tensor([1., 1., 0., 1., 0., 0., 0., 1., 1.], grad_fn=&lt;AddBackward0&gt;)
True
</code></pre>