Intersection between to vectors/tensors

supersymmetry: <p>Can pytorch find the intersection of two vectors?</p>
<p>Similar to the Matlab “intersection” function:<br>
<a href="https://www.mathworks.com/help/matlab/ref/double.intersect.html;jsessionid=0f4b1f0e0ff110c50bb8a0076587" class="onebox" target="_blank" rel="nofollow noopener">https://www.mathworks.com/help/matlab/ref/double.intersect.html;jsessionid=0f4b1f0e0ff110c50bb8a0076587</a></p>
<p>What about other set operations? e.g., union, setdiff?</p>
<p>Thanks</p>

jetcai1900: <p>Do you have any answers? Thanks.</p>

WangXin93: <p>I used <a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.intersect1d.html" rel="nofollow noopener"><code>np.intersect1d</code></a> like:</p>
<pre><code class="lang-python">&gt;&gt;&gt; import torch
&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; a = torch.tensor([1, 2, 3, 6])
&gt;&gt;&gt; b = torch.tensor([0, 2, 3, 7])
&gt;&gt;&gt; np.intersect1d(a, b)
array([2, 3])
</code></pre>
<p>It seems that tensorflow 2.0 has a function named <a href="https://www.tensorflow.org/api_docs/python/tf/sets/intersection" rel="nofollow noopener"><code>tf.sets.intersection</code></a></p>

braindotai: <p>I’ve also been trying to figure it out lately, and I think I’ve found a good solution. And it works without numpy, so no need to transfer your tensors to cpu (which is really slower if while training models). Check this out</p>
<pre><code class="lang-python">&gt;&gt;&gt; import torch
&gt;&gt;&gt; a = torch.tensor([1, 2, 3, 6]).cuda()
&gt;&gt;&gt; b = torch.tensor([0, 2, 3, 7]).cuda()
&gt;&gt;&gt; intersection = (a * (a == b).float()).nonzero().flatten()
&gt;&gt;&gt; intersection
tensor([1, 2], device='cuda:0')
</code></pre>

alokpathy: <p>I think this only works if the matching elements are at the same indices. If I rearrange  <code>a</code>, I’m getting</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; a = torch.tensor([6, 1, 2, 3]).cuda()
&gt;&gt;&gt; b = torch.tensor([0, 2, 3, 7]).cuda()
&gt;&gt;&gt; intersection = (a * (a == b).float()).nonzero().flatten()
&gt;&gt;&gt; intersection
tensor([], device='cuda:0')
</code></pre>
<p>I’ve had some success with the second answer on this SO post: <a href="https://stackoverflow.com/questions/55110047/finding-non-intersection-of-two-pytorch-tensors" rel="nofollow noopener">https://stackoverflow.com/questions/55110047/finding-non-intersection-of-two-pytorch-tensors</a> . The <code>repeat</code> call isn’t ideal memory-wise but it works at least.</p>

alokpathy: <p>You can also modify the first answer like so to get intersection</p>
<pre><code class="lang-auto">&gt;&gt;&gt; t1 = torch.tensor([1, 9, 12, 5, 24], device = 'cuda')
&gt;&gt;&gt; t2 = torch.tensor([1, 24], device = 'cuda')
&gt;&gt;&gt; indices = torch.zeros_like(t1, dtype = torch.uint8, device = 'cuda')
&gt;&gt;&gt; for elem in t2:
&gt;&gt;&gt;     indices = indices | (t1 == elem)  
&gt;&gt;&gt; intersection = t1[indices]  
</code></pre>

braindotai: <p>That’s better. Thanks</p>

Deeply: <aside class="quote no-group" data-username="alokpathy" data-post="5" data-topic="50364">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alokpathy/48/22559_2.png" class="avatar"> alokpathy:</div>
<blockquote>
<p>h the second ans</p>
</blockquote>
</aside>
<p>Not sure if this would help, as the code hoovers over all <code>t2</code> elements in a for-loop. Hence, would not benefit from the GPU. In fact, <code>numpy</code> intersect is much faster.</p>
<pre><code class="lang-auto">def tensor_intersect(t1, t2):
    t1=t1.cuda()
    t2=t2.cuda()
    indices = torch.zeros_like(t1, dtype = torch.bool, device = 'cuda')
    for elem in t2:
        indices = indices | (t1 == elem)  
        intersection = t1[indices]  
    return intersection
</code></pre>
<pre><code class="lang-auto">t1= np.random.randint( 1,1e9, 10000)
t2= np.random.randint( 1,1e9, 10000)
tic = time.time()
np.intersect1d(t1, t2) 
print(time.time()-tic)
0.0009970664978027344

tic = time.time()
tensor_intersect( torch.tensor(t1), torch.tensor(t2))
print(time.time()-tic)
1.426218032836914
</code></pre>
<p>NB. indices should be changed to:<br>
<code>indices = torch.zeros_like(t1, dtype = torch.bool, device = 'cuda')</code></p>

Deeply: <p>Here’s a tweak that’s 15 to 20 times faster than <code>numpy intersect</code> (for large sets). For small sets, it is better to work with <code>numpy</code>, until someone writes a better <code>torch</code> algorithm based on search:</p>
<pre><code class="lang-auto">
t1= torch.randint(0, 99999, (1000000,))
t2= torch.randint(0, 99999, (100000000,))

def torch_intersect(t1, t2, use_unique=False):
    t1 = t1.cuda()
    t2 = t2.cuda()
    t1 = t1.unique()
    t2 = t2.unique()
        
    return torch.tensor(np.intersect1d(t1.cpu().numpy(), t2.cpu().numpy()))


def torch_intersect2(t1, t2, use_unique=False):
    t1 = t1.cuda()
    t2 = t2.cuda()
    t1 = t1.unique()
    t2 = t2.unique()
    t1=set(t1.cpu().numpy())
    t2=set(t2.cpu().numpy())    
        
    return t1.intersection(t2)

tic= time.time()
res = np.intersect1d(t1.cpu().numpy(), t2.cpu().numpy())    
print('Numpy Intersect Time:', time.time()-tic)

    
tic= time.time()
res = torch_intersect(t1, t2)
print('Tensor Intersect Time:', time.time()-tic)
</code></pre>
<pre><code class="lang-auto">Numpy Intersect Time: 6.87971830368042
Tensor Intersect Time:  0.3944394588470459
</code></pre>

shinstra: <p>Nifty solution I worked out today:</p>
<pre><code class="lang-auto">import torch
a = torch.tensor([1, 2, 3, 6]).cuda()
b = torch.tensor([0, 2, 3, 4]).cuda()
a_cat_b, counts = torch.cat([a, b]).unique(return_counts=True)
intersection = a_cat_b[torch.where(counts.gt(1))]
</code></pre>
<p>a and b can be of different lengths and common values can be at different positions.</p>

Amin_Jun: <p>For a one line answer I would do</p>
<pre><code class="lang-auto">import torch
first = torch.Tensor([1, 2, 3, 4, 5, 6])
second = torch.Tensor([7, 3, 9, 1])
intersection=first[(first.view(1, -1) == second.view(-1, 1)).any(dim=0)]
</code></pre>

Yuhao_Huang: <p>Thanks for sharing. Your solution could even expand to N-dim vectors, e.g., 3D coordinates. In addition, the index of the intersection could also be acquired in this way, if the <code>return_inverse</code> is set to True in the <code>torch.unique</code> function.</p>

JuyiLin: <p>This way is  100- 1000 times faster than  nid1[(nid1.view(1, -1) == nid2.view(-1, 1)).any(dim=0)] in my case</p>

jwillette: <p>This is super slick. Thanks for sharing</p>