Help with Efficient Negative Sampling

PopHSKE: <p>Im experimenting with negative sampling for a recommendation (ranking) task. Im hoping for some advice on how to most efficiently implement this is Pytorch.</p>
<p><strong>Is the best place to create negative samples in the data loader?</strong></p>
<p>My current attempt looks like below.</p>
<p><strong>array</strong> : is the raw data which are interactions - user and item ids that have been observed. There are around 5 million rows. There are around 600,000 users and 1500 items.</p>
<p><strong>sparse_user_item</strong> : This is a sparse matrix of the interaction data and I am using it to look up a user and get the indices for the items they have interacted with.</p>
<p><strong>max_user:</strong> This is just the highest index of users which i need to add to the item ids based on how the model is actually set up (users and items are both included in the same embedding matrix).</p>
<p><strong>Is there anything glaring that could be improved in how it is set up or high level, how the sampling is done?</strong></p>
<pre><code class="lang-auto">class BPR_NegSampleDataset(Dataset):
    def __init__(self, array, min_item_id, max_item_id, check_neg = False, sparse_user_item = None, max_user = None ):
        
        self.x = torch.LongTensor(array) #  user and item
        self.check_neg = check_neg
        self.min_item_id = min_item_id
        self.max_item_id = max_item_id
        
        self.array_items = np.arange(min_item_id, max_item_id)
        self.sparse_user_item = sparse_user_item
        self.max_user = max_user
        
    def __len__(self):
        return len(self.x)  # number interactions
    
    #https://medium.com/@2j/negative-sampling-in-numpy-18a9ad810385
    def negsamp_vectorized_bsearch_preverif(self, pos_inds, n_samp=1):
        """ Pre-verified with binary search
            `pos_inds` is assumed to be ordered
        """
        raw_samp = np.random.randint(self.min_item_id, self.max_item_id - len(pos_inds), size=n_samp)
        pos_inds_adj = pos_inds - np.arange(len(pos_inds))
        neg_inds = raw_samp + np.searchsorted(pos_inds_adj, raw_samp, side='right')
        return neg_inds
    

    def __getitem__(self, idx):
        
        pos_interation = self.x[idx] # userID and ItemID that was purchased
        
        if self.check_neg:
            pos_indxs = self.sparse_user_item[pos_interation[0]].nonzero()[1] + self.max_user +1
            neg_item = np.random.choice(np.setdiff1d(self.array_items, pos_indxs))
        
        else:
            neg_item =  torch.randint(self.min_item_id, self.max_item_id, size = (1,)) 
            
  
        neg_interaction = torch.hstack([pos_interation[0],torch.LongTensor(np.array([neg_item]))])

        return pos_interation, neg_interaction
</code></pre>
<p>Its very slow to confirm negatives are not actually positives for a user:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/6/566ec05d0d432786d8a2ec9096b97179f86a280f.png" data-download-href="https://discuss.pytorch.org/uploads/default/566ec05d0d432786d8a2ec9096b97179f86a280f" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/6/566ec05d0d432786d8a2ec9096b97179f86a280f.png" alt="image" data-base62-sha1="ckClt6xaVFOX6BhQ1opySpr4q75" width="689" height="262" data-dominant-color="F4F4F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1106×420 22.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>but even without checking its slow:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f90b02c882d5603665a95b98215efa6a99cd01c8.png" data-download-href="https://discuss.pytorch.org/uploads/default/f90b02c882d5603665a95b98215efa6a99cd01c8" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f90b02c882d5603665a95b98215efa6a99cd01c8.png" alt="image" data-base62-sha1="zx8tOioncTzL4LZRZgn7JNI35a0" width="690" height="245" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1102×392 21.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>