Sample from tensor

username: Marco_Piccoli
cooked: <p>Hi everyone <img src="https://discuss.pytorch.org/images/emoji/apple/wave/3.png?v=12" title=":wave:t3:" class="emoji" alt=":wave:t3:" loading="lazy" width="20" height="20"></p>
<p>I need to create a layer that samples data from a tensor that keeps the computation graph in order to backpropagate properly.<br>
More precisely, I have objects represented as point-cloud from the ModelNet dataset and I have to draw some points from the cloud.</p>
<p>The layer I came up with is:</p>
<pre><code class="lang-auto">class Sampler(nn.Module):
    def __init__(self, N: int):
        super(Sampler, self).__init__()
        # Number of points to extract
        self.N = N
    
    def forward(self, x):
        r''' source: https://discuss.pytorch.org/t/take-random-sample-of-long-tensor-create-new-subset/36244/3
        author: @rasbt
        '''
        rand_columns = torch.randperm(x.shape[1])[:self.N]
        out = x.clone().detach()[:, rand_columns, :].requires_grad_(True)
        return out
</code></pre>
<p>Where x is a tensor of shape [batch_size, n_points, n_features] and N &lt; n_points<br>
So the out tensor is of shape [batch_size, N, n_features]</p>
<p>Everything seems fine but I keep getting the <strong>UserWarning</strong> alert about copying tensors:<br>
UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor)<br>
even tho I tried to use the clone-detach on the x tensor. What am I missing here?</p>
<p>The source for the forward function is taken from a reply by <a class="mention" href="/u/rasbt">@rasbt</a></p><aside class="quote quote-modified" data-post="3" data-topic="36244">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/90db22/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/take-random-sample-of-long-tensor-create-new-subset/36244/3">Take random sample of long tensor, create new subset</a> 
  </div>
  <blockquote>
    <a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/9/9095d6af7e012afda56825501733506a50f49131.png" data-download-href="https://discuss.pytorch.org/uploads/default/9095d6af7e012afda56825501733506a50f49131" title="outputs_hw1.png">[outputs_hw1]</a> 
ouputs are from final layer and are of shape 1000x10. outputs 2 are pulled 2nd from last layer and are of shape 1000x1024. I want to randomly sample out of my outputs2 to get 1000x10. 
I tried this, and it looks like tensors wrapped in a tensor… 
 <a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/ca690354a937b28ad8e961f9626ac970d3fed3c4.png" data-download-href="https://discuss.pytorch.org/uploads/default/ca690354a937b28ad8e961f9626ac970d3fed3c4" title="output_test4.png">[output_test4]</a>
  </blockquote>
</aside>


username: rasbt
cooked: <p>Also not sure why you’d get the warning. What you are doing seems to be right. Maybe it’s just a  false positive warning, I dunno. What you could try is</p>
<pre><code class="lang-python">out = torch.tensor(x.clone().detach()[:, rand_columns, :], requires_grad=True)
</code></pre>
<p>instead of</p>
<pre><code class="lang-python">out = x.clone().detach()[:, rand_columns, :].requires_grad_(True)
</code></pre>