Implementing a low level function

username: Sia_Rezaei
cooked: <p>Hi,</p>
<p>I would like to work on this issue:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/67745">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/67745" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/67745" target="_blank" rel="noopener nofollow ugc">A `sieve` operation for separating values of a tensor into two tensors based on a condition.</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-11-03" data-time="03:43:59" data-timezone="UTC">03:43AM - 03 Nov 21 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/siarez" target="_blank" rel="noopener nofollow ugc">
          <img alt="siarez" src="https://avatars.githubusercontent.com/u/636471?v=4" class="onebox-avatar-inline" width="20" height="20">
          siarez
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          enhancement
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: ux
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: viewing and reshaping
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🚀 Feature
This is an operation that is almost the inverse of `where`. 
Let'<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">s call it `sieve`.
It takes as input a tensor `A` and a boolean tensor `B` of the same shape and a "fill" value `c`.
It returns two tensors `O1` and `O2` of the same shape as `A`.
`O1` is equal to `A` where `B` is True and is equal `c` where `B` is False.
`O2` is equal to `A` where `B` is False and is equal `c` where `B` is True.

For example, let's say you want to separate positive and non-positive elements of `A` into two tensors.:
``` pos, neg = torch.sieve(A, B=(X &gt; 0), fill=0) ```

## Motivation
This can be made twice faster than current solutions.

## Pitch
This obviously can already be done in various ways in PyTorch currently.
But it can be theoretically be twice as fast when done in one operation.
That is because **ALL** current ways of achieving this have to go over `A` twice. 
Once to get `O1` and once to get `O2`. 
But we know if something does not go to `O1`, is has to go to `O2`.
Therefore, we can do this in one pass of `A`.

## Alternatives

For the example mentioned above, one current solution is to do.
```
neg = torch.minimum(X, 0)
pos = torch.maximum(X, 0)
```
Here, we are comparing each element of X with zero twice, but we don't have to.

## Context
I use this in a custom activation function. So having this operation means my activation functions will be 2x faster, and that makes my runs noticeably faster.

I'm willing to dedicate some time to this, if someone is willing to guide me, as I haven't written custom CPU or Kernel code.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>It is a simple function, but I think it needs to be implement at a very low level, since none of the primitives in the current API can address the issue.</p>
<p>Looking at the implementation of the <code>where</code> and <code>relu</code> functions should be helpful. But I can’t quite located their <strong>low-level</strong> implementations in the repo. It would be great if someone could point them out to me.</p>
<p>Any other pointers are also much appreciated.</p>

username: InnovArul
cooked: <p>Maybe start creating it as a pytorch extension?<br>
<a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" class="onebox" target="_blank" rel="noopener">https://pytorch.org/tutorials/advanced/cpp_extension.html</a></p>
<p>It doesn’t need to be written inside core pytorch in the first go,  I guess.</p>

username: Sia_Rezaei
cooked: <p>Can you point me to the implementation of <code>where</code> and <code>relu</code> functions?</p>