Error when building an extension

username: Eta_C
cooked: <p>Here’s my code.</p>
<p>file: <code>q_cu.cpp</code></p>
<pre><code class="lang-cpp">#include &lt;torch/torch.h&gt;
#include &lt;cmath&gt;

void run_cu(at::Tensor&amp; out, const at::Tensor&amp; x, int bw, int fl);

at::Tensor run(const at::Tensor x, int bw, int fl) {
    at::Tensor out = at::empty(x.sizes(), x.options());
    run_cu(out, x, bw, fl);
    return out;
}

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
    m.def("run", &amp;run, "run");
}
</code></pre>
<p>file: <code>q_cu_core.cu</code></p>
<pre><code class="lang-cpp">#include &lt;ATen/ATen.h&gt;
#include &lt;ATen/cuda/CUDAApplyUtils.cuh&gt;
#include &lt;cmath&gt;

void run_cu(at::Tensor&amp; out, const at::Tensor&amp; x, int bw, int fl) {
    int64_t q_max = (1 &lt;&lt; (bw - 1)) - 1;
    int64_t q_min = -(1 &lt;&lt; (bw - 1));
    float scale = pow(2.0, fl);
    float inv_scale = 1.0 / scale;
    at::cuda::CUDA_tensor_apply2&lt;float, float&gt;(
        x, out, [=] __device__(const float&amp; src, float&amp; dst) {
            dst =(fminf(q_max,
                        fmaxf(q_min,
                              static_cast&lt;int64_t&gt;(std::round(val * scale))
                        )
                 ) * inv_scale;
        }
    );
}
</code></pre>
<p>file: <code>setup.py</code></p>
<pre><code class="lang-python">from setuptools import setup
from torch.utils.cpp_extension import CUDAExtension, BuildExtension

setup(
    name="my_method",
    ext_modules=[
      CUDAExtension("my_method2",
                    ["q_cu.cpp", "q_cu_core.cu"],
                    extra_compile_args=["-expt-extened-lambda"])
    ]
    cmdclass={"build_ext": BuildExtension}
)
</code></pre>
<p>There is no error when I compile this extension. But if I import it,</p>
<pre><code class="lang-python">import torch
import my_method2

# output
# ImportError...
# undefined symbol: 
# _ZN2at6native6legacy4cuda27_th_copy_ignoring_overlaps_ERNS_6TensorERKS3_
</code></pre>
<p>How can I sovle this problem?</p>

username: Eta_C
cooked: <p>BTW, I just want to make an extension like <code>fake_quantize_slice_cuda</code>, which use <code>std::round</code> instead of <code>std::nearbyint</code>.<img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=9" title=":sob:" class="emoji" alt=":sob:"></p>

username: Eta_C
cooked: <p>It seems that <code>at::native::legacy::cuda::_th_copy_ignoring_overlaps_</code> is not found.<br>
My PyTorch version is 1.3.1.</p>

username: Eta_C
cooked: <p>Does it mean I should not use <code>at::cuda::CUDA_tensor_apply2</code> in my extension?</p>

username: Eta_C
cooked: <p>Finally, I found this topic: <a href="https://discuss.pytorch.org/t/cuda-tensor-apply-in-extension-gives-undefined-symbol/56736" class="inline-onebox">CUDA_tensor_apply in extension gives undefined symbol</a></p>
<p>As I speculated above, this problem could be solved after I replace the <code>_th_copy_ignoring_overlaps_</code> with <code>at::_copy_from</code>. (I will verify its correctness later <img src="https://discuss.pytorch.org/images/emoji/apple/mask.png?v=9" title=":mask:" class="emoji" alt=":mask:">)</p>

username: TomB
cooked: <p>Yes, I think it should be fine to replace with a standard copy function (I used <code>Tensor::copy_</code> as you can see <a href="https://github.com/thomasbrandon/mish-cuda/commit/56fd30d9581abab12e565e2e715b2a8102944a36" rel="nofollow noopener">here</a> which seems to work fine, I explicitly tested overlapping tensors where it’s used).<br>
I believe the need for <code>_th_copy_ignoring_overlaps_</code> is a legacy thing that doesn’t apply anymore. In the original THC code the copy functions used the tensor apply function from which <code>CUDA_tensor_apply</code> is derived. So if the tensor apply function called copy then you’d get an infinite loop. Thus it calls <code>_th_copy_ignoring_overlaps</code> which avoids this recursion through a bit of a hack. In aten the copy functions now use a different internal pointwise apply function so there is no infinite loop and standard copy functions can be used.</p>

username: Eta_C
cooked: <p>Thanks for your analysis. I have roughly checked your <a href="https://github.com/thomasbrandon/mish-cuda" rel="nofollow noopener">repository</a> and use <code>copy_</code> instead of <code>_th_copy_ignoring_overlaps_</code> in my code.</p>
<p>But now another <a href="https://discuss.pytorch.org/t/whats-the-difference-between-cuda-and-to-device/64488">problem</a> arises. My extension works fine only when the tensor is in the current device.</p>
<pre><code class="lang-python">y = fun(x.to(0))  # work fine
y = fun(x.to(1))  # raise error
</code></pre>
<p>I’m not asking you to solve my problem. I will make a careful comparison and do more test later.</p>