Coredump when registering a dispatched Operator in C++ with cuda backend only

401qingkong: <p>I register an op in c++ with the reference of <a>https://pytorch.org/tutorials/advanced/dispatcher.html#for-operators-that-do-not-need-autograd</a>.I implement cuda device backend  only and the autograd is also impl in c++. But I notice the autograd kernel needs to <em>redispatch</em> : it needs to call back into the dispatcher to get to the inference kernels, e.g. CPU or CUDA implementations.</p>
<p>I complie th c++ extension successfully. But when I import the new python module , coredump occures. I debug it with gdb and the stack information show as the following:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bffd31bace9ec729744d68155e15d91d74242785.png" data-download-href="https://discuss.pytorch.org/uploads/default/bffd31bace9ec729744d68155e15d91d74242785" title="1673920966805"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bffd31bace9ec729744d68155e15d91d74242785.png" alt="1673920966805" data-base62-sha1="ropF6L7ESHlic5wDmMmrxTPsmB7" width="690" height="312" data-dominant-color="293832"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1673920966805</span><span class="informations">1665×754 93 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
torch_softmax.cpp is the file defining the autograd  kernel calling. part of its information just like this</p>
<pre><code class="lang-auto">#include "torch_softmax.h"

#include &lt;ATen/core/dispatch/Dispatcher.h&gt;
#include &lt;torch/library.h&gt;
#include &lt;torch/types.h&gt;

namespace at {
namespace native {

Tensor fuse_softmax_forward(const Tensor &amp;input, const int64_t dim, const bool half_to_float) {
  // C10_LOG_API_USAGE_ONCE("lightop.csrc.torch_softmax.fuse_softmax_forward");
  static auto op = c10::Dispatcher::singleton()
                       .findSchemaOrThrow("myops::fuse_softmax_forward", "")
                       .typed&lt;decltype(fuse_softmax_forward)&gt;();
  return op.call(input, dim, half_to_float);
}

namespace detail {

Tensor fuse_softmax_backward(const Tensor&amp; grad, const Tensor&amp; output, int64_t dim, const Tensor&amp; input) {
  static auto op =
      c10::Dispatcher::singleton()
          .findSchemaOrThrow("myops::fuse_softmax_backward", "")
          .typed&lt;decltype(fuse_softmax_backward)&gt;();
  return op.call(
      grad,
      output,
      dim,
      input);
}

} // namespace detail

TORCH_LIBRARY_FRAGMENT(myops, m) {
  m.def("fuse_softmax_forward(const Tensor &amp;input, const int64_t dim, const bool half_to_float) -&gt; Tensor");

  m.def("fuse_softmax_backward(const Tensor&amp; grad, const Tensor&amp; output, int64_t dim, const Tensor&amp; input) -&gt; Tensor");

}

} // namespace ops
} // namespace vision

</code></pre>
<p>I want to keep the autograd module with c++ implementation to improve performance. The stack info shows the op calls in jit method. I suspect this problem is related to the dynamic dispatch selection. But I don’t know how to call the definite cuda backend or how to debug it furthermore. Could someone give me some advice?</p>