Two questions about THPVariable_to and tensor_ctor

jiangxiaobin96: <p>I met two questions when reading source code.<br>
First is about tensor_ctor function. As we all known, tensor_ctor run internal_new_from_data to create a new tensor. Like following source code, typeIdWithDefault and deviceOptional function will all set device_idx=2, so why here set device=2? If I want create a tensor and transfer to device 0 here, where will the code run?</p>
<pre><code class="lang-c++">auto new_tensor = internal_new_from_data(
        typeIdWithDefault(r, 2, dispatch_key), 
        r.scalartypeWithDefault(1, scalar_type),
        r.deviceOptional(2),
        data,
        /*copy_variables=*/true,
        /*copy_numpy=*/true,
        /*type_inference=*/type_inference,
        pin_memory);
</code></pre>
<p>Second question is also about <code>to</code> function. when I run tensor.to(0), function THPVariable_to will run in c++ level. And the function call chain here is THPVariable_to-&gt;dispatch_to. But here I can not find the next function after dispatch_to function.</p>
<pre><code class="lang-c++">tatic Tensor dispatch_to(const Tensor &amp; self, Device device, bool non_blocking, bool copy, c10::optional&lt;c10::MemoryFormat&gt; optional_memory_format) {
  pybind11::gil_scoped_release no_gil;
  return self.to(self.options().device(device).memory_format(optional_memory_format), non_blocking, copy);
} 
</code></pre>
<p>Because to function is a cudaMemcpyHostToDevice function, so I want to search the to fucntion from the ending to beginning. But only can find copy_ function(THPVariable_copy_). So what’s function after dispatch_to.</p>
<p>I just started learning the source code. Thanks for your reply.</p>

ptrblck: <p>Could you post the code reference where the device is hard-coded in the PyTorch code base?</p>
<p>The copy kernel can be found <a href="https://github.com/pytorch/pytorch/blob/236472b32a13889afacb2b1eb595a88a779f80a7/aten/src/ATen/native/cuda/Copy.cu#L101">here</a>.</p>

jiangxiaobin96: <p>Thanks for your reply. I will give more details about question.<br>
First question, when I run torch.tensor, the function call chain is <code>THPVariable_tensor-&gt;tensor_ctor-&gt; internal_new_from_data</code>.</p>
<pre><code class="lang-c++">Tensor internal_new_from_data(
    c10::TensorOptions options,
    at::ScalarType scalar_type,
    c10::optional&lt;Device&gt; device_opt,
    PyObject* data,
    bool copy_variables,
    bool copy_numpy,
    bool type_inference,
    bool pin_memory = false) {
</code></pre>
<p>The first and third input is options and device_opt which is related to <code>typeIdWithDefault(r, 2, dispatch_key)</code> and <code>r.deviceOptional(2)</code>. Here we can see there are both value <code>2</code>.</p>
<pre><code class="lang-c++">c10::TensorOptions typeIdWithDefault(
    PythonArgs&amp; r,
    int64_t device_idx,
    c10::DispatchKey dispatch_key) {
  auto options = dispatchKeyToTensorOptions(dispatch_key);
  if (!r.isNone(static_cast&lt;int&gt;(device_idx))) {
    // TODO: This line doesn't seem to be exercised at all in tests
    options = options.device(r.device(static_cast&lt;int&gt;(device_idx)).type());
  }
  return options;
}

inline c10::optional&lt;at::Device&gt; PythonArgs::deviceOptional(int i) {
  if (!args[i])
    return c10::nullopt;
  return device(i);
}
</code></pre>
<p>As the source code above, device_idx=2 will be set to device(2) which will finally influence the tensor.to(device) in internal_new_from_data.</p>
<pre><code class="lang-c++">auto device = device_opt.has_value() ? *device_opt : options.device();
...
tensor = tensor.to(
        device, inferred_scalar_type, /*non_blocking=*/false, /*copy=*/false);
</code></pre>
<p>So why here we set device_idx=2?</p>
<p>About the copy kernel, I found the function call chain: <code>THPVariable_copy_-&gt;dispatch_copy_-&gt;copy_-&gt;copy_impl-&gt;copy_stub-&gt;copy_kernel_cuda</code>. But here start with <code>THPVariable_copy_</code>. How about <code>THPVariable_to</code>, I think it is also a function which use to copy data HTD or DTH. I just find the function call chain: <code>THPVariable_to-&gt;dispatch_to</code>, and what is the next function next to dispatch_to?</p>

jiangxiaobin96: <p><code>THPVariable_to-&gt;dispatch_to-&gt;Tensor to(aten/src/Aten/native/TensorConversions.cpp)-&gt;to_impl-&gt;_to_copy-&gt;copy_</code>. Finally THPVariable_to run copy_ function similar with THPVariable_copy_.</p>

jiangxiaobin96: <p>I found tensor_ctor use default dispatch key and default_backend is DispatchKey::CPU. So does this mean creating a new tensor will store data in CPU memory first when running torch.tensor?</p>
<pre><code class="lang-c++">torch::utils::tensor_ctor(
      torch::tensors::get_default_dispatch_key(),
      torch::tensors::get_default_scalar_type(),
      r)
</code></pre>