How to pass Python device and dtype to C++

G.M: <p>I’m trying to implement a C++ function that has the arguments of a Device and Dtype and will be called in Python. But I got error trying to achieve this.<br>
Here’s my C++ code:</p>
<pre><code class="lang-auto">#define tensor torch::Tensorv
tensor fun1(tensor x,torch::Device device,torch::Dtype dtype) {
	auto opt = torch::TensorOptions(device).dtype(dtype);
	return x.to(opt);
}
</code></pre>
<p>Here’s the Python part:</p>
<pre><code class="lang-auto">a=torch.tensor([1])
fun1(a,torch.device('cpu'),torch.float)
</code></pre>
<p>Here’s the error:</p>
<pre><code class="lang-auto">    fun1(a,torch.device('cpu'),torch.float)
TypeError: fun1(): incompatible function arguments. The following argument types are supported:
    1. (arg0: at::Tensor, arg1: c10::Device, arg2: c10::ScalarType) -&gt; at::Tensor

Invoked with: tensor([1]), device(type='cpu'), torch.float32
</code></pre>

tom: <p>I don’t think this works directly right now.</p>
<p>If you have compiled torch yourself, you see that in torch/csrc/autograd/generated/python_torch_functions.cpp the factory functions take the Python objects and unpacks them.</p>
<p>The PyBind converters are only present for IntArrayRef and Tensor (<a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/utils/pybind.h" rel="nofollow noopener">toch/csrc/utils/pybind.h</a>). You could try to register your own for torch::Device and torch::Dtype – or submit that PR (1.5 years ago, it was only Tensors).</p>
<p>A standard workaround could be to just pass a (potentially empty) tensor and take the device and dtype of that.</p>
<p>Best regards</p>
<p>Thomas</p>

G.M: <p>Ok, thank you very much. I’ll try that.</p>

G.M: <p>Actually, I found a way to pass dtype to C++.<br>
In TouchScript(JIT), <code>torch.dtype</code> type  are treated as python <code>int</code>, which corressponds to what happens in C++, as the dtypes are all <code>ScalarType</code>. So I can just create an C++ function that takes in an integer and make it an <code>ScalarType</code> to pass the dtype. But this method can only be called in TouchScript because only in TouchScript are dtypes treated as python <code>int</code>.</p>

kaihsinwu: <p>It’s just an follow-up. I recently encounter the similar situation. The workaround solution that I did is to use the str() to cast the torch.dtype into a C++ string, and in the python end write a wrapper around it:</p>
<p>Here is the C++ part:</p>
<pre><code class="lang-auto">torch::Tensor func1(const vector&lt;int64_t&gt; &amp;shape, const std::string &amp;dtype){
   // convert the string -&gt; torch::ScalarType here. 
   auto options = torch.TensorOptions();
   if(dtype=="torch.float32"){
       options.dtype(torch::kFloat32);
   } // ... extend this

   return torch::Tensor(shape,options);

}

</code></pre>
<p>Here is the pybind wrapper part :</p>
<pre><code class="lang-auto">PYBIND11_MODULE(cytnx,m){
    m.def("c_func1",&amp;func1);
}
</code></pre>
<p>Here is the python module in the directory structure:<br>
cytnx/<br>
|- __ init __.py<br>
|- [the dynamic linking file compiled from pybind]</p>
<p>Inside __ init __.py:</p>
<pre><code class="lang-auto">from .cytnx import *

## this is the API expose to user
def func1(shape, dtype):
     # calling the wrapper from pybind
     return c_func1(shape,str(dtype),str(device))
</code></pre>
<p>I would hope there will be a direct support over this in the future</p>

kaihsinwu: <p>Another way to do this, which I realized later-on is you can pass the py::object, and then cast to the torch::Dtype (torch::ScalarType):</p>
<pre><code class="lang-auto">PYBIND11_MODULE(cytnx,m){
    m.def("func1",[](py::object dtype){
                   torch::ScalarType type = torch::python::detail::py_object_to_dtype(dtype);
             });
}
</code></pre>

tom: <p>Thank you for sharing!</p>

esoha-nvidia: <p>If you cast the object to a dtype, you should first check that the conversion is valid.</p>
<p>if (THPDtype_Check(obj.ptr())) {<br>
torch::ScalarType scalar_type = reinterpret_cast&lt;THPDtype*&gt;(obj.ptr())-&gt;scalar_type;<br>
}</p>