Generating and Calling custom C++ LibTorch Functions in Python script via SetupTools/CMakesLists.txt

Agnij_Dave: <p>In General, I need a shared library file generated (<strong>.so</strong>) which I can load using &lt;<strong>torch.ops.load_library()</strong>&gt; function from a python script.</p>
<p><strong>Working on Windows 10: 64 bit-operating system</strong></p>
<p>Tried 2 approaches- Neither generate this File</p>
<p><strong>1&gt;</strong> CMakesList.txt - The build is succesful and I get a <strong>.dll and .lib</strong> file which I tried loading using <strong>python ctypes - cdll.load_library()</strong>, but no functions present, cross checked using <strong>dumpbin via vs code.</strong></p>
<pre><code class="lang-auto">cmake_minimum_required (VERSION 3.8)
    find_package(Torch REQUIRED)
    project(reductionResNet VERSION 1.0 DESCRIPTION "Deep_Learning")
    
    # add_executable (customOperator "customOperator.cpp" "customOperator.h")
    # Define our library target
    add_library(customOperator SHARED customOperator.cpp)
    # Enable C++14
    target_compile_features(customOperator PRIVATE cxx_std_14)
    # Link against LibTorch
    target_link_libraries(customOperator "${TORCH_LIBRARIES}")
    
    if (MSVC)
    	file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
    	add_custom_command(TARGET customOperator
    						POST_BUILD
    						COMMAND ${CMAKE_COMMAND} -E copy_if_different
    						${TORCH_DLLS}
    						$&lt;TARGET_FILE_DIR:customOperator&gt;)
    endif (MSVC)
</code></pre>
<p><strong>2&gt;</strong> Python Setuptools.py - Throws the error → LINK : error LNK2001: unresolved external symbol PyInit_reduction.</p>
<pre><code class="lang-auto"> from setuptools import setup, Extension
    from torch.utils import cpp_extension
    
    setup(name='customOperator',
          ext_modules=[cpp_extension.CppExtension('customOperator', ['customOperator.cpp'],
          include_dirs = ['C:/Libtorch/libtorch-win-shared-with-deps-1.8.1+cpu/libtorch'])],
          cmdclass={'build_ext':cpp_extension.BuildExtension.with_options(no_python_abi_suffix=True)}  )
</code></pre>
<p>These are the tutorials I have been following <a href="https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html#building-with-cmake" rel="noopener nofollow ugc">pytorchDocs</a> &amp; <a href="https://github.com/onnx/tutorials/tree/master/PyTorchCustomOperator" rel="noopener nofollow ugc">gitTutorial</a></p>
<p>This is the <strong>Tested .cpp</strong> file that holds the two functions - <strong>reduction and repeatInterleave</strong></p>
<pre><code class="lang-auto"> #include "customOperator.h"
    #include &lt;torch/torch.h&gt;
    
    using namespace std;
    
    torch::Tensor repeatInterleave(
        torch::Tensor input,
        int val
    ) {
        auto output_ = torch::repeat_interleave(input, val);
        return output_;
    }
    
    torch::Tensor reduction(
        torch::Tensor layerOne,
        torch::Tensor layerTwo,
        torch::Tensor layerThree,
        torch::Tensor layerFour) {
    
    
        auto layerOne_ = repeatInterleave(layerOne, 8);
        auto layerTwo_ = repeatInterleave(layerTwo, 4);
        auto layerThree_ = repeatInterleave(layerThree, 2);
    
        int len = layerFour.sizes()[0];
        //cout &lt;&lt; len &lt;&lt; endl;
    
        torch::Tensor arr[512] = {};
    
        //torch::Tensor* arr = new torch::Tensor[len];
        //std::vector&lt;std::string&gt; x = { "a", "b", "c" };
        //x.push_back("d");
        //std::vector &lt;torch::Tensor&gt; arr = {};
    
        for (int i = 0; i &lt; len; i += 1) {
            arr[i] = (layerOne_[i] + layerTwo_[i] + layerThree_[i] + layerFour[i]) / 4;
    
            //arr.push_back((layerOne_[i] + layerTwo_[i] + layerThree_[i] + layerFour[i]) / 4);
            //cout &lt;&lt; arr[i] &lt;&lt; endl;
        }
    
        //cout &lt;&lt; arr &lt;&lt; endl;
        //torch::Tensor output = torch::zeros(layerFour.sizes());
        //delete[] arr;
    
        auto ouput = torch::stack(arr);
        return ouput;
    }
    
    int main()
    {
    	cout &lt;&lt; "Hello CMake." &lt;&lt; endl;
    	return 0;
    }
</code></pre>

tom: <p>There are several methods to achieve this, but I think you need to decide on one and then add the required interface declarations. (I think it’s a DLL on Python, not a <code>.so</code> btw.)</p>
<ul>
<li>
<p>If you want to do C++ and Python, the custom operator approach you mention is a nice way.<br>
For this, too you would want to add a the registration part, this can look like <a href="https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html#registering-the-custom-operator-with-torchscript" rel="noopener nofollow ugc">the one in the custom op tutorial</a>:</p>
<pre><code class="lang-auto">TORCH_LIBRARY(my_ops, m) {
   m.def("warp_perspective", warp_perspective);
}
</code></pre>
<p>This can be loaded with <code>torch.ops.load_library</code>.</p>
</li>
<li>
<p>If you want to call from Python: Write a torch extension module.<br>
In this case you need a declaration like this one from <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html#binding-to-python" rel="noopener nofollow ugc">the tutorial</a>:</p>
<pre><code class="lang-auto">PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
 m.def("forward", &amp;lltm_forward, "LLTM forward");
 m.def("backward", &amp;lltm_backward, "LLTM backward");
}
</code></pre>
<p>This seems to be missing in method 2 and the error messages is a cryptic way of telling you “this isn’t a complete python module”.</p>
</li>
</ul>
<p>As an aside:</p>
<ul>
<li>If you want to call from C++ only: you can skip the PYBIND11_MODULE  likely need to declare your symbols as exported. This is particularly important on windows. (<code>__declspec(dllexport)</code> in the exporting library, <code>__declspec(dllimport)</code> in the programs using it). This seems to be the problem of of no symbols find in method 1. You don’t need a main for libraries (I think).  I’m labelling this “C++ only” because I don’t know how well passing tensors via ctypes would work. You could try to go through DLPack if you wanted. I do use DLPack to get tensors to get third party C code (e.g. Apache TVM-generated kernels) in my courses, but even there I do the conversion in a custom PyTorch C++ extension module.</li>
</ul>
<p>Best regards</p>
<p>Thomas</p>