Using both libtorch c++ and python API

SimonRelu: <p>Hey,</p>
<p>I’ve written a wrapper around a pytorch operator to use it in ORT. The reason being that one ORT operator is significantly slower than the pytorch one (see <a href="https://github.com/microsoft/onnxruntime/issues/14596" class="inline-onebox" rel="noopener nofollow ugc">[Performance] Pytorch is faster than ONNX when running inference multiple times · Issue #14596 · microsoft/onnxruntime · GitHub</a>). For this, I use the libtorch C++ API (just wrap the ORT tensors in a torch tensor with <code>at::from_blob</code>). I load this custom ORT kernel in python using a <code>.so</code>/<code>.dll</code> file. Before doing so, I also load the necessary libtorch DLLs with <code>ctypes.cdll.LoadLibrary</code> (alternatively I could set the DLL env but I found this approach easier).</p>
<p>This works fine, however, after importing the libtorh DLLs I can’t import torch anymore in python. Doing so with <code>import torch</code> simply results in a segfault. When I first import torch and then the torchlib DLLs I get the following error when loading my ORT costum operator:</p>
<pre><code class="lang-auto">onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Failed to load library /path/to/libORT.so with error: /path/to/libORT.so: undefined symbol: _ZN3c106detail14torchCheckFailEPKcS2_jRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
</code></pre>
<p>The reason for this I believe is because of a version mismatch between the python install and pytorhlib. I install the python version with <code>pip3 install torch==2.0.0+c118 -f https://download.pytorch.org/whl/torch_stable.html</code> and download torchlib from <code>https://download.pytorch.org/libtorch/cu118/libtorch-cxx11-abi-shared-with-deps-2.0.0%2Bcu118.zip</code>. I thought this should be fine since they are the same version but maybe they are from different builds?</p>
<p>So I was wondering how I best deal with this. Would it be possible to use the DLLs in <code>/usr/local/lib/python/dist_packages/torch/lib</code> for my C++ wrapper. They seem to be the same as the ones from libtorch or will this cause issues? My goal is to be able to ship a version of the code with only the necessary pytorchlib for inference to keep the install size small but have the full pytorch python API for development and training.</p>

Hanshuo_WANG: <p>Hi,</p>
<p>I’ve got the same issue, i’m wondering if you have found the solutions?<br>
Any suggestions and help are appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Hanshuo_WANG: <p>Hey there,<br>
I solved this problem by myself.</p>
<p>Actually, there is no need to download c++ libtorch if you have already downloaded pytorch using pip. Assume that you are using conda, libtorch.so will be automatically installed in “/root/miniconda3/envs/your_env/lib/python3.10/site-packages/torch/lib”, alongside with many other libraries that c++ can use. Using this libtorch directly in your c++ code and cmake will cause no version confliction when importing it to python by pybind11, cause they come from the same place. And pybind11 has no version issue, so any download method will be fine (git clone from source or pip).</p>
<p>Hope this could help anyone that have same issues. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>