Building libtorch with UE4 - Stuck on a couple of errors

Chozabu: <p>Hi all,<br>
First off - Having a Great time with pytorch, and it’s working well with UnrealEngine 4.23 - but with performance in mind…<br>
I’m trying to build libtorch with UE4 - but running into some issues during compilation</p>
<pre><code class="lang-auto">Error	C4273	'torch::jit::tracer::addInputs': inconsistent dll linkage	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\jit\tracer.h	292	
Error	C4273	'torch::jit::tracer::addInputs': inconsistent dll linkage	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\jit\tracer.h	299	
Error	C2280	'std::shared_ptr&lt;Contained&gt; std::dynamic_pointer_cast&lt;Derived,_Ty&gt;(std::shared_ptr&lt;_Ty&gt; &amp;&amp;) noexcept': attempting to reference a deleted function	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch\nn\cloneable.h	85	

</code></pre>
<p>I’m on win10, using VS2019 (which calls UBT - unreals own build system instead of using its own, but still compiles with MSVC)</p>
<p>What I have tried:</p>
<ul>
<li>building using libtorch 1.4 release, 1.4 debug, and latest debug</li>
<li>surpressing errors using pragma <code>__pragma(warning(disable: 4273))</code>
</li>
<li>enabling permissive mode</li>
<li>linking to only the libs, dlls, both or neither (This actually seems to have no effect)</li>
<li>ensuring the build path has no spaces</li>
</ul>
<p>The only ideas I really have left are to use vs2017 or try building libtorch from source, but not too confident either option will work.</p>
<p>Any ideas? I am happy to provide any more details or share the project.</p>
<p>Thanks!</p>

Maindz: <p>I am trying to get the same thing to work, and have already tried all the things you have tried with no success and I get the same errors you get. In addition I can say that changing to vs2017 will not make a difference as I have tried. Might be a good idea to try to build from source and make sure you follow these steps <a href="https://wiki.unrealengine.com/Creating_%26_Linking_Static_Libraries_And_Make_Your_Own_Blueprint_Node_With_VS_2017_%26_UE4" rel="nofollow noopener">https://wiki.unrealengine.com/Creating_%26_Linking_Static_Libraries_And_Make_Your_Own_Blueprint_Node_With_VS_2017_%26_UE4</a>, in “Customizations for Targeting UE4 Modules”, I will most likely try that next week. So if you ever figure this one out please let me know.<br>
Thanks.</p>

Chozabu: <p><a class="mention" href="/u/maindz">@Maindz</a> Good to know someone else is on the same path!</p>
<p>Worth asking - what version of UE4 and MSVC are you working with? I’m on MSVC <code>14.24.28314</code></p>
<p>Actually - if you can share your build output, could be rather useful to compare - here is mine:</p>
<pre><code class="lang-auto">1&gt;------ Build started: Project: UETorchC, Configuration: Development_Editor x64 ------
1&gt;Parsing headers for UETorchCEditor
1&gt;  Running UnrealHeaderTool "E:\UETorchC\UETorchC.uproject" "E:\UETorchC\Intermediate\Build\Win64\UETorchCEditor\Development\UETorchCEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -installed
1&gt;Reflection code generated for UETorchCEditor in 6.2185253 seconds
1&gt;Using Visual Studio 2019 14.24.28314 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314) and Windows 10.0.18362.0 SDK (C:\Program Files (x86)\Windows Kits\10).
1&gt;Building 4 actions with 4 processes...
1&gt;  [1/4] UETorch.cpp
1&gt;E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(292): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1&gt;  E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(276): note: see previous definition of 'addInputs'
1&gt;E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(299): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1&gt;  E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(279): note: see previous definition of 'addInputs'
1&gt;E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(85): error C2280: 'std::shared_ptr&lt;Contained&gt; std::dynamic_pointer_cast&lt;Derived,_Ty&gt;(std::shared_ptr&lt;_Ty&gt; &amp;&amp;) noexcept': attempting to reference a deleted function
1&gt;          with
1&gt;          [
1&gt;              Contained=torch::nn::GroupNormImpl,
1&gt;              Derived=torch::nn::GroupNormImpl,
1&gt;              _Ty=torch::nn::Module
1&gt;          ]
1&gt;  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\INCLUDE\memory(1465): note: see declaration of 'std::dynamic_pointer_cast'
1&gt;  E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(81): note: while compiling class template member function 'void torch::nn::Cloneable&lt;torch::nn::GroupNormImpl&gt;::clone_(torch::nn::Module &amp;,const c10::optional&lt;c10::Device&gt; &amp;)'
1&gt;  E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/modules/normalization.h(106): note: see reference to class template instantiation 'torch::nn::Cloneable&lt;torch::nn::GroupNormImpl&gt;' being compiled
1&gt;  E:\UETorchC\Source\ThirdParty\libtorch\include\c10/core/MemoryFormat.h(55): note: see reference to class template instantiation 'c10::ArrayRef&lt;int64_t&gt;' being compiled
1&gt;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""E:\Epic Games\UE_4.23\Engine\Build\BatchFiles\Build.bat" UETorchCEditor Win64 Development -Project="E:\UETorchC\UETorchC.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command.
1&gt;Done building project "UETorchC.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
</code></pre>
<p>strongly suspect issues are MSVC version related, I’ll give this a go on my linux box - avoiding the MSVC part of the toolchain may well sidestep the issues…</p>

Maindz: <p>I have tried different MSVC versions but the one I am using now is 2017 14.16.27023.<br>
Build output:</p>
<pre><code class="lang-auto">1&gt;------ Build started: Project: IAF, Configuration: Development_Editor x64 ------
1&gt;Using 'git status' to determine working set for adaptive non-unity build (C:\Users\mariu\Documents\Bachelor\IAF).
1&gt;Invalidating makefile for IAFEditor (source file removed)
1&gt;Parsing headers for IAFEditor
1&gt;  Running UnrealHeaderTool "C:\Users\mariu\Documents\Bachelor\IAF\IAF.uproject" "C:\Users\mariu\Documents\Bachelor\IAF\Intermediate\Build\Win64\IAFEditor\Development\IAFEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -installed
1&gt;Reflection code generated for IAFEditor in 6,4612741 seconds
1&gt;Using Visual Studio 2017 14.16.27023 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023) and Windows 10.0.17763.0 SDK (C:\Program Files (x86)\Windows Kits\10).
1&gt;Building 12 actions with 8 processes...
1&gt;  [1/12] SharedPCH.Engine.NoUndef.cpp
1&gt;  [2/12] IAF.cpp
1&gt;  [3/12] RL.cpp
1&gt;  [4/12] PPOActor.cpp
1&gt;  [5/12] IAF.init.gen.cpp
1&gt;  [6/12] AgentPPO.gen.cpp
1&gt;  [7/12] RL.gen.cpp
1&gt;  [8/12] PPOActor.gen.cpp
1&gt;  [9/12] AgentPPO.cpp
1&gt;C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/flat_hash_map.h(1308): warning C4458: declaration of 'shift' hides class member
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/flat_hash_map.h(1318): note: see declaration of 'ska::fibonacci_hash_policy::shift'
1&gt;C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\ATen/core/TensorBody.h(1105): error C4522: 'at::Tensor': multiple assignment operators specified
1&gt;C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/order_preserving_flat_hash_map.h(1427): warning C4458: declaration of 'shift' hides class member
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/order_preserving_flat_hash_map.h(1437): note: see declaration of 'ska_ordered::fibonacci_hash_policy::shift'
1&gt;C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(289): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(277): note: see previous definition of 'addInputs'
1&gt;C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(296): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(283): note: see previous definition of 'addInputs'
1&gt;C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(85): error C2280: 'std::shared_ptr&lt;Contained&gt; std::dynamic_pointer_cast&lt;Derived,_Ty&gt;(const std::shared_ptr&lt;_Ty&gt; &amp;) noexcept': attempting to reference a deleted function
1&gt;          with
1&gt;          [
1&gt;              Contained=torch::nn::GroupNormImpl,
1&gt;              Derived=torch::nn::GroupNormImpl,
1&gt;              _Ty=torch::nn::Module
1&gt;          ]
1&gt;  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\INCLUDE\memory(1770): note: see declaration of 'std::dynamic_pointer_cast'
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(81): note: while compiling class template member function 'void torch::nn::Cloneable&lt;torch::nn::GroupNormImpl&gt;::clone_(torch::nn::Module &amp;,const c10::optional&lt;c10::Device&gt; &amp;)'
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/modules/normalization.h(106): note: see reference to class template instantiation 'torch::nn::Cloneable&lt;torch::nn::GroupNormImpl&gt;' being compiled
1&gt;  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/core/MemoryFormat.h(53): note: see reference to class template instantiation 'c10::ArrayRef&lt;int64_t&gt;' being compiled
1&gt;C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""C:\Program Files\Epic Games\UE_4.23\Engine\Build\BatchFiles\Build.bat" IAFEditor Win64 Development -Project="C:\Users\mariu\Documents\Bachelor\IAF\IAF.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command.
1&gt;Done building project "IAF.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

</code></pre>
<p>AgentPPO is the first place I include libtorch.</p>
<p>If I remember correctly I think the c4458 warnings and the c4522 error disappeared when I changed to libtorch gpu version, used libtorch 1.4 cpu when I tried to build this, though I have tried pretty much every version from 1.2 both cpu and gpu.</p>
<p>Probably also worth mentioning that I tried to use libtorch in visual studio without UE4 just a plain c++ project, and that worked.</p>

Selcuk_Caglar: <p>Have got any progress? What is the problem with GPU version?</p>

Chozabu: <p>I did try compilation from linux - and got a different error, seemed more understandable but not yet looked into it deeply.</p>
<p>I’ve been working more with the regular python version of pytorch + ue4 for now: <a href="https://gyazo.com/5adf2471d03be02aefe4a5a1ea4d5a3e" rel="nofollow noopener">https://gyazo.com/5adf2471d03be02aefe4a5a1ea4d5a3e</a></p>
<p>Still intending to revisit compiling libtorch with UE4, but not sure when.<br>
Also considering other options - like a lightweight inference system - similar to barracuda</p>

David_Yanni: <p>I got help for this from someone at work who is a ue4 pro. Try enabling RTTI.</p>
<p>Edit also the warnings can be gotten around with</p>
<pre><code class="lang-auto">THIRD_PARTY_INCLUDES_START
#include "torch/torch.h"
THIRD_PARTY_INCLUDES_END
</code></pre>

Chozabu: <p><a class="mention" href="/u/david_yanni">@David_Yanni</a> Interesting! Am I to understand you have libtorch up and running with ue4?</p>

Loris_Donovan: <p>So after following <a class="mention" href="/u/david_yanni">@David_Yanni</a>’s steps and adding a few changes in the definition of THIRD_PARTY_INCLUDES_START i.e, after I adding the following lines in the definition:</p>
<pre><code class="lang-auto">__pragma(warning(disable: 4273))\
__pragma(warning(disable: 4582))\
__pragma(warning(disable: 4583))\
__pragma(warning(disable: 4018))\
</code></pre>
<p>And also copying all the .dll files in libtorch/lib folder to Binaries/Win64, I managed to get the project compiling. But I don’t seem to be able to “Play” after adding blueprint class which simply logs the output of a neural network. I get a breakpoint saying “ntdll.pdb not loaded”.<br>
Since I am new to Unreal Engine may be I did something wrong idk so I would like you to try it out. If you get it working please let me know. Thanks!</p>

Fhrozen: <p>Hello there, interesting project going here.<br>
I just finished to test an integration btw Unity and Libtorch following this recipe: <a href="https://www.goodai.com/neural-networks-in-unity-using-native-libraries/" rel="nofollow noopener">https://www.goodai.com/neural-networks-in-unity-using-native-libraries/</a>.<br>
Then, I would suggest to build an external library similar to the link and then just import the built library into UE4. I would try this also for a project later.<br>
Let me know if it works.</p>
<p>Best</p>

NeuralVFX: <p><a class="mention" href="/u/fhrozen">@Fhrozen</a> Did you ever happen to get Unreal+Librorch working? I just followed this “<a href="http://goodai.com" rel="nofollow noopener">goodai.com</a>” tutorial you posted, except using Unreal Engine instead of Unity, and it works. My test scene runs salient object detection on a live video feed, passes the output through the DLL, and updates a texture in the scene every tick.</p>
<p>I don’t have a GPU at my disposal… so I used “libtorch-win-shared-with-deps-1.5.1+cpu.zip”</p>
<p>I’m curious if you or anyone here can confirm if the same thing works with the GPU version.</p>

Fhrozen: <p>I have no problem with the GPU version.<br>
In my case, tested on Unity, I added the line</p>
<pre><code class="lang-auto">network.to(at::kCUDA); // If we're doing this on GPU

</code></pre>
<p>in the network.cpp file and it worked. I supposed that this should work for UE4 too</p>

ava6969: <p>Hi Nelson where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.</p>

ava6969: <p>Hi loris where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.</p>

ava6969: <p>Hi david where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.</p>

Fhrozen: <p>I did not try to run with unreal and libtorch seamlessly. I only tested on Unity. Sadly, I do not have time for test it.</p>

ava6969: <p>Can you possibly share a git link also for your unity project. if its public. i had a hard time following the instruction.</p>

Fhrozen: <p>I did not have an open git for that project, but I copied from here <a href="https://www.goodai.com/neural-networks-in-unity-using-native-libraries/" rel="nofollow noopener">https://www.goodai.com/neural-networks-in-unity-using-native-libraries/</a>. I will try to upload the code later.</p>

NeuralVFX: <p>In case anyone’s interested, I’ve made an example UE4 project based on the <a href="https://www.goodai.com/neural-networks-in-unity-using-native-libraries/" rel="nofollow noopener">goodai</a> tutorial:</p>
<p>DLL Project: <a href="https://github.com/NeuralVFX/basic-libtorch-dll" rel="nofollow noopener">https://github.com/NeuralVFX/basic-libtorch-dll</a></p>
<p>Unreal Project: <a href="https://github.com/NeuralVFX/basic-unreal-libtorch-plugin" rel="nofollow noopener">https://github.com/NeuralVFX/basic-unreal-libtorch-plugin</a></p>
<p>The example uses the CPU, however, if <a class="mention" href="/u/fhrozen">@Fhrozen</a> is right, changing a couple lines of code will make it GPU compatible.</p>

xindelvcheng: <p>This problem can be solve by add "bUseRTTI = true;"to *.build.cs。I dumpbin *.dll (to search dependency)and link successfully，but it work abnormally，every tensor is filled by 0。</p>