C10/macros/cmake_macros.h not exists

Aviais: <p>Hi,<br>
I’m trying to include pytorch/script.h in my CPP android project.<br>
I’m using ndk21 and C++14.<br>
I cloned the repository and used “git submodule update --init --recursive” to update sub=modules.<br>
I’m getting the following compilation error:</p>
<p>In file included from C:\REPOS\pytorch\torch/script.h:3:<br>
In file included from C:\REPOS\pytorch\torch/csrc/api/include/torch/types.h:3:<br>
In file included from C:\REPOS\pytorch/aten/src\ATen/ATen.h:3:<br>
In file included from C:\REPOS\pytorch\c10/core/Allocator.h:6:<br>
In file included from C:\REPOS\pytorch\c10/core/Device.h:3:<br>
In file included from C:\REPOS\pytorch\c10/core/DeviceType.h:8:<br>
C:\REPOS\pytorch\c10/macros/Macros.h:21:10: fatal error:  <strong>‘c10/macros/cmake_macros.h’ file not found</strong><br>
<span class="hashtag">#include</span> &lt;c10/macros/cmake_macros.h&gt;<br>
^~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
1 error generated.</p>
<p>I do see that there is a file c10/macros/cmake_macros.h**.in**, an it specifically says:<br>
<em>Do not include this file directly. Instead, include c10/macros/Macros.h.</em><br>
So what is the problem?</p>

glaringlee: <p><a class="mention" href="/u/aviais">@Aviais</a><br>
You can’t include a single header file from pytorch codebase directly, there are many dependencies.<br>
Can you tell me the details about what you are trying to do here?</p>

Aviais: <ol>
<li>
<p>I have a CPP android project. I’m compiling it using ndk-build (ndk21 and C++14).</p>
</li>
<li>
<p>I have a file named “main.cpp” and I’m trying to  <strong><span class="hashtag">#include</span> “torch/script.h”</strong> .</p>
</li>
<li>
<p>I got the error above when I git cloned pytorch and updated sub-modules as reported below.</p>
</li>
</ol>
<p>When I downloaded a different libtorch version using this command,<br>
<strong>wget <a href="https://github.com/Kitware/CMake/releases/download/v3.15.2/cmake-3.15.2.tar.gz" rel="nofollow noopener">https://github.com/Kitware/CMake/releases/download/v3.15.2/cmake-3.15.2.tar.gz</a></strong><br>
I saw a different structure of the files and I had different compilations error, as I reported in:<br>
<a href="https://discuss.pytorch.org/t/compilation-error-in-jit-ir-ir-h/80694" class="inline-onebox">Compilation error in 'jit/ir/ir.h'</a></p>
<ol start="4">
<li>In my Android.mk file, I’m giving the path to:<br>
LOCAL_C_INCLUDES += $(TORCH_REPO_PATH)<br>
$(TORCH_REPO_PATH)/include<br>
$(TORCH_REPO_PATH)/include/torch<br>
$(TORCH_REPO_PATH)/include/torch/csrc/api/include</li>
</ol>
<p>Please help.<br>
Thanks for your response.</p>

glaringlee: <p><a class="mention" href="/u/aviais">@Aviais</a><br>
As I said, you can not include a single header file in your program, it won’t work even you added include path in your mk file, basically you need dependencies, shared libs, at least libc10, libtorch, libtorch_cpu, libtorch_gpu, etc. What I recommend to do is download libtorch zip package and follow the instruction here:<br>
<a href="https://pytorch.org/cppdocs/installing.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/cppdocs/installing.html</a><br>
And you can extend your makefile based on instruction. Check the CMakeList.txt that provided in the instruction.</p>
<p>If you don’t want to use cmake and want to build with your own makefile or even build with command line, check here:<br>
‘<a href="https://discuss.pytorch.org/t/how-do-i-to-create-a-new-project-in-c-to-run-libtorch-without-cmake/79046" class="inline-onebox">How do i to create a new project in C++ to run libtorch without cmake?</a>’<br>
I posted a command line which you can use to build libtorch. And you can put those minimal requirements in the commandl line (libs, includes, etc) into your make file. But libtorch is changing fast, I do recommend using our cmake solution.</p>
<p>FYI, for torch script, please follow here:<br>
<a href="https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html</a><br>
and this is the c++ part instruction:<br>
<a href="https://pytorch.org/tutorials/advanced/cpp_export.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_export.html</a></p>

Aviais: <aside class="quote no-group" data-username="glaringlee" data-post="4" data-topic="80859">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/9de053/48.png" class="avatar"> glaringlee:</div>
<blockquote>
<p>What I recommend to do is download libtorch zip package and follow the instruction here:</p>
</blockquote>
</aside>
<p>**Thank you for the detailed answer. **<br>
<strong>I followed all the instructions you shared with me and did it successfully. I used CMake, built and loaded my model got input and output via CPP on my PC.</strong></p>
<p>I would like to include <strong>torch/script.h</strong> in my existing CPP Android project, so I can load my model and execute it from my exisiting code.</p>
<p><strong>I added to my Andorid.mk file the shared libraries you mentioned + the includes above.</strong><br>
<strong>LOCAL_LDLIBS += -L$(TORCH_REPO_PATH)/lib -ltorch -ltorch_cpu -lc10 -ltorch_global_deps</strong></p>
<p>I’m still getting these errors:</p>
<p>C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:67:17: error: no namespace named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?<br>
using namespace ::c10::prim;<br>
^~~~~~~~~~~<br>
prim<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:66:11: note: namespace ‘prim’ defined here<br>
namespace prim {<br>
^<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:70:17: error: no namespace named ‘attr’ in namespace ‘c10’; did you mean simply ‘attr’?<br>
using namespace ::c10::attr;<br>
^~~~~~~~~~~<br>
attr<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:69:11: note: namespace ‘attr’ defined here<br>
namespace attr {<br>
^<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:73:17: error: no namespace named ‘aten’ in namespace ‘c10’; did you mean simply ‘aten’?<br>
using namespace ::c10::aten;<br>
^~~~~~~~~~~<br>
aten<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:72:11: note: namespace ‘aten’ defined here<br>
namespace aten {<br>
^<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1309:34: error: no member named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?<br>
static constexpr Symbol Kind = ::c10::prim::profile;<br>
^~~~~~~~~~~<br>
prim<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:66:11: note: ‘prim’ declared here<br>
namespace prim {<br>
^<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1309:47: error: no member named ‘profile’ in namespace ‘torch::jit::prim’<br>
static constexpr Symbol Kind = ::c10::prim::profile;<br>



^<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1311:21: error: no member named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?<br>
: Node(graph, ::c10::prim::profile), callback_(callback) {}<br>
^


~~~~<br>
prim<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:66:11: note: ‘prim’ declared here<br>
namespace prim {<br>
^<br>
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1311:34: error: no member named ‘profile’ in namespace ‘torch::jit::prim’<br>
: Node(graph, ::c10::prim::profile), callback_(callback) {}<br>
~~~~~~^<br>
7 errors generated.</p>
<p><strong>What am I missing?</strong></p>

glaringlee: <p><a class="mention" href="/u/aviais">@Aviais</a><br>
What’s your env? Windows? or Linux?</p>
<p>To cross build, you need to make sure the libtorch you downloaded is compiled with the same compiler with your android project. You can download a correct build from here:<br>
‘<a href="https://pytorch.org/" rel="nofollow noopener">https://pytorch.org/</a>’ (select package ‘libtorch’, choose correct operating system you have)</p>
<p>If you are under linux and the libtorch you downloaded is non-abi build, then ABI must set to be false.<br>
D_GLIBCXX_USE_CXX11_ABI=0.<br>
And you need to tell <strong>linker</strong> the location of the shared library. The command line I posted here:<br>
‘<a href="https://discuss.pytorch.org/t/how-do-i-to-create-a-new-project-in-c-to-run-libtorch-without-cmake/79046" class="inline-onebox">How do i to create a new project in C++ to run libtorch without cmake?</a>’<br>
has ‘-Wl,R’ before -ltorch -ltorch_cpu -lc10 so linker knows where to find those shared libs.</p>

Aviais: <p>I did all of the above but I still get the same 7 errors from /include\torch/csrc/jit/ir/ir.h<br>
I’m working on Windows.</p>

peterjc123: <aside class="quote no-group" data-username="Aviais" data-post="5" data-topic="80859">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aviais/48/12808_2.png" class="avatar"> Aviais:</div>
<blockquote>
<p>CPP Android</p>
</blockquote>
</aside>
<p>If you want to compile for Android, then you should not use the Windows libraries. What you are trying to do is called cross compiling, which is to compile for a different target on a host machine. You should choose the libraries that are designed for Android and use GCC instead of MSVC.</p>

Aviais: <p>Thank you for responding.</p>
<p>It is important to mention that I was able to load a torch script model on my PC after I followed the tutorial (using CMake) in: <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_export.html</a></p>
<p>Can you please send me a link to a guide that explains how to include pytorch in an existing CPP android project? I tried both linux/windows versions and I get the same 7 errors I reported in this post. I also tried various pytorch versions.<br>
I don’t understand why it doesn’t work…</p>

peterjc123: <aside class="quote no-group" data-username="Aviais" data-post="5" data-topic="80859">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aviais/48/12808_2.png" class="avatar"> Aviais:</div>
<blockquote>
<p>namespace named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?</p>
</blockquote>
</aside>
<p>I found a related post. <a href="https://discuss.pytorch.org/t/compilation-error-in-jit-ir-ir-h/80694/2" class="inline-onebox">Compilation error in 'jit/ir/ir.h' - #2 by Aviais</a></p>