Compiling pytorch C++ for android

divinho: <p>I want to use the new C++ frontend on android and so need to compile for ARM (32 bit). How can I do that? I saw there is a <code>script/build_android.sh</code> script but as I understand that is for caffe2. I saw there is a <code>BUILD_ATEN_MOBILE</code> option in <code>CMakeLists.txt</code>, but that can’t be enough (I would at least expect I need to provide a path to the android ndk I’m using).</p>
<p>Anyone have a suggestion how to do this?</p>

divinho: <p>I think I’ve made some progress, but I’m stuck on something and hope someone knows the answer.</p>
<p>The approach I took was taking <code>tools/build_libtorch.py</code> and <code>tools/build_pytorch_libs.sh</code>, and making changes to the cmake arguments with the help of looking at what <code>scripts/build_android.sh</code> was doing. So setting <code>ANDROID_NDK</code>, <code>CMAKE_TOOLCHAIN_FILE</code> and so on.</p>
<p>There were some errors that I managed to get around by setting <code>-DCMAKE_CROSSCOMPILING=1</code>, <code>-DRUN_HAVE_STD_REGEX=0</code>,  <code>-DANDROID_STL="c++_static"</code> and <code>LDFLAGS="-llog"</code>.</p>
<p>But now I’m getting errors related to protobuf. I noticed a warning</p>
<pre><code class="lang-auto">WARNING: Target "libprotoc" has EXCLUDE_FROM_ALL set and will not be built by default but an install rule has been provided for it.  CMake does not define behavior for this case.
WARNING: Target "protoc" has EXCLUDE_FROM_ALL set and will not be built by default but an install rule has been provided for it.  CMake does not define behavior for this case.
</code></pre>
<p>And the (I think) <a href="https://github.com/pytorch/pytorch/blob/master/cmake/ProtoBuf.cmake#L72" rel="nofollow noopener">relevant code has a comment</a> saying that pending a MR the behaviour could be changed. <strong>And the MR has been merged. But I’m not knowledgeable enough to know how to change the code…</strong> edit: <a href="https://github.com/pytorch/pytorch/pull/7085" rel="nofollow noopener">found the issue</a></p>
<p>The errors I’m still getting are</p>
<pre><code class="lang-auto">[...]/t/pytorch-cpu/third_party/onnx/onnx/onnx_onnx_torch.pb.h:38:3: error: expected expression
  static const ::google::protobuf::internal::ParseTableField entries[];
  ^
[...]/pytorch-cpu/third_party/onnx/onnx/onnx_onnx_torch.pb.h:37:17: error: variable has incomplete type 'struct ONNX_API'
struct ONNX_API TableStruct {
                ^
[...]/pytorch-cpu/third_party/onnx/onnx/onnx_onnx_torch.pb.h:37:8: note: forward declaration of 'protobuf_onnx_2fonnx_5fonnx_5ftorch_2eproto::ONNX_API'
struct ONNX_API TableStruct {
       ^
[...]/pytorch-cpu/third_party/onnx/onnx/onnx_onnx_torch.pb.h:45:6: error: variable has incomplete type 'void'
void ONNX_API AddDescriptors();
     ^
[...]/pytorch-cpu/third_party/onnx/onnx/onnx_onnx_torch.pb.h:45:14: error: expected ';' after top level declarator
void ONNX_API AddDescriptors();
             ^
             ;
</code></pre>

santi-pdp: <p>Hi! I am very interested on PyTorch compilation for Android. Did you have any further progress?</p>
<p>Thanks!</p>

divinho: <p>Nope. None at all. Caffe2 didn’t work for me as well (I suspect it’s because I’m using RNNs). I switched to Tensorflow (to my great displeasure).</p>

santi-pdp: <p>Argh I see, I am trying to make the Caffe2 option work with onnx but no luck so far. Thanks!</p>

tom: <p>I wrote a bit about <a href="https://lernapparat.de/pytorch-android/" rel="nofollow noopener">how to get Caffe2 (the AICamera example) to work on android</a>. As you can tell from the other bits of the blog post, I think that libtorch would be neat to have, too, and while I have a quick and dirty “feasibility study” port, I think such a project would need commercial backing. If that works out, we could have libtorch on android by the end of the year.</p>
<p>Best regards</p>
<p>Thomas</p>

divinho: <p><a class="mention" href="/u/tom">@tom</a></p>
<p>Thanks for replying to the message I sent (program hangs when calling <code>new caffe2::Predictor</code>, not something you had experienced for anyone else). I’m also curious what commit of pytorch and what version of the ndk you used (15/16/17?) if you wouldn’t mind sharing?</p>

tom: <p>I used some (arbitrary) master version <a href="https://github.com/t-vi/pytorch/tree/caffe2_android" rel="nofollow noopener">plus, as described in the blog post, a few tweaks</a>. I used ndk v18 as that was the one AndroidStudio happened to download. I did adapt the predictor calling in <a href="https://github.com/t-vi/AICamera/" rel="nofollow noopener">my fork of AICamera</a>, but once it compiled, I didn’t observe hanging.</p>
<p>Best regards</p>
<p>Thomas</p>

divinho: <p>Seems there are a lot of people <a href="https://github.com/caffe2/AICamera/issues/68" rel="nofollow noopener">1</a>,<a href="https://github.com/caffe2/caffe2/issues/2565" rel="nofollow noopener">2</a>,<a href="https://github.com/caffe2/AICamera/issues/59" rel="nofollow noopener">3</a> with this problem, but of course no responses.</p>

tom: <p>But does the master version of caffe2 work for you? I used that and it went reasonably well. (I must admit I haven’t spent much time with caffe2/android after I found libtorch/android to be feasible because I think that is a future I prefer. In a week we know a bit more about how soon that’s going to happen.)</p>
<p>Best regards</p>
<p>Thomas</p>

divinho: <p>No it does not. In fact I had to make some modifications to the caffe2 source so it would even compile (had error <code>no member named 'set_device_id' in 'caffe2::DeviceOption'</code>). I’m guessing you haven’t actually tried to use any of your own models, I’ve found comments talking about how even when the demo worked using other models does not work. <a href="https://twitter.com/cedric_chee/status/1059683939944525825" rel="nofollow noopener">This twitter thread was quite revealing</a>.</p>

tom: <p>Hm. No, I didn’t add my own model and it would be quite a bummer if that would not work. Is the model you’re trying to use available somewhere? Then I could drop it into my version.<br>
I didn’t have to change the caffe2 source, so that could probably be something where things went wrong for you. How did you try to compile? I put my procedure in the README.md of my AICamera fork.<br>
That said, I cannot understand the outrage from the difference between having bias in the convolution layer vs. adding it separately. Yes, performance wants fused kernels, but hey, for a toy app…<br>
But thanks for the advertisement for how doing this with only PyTorch is much easier. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>
<p>Best regards</p>
<p>Thomas</p>

divinho: <p>Okay. I’m first going to talk getting your app to work. Then trying with mine. I’m using ndk18 now by the way.</p>
<p>I compile using <code>git clone --recursive</code> and <code>ANDROID_NDK=&lt;path/to/Android/Sdk/ndk-bundle&gt; scripts/build_android.sh -DCTOOLCHAIN=clang</code>.</p>
<p>Using pytorch from your github link leads to the problem that it tries to get eigen from <code>'https://github.com/RLovelett/eigen.git</code> which doesn’t exist. I fix it by getting eigen from the current remote repo. Then when compiling an <code>error: call to 'stod' is ambiguous</code>, of course there is a <a href="https://github.com/pytorch/pytorch/issues/7589" rel="noopener nofollow ugc">github issue on this with no response</a>. When finished, not all requested libraries have been built, and I realize that for some reason <code>pytorch/c10</code> doesn’t exist. <code>git submodule update</code> does nothing, so I give up and try and just use the latest pytorch.</p>
<p>So I get that and compile it (and it worked! amazing). Obviously I change the paths to the static libraries to where I have them. The first build attempt fails, I need to add libqnnpack. I am by the way using the header files you’ve included (so potentially different). I get an error:</p>
<pre><code class="lang-auto">../../../../src/main/cpp/ATen/core/TensorImpl.h:755: error: undefined reference to 'caffe2::GetAllocator(at::DeviceType const&amp;)'
../../../../src/main/cpp/ATen/core/TensorImpl.h:763: error: undefined reference to 'at::PlacementDeleteContext::makeDataPtr(at::DataPtr&amp;&amp;, void (*)(void*, unsigned int), unsigned int, at::Device)'
</code></pre>
<p>So I stop using your headers and instead include my own:</p>
<pre><code class="lang-auto">include_directories(git/pytorch-android git/pytorch-android/aten/src
   git/pytorch-android/torch/lib/include git/pytorch-android/build_android
    git/pytorch-android/build_host_protoc/include git/pytorch/build/aten/src/)
</code></pre>
<p>Note the <strong>git/pytorch/build/aten/src/</strong>, I had to link to the build of pytorch I had for local use because running <code>scripts/build_android.sh</code> doesn’t actually create all the required ATen header files.</p>
<p>And after that it worked!</p>
<p>Now about my app.<br>
Because of a ‘multiple definitions’ error with another library (openfst) I compile normally and then with <code>-DBUILD_SHARED_LIBS=YES</code> (after renaming the <code>build_android</code> dir of course), which doesn’t complete, but gets far enough to give me a <code>libc10.so</code> which I use to avoid the error (I’m mentioning this just to give a fuller picture of what I’m doing).</p>
<p>Then the app runs but can’t instantiate the Predictor (fails on <code>new caffe2::Predictor(init_net, predict_net)</code>). I don’t get any error message. I used <code>onnx_graph_to_caffe2_net</code> to export the model, <code>mobile_exporter</code> doesn’t like having Long inputs it seems, and if I use a model that has floats I get the same error in the end anyways.</p>
<p>My model is not related to the camera thing at all, but it’s quite small (6MB) and simple to use. See <a href="https://gist.github.com/Nimitz14/95889f250915e35cfeab5d8353aaae14" rel="noopener nofollow ugc">this gist</a> for how I run it (using pytorch built for my PC, just to check whether caffe2 works locally). <a href="https://files.fm/u/zfz7smvk" rel="noopener nofollow ugc">Model is here</a>.<br>
<strong>EDIT</strong>: But the model is irrelevant I think. Even just a single fully connected layer as a model results in the same problem.</p>
<p>I’d say the ‘outrage’ is just regarding the fact that there are a lot of people having problems but noone seems to get any responses. Even just a</p>
<blockquote>
<p>I didn’t have to change the caffe2 source, so that could probably be something where things went wrong for you.</p>
</blockquote>
<p>is very helpful (so even if you just say “this seems like you did something wrong”). But most people aren’t getting anything at all.</p>

tom: <p>Before I start getting into technical things, what are you trying to achieve?<br>
If you just want to vent, I don’t have to judge that, but I’d rather not engage with it. I’m just a random guy on the internet trying to be help out other people every once in a while, and to be honest I’m asking myself why sometimes.</p>
<aside class="quote no-group" data-username="divinho" data-post="13" data-topic="27384">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/9dc877/48.png" class="avatar"> divinho:</div>
<blockquote>
<p>Using pytorch from your github link leads to the problem that it tries to get eigen from <code>'https://github.com/RLovelett/eigen.git</code> which doesn’t exist. I fix it by getting eigen from the current remote repo. Then when compiling an <code>error: call to 'stod' is ambiguous</code> , of course there is a <a href="https://github.com/pytorch/pytorch/issues/7589" rel="noopener nofollow ugc">github issue on this with no response</a>. When finished, not all requested libraries have been built, and I realize that for some reason <code>pytorch/c10</code> doesn’t exist. <code>git submodule update</code> does nothing, so I give up and try and just use the latest pytorch.</p>
</blockquote>
</aside>
<p>Now, the eigen repository changed URL quite a while ago (seams not unreasonable to me) and for some reason you picked up the old one (maybe because I don’t use the master branch in my github repo and it is old enough to still have that - but I don’t know and git submodule apparently does not handle changes in the upstream repository - probably because no-one who hit the problem showed up to fix it, I certainly am guilty of that). Similarly, I could point out that we’re all doing <code>git submodule update --init --recursive</code> or so these days.</p>
<p>Probably it’s fair to say that some of the things that are happening in the following are from mixing different versions and builds. Including the headers seems like a not so stable recipe when people might use different versions of the lib, and I must admit I am hoping that someone else finds out about how to get all the headers in one place nicely and tells me.</p>
<p>I am not quite able to discern what you did from your description. It’s no wonder that typical bug report templates ask people to provide as exact instructions to reproduce. The issues you are seeing likely stem from the complexity of the interactions of various bits so that apparently the instructions such as the ones I put out don’t allow you to replicate what I did. But anyone trying to look into your issue will have the <em>exact same problem</em>, so it would seem a natural reaction to say <em>yeah well, I’ll answer someone who provided more detail instead</em>. It might be nicer to drop a note of that sort into the bug report, but I’ll tell you from experience, that it’s as frustrating for the people trying to find out what went wrong as for the reporter, so it’s easy for incomplete bug reports to fall through the grid.</p>
<p>To me, your report sounds like <em>I started with what you said and then I did all sorts of crazy things and then it turned out wrong and now I’m angry</em> and by the time I could have looked at more details I have zoomed out and walked away.</p>
<p>Good luck with your project!</p>
<p>Best regards</p>
<p>Thomas</p>

divinho: <p>My goal was to get to the same point others had, being able to build the AICamera demo but not use my own model. Because doing so was not trivial, I wrote down what I had to do to get to that point. I can see now that the way I wrote it, it may have come across as confrontational. Sorry about that. Thank you for taking the time to write the posts that you did. Hope I’m not the reason you stop helping random strangers on the internet. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=6" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

tom: <p>Hey, no worries.</p>
<p>Based on your feedback, I updated my master branch and I’ll suggest to do</p>
<pre><code class="lang-auto">git clone --recursive -b caffe2_android https://github.com/t-vi/pytorch.git
</code></pre>
<p>to get the repo.<br>
I’ll try to build from that and see if I also hit either the stod or c10 thing.</p>
<p>For the model use:<br>
Do I understand correctly, that the demo model works for you (but your own doesn’t)?<br>
Does the Predictor instantiation fail on an x86 android virtual device as well?<br>
Does it raise an exception when in debug mode?<br>
I sometimes found error messages in logcat or have enabled catching native exceptions (in the “edit all breakpoints” in android, but be sure to set a breakpoint on the invocation you’re interested in and do “disable until breakpoint is hit” or so to not drown in exceptions during setup) then one can try to inspect the exception message in the local variables or so.</p>
<p>Best regards</p>
<p>Thomas</p>

divinho: <p>Yes, the demo model worked, but my own did not.</p>
<p>I appreciate you asking more clarifying questions, but I’ve moved to tensorflow (I said the same thing a week ago, I came back to try pytorch/caffe2 on android without RNNs but obviously my attempt with that failed as well) and do not plan on sinking any more time into trying to solve this. Thanks again for the time you put in trying to help me anyways!</p>

tom: <p>Thanks!</p>
<p>Yes, I can understand that you’d not want to invest more into something you don’t intend to use.</p>
<p>Best of luck, I hope PyTorch get’s the chance to provide you with a better experience next year.</p>
<p>Best regards</p>
<p>Thomas</p>

black0017: <p>Hello <a class="mention" href="/u/divinho">@divinho</a> , <a class="mention" href="/u/tom">@tom</a> is there any update on this issue ?</p>
<p>I have exported a custom CNN-LSTM model from Pytorch (v1.1) using the approach: pytorch–&gt;ONNX–&gt; caffe2. I have used ndk 19 and ONNX 1.3 and started from the official tutorial <a href="https://github.com/caffe2/AICamera" class="inline-onebox" rel="noopener nofollow ugc">GitHub - facebookarchive/AICamera: Demonstration of using Caffe2 inside an Android application.</a>.</p>
<p>I am kind of stuck with loading RNN layers in caffe2 similar to the above problem that has been referenced here.</p>
<aside class="quote no-group" data-username="divinho" data-post="7" data-topic="27384">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/9dc877/48.png" class="avatar"> divinho:</div>
<blockquote>
<p>program hangs when calling <code>new caffe2::Predictor</code> , not something you had experienced for anyone else)</p>
</blockquote>
</aside>
<p>I would like to know if there is any progress either from caffe2 or from directly working with c++ front-end (LibTorch) for android using RNNs ?</p>
<p>Thanks in advance and thank you for your time.</p>
<p>black0017.</p>

divinho: <p>I just asked about this <a href="https://discuss.pytorch.org/t/whats-the-state-of-pytorch-on-android/46873">here</a> but received no reply. <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"></p>