aten/src/ATen/Tensor.h:3:10: fatal error: 'ATen/core/TensorBody.h' file not found

username: smpurkis
cooked: <p>Hello,<br>
I’m trying to build pytorch v1.10 and get the <code>speed_benchmark_torch</code> binary described in <a href="https://pytorch.org/tutorials/recipes/mobile_perf.html#android-benchmarking-setup" class="inline-onebox" rel="noopener nofollow ugc">Pytorch Mobile Performance Recipes — PyTorch Tutorials 1.10.0+cu102 documentation</a>. Using that command:</p>
<pre><code class="lang-auto">BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_android.sh -DBUILD_BINARY=ON
</code></pre>
<p>(I am using AndroidNDK-r21, with set <code>ANDROID_NDK</code>)</p>
<p>I am getting this error</p>
<pre><code class="lang-auto">aten/src/ATen/Tensor.h:3:10: fatal error: 'ATen/core/TensorBody.h' file not found
</code></pre>
<p>(I’ve seen <a href="https://discuss.pytorch.org/t/build-libtorch-for-android-error-aten-core-tensorbody-h-file-not-found/100526" class="inline-onebox">Build libtorch for android error, ‘ATen/core/TensorBody.h’ file not found</a>, but it is using a different build command. I tried</p>
<pre><code class="lang-auto">BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_pytorch_android.sh -DBUILD_BINARY=ON
</code></pre>
<p>but it doesn’t seem to support building the binary.)</p>
<p>Tried uploading the full log, but it said only image attachments are supported?!<br>
So here is some of the log:</p>
<pre><code class="lang-auto">-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.16.3
--   CMake command         : /usr/bin/cmake
--   System                : Android
--   C++ compiler          : /home/sam/CK-SDKs/AndroidNDK-r21/android-ndk-r21e/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++
--   C++ compiler id       : Clang
--   C++ compiler version  : 9.0
--   Using ccache if found : ON
--   Found ccache          : CCACHE_PROGRAM-NOTFOUND
--   CXX flags             : -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -frtti -fexceptions  -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DUSE_VULKAN_WRAPPER -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DBUILD_LITE_INTERPRETER -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -g0
--   Build type            : Release
--   Compile definitions   : 
--   CMAKE_PREFIX_PATH     : /home/sam/envs/py8/lib/python3.8/site-packages;/home/sam/CK-SDKs/AndroidNDK-r21/android-ndk-r21e/toolchains/llvm/prebuilt/linux-x86_64
--   CMAKE_INSTALL_PREFIX  : /home/sam/tmp/pytorch-v1.10.0/build_android/install
--   USE_GOLD_LINKER       : OFF
-- 
--   TORCH_VERSION         : 1.10.0
--   CAFFE2_VERSION        : 1.10.0
--   BUILD_CAFFE2          : ON
--   BUILD_CAFFE2_OPS      : OFF
--   BUILD_CAFFE2_MOBILE   : OFF
--   BUILD_STATIC_RUNTIME_BENCHMARK: OFF
--   BUILD_TENSOREXPR_BENCHMARK: OFF
--   BUILD_BINARY          : ON
--   BUILD_CUSTOM_PROTOBUF : OFF
--     Protobuf compiler   : 
--     Protobuf includes   : 
--     Protobuf libraries  : 
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : OFF
--   BUILD_SHARED_LIBS     : OFF
--   CAFFE2_USE_MSVC_STATIC_RUNTIME     : ON
--   BUILD_TEST            : OFF
--   BUILD_JNI             : OFF
--   BUILD_MOBILE_AUTOGRAD : OFF
--   BUILD_LITE_INTERPRETER: ON
--   INTERN_BUILD_MOBILE   : ON
--   USE_BLAS              : 1
--     BLAS                : 
--   USE_LAPACK            : 0
--   USE_ASAN              : OFF
--   USE_CPP_CODE_COVERAGE : OFF
--   USE_CUDA              : OFF
--   USE_ROCM              : OFF
--   USE_EIGEN_FOR_BLAS    : ON
--   USE_FBGEMM            : OFF
--     USE_FAKELOWP          : OFF
--   USE_KINETO            : ON
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : OFF
--   USE_GLOG              : OFF
--   USE_LEVELDB           : OFF
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : OFF
--   USE_METAL             : OFF
--   USE_PYTORCH_METAL     : OFF
--   USE_PYTORCH_METAL_EXPORT     : OFF
--   USE_FFTW              : OFF
--   USE_MKL               : 
--   USE_MKLDNN            : OFF
--   USE_NCCL              : OFF
--   USE_NNPACK            : ON
--   USE_NUMPY             : ON
--   USE_OBSERVERS         : OFF
--   USE_OPENCL            : OFF
--   USE_OPENCV            : OFF
--   USE_OPENMP            : OFF
--   USE_TBB               : OFF
--   USE_VULKAN            : ON
--     USE_VULKAN_FP16_INFERENCE    : OFF
--     USE_VULKAN_RELAXED_PRECISION : OFF
--     USE_VULKAN_SHADERC_RUNTIME   : OFF
--   USE_PROF              : OFF
--   USE_QNNPACK           : OFF
--   USE_PYTORCH_QNNPACK   : ON
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF
--   USE_DISTRIBUTED       : OFF
--   USE_DEPLOY           : OFF
--   USE_BREAKPAD         : ON
--   Public Dependencies  : Threads::Threads
--   Private Dependencies : eigen_blas;pthreadpool;cpuinfo;pytorch_qnnpack;nnpack;XNNPACK;VulkanWrapper;fp16;log;fmt::fmt-header-only;kineto;dl
--   USE_COREML_DELEGATE     : OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /home/sam/tmp/pytorch-v1.10.0/build_android
Will install headers and libs to /home/sam/tmp/pytorch-v1.10.0/build_android/install for further Android project usage.
Scanning dependencies of target clog
Scanning dependencies of target pthreadpool
Scanning dependencies of target fmt
Scanning dependencies of target VulkanWrapper
[  1%] Building C object confu-deps/cpuinfo/deps/clog/CMakeFiles/clog.dir/src/clog.c.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/legacy-api.c.o
[  1%] Building CXX object third_party/fmt/CMakeFiles/fmt.dir/src/format.cc.o
[  1%] Building CXX object CMakeFiles/VulkanWrapper.dir/home/sam/CK-SDKs/AndroidNDK-r21/android-ndk-r21e/sources/third_party/vulkan/src/common/vulkan_wrapper.cpp.o
Scanning dependencies of target eigen_blas
[  1%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/single.cpp.o
[  1%] Linking C static library ../../../../lib/libclog.a
[  1%] Built target clog
[  1%] Linking CXX static library lib/libVulkanWrapper.a
Scanning dependencies of target breakpad_common
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/convert_UTF.cc.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/portable-api.c.o
[  1%] Built target VulkanWrapper
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/file_id.cc.o
Scanning dependencies of target libkineto_defs.bzl
[  1%] Built target libkineto_defs.bzl
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/guid_creator.cc.o
Scanning dependencies of target kineto_api
[  1%] Building CXX object third_party/kineto/libkineto/CMakeFiles/kineto_api.dir/src/ThreadUtil.cpp.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/memory_mapped_file.cc.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/memory.c.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/linux_libc_support.cc.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/pthreads.c.o
[  1%] Building C object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/breakpad_getcontext.S.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/elfutils.cc.o
[  1%] Linking C static library ../../lib/libpthreadpool.a
[  1%] Built target pthreadpool
Scanning dependencies of target c10
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/Allocator.cpp.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/safe_readlink.cc.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/string_conversion.cc.o
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/AutogradState.cpp.o
[  1%] Linking CXX static library ../../lib/libbreakpad_common.a
[  1%] Built target breakpad_common
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/CPUAllocator.cpp.o
Scanning dependencies of target ATEN_CPU_FILES_GEN_TARGET
[  1%] Generating ../aten/src/ATen/CPUFunctions.h, ../aten/src/ATen/CPUFunctions_inl.h, ../aten/src/ATen/CompositeExplicitAutogradFunctions.h, ../aten/src/ATen/CompositeExplicitAutogradFunctions_inl.h, ../aten/src/ATen/CompositeImplicitAutogradFunctions.h, ../aten/src/ATen/CompositeImplicitAutogradFunctions_inl.h, ../aten/src/ATen/Declarations.yaml, ../aten/src/ATen/Functions.cpp, ../aten/src/ATen/Functions.h, ../aten/src/ATen/NativeFunctions.h, ../aten/src/ATen/NativeMetaFunctions.h, ../aten/src/ATen/Operators.h, ../aten/src/ATen/Operators_0.cpp, ../aten/src/ATen/Operators_1.cpp, ../aten/src/ATen/Operators_2.cpp, ../aten/src/ATen/Operators_3.cpp, ../aten/src/ATen/Operators_4.cpp, ../aten/src/ATen/RedispatchFunctions.h, ../aten/src/ATen/RegisterBackendSelect.cpp, ../aten/src/ATen/RegisterCPU.cpp, ../aten/src/ATen/RegisterCompositeExplicitAutograd.cpp, ../aten/src/ATen/RegisterCompositeImplicitAutograd.cpp, ../aten/src/ATen/RegisterQuantizedCPU.cpp, ../aten/src/ATen/RegisterSchema.cpp, ../aten/src/ATen/RegistrationDeclarations.h, ../aten/src/ATen/core/ATenOpList.cpp, ../aten/src/ATen/core/TensorBody.h, ../aten/src/ATen/core/TensorMethods.cpp
[  1%] Building CXX object third_party/fmt/CMakeFiles/fmt.dir/src/os.cc.o
[  1%] Linking CXX static library ../../lib/libfmt.a
[  1%] Built target fmt
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/CopyBytes.cpp.o
[  1%] Building CXX object third_party/kineto/libkineto/CMakeFiles/kineto_api.dir/src/libkineto_api.cpp.o
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/DefaultDtype.cpp.o
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/Device.cpp.o
Scanning dependencies of target aot_compiler
[  1%] Building CXX object caffe2/torch/CMakeFiles/aot_compiler.dir/csrc/jit/mobile/nnc/aot_compiler.cpp.o
[  1%] Built target kineto_api
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/DeviceType.cpp.o
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/mobile/nnc/aot_compiler.cpp:1:
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/mobile/nnc/aot_compiler.h:4:
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/ir/ir.h:3:
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/ir/attributes.h:2:
In file included from /home/sam/tmp/pytorch-v1.10.0/aten/src/ATen/ATen.h:9:
In file included from /home/sam/tmp/pytorch-v1.10.0/aten/src/ATen/Context.h:4:
e[1m/home/sam/tmp/pytorch-v1.10.0/aten/src/ATen/Tensor.h:3:10: e[0me[0;1;31mfatal error: e[0me[1m'ATen/core/TensorBody.h' file not founde[0m
#include &lt;ATen/core/TensorBody.h&gt;
e[0;1;32m         ^~~~~~~~~~~~~~~~~~~~~~~~
e[0m[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/DispatchKey.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/DispatchKeySet.cpp.o
1 error generated.
make[2]: *** [caffe2/torch/CMakeFiles/aot_compiler.dir/build.make:63: caffe2/torch/CMakeFiles/aot_compiler.dir/csrc/jit/mobile/nnc/aot_compiler.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1460: caffe2/torch/CMakeFiles/aot_compiler.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/double.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/GeneratorImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/GradMode.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/InferenceMode.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/Scalar.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/Storage.cpp.o
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/complex_single.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/StorageImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/Stream.cpp.o
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/complex_double.cpp.o
[  2%] Built target ATEN_CPU_FILES_GEN_TARGET
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/TensorImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/TensorOptions.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/UndefinedTensorImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/impl/DeviceGuardImplInterface.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/impl/LocalDispatchKeySet.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/impl/SizesAndStrides.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/thread_pool.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/mobile/CPUCachingAllocator.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/mobile/CPUProfilingAllocator.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/Array.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/Backtrace.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/C++17.cpp.o
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/xerbla.cpp.o
[  2%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/srotm.c.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/DeadlockDetection.cpp.o
[  2%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/srotmg.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/drotm.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/drotmg.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/lsame.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/dspmv.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/ssbmv.c.o
[  3%] Building CXX object c10/CMakeFiles/c10.dir/util/Exception.cpp.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/chbmv.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/sspmv.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/zhbmv.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Half.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/chpmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/dsbmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/zhpmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/dtbmv.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/LeftRight.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/stbmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/ctbmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/ztbmv.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Logging.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/d_cnjg.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/r_cnjg.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/MathConstants.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/complexdots.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Metaprogramming.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Optional.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/SmallVector.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/StringUtil.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/ThreadLocalDebugInfo.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Type.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/TypeTraits.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/TypeList.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Unicode.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/UniqueVoidPtr.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/complex_math.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/flags_use_gflags.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/flags_use_no_gflags.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/intrusive_ptr.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/numa.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/signal_handler.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/thread_name.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/typeid.cpp.o
[  4%] Linking CXX static library lib/libeigen_blas.a
[  4%] Built target eigen_blas
[  4%] Linking CXX static library ../lib/libc10.a
[  4%] Built target c10
make: *** [Makefile:141: all] Error 2
</code></pre>

username: dhruvbird
cooked: <p><a class="mention" href="/u/cccclai">@cccclai</a> do you have some idea around why this may be failing to build?</p>

username: cccclai
cooked: <p>I’d recommend to follow guidance <a href="https://github.com/pytorch/pytorch#from-source" class="inline-onebox" rel="noopener nofollow ugc">GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration</a> to make sure we get all needed dependency.</p>
<p>Also is it the first time to build for android?</p>

username: pallgeuer
cooked: <p>I am currently getting the same issue with <code>ATen/core/TensorBody.h</code> not being found while trying to do a normal source compile of PyTorch 1.10.0 (for Linux, not a mobile build). PyTorch 1.9.1 and below build fine for me with the same setup.</p>
<p>I notice the file trying to be compiled when the error happens has ‘mobile’ in the path, so surely there is an issue that needs to be resolved?</p>
<p>I have been very careful with git and submodules etc, that is definitely not the problem.</p>

username: smpurkis
cooked: <p>My aim is to get the <code>speed_benchmark_torch</code> binary to benchmark pytorch models on Android devices.<br>
I’ve been able to do this with v1.6, v1.7, v1.8, v1.9 using</p>
<pre><code class="lang-auto">BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_android.sh -DBUILD_BINARY=ON
</code></pre>
<p>I’ve tried</p>
<pre><code class="lang-auto">bash ./scripts/build_pytorch_android.sh arm64-v8a
</code></pre>
<p>however is doesn’t seem to build the binaries, so for me this is a Blocker issue.<br>
Are there any arguements I can pass to <code>/scripts/build_pytorch_android.sh</code> in order to build the binaries?</p>
<p>The link that you mention doesn’t seem to have anything to do with building <code>speed_benchmark_torch</code> binary.</p>

username: pallgeuer
cooked: <p>It seems the <code>ATen/core/TensorBody.h</code> header file is generated during the build and ends up inside the build folder. When I first tried to compile I got the file not found problem and compilation stopped. Simply resuming the compilation a few times (WITHOUT cleaning the build in between so that it keeps whatever it has built so far) eventually got the whole thing to build. There is evidently an explicit build dependency missing somewhere that ensures that <code>ATen/core/TensorBody.h</code> is generated prior to it being required. Because I use 16 parallel jobs for my build, eventually the header gets built anyway and allows the rest to start succeeding. For your mobile scenario, if repeated retriggers of the build process don’t eventually let it succeed then I suspect that the missing explicit build dependency means that whatever generates <code>ATen/core/TensorBody.h</code> is never pulled into the build at all? Not sure.</p>

username: smpurkis
cooked: <p>When you say build, do you mean building with <code>scripts/build_android.sh</code>? Or building it normally then running <code>/scripts/build_android.sh</code>?</p>
<p>I tried repeatedly running</p>
<pre><code class="lang-auto">BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_android.sh -DBUILD_BINARY=ON
</code></pre>
<p>and it now crashes with</p>
<pre><code class="lang-auto">aot_compiler.cpp:(.text._ZN5torch3jit10tensorexpr16TensorExprKernelD2Ev[_ZN5torch3jit10tensorexpr16TensorExprKernelD2Ev]+0xf8): undefined reference to `c10::UndefinedTensorImpl::_singleton'
aot_compiler.cpp:(.text._ZN5torch3jit10tensorexpr16TensorExprKernelD2Ev[_ZN5torch3jit10tensorexpr16TensorExprKernelD2Ev]+0xfc): undefined reference to `c10::UndefinedTensorImpl::_singleton'
aot_compiler.cpp:(.text._ZN5torch3jit10tensorexpr16TensorExprKernelD2Ev[_ZN5torch3jit10tensorexpr16TensorExprKernelD2Ev]+0x244): undefined reference to `torch::jit::Code::~Code()'
CMakeFiles/aot_compiler.dir/csrc/jit/mobile/nnc/aot_compiler.cpp.o: In function `torch::jit::mobile::nnc::Function::Function()':              
aot_compiler.cpp:(.text._ZN5torch3jit6mobile3nnc8FunctionC2Ev[_ZN5torch3jit6mobile3nnc8FunctionC2Ev]+0x44): undefined reference to `c10::AnyType::get()'
[ 10%] Building C object confu-deps/pytorch_qnnpack/CMakeFiles/pytorch_qnnpack.dir/src/x8zip/x3-neon.c.o                                      
[ 10%] Building C object confu-deps/pytorch_qnnpack/CMakeFiles/pytorch_qnnpack.dir/src/x8zip/x4-neon.c.o                                      
clang++: error: linker command failed with exit code 1 (use -v to see invocation)                                                             
make[2]: *** [caffe2/torch/CMakeFiles/aot_compiler.dir/build.make:84: lib/libaot_compiler.so] Error 1                                         
make[1]: *** [CMakeFiles/Makefile2:1460: caffe2/torch/CMakeFiles/aot_compiler.dir/all] Error 2                                                
make[1]: *** Waiting for unfinished jobs....                                                        
</code></pre>

username: kimishpatel
cooked: <p><a class="mention" href="/u/ljk53">@ljk53</a> is oss build of pt mobile by default also build aot_compiler stuff?</p>

username: smpurkis
cooked: <p>To better debug and give a reproducible environment I’ve made this Dockerfile. It goes through the steps that worked with 1.6-1.9. But doesn’t work for 1.10</p>
<pre><code class="lang-auto">FROM ubuntu:20.04

# Install the required libraries
RUN apt-get update &amp;&amp; DEBIAN_FRONTEND=noninteractiv apt-get install -y  \
    python3-dev \
    python3-pip \
    python-is-python3 \
    cmake  \
    git  \
    sudo  \
    tzdata  \
    android-sdk \
    curl \
    unzip \
    &amp;&amp; apt-get clean

# Download Android NDK 21
RUN curl https://dl.google.com/android/repository/android-ndk-r21e-linux-x86_64.zip --output android-ndk-r21e-linux-x86_64.zip
RUN unzip android-ndk-r21e-linux-x86_64.zip
CMD export ANDROID_NDK="~/android-ndk-r21e/"

# Clone PyTorch Repo and all required submodules
RUN git clone --recursive -b v1.10.0 https://github.com/pytorch/pytorch.git
WORKDIR ./pytorch

# Install PyTorch requirements
RUN echo "precedence ::ffff:0:0/96  100" &gt;&gt; "/etc/gai.conf"
RUN pip install -r requirements.txt

# Set Android NDK Path and Build the Binaries
CMD export ANDROID_NDK="/android-ndk-r21e/" &amp;&amp; BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_android.sh -DBUILD_BINARY=ON
</code></pre>
<p>As you can see from running it, it gives</p>
<pre><code class="lang-auto">/pytorch/aten/src/ATen/Tensor.h:3:10: fatal error: 'ATen/core/TensorBody.h' file not found
</code></pre>
<p>as the error.</p>