How to compile CUDA with torch also statically linked?

aviezab: <p>Hello Torch Team &amp; Torch Enthusiasts,</p>
<p>I am compiling torch with given  <code>scripts/build_local.sh</code>  in Linux 64 bit.<br>
I am succeed compiled torch statically but CUDA not.<br>
So I repeated the steps to see what the options.</p>
<p>I found this:<br>
<a href="https://user-images.githubusercontent.com/34294278/90263086-5bc0ea00-de79-11ea-99f7-7312b9b9c2d9.png" rel="nofollow noopener">CUDA Static still OFF but Torch Static On</a></p>
<p>How to set CUDA static link to ON?</p>
<p>This is my CMake Args I made in  <code>build_local.sh</code>  :</p>
<pre><code class="lang-auto">CMAKE_ARGS+=("-DCMAKE_PREFIX_PATH=$(python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())')")
CMAKE_ARGS+=("-DPYTHON_EXECUTABLE=$(python -c 'import sys; print(sys.executable)')")
CMAKE_ARGS+=("-DBUILD_CUSTOM_PROTOBUF=OFF")
CMAKE_ARGS+=("-DBUILD_SHARED_LIBS=OFF")

CMAKE_ARGS+=("-DCUDA_TOOLKIT_ROOT_DIR=/opt/cuda-10.1")
CMAKE_ARGS+=("-DCUDA_INCLUDE_DIRS=/opt/cuda-10.1/bin/include")
CMAKE_ARGS+=("-DCUDA_CUDART_LIBRARY=/opt/cuda-10.1/bin/lib64/libcudart_static.a")
CMAKE_ARGS+=("-DUSE_CUDA=ON")
CMAKE_ARGS+=("-DCMAKE_C_COMPILER=/usr/bin/gcc-8")
CMAKE_ARGS+=("-DCMAKE_CXX_COMPILER=/usr/bin/g++-8")
</code></pre>
<p>Error when compiling (I assume because cuda also included dynamic libs):</p>
<pre><code class="lang-auto">[363/366] Linking CXX shared library lib/libtorch_python.so
FAILED: lib/libtorch_python.so 
: &amp;&amp; /usr/bin/g++-8 -fPIC -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG  -Wl,--no-as-needed -rdynamic -shared -Wl,-soname,libtorch_python.so -o lib/libtorch_python.so caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/generated/python_functions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/generated/python_variable_methods.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/generated/python_torch_functions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/generated/python_nn_functions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/generated/python_fft_functions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/generated/python_linalg_functions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/CudaIPCTypes.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/DataLoader.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Device.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Dtype.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/DynamicTypes.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Exceptions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Generator.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Layout.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/MemoryFormat.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/QScheme.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Module.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/PtrWrapper.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/python_dimname.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Size.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/Storage.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/TypeInfo.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/api/src/python/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/functions/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_anomaly_mode.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_cpp_function.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_engine.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_function.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_hook.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_legacy_variable.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_variable.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/autograd/python_variable_indexing.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/backends/backend_init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/backends/backend_resolver.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/cast_all_constant_to_floating.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/eval_peephole.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/constant_fold.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/eliminate_unused_items.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/fixup_onnx_controlflow.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/function_substitution.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/helper.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/peephole.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/preprocess_for_onnx.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/prepare_division_for_onnx.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/scalar_type_analysis.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/unpack_quantized_weights.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/passes/onnx/prepare_inplace_ops_for_onnx.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_arg_flatten.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_custom_class.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_interpreter.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_ir.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_tracer.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/script_init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/frontend/concrete_module_type.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_sugared_value.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/python/python_tree_views.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/multiprocessing/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/onnx/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/serialization.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/tensor/python_tensor.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/throughput_benchmark.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/cuda_lazy_init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/invalid_arguments.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/object_ptr.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/python_arg_parser.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/python_dispatch.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/structseq.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_apply.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_dtypes.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_layouts.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_memoryformats.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_qschemes.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_list.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_new.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_numpy.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/tensor_types.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/utils/disable_torch_function.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/Event.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/Module.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/python_comm.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/Storage.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/Stream.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/serialization.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/shared/cudart.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/shared/nvtx.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/utils.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/autograd/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/c10d/comm.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/c10d/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/c10d/reducer.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/process_group_agent.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/py_rref.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/python_functions.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/python_rpc_handler.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/request_callback_impl.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/tensorpipe_agent.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/testing/faulty_process_group_agent.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/testing/init.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/unpickled_python_call.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/distributed/rpc/unpickled_python_remote_call.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/jit/runtime/register_distributed_ops.cpp.o caffe2/torch/CMakeFiles/torch_python.dir/csrc/cuda/python_nccl.cpp.o  -Wl,-rpath,/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib:/opt/cuda-10.1/lib:/opt/cuda-10.1/lib64:/usr/lib/openmpi:  lib/libshm.so  /opt/cuda-10.1/lib/libnvToolsExt.so  lib/libc10d.a  lib/libtensorpipe.a  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libtorch.a" -Wl,--no-whole-archive  lib/libtorch.a  lib/libtensorpipe.a  lib/libtensorpipe_uv.a  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libtorch_cpu.a" -Wl,--no-whole-archive  /usr/lib/gcc/x86_64-pc-linux-gnu/8.4.0/libgomp.so  /usr/lib/libpthread.so  /usr/lib/gcc/x86_64-pc-linux-gnu/8.4.0/libgomp.so  /usr/lib/libpthread.so  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libcaffe2_protos.a" -Wl,--no-whole-archive  lib/libqnnpack.a  lib/libpytorch_qnnpack.a  lib/libnnpack.a  lib/libXNNPACK.a  lib/libpthreadpool.a  -lm  lib/libfbgemm.a  lib/libcpuinfo.a  lib/libclog.a  lib/libasmjit.a  lib/libfoxi_loader.a  -lgcc_s  -lgcc  /usr/lib/libblas.so  /usr/lib/liblapack.so  -lm  lib/libdnnl.a  -ldl  sleef/lib/libsleef.a  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libonnx.a" -Wl,--no-whole-archive  lib/libonnx_proto.a  /usr/local/lib/libprotobuf.a  -pthread  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libCaffe2_perfkernels_avx.a" -Wl,--no-whole-archive  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libCaffe2_perfkernels_avx2.a" -Wl,--no-whole-archive  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libCaffe2_perfkernels_avx512.a" -Wl,--no-whole-archive  -Wl,--whole-archive,"/run/media/aviezab/Binary/SourceCode/cpp/pytorch/build/lib/libtorch_cuda.a" -Wl,--no-whole-archive  lib/libc10_cuda.a  lib/libc10.a  /opt/cuda-10.1/lib/libnvToolsExt.so  /opt/cuda-10.1/lib64/libcusparse.so  /opt/cuda-10.1/lib64/libcurand.so  /opt/cuda-10.1/lib64/libcufft.so  /opt/cuda-10.1/lib64/libcurand.so  /opt/cuda-10.1/lib64/libcublas.so  nccl/lib/libnccl_static.a  -lrt  lib/libgloo_cuda.a  lib/libgloo.a  /usr/lib/openmpi/libmpi_cxx.so  /usr/lib/openmpi/libmpi.so  -lpthread  /opt/cuda-10.1/lib64/libcudart.so &amp;&amp; :
/usr/bin/ld: lib/libtensorpipe.a(context.cc.o): relocation R_X86_64_PC32 against symbol `_ZGVZ24TensorPipeVerbosityLevelvE5level' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
[365/366] Building NVCC (Device) object modules/detectron/CMakeFiles/caffe2_detectron_ops_gpu.dir/caffe2_detectron_ops_gpu_generated_upsample_nearest_op.cu.o
ninja: build stopped: subcommand failed.
</code></pre>
<p>Am I supposed to pass  <code>nvcc</code>  build options too?</p>
<p>Thank You for your kindness.</p>

glaringlee: <p><a class="mention" href="/u/aviezab">@aviezab</a><br>
Check this<br>
‘<a href="https://github.com/pytorch/pytorch/blob/31ed46890546f1dce9f730037136d58c411652a4/cmake/public/cuda.cmake#L25" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/31ed46890546f1dce9f730037136d58c411652a4/cmake/public/cuda.cmake#L25</a>’</p>
<p>I am not sure how to statically link this, you can create an issue in github pytorch repo, label it as ‘build’, someone will answer you there.</p>