Failed to build torchaudio Python

jiapei100: <p>I successfully built <strong>torchaudio</strong> by <strong>cmake</strong>, but it looks when I tried to <code>TORCH_CUDA_ARCH_LIST="8.6" python setup.py build</code>, it seems <strong>mkl libraries are NOT found</strong>, which I’m sure are under <strong>LD_LIBRARY_PATH</strong>.<br>
Can anybody help please?</p>
<pre data-code-wrap="console"><code class="lang-nohighlight">➜  audio git:(main) TORCH_CUDA_ARCH_LIST="8.6" python setup.py bdist_wheel
-- Git branch: main
-- Git SHA: b076abd17bb93a61ef9cc7d4265dab8b1e154dd8
-- Git tag: None
-- PyTorch dependency: torch
-- Building version 0.13.0a0+b076abd
 --- Initializing submodules
 --- Initialized submodule
running bdist_wheel
running build
running build_py
copying torchaudio/version.py -&gt; build/lib.linux-x86_64-cpython-310/torchaudio
running build_ext
-- Caffe2: Found protobuf with new-style protobuf targets.
-- Caffe2: Protobuf version 3.13.0.0
-- Caffe2: CUDA detected: 11.7
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 11.7
-- Found cuDNN: v8.5.0  (include: /usr/local/cuda/include, library: /usr/local/cuda/lib/libcudnn.so)
-- /usr/local/cuda/lib/libnvrtc.so shorthash is 581f1f99
CMake Warning at ~/.local/lib/python3.10/site-packages/torch/share/cmake/Caffe2/public/utils.cmake:385 (message):
  In the future we will require one to explicitly pass TORCH_CUDA_ARCH_LIST
  to cmake instead of implicitly setting it as an env variable.  This will
  become a FATAL_ERROR in future version of pytorch.
Call Stack (most recent call first):
  ~/.local/lib/python3.10/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:437 (torch_cuda_get_nvcc_gencode_flag)
  ~/.local/lib/python3.10/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:92 (include)
  ~/.local/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  cmake/TorchAudioHelper.cmake:1 (find_package)
  CMakeLists.txt:88 (include)


-- Added CUDA NVCC flags for: -gencode;arch=compute_86,code=sm_86
-- MKL_ARCH: intel64
-- MKL_ROOT /opt/intel/oneapi/mkl/2022.2.0
-- MKL_LINK: dynamic
-- MKL_INTERFACE_FULL: intel_ilp64
-- MKL_THREADING: intel_thread
-- MKL_MPI: intelmpi
-- Configuring done
-- Generating done
-- Build files have been written to: ....../audio/build/temp.linux-x86_64-cpython-310
[1/3] : &amp;&amp; /usr/bin/c++ -fPIC -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG   -shared -Wl,-soname,libtorchaudio.so -o torchaudio/csrc/libtorchaudio.so torchaudio/csrc/CMakeFiles/libtorchaudio.dir/lfilter.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/overdrive.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/utils.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/cpu/compute_alphas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/cpu/compute_betas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/cpu/compute.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/compute_alphas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/compute_betas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/compute.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/autograd.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/gpu/compute_alphas.cu.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/gpu/compute_betas.cu.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/gpu/compute.cu.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/kaldi.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/io.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/utils.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/effects.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/effects_chain.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/types.cpp.o -L/usr/local/cuda/targets/x86_64-linux/lib/stubs   -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,~/.local/lib/python3.10/site-packages/torch/lib:/usr/local/cuda/lib64:/usr/local/cuda/lib:  ~/.local/lib/python3.10/site-packages/torch/lib/libtorch.so  ~/.local/lib/python3.10/site-packages/torch/lib/libc10_cuda.so  /usr/local/cuda/lib64/libcudart.so  third_party/kaldi/libkaldi.a  -Wl,--no-as-needed,"~/.local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so" -Wl,--as-needed  /usr/local/lib/libprotobuf.a  -lmkl_intel_ilp64  -lmkl_core  -lmkl_intel_thread  -Wl,--no-as-needed,"~/.local/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so" -Wl,--as-needed  ~/.local/lib/python3.10/site-packages/torch/lib/libc10_cuda.so  ~/.local/lib/python3.10/site-packages/torch/lib/libc10.so  /usr/local/cuda/lib/libnvToolsExt.so  /usr/local/cuda/lib64/libcufft.so  /usr/local/cuda/lib64/libcurand.so  /usr/local/cuda/lib64/libcublas.so  /usr/local/cuda/lib/libcudnn.so  /usr/local/cuda/lib64/libcudart.so  ../../third_party/sox/../install/lib/libsox.a  ../../third_party/sox/../install/lib/libopencore-amrnb.a  ../../third_party/sox/../install/lib/libopencore-amrwb.a  ../../third_party/sox/../install/lib/libmp3lame.a  ../../third_party/sox/../install/lib/libFLAC.a  ../../third_party/sox/../install/lib/libopusfile.a  ../../third_party/sox/../install/lib/libopus.a  ../../third_party/sox/../install/lib/libvorbisenc.a  ../../third_party/sox/../install/lib/libvorbisfile.a  ../../third_party/sox/../install/lib/libvorbis.a  ../../third_party/sox/../install/lib/libogg.a  /usr/lib/gcc/x86_64-linux-gnu/11/libgomp.so  /usr/lib/x86_64-linux-gnu/libpthread.a  -lcudadevrt  -lcudart_static  -lrt  -lpthread  -ldl &amp;&amp; :
FAILED: torchaudio/csrc/libtorchaudio.so 
: &amp;&amp; /usr/bin/c++ -fPIC -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG   -shared -Wl,-soname,libtorchaudio.so -o torchaudio/csrc/libtorchaudio.so torchaudio/csrc/CMakeFiles/libtorchaudio.dir/lfilter.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/overdrive.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/utils.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/cpu/compute_alphas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/cpu/compute_betas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/cpu/compute.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/compute_alphas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/compute_betas.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/compute.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/autograd.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/gpu/compute_alphas.cu.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/gpu/compute_betas.cu.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/rnnt/gpu/compute.cu.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/kaldi.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/io.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/utils.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/effects.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/effects_chain.cpp.o torchaudio/csrc/CMakeFiles/libtorchaudio.dir/sox/types.cpp.o -L/usr/local/cuda/targets/x86_64-linux/lib/stubs   -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,~/.local/lib/python3.10/site-packages/torch/lib:/usr/local/cuda/lib64:/usr/local/cuda/lib:  ~/.local/lib/python3.10/site-packages/torch/lib/libtorch.so  ~/.local/lib/python3.10/site-packages/torch/lib/libc10_cuda.so  /usr/local/cuda/lib64/libcudart.so  third_party/kaldi/libkaldi.a  -Wl,--no-as-needed,"~/.local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so" -Wl,--as-needed  /usr/local/lib/libprotobuf.a  -lmkl_intel_ilp64  -lmkl_core  -lmkl_intel_thread  -Wl,--no-as-needed,"~/.local/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so" -Wl,--as-needed  ~/.local/lib/python3.10/site-packages/torch/lib/libc10_cuda.so  ~/.local/lib/python3.10/site-packages/torch/lib/libc10.so  /usr/local/cuda/lib/libnvToolsExt.so  /usr/local/cuda/lib64/libcufft.so  /usr/local/cuda/lib64/libcurand.so  /usr/local/cuda/lib64/libcublas.so  /usr/local/cuda/lib/libcudnn.so  /usr/local/cuda/lib64/libcudart.so  ../../third_party/sox/../install/lib/libsox.a  ../../third_party/sox/../install/lib/libopencore-amrnb.a  ../../third_party/sox/../install/lib/libopencore-amrwb.a  ../../third_party/sox/../install/lib/libmp3lame.a  ../../third_party/sox/../install/lib/libFLAC.a  ../../third_party/sox/../install/lib/libopusfile.a  ../../third_party/sox/../install/lib/libopus.a  ../../third_party/sox/../install/lib/libvorbisenc.a  ../../third_party/sox/../install/lib/libvorbisfile.a  ../../third_party/sox/../install/lib/libvorbis.a  ../../third_party/sox/../install/lib/libogg.a  /usr/lib/gcc/x86_64-linux-gnu/11/libgomp.so  /usr/lib/x86_64-linux-gnu/libpthread.a  -lcudadevrt  -lcudart_static  -lrt  -lpthread  -ldl &amp;&amp; :
/usr/bin/ld: cannot find -lmkl_intel_ilp64: No such file or directory
/usr/bin/ld: cannot find -lmkl_core: No such file or directory
/usr/bin/ld: cannot find -lmkl_intel_thread: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "....../audio/setup.py", line 183, in &lt;module&gt;
    _main()
  File "....../audio/setup.py", line 147, in _main
    setup(
  File "~/.local/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "~/.local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
    self.run_command('build')
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "~/.local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 132, in run
    self.run_command(cmd_name)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "~/.local/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "....../audio/tools/setup_helpers/extension.py", line 78, in run
    super().run()
  File "~/.local/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
    _build_ext.run(self)
  File "~/.local/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
    self.build_extensions()
  File "~/.local/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
    self._build_extensions_serial()
  File "~/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
    self.build_extension(ext)
  File "....../audio/tools/setup_helpers/extension.py", line 153, in build_extension
    subprocess.check_call(["cmake", "--build", "."] + build_args, cwd=self.build_temp)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install']' returned non-zero exit status 1.
</code></pre>

ptrblck: <aside class="quote no-group" data-username="jiapei100" data-post="1" data-topic="162811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jiapei100/48/79_2.png" class="avatar"> jiapei100:</div>
<blockquote>
<pre><code class="lang-auto">/usr/bin/ld: cannot find -lmkl_intel_ilp64: No such file or directory
/usr/bin/ld: cannot find -lmkl_core: No such file or directory
/usr/bin/ld: cannot find -lmkl_intel_thread: No such file or directory
collect2: error: ld returned 1 exit status
</code></pre>
</blockquote>
</aside>
<p>It seems your build expects to find MKL and fails to do so.</p>

jiapei100: <p>Yes… That’s obvious…<br>
As mentioned in my previous message, <strong>mkl is already under my LD_LIBRARY_PATH</strong> …</p>
<p>How to let <strong>mkl</strong> be found by <code>TORCH_CUDA_ARCH_LIST="8.6"  python setup.py build</code> ?<br>
By <strong>cmake</strong>, I can specify <strong>mkl</strong> paths…</p>

ptrblck: <p><code>LD_LIBRARY_PATH</code> will search for shared libs during the runtime, so try to use <code>LIBRARY_PATH</code>.</p>

jiapei100: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a> .</p>
<pre data-code-wrap="console"><code class="lang-nohighlight">LIBRARY_PATH="/opt/intel/oneapi/mkl/latest/lib/intel64" TORCH_CUDA_ARCH_LIST="8.6" python setup.py build
</code></pre>
<p>works…</p>
<p>However, I still failed to <code>import torchaudio</code>. Please refer to <a href="https://discuss.pytorch.org/t/torchaudio-installed-but-failed-to-import/163157">https://discuss.pytorch.org/t/torchaudio-installed-but-failed-to-import/163157</a> .</p>
<p>Thank you</p>