Pytorch C++ with Java Native Interface (JNI) linking error

jparsert: <p>Hi, I am running into a linking problem when running Torch with JNI. I am not sure if this is the right forum as it might not be directly related to torch but maybe someone might be able to help me:</p>
<p>I have a java program Main.java that has native functions declared which are implemented in torchJNI.cpp and defined in torchJNI.h. Of course, for this to work I also load torchJNI with <code>System.loadLibrary("torchJNI");</code><br>
This works when I create tensors in the native functions and print them etc. The problem comes, when I do more complex things (loss.backward() and optim.step()). In particular, functions that depend on the MKL shared library where the .so files are in /opt/intel/…/.<br>
I implemented a main function in torchJNI.cpp which does training. When compiling and running the torchJNI.cpp file this all of this works perfectly fine. However, when I make a shared library out of torchJNI.cpp/torchJNI.h which gets loaded with JNI and4 I try to run the native method that “has the exact same code” as the main function I get the following error:</p>
<pre><code class="lang-auto">INTEL MKL ERROR: /opt/intel/mkl/lib/intel64/libmkl_vml_avx2.so: undefined symbol: mkl_lapack_dspevd.    
Intel MKL FATAL ERROR: cannot load libmkl_vml_avx2.so or libmkl_vml_def.so.
</code></pre>
<p>now I though okay I have to do the same thing to load this shared library, so I added the following lines to the java file:</p>
<pre><code class="lang-auto"> System.loadLibrary("mkl_vml_avx2");     
 System.loadLibrary("mkl_vml_def");
</code></pre>
<p>Now, when I try to run the java file I get:</p>
<pre><code class="lang-auto">Exception in thread "main" java.lang.UnsatisfiedLinkError: /opt/intel/mkl/lib/intel64/libmkl_vml_avx2.so: /opt/intel/mkl/lib/intel64/libmkl_vml_avx2.so: undefined symbol: mkl_lapack_dspevd ....
</code></pre>
<p>note, that I run Java with -Djava.library.path=:./path/to/torchJNI/:/opt/intel/mkl/lib/intel64<br>
I looked, and both libmkl_vml_avx2.so and libmkl_vml_def.so are present in /opt/intel/mkl/lib/intel64.</p>
<p>To summarize:</p>
<ol>
<li>
<p>Everything seems to work fine (compilation and runnning) as long as I stay completely in C++.</p>
</li>
<li>
<p>When I invoke a native function from java that only does simple Torch things that do not require mkl everything also works.</p>
</li>
<li>
<p>When I invoke a native function from java that uses the torch function backward (for example) I get an INTEL MKL ERROR … undefined symbol: mkl_lapack_dspevd</p>
</li>
<li>
<p>Trying to fix 3. I tried loading  libmkl_vml_avx2.so or libmkl_vml_def.so. in Java manually then the error messages changes to a java exception.</p>
</li>
</ol>
<p>Can anyone help me out or take a guess what the problem may be?</p>

saudet: <p>This is more of a FYI, but I’ve recently started working on the JavaCPP Presets for PyTorch and that takes care of all those linking issues with MKL and what not by bundling everything in JAR files (actually Maven artifacts):</p><aside class="onebox githubfolder">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars.githubusercontent.com/u/5856852?s=400&amp;amp;v=4" class="thumbnail onebox-full-image" width="60" height="60">

<h3><a href="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch" target="_blank" rel="noopener nofollow ugc">bytedeco/javacpp-presets</a></h3>

<p><a href="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch" target="_blank" rel="noopener nofollow ugc">master/pytorch</a></p>

  <p><span class="label1">The missing Java distribution of native C++ libraries</span></p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>