Repository: spack/spack

Issue:
Linking R with MKL does currently not work (needs mkl_gf_lp64)
Hi,

to use (a sequential version of) the MKL with R when building it with GCC (and GFortran), it seems necessary to link to three shared MKL libraries: mkl_gf_lp64, mkl_core and mkl_sequential (see https://cran.r-project.org/doc/manuals/r-release/R-admin.html#MKL and https://software.intel.com/en-us/articles/using-intel-mkl-with-r para. B).
The intel-mkl package in Spack does expose the latter two, but not the first (mkl_gf_lp64). Instead, it only provides mkl_intel_lp64 (or mkl_intel_ilp64 when requesting the ilp64 variant). The difference seems to be relevant when GFortran is involved in the build process, see https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/558132.
Unfortunately, this does not raise any immediate (fatal) errors (only some hints) during compile time, but the resulting R installation is not linking to the MKL, falling back to the internal BLAS/LAPACK implementation instead:

Current spack behavior (selected/shortened output):

```
$ spack install -v r@3.5.0 +X +external-lapack ^intel-mkl
==> Installing r
==> Executing phase: 'configure'
==> '/path/to/configure' '--prefix=/path/to/gcc/r' '--libdir=/path/to/rlib' '--enable-R-shlib' '--enable-BLAS-shlib' '--enable-R-framework=no' '--with-tcl-config=/path/to/tclConfig.sh' '--with-tk-config=/path/to/tkConfig.sh' '--with-blas=/path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_lp64.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_sequential.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_core.so /usr/lib64/libpthread.so /usr/lib64/libm.so /usr/lib64/libdl.so' '--with-lapack' '--with-x'
...
checking for dgemm_ in /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_lp64.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_sequential.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_core.so /usr/lib64/libpthread.so /usr/lib64/libm.so /usr/lib64/libdl.so... yes
checking whether double complex BLAS can be used... no
checking iconv.h usability... yes
...
R is now configured for x86_64-pc-linux-gnu
External libraries:        readline, curl
Options enabled:           shared R library, shared BLAS, R profiling
```

and calling sessionInfo() in R:
```
> sessionInfo()
BLAS: /path/to/r/rlib/R/lib/libRblas.so
LAPACK: /path/to/r/rlib/R/lib/libRlapack.so
```

A hacky patch to the R package

```python
        if '+external-lapack' in spec:
            if 'intel-mkl' in spec:
                config_args.extend([
                    '--with-blas={0}{1} {0}{2} {0}{3}'.format(spec['intel-mkl'].prefix.compilers_and_libraries.linux.mkl.lib.intel64, '/libmkl_gf_lp64.so', '/libmkl_core.so', '/libmkl_sequential.so'),
                    '--with-lapack'
                ])

            else:
                config_args.extend([
                    '--with-blas={0}'.format(spec['blas'].libs),
                    '--with-lapack'
                ])
```

leads to:
```
$ spack install -v r@3.5.0 +X +external-lapack ^intel-mkl
==> Installing r
==> Executing phase: 'configure'
==> '/path/to/configure' '--prefix=/path/to/gcc/r' '--libdir=/path/to/rlib' '--enable-R-shlib' '--enable-BLAS-shlib' '--enable-R-framework=no' '--with-tcl-config=/path/to/tclConfig.sh' '--with-tk-config=/path/to/tkConfig.sh' '--with-blas=/path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_gf_lp64.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_core.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_sequential.so' '--with-lapack' '--with-x'
...
checking for dgemm_ in /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_gf_lp64.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_core.so /path/to/mkl/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_sequential.so... yes
checking whether double complex BLAS can be used... yes
checking whether the BLAS is complete... yes
checking for dpstrf_... no
checking for dpstrf_ in -llapack... no
checking iconv.h usability... yes
...
R is now configured for x86_64-pc-linux-gnu
External libraries:        readline, BLAS(generic), curl
Options enabled:           shared R library, shared BLAS, R profiling
```

and

```
> sessionInfo()
`BLAS/LAPACK: /path/to/mkl/compilers_and_libraries_YEAR.X.XXX/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so
```

I think this should probably be handled by the intel-mkl package (possibly generally providing mkl_gf_lp64 instead of mkl_intel_lp64 when GCC/GFortran are used?), but if this breaks other packages/configurations, an R specific solution might be preferred.

I'm happy to provide additional input/information if needed.

Thanks,
Johannes



Discussion:
> possibly generally providing mkl_gf_lp64 instead of mkl_intel_lp64 when GCC/GFortran are used?

I think it's even worse:

> For GNU C compiler, we still go via Intel Compiler, to use mkl_intel_lp64. 
>
> For GNU fortran compiler, we use mkl_gf_lp64,  which is only for GNU fortran compiler and Absoft compiler. 

that's also what link-advisor says as of now if you flip between GNU C/C++ and GNU Fortran  for `MKL2018+Linux+None+GCC+Intel64+Dynamic+32bit+Sequential` https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/  

So you can have a setup with GNU compilers in Spack for both C/C++ and Fortran, but then it depends on the target package being built, if it has Fortran or C/C++ code. And what if it builds both?!.. What a PITA..

Any modifications to Intel packages should wait until https://github.com/spack/spack/pull/7469 is merged. We probably need additional switch in blas/lapack interface to specify whether we intend to build C/C++ or Fortran code. And then, only if GNU is involved and Fortran is _requested_, a different lib is to be returned. @alalazo what do you think?

p.s. I build a whole stack of software with GNU compilers + MKL (petsc,slepc,trilinos,dealii, suite-sparse, etc) and never came across this issue.
@alalazo recently worked on #8449, but it seems it didn't go far enough. I agree that any changes should either go into @mgsternberg's #7469 or after it is merged.
The issue became also apparent when building Octave (with GCC/GFortran) linking against the MKL...

> 1 error found in build log:
     384    checking whether ISAMAX is called correctly from Fortran... yes
     385    checking whether SDOT is called correctly from Fortran... no
     386    checking whether DDOT is called correctly from Fortran... yes
     387    checking whether CDOTU is called correctly from Fortran... yes
     388    checking whether ZDOTU is called correctly from Fortran... yes
     389    checking whether the integer size is correct... no
     390    configure: error: A BLAS library was detected but found incompatible with your Fortran 77 compiler settings.

... and can be resolved by explicitely providing mkl_gf_lp64 (as with R).
can this be merged? this issue doesnt appear to be resolved in any of the referenced commits. and the hacky patch works for me. so shrugs.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/r/package.py b/var/spack/repos/builtin/packages/r/package.py
--- a/var/spack/repos/builtin/packages/r/package.py
+++ b/var/spack/repos/builtin/packages/r/package.py
@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
 
 import os
+import re
 
 from spack import *
 
@@ -70,9 +71,10 @@ class R(AutotoolsPackage):
     depends_on('pango~X', when='~X')
     depends_on('freetype')
     depends_on('tcl')
-    depends_on('tk')
+    depends_on('tk', when='+X')
     depends_on('libx11', when='+X')
     depends_on('libxt', when='+X')
+    depends_on('libxmu', when='+X')
     depends_on('curl')
     depends_on('pcre')
     depends_on('java')
@@ -92,7 +94,6 @@ def configure_args(self):
         prefix = self.prefix
 
         tcl_config_path = join_path(spec['tcl'].prefix.lib, 'tclConfig.sh')
-        tk_config_path = join_path(spec['tk'].prefix.lib, 'tkConfig.sh')
 
         config_args = [
             '--libdir={0}'.format(join_path(prefix, 'rlib')),
@@ -100,14 +101,27 @@ def configure_args(self):
             '--enable-BLAS-shlib',
             '--enable-R-framework=no',
             '--with-tcl-config={0}'.format(tcl_config_path),
-            '--with-tk-config={0}'.format(tk_config_path),
+            'LDFLAGS=-L{0} -Wl,-rpath,{0}'.format(join_path(prefix, 'rlib',
+                                                            'R', 'lib')),
         ]
+        if '^tk' in spec:
+            tk_config_path = join_path(spec['tk'].prefix.lib, 'tkConfig.sh')
+            config_args.append('--with-tk-config={0}'.format(tk_config_path))
 
         if '+external-lapack' in spec:
-            config_args.extend([
-                '--with-blas={0}'.format(spec['blas'].libs),
-                '--with-lapack'
-            ])
+            if '^mkl' in spec and 'gfortran' in self.compiler.fc:
+                mkl_re = re.compile(r'(mkl_)intel(_i?lp64\b)')
+                config_args.extend([
+                    mkl_re.sub(r'\g<1>gf\g<2>',
+                               '--with-blas={0}'.format(
+                                   spec['blas'].libs.ld_flags)),
+                    '--with-lapack'
+                ])
+            else:
+                config_args.extend([
+                    '--with-blas={0}'.format(spec['blas'].libs.ld_flags),
+                    '--with-lapack'
+                ])
 
         if '+X' in spec:
             config_args.append('--with-x')


Test patch: