Repository: numba/numba

Issue:
Detect whether LLVM has SVML enabled and override SVML setting if not present
The bug in #2998 (and likely the one reported in the discussion of #2938) has revealed a situation where array expressions will give the wrong answer.  Triggering this bug requires:

* The ICC runtime (specifically libsvml) is present in the user's environment
* The user is using an llvmlite statically linked against a version of LLVM that has not been patched with SVML support.
* [Edit: Forgot one more condition] The platform is 64-bit.

We incorrectly assumed that this combination would be extremely unlikely, and if did happen, LLVM would raise an error.  This was a mistake, as it seems:

1. There are a number of Windows users getting llvmlite from the [Chris Gohlke packages](https://www.lfd.uci.edu/~gohlke/pythonlibs/) or from conda-forge, which do not apply all of our LLVM patches.  (Note that anyone who gets llvmlite from the Numba channel on Anaconda Cloud, the default channel from Anaconda, or pip installs the wheels posted to PyPI are unaffected by this problem.)
2. Many Windows users of Numba have the ICC runtime present as Intel Fortran is commonly used to compile SciPy for Windows due to challenges getting other Fortran compilers to work on Windows.
3. Unpatched LLVM does not raise an error when we enable SVML autovectorization, but rather seems to generate incorrect code instead.

To correct this, we're going to add a function to llvmlite to verify if the LLVM being used has the patch.  (Note, as LLVM is statically linked to llvmlite, it does not matter what LLVM is present on the end user system, only which LLVM is used to build llvmlite.)  Numba will look for this function, and if it is not present or returns false (SVML support not available), it will disable the SVML code generation on that platform, regardless of whether the ICC runtime is present.

Although it affects a small number of users, we view this as a critical bug, and will be making patch releases of Numba 0.38 and llvmlite 0.23 outside of our normal release cycle.
Wrong results when calculating power with an array
Since Numba 0.38.0, the following function leads to wrong results:

```python
import numpy as np
import numba as nb

def test(a, b):
    return (1. / a) ** b

test_nb = nb.njit(test)
```

Example:

```python
>>> test_nb(0.1 * np.ones(50), np.ones(50))
array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
        1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
        1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
        1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., 10., 10.])
```

Expected output (with NumPy):

```python
>>> test(0.1 * np.ones(50), np.ones(50))
array([10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10.,
       10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10.,
       10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10.,
       10., 10., 10., 10., 10., 10., 10., 10., 10., 10., 10.])
```


I have only encountered this bug on Windows, and it does not occur on Numba 0.37.0. Relevant output from `conda list`:

```
llvmlite                  0.23.0                   py36_1    conda-forge
numba                     0.38.0           py36h830ac7b_0
numpy                     1.14.2           py36h5c71026_0
python                    3.6.0                         0
```

Let me know if you need more information.


Discussion:
Note that as mentioned in the linked issue, the workaround before these new releases is to set:
```
NUMBA_DISABLE_INTEL_SVML=1
```
in your environment.
FWIW, I rebuilt the llvmlite binaries at https://www.lfd.uci.edu/~gohlke/pythonlibs/#llvmlite using "the patches".
@dionhaefner thanks for the report. Given what you present, please could you try setting the environment variable `NUMBA_DISABLE_INTEL_SVML` to non-zero and re-running. I'm suggesting this as Windows 64 bit is the only platform with SVML by default and it appeared in 0.38. Thanks.
Setting `NUMBA_DISABLE_INTEL_SVML=1` indeed fixes the issue. Thanks for the quick reaction.
Great, glad that worked, and thanks for getting back to us. I also note that you are using a `conda-forge` llvmlite, does the same happen if you use `llvmlite` from defaults?

FTR, cannot replicate on linux with SVML present.
If this turns out to be a bug that also affects our Windows builds of llvmlite, we will want to put out a patch release that defaults `NUMBA_DISABLE_INTEL_SVML=0`.  If the problem is only with conda-forge builds, we'll need to escalate it to whoever manages that recipe ASAP.
Installing `llvmlite==0.23.1` from `defaults` also fixed the issue. So it seems to be `conda-forge` specific.
OK, thanks for checking.  I'm going to keep this issue open while we figure out what the best solution with conda-forge is.
RE: https://github.com/numba/numba/issues/2998#issuecomment-392871302 I wonder whether, for cross platform consistency, we ought to switch `SVML` off (by default) on windows?
Are we sure that the problem is only Windows?  If the conda-forge llvmlite is installed on a Linux box with SVML present, what happens?

I'm worried that instead we need to be able to detect when llvmlite has been linked against an unpatched LLVM (regardless of platform), and then disable SVML support in that case, regardless of whether the library is present.

Resolving patch:
diff --git a/numba/__init__.py b/numba/__init__.py
--- a/numba/__init__.py
+++ b/numba/__init__.py
@@ -6,6 +6,7 @@
 import platform
 import re
 import sys
+import warnings
 
 from . import config, errors, runtests, types
 
@@ -118,10 +119,36 @@ def _try_enable_svml():
                 llvmlite.binding.load_library_permanently("svml_dispmd")
             else:
                 return False
+            # The SVML library is loaded, therefore SVML *could* be supported.
+            # Now see if LLVM has been compiled with the SVML support patch.
+            # If llvmlite has the checking function `has_svml` and it returns
+            # True, then LLVM was compiled with SVML support and the the setup
+            # for SVML can proceed. We err on the side of caution and if the
+            # checking function is missing, regardless of that being fine for
+            # most 0.23.{0,1} llvmlite instances (i.e. conda or pip installed),
+            # we assume that SVML was not compiled in. llvmlite 0.23.2 is a
+            # bugfix release with the checking function present that will always
+            # produce correct behaviour. For context see: #3006.
+            try:
+                if not getattr(llvmlite.binding.targets, "has_svml")():
+                    # has detection function, but no svml compiled in, therefore
+                    # disable SVML
+                    return False
+            except AttributeError:
+                if platform.machine() == 'x86_64' and config.DEBUG:
+                    msg = ("SVML was found but llvmlite >= 0.23.2 is "
+                           "needed to support it.")
+                    warnings.warn(msg)
+                # does not have detection function, cannot detect reliably,
+                # disable SVML.
+                return False
+
+            # All is well, detection function present and reports SVML is
+            # compiled in, set the vector library to SVML.
             llvmlite.binding.set_option('SVML', '-vector-library=SVML')
             return True
         except:
-            if platform.machine == 'x86_64' and config.DEBUG:
+            if platform.machine() == 'x86_64' and config.DEBUG:
                 warnings.warn("SVML was not found/could not be loaded.")
     return False
 


Test patch: