Repository: numpy/numpy

Issue:
MinGW build broken in einsum.c.src
Standard MinGW 3.4.5 build chain as used for releases:

```
gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes -DHAVE_NPY_CONFIG_H=1 -DNPY_MINGW_USE_CUSTOM_MSVCR -D__MSVCRT_VERSION__=0x0900 -Inumpy\core\include -Ibuild\src.win32-3.2\numpy\core\include/numpy -Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -Inumpy\core\include -IC:\Python32\include -IC:\Python32\PC -c build\src.win32-3.2\numpy\core\src\multiarray\einsum.c -o build\temp.win32-3.2\Release\build\src.win32-3.2\numpy\core\src\multiarray\einsum.o
In file included from C:/Python32/include/Python.h:112,
                 from numpy\core\src\multiarray\einsum.c.src:12:
C:/Python32/include/pythonrun.h:172: warning: function declaration isn't a prototype
numpy\core\src\multiarray\einsum.c.src: In function `double_sum_of_products_contig_outstride0_one':
numpy\core\src\multiarray\einsum.c.src:1590: error: `__m128d' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:1590: error: (Each undeclared identifier is reported only once
numpy\core\src\multiarray\einsum.c.src:1590: error: for each function it appears in.)
numpy\core\src\multiarray\einsum.c.src:1590: error: syntax error before "a"
numpy\core\src\multiarray\einsum.c.src:1120: warning: implicit declaration of function `_mm_prefetch'
numpy\core\src\multiarray\einsum.c.src:1120: error: `_MM_HINT_T0' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:1124: error: `accum_sse' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:1124: warning: implicit declaration of function `_mm_add_pd'
numpy\core\src\multiarray\einsum.c.src:1124: warning: implicit declaration of function `_mm_load_pd'
numpy\core\src\multiarray\einsum.c.src:1130: error: `a' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:1130: warning: implicit declaration of function `_mm_shuffle_pd'
numpy\core\src\multiarray\einsum.c.src:1130: warning: implicit declaration of function `_MM_SHUFFLE2'
numpy\core\src\multiarray\einsum.c.src:1132: warning: implicit declaration of function `_mm_store_sd'
numpy\core\src\multiarray\einsum.c.src:1165: warning: implicit declaration of function `_mm_loadu_pd'
numpy\core\src\multiarray\einsum.c.src: In function `double_sum_of_products_stride0_contig_outcontig_two':
numpy\core\src\multiarray\einsum.c.src:371: error: `__m128d' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:371: error: syntax error before "a"
numpy\core\src\multiarray\einsum.c.src:424: error: `value0_sse' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:424: warning: implicit declaration of function `_mm_set1_pd'
numpy\core\src\multiarray\einsum.c.src:432: error: `a' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:432: warning: implicit declaration of function `_mm_mul_pd'
numpy\core\src\multiarray\einsum.c.src:433: error: `b' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:434: warning: implicit declaration of function `_mm_store_pd'
numpy\core\src\multiarray\einsum.c.src:468: warning: implicit declaration of function `_mm_storeu_pd'
numpy\core\src\multiarray\einsum.c.src: In function `double_sum_of_products_contig_contig_outstride0_two':
numpy\core\src\multiarray\einsum.c.src:584: error: `__m128d' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:584: error: syntax error before "a"
numpy\core\src\multiarray\einsum.c.src:651: error: `_MM_HINT_T0' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:655: error: `a' undeclared (first use in this function)
numpy\core\src\multiarray\einsum.c.src:656: error: `accum_sse' undeclared (first use in this function)
error: Command "gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes -DHAVE_NPY_CONFIG_H=1 -DNPY_MINGW_USE_CUSTOM_MSVCR -D__MSVCRT_VERSION__=0x0900 -Inumpy\core\include -Ibuild\src.win32-3.2\numpy\core\include/numpy -Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -Inumpy\core\include -IC:\Python32\include -IC:\Python32\PC -c build\src.win32-3.2\numpy\core\src\multiarray\einsum.c -o build\temp.win32-3.2\Release\build\src.win32-3.2\numpy\core\src\multiarray\einsum.o" failed with exit status 1
```



Discussion:
Looks like it is missing some header files. The relevant changes are

```
-#ifdef __SSE__
+#ifdef HAVE_XMMINTRIN_H
 #define EINSUM_USE_SSE1 1
 #else
 #define EINSUM_USE_SSE1 0
@@ -31,7 +32,7 @@
 /*
  * TODO: Only some SSE2 for float64 is implemented.
  */
-#ifdef __SSE2__
+#ifdef HAVE_EMMINTRIN_H
 #define EINSUM_USE_SSE2 1
 #else
 #define EINSUM_USE_SSE2 0

```

So the configuration is failing. I assume you aren't using bento. @juliantaylor made the changes, but why the configuration is failing I don't know. Are you running on an actual Windows box or in WINE?

does `#include <emmintrin.h>` work with mingw?
please try with -msse2 and -mno-sse2, the latter should not compile.

if its available but incomplete in mingw we might need more detailed checks for intrinsics in setup.py.

Still, it looks like this may have uncovered some previously undetected bugs in the code.

I explicitly set the configuration checks to zero and it still compiled without error messages, and because I don't see a message about failing include, I'm going to guess that there is something different about the include file. The question is, why did it work before? Maybe the sse stuff wasn't used. How much we can assume about sse1/sse2? They are both very old, but perhaps in distributing binaries we should be explicit about using them via compiler flags in the 32 bit case. The automatic configuration might be insufficiently generic for that situation. IIRC, the old 32 bit generic ATLAS binaries distributed by Debian omitted both, the SSE versions were available separately.

So I think the thing to do here is go back to the old macros.

Or possibly use both with && so that the flags could override the detection.

sse2 was working before. I built the superpack installers with Wine + MinGW 3.4.5, those include full builds with sse2 and sse3.

are you sure about that?
I tried mingw32 4.2.1 and it doesn't define `__SSE2__`, so it wasn't using the code.

emmintrin.h includes on mingw 4.2.1 even if the sse2 is disabled which explains the error one gets now
so we need finer grained checks for it.

I haven't looked into those details myself but am going by https://github.com/numpy/vendor/tree/master/binaries. I assume that that build config info is correct and we're not shipping 3x the exact same binaries in the superpack. Maybe @cournape can confirm.

@charris good question about whether we can assume SSE2 being present now. It would make things a bit simpler.


Resolving patch:
diff --git a/numpy/core/setup.py b/numpy/core/setup.py
--- a/numpy/core/setup.py
+++ b/numpy/core/setup.py
@@ -165,8 +165,14 @@ def check_funcs(funcs_name):
         if config.check_func("", decl=False, call=False, headers=[h]):
             moredefs.append((fname2def(h).replace(".", "_"), 1))
 
-    for f, args in OPTIONAL_INTRINSICS:
-        if config.check_func(f, decl=False, call=True, call_args=args):
+    for tup in OPTIONAL_INTRINSICS:
+        headers = None
+        if len(tup) == 2:
+            f, args = tup
+        else:
+            f, args, headers = tup[0], tup[1], [tup[2]]
+        if config.check_func(f, decl=False, call=True, call_args=args,
+                             headers=headers):
             moredefs.append((fname2def(f), 1))
 
     for dec, fn in OPTIONAL_GCC_ATTRIBUTES:
diff --git a/numpy/core/setup_common.py b/numpy/core/setup_common.py
--- a/numpy/core/setup_common.py
+++ b/numpy/core/setup_common.py
@@ -107,7 +107,8 @@ def check_api_version(apiversion, codegen_dir):
                 "emmintrin.h", # SSE2
 ]
 
-# optional gcc compiler builtins and their call arguments
+# optional gcc compiler builtins and their call arguments and optional a
+# required header
 # call arguments are required as the compiler will do strict signature checking
 OPTIONAL_INTRINSICS = [("__builtin_isnan", '5.'),
                        ("__builtin_isinf", '5.'),
@@ -115,6 +116,8 @@ def check_api_version(apiversion, codegen_dir):
                        ("__builtin_bswap32", '5u'),
                        ("__builtin_bswap64", '5u'),
                        ("__builtin_expect", '5, 0'),
+                       ("_mm_load_ps", '(float*)0', "xmmintrin.h"), # SSE
+                       ("_mm_load_pd", '(double*)0', "emmintrin.h"), # SSE2
                        ]
 
 # gcc function attributes


Test patch: