Repository: numpy/numpy

Issue:
Building numpy on AIX (xlc and xlf)
<!-- Please describe the issue in detail here, and fill in the fields below -->
build for AIX is not working.

A couple of issues - maybe all simple to fix - still researching the last one...

First one: numpy/fft/_pocketfft.c does not compile -- resolution: `export CFLAGS=-D_LARGE_FILES`

Second issue: Does not find ld_so_aix - expects, it seems, a symbolic link to the correct directory. e.g., calls `/opt/lib/python3.6/config/ld_so_aix` when the actual path is `/opt/lib/python3.6/config-3.6m/ld_so_aix`

Third issue: same issue as issue #6609 - undefined symbols:
```
ld: 0711-317 ERROR: Undefined symbol: .zungqr_
ld: 0711-317 ERROR: Undefined symbol: .zgeqrf_
ld: 0711-317 ERROR: Undefined symbol: .zgelsd_
ld: 0711-317 ERROR: Undefined symbol: .dorgqr_
ld: 0711-317 ERROR: Undefined symbol: .dgeqrf_
ld: 0711-317 ERROR: Undefined symbol: .dgelsd_
```

The first issues are easy to get around - export CFLAGS and add a symbolic link for the generic path.

For the final issue I am going to guess that there is a macro involved - that is adding an underscore to the function being called - as they do exist in liblapack.a

### Reproducing code example:

`python setup.py build`

<!-- Remove these sections for a feature request -->

### Error message:
More detail:
```
ld: 0711-318 ERROR: Undefined symbols were found.
        The following symbols are in error:
 Symbol                    Inpndx  TY CL Source-File(Object-File) OR Import-File{Shared-object}
                              RLD: Address  Section  Rld-type Referencing Symbol
 ----------------------------------------------------------------------------------------------
 .zungqr_                  [124]   ER PR numpy/linalg/lapack_litemodule.c(build/temp.AIX.6106.1043.64-3.6/numpy/linalg/lapack_litemodule.o)
                                   000002c0 .text    R_RBR    [10]    <.lapack_lite_zungqr>
 .zgeqrf_                  [128]   ER PR numpy/linalg/lapack_litemodule.c(build/temp.AIX.6106.1043.64-3.6/numpy/linalg/lapack_litemodule.o)
                                   00000454 .text    R_RBR    [12]    <.lapack_lite_zgeqrf>
 .zgelsd_                  [130]   ER PR numpy/linalg/lapack_litemodule.c(build/temp.AIX.6106.1043.64-3.6/numpy/linalg/lapack_litemodule.o)
                                   000006fc .text    R_RBR    [14]    <.lapack_lite_zgelsd>
 .dorgqr_                  [132]   ER PR numpy/linalg/lapack_litemodule.c(build/temp.AIX.6106.1043.64-3.6/numpy/linalg/lapack_litemodule.o)
                                   00000920 .text    R_RBR    [16]    <.lapack_lite_dorgqr>
 .dgeqrf_                  [134]   ER PR numpy/linalg/lapack_litemodule.c(build/temp.AIX.6106.1043.64-3.6/numpy/linalg/lapack_litemodule.o)
                                   00000a94 .text    R_RBR    [18]    <.lapack_lite_dgeqrf>
 .dgelsd_                  [136]   ER PR numpy/linalg/lapack_litemodule.c(build/temp.AIX.6106.1043.64-3.6/numpy/linalg/lapack_litemodule.o)
                                   00000d3c .text    R_RBR    [20]    <.lapack_lite_dgelsd>
ER: The return code is 8.


```

Note: They do exist - just without the trailing underscore (_)

```
(py36) root@x066:[/data/prj/python/numpy-1.17.4]nm -Ae /usr/lib/liblapack.a | egrep "zungqr|zgeqrf|zgelsd|dorgqr|dgeqrf|dgelsd"_ | grep -v " U " | egrep ": \.[a-z]"
/usr/lib/liblapack.a[dgeqrf.o]: .dgeqrf              T           0
/usr/lib/liblapack.a[dgeqrfp.o]: .dgeqrfp             T           0
/usr/lib/liblapack.a[dorgqr.o]: .dorgqr              T           0
/usr/lib/liblapack.a[zgelsd.o]: .zgelsd              T           0
/usr/lib/liblapack.a[zgeqrf.o]: .zgeqrf              T           0
/usr/lib/liblapack.a[zgeqrfp.o]: .zgeqrfp             T           0
/usr/lib/liblapack.a[zungqr.o]: .zungqr              T           0

```

* UPDATE: this is resolved by adding -DNO_APPEND_FORTRAN to CFLAGS

* Now I am open to suggestions on where and how to best get these automated in setup.py process.


Discussion:
Thanks for the report @aixtools. Please have a look at how `_LARGE_FILES` is special-cased for AIX in `numpy/random/setup.py`. I think you can apply these fixes in the same way.
I have a draft - for a PR - but needs refinement. See https://github.com/aixtools/numpy/tree/15801-fft-largefiles.

Not looked for the first issue (finding the correct config directory), but the flags are 'corrected'.

The patch, asis, works, but is cumbersome.

If defs=[] (for non-AIX) is it still proper/safe to pass `define_macros=defs`

In short, would something like:

```
import sys

def configuration(parent_package='',top_path=None):
    from numpy.distutils.misc_util import Configuration
    config = Configuration('fft', parent_package, top_path)

    config.add_data_dir('tests')

    # AIX needs to be told to use large file support - at all times
    defs = [('_LARGE_FILES', None)] if sys.platform[:3] == "aix" else []
    # Configure pocketfft_internal
    config.add_extension('_pocketfft_internal',
                         sources=['_pocketfft.c'],
                         define_macros=defs
                         )

    return config
```
be both acceptable code as well as more concise.
That is fine but no need to be so concise
```
define_macros = []
if sys.platform[:3] == 'aix':
    define_macros.append(('_LARGE_FILES', None))
...
define_macros = define_macros
```
Is the problem really aix-specific or is it a version of the compiler that needs the macro?
Also, see [this comment](https://github.com/numpy/numpy/issues/6609#issuecomment-172197438)
which suggests
```
export 'CC=gcc -lgcc'
export LDSHARED="/opt/freeware/lib/python2.7/config/ld_so_aix gcc -bI:/opt/freeware/lib/python2.7/config/python.exp"
export CFLAGS="-pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -fno-tree-dominator-opts"
```
> Is the problem really aix-specific or is it a version of the compiler that needs the macro?

Yes, AIX specific as it is a label AIX introduced in late 1996 - early 1997 (AIX 4.2) when it started supporting files larger than 2G - even though the kernel was still 32-bit. Hence _LARGE_FILES in "32-bit" land. 64-bit kernel came, roughly, in 1998 (AIX 4.3).
> 
> 
> Also, see [this comment](https://github.com/numpy/numpy/issues/6609#issuecomment-172197438)
> which suggests
> 
> ```
> export 'CC=gcc -lgcc'
> export LDSHARED="/opt/freeware/lib/python2.7/config/ld_so_aix gcc -bI:/opt/freeware/lib/python2.7/config/python.exp"
> export CFLAGS="-pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -fno-tree-dominator-opts"
> ```

Is the issue title I reference that this is with xlc and xlf, not gcc. Also, I am working with Python3, not Python2 (although it should not matter much for packaging).

I'll have to look at the way Python2 builds - to see if it does NOT have a version specific config directory. There is a big chance that for Python2 they did not versionize the config directory.

As your reference to issue #6609 - that refers to gcc and gfortran and openblas. I am using libblas.a provided by AIX, not OpenBLAS.
* So, added a PR - that is still waiting for CI to finish (after 15 hours).

* I have been thinking about how the build process runs (attaching the screen text captured using script)
[numpy-dev-build.txt](https://github.com/numpy/numpy/files/4385828/numpy-dev-build.txt)

* In this, there is also this section appearing - that I would like some help addressing - as there is a system blas library - but, just maybe, the test is only looking for routines with an underscore (_) appended.
```
/data/prj/python/numpy-dev/numpy/distutils/system_info.py:1914: UserWarning:
    Optimized (vendor) Blas libraries are not found.
    Falls back to netlib Blas library which has worse performance.
    A better performance should be easily gained by switching
    Blas library.
  if self._calc_info(blas):
blas_info:
C compiler: xlc_r -DNDEBUG -O -I/opt/include    -O2 -qmaxmem=-1 -qarch=pwr5 -q64 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -q64

```

* If so, is it (also) creating a throw-away site.cfg file, if yes, is there a way to save that: a) for debugging; b) for efficiency for the future?

* I do not know distutils all that well, and am trying to a negotiate aka learn how it works, b) looking for a place to (ideally) add some logic for AIX (and maybe other platforms that have more than one compiler family) to distinguish between gcc and "not-gcc".

* @rgommers - many thanks for your initial hint. The principle is working. While comfortable with adding -D_LARGE_FILES for all compilers I am not so comfortable with adding -DNO_APPEND_FORTRAN

* Final comment: in the script file there are many other warnings about incorrect arguments to compilers. If interested - can take those up at a later time.
Getting better at finding things - so, how dependent is numpy on the routine cblas_ddot()?

As in this test in: `./numpy/distutils/system_info.py`
```
        s = textwrap.dedent("""\
            #include <cblas.h>
            int main(int argc, const char *argv[])
            {
                double a[4] = {1,2,3,4};
                double b[4] = {5,6,7,8};
                return cblas_ddot(4, a, 1, b, 1) > 10;
            }""")
        src = os.path.join(tmpdir, 'source.c')
        try:
            with open(src, 'wt') as f:
                f.write(s)

            try:
                # check we can compile (find headers)
                obj = c.compile([src], output_dir=tmpdir,
                                include_dirs=self.get_include_dirs())
            except (distutils.ccompiler.CompileError, distutils.ccompiler.LinkError):
                return None

```

On AIX this is always failing:
a) not cblas.h
b) no cblas_ddot()
However, AIX does have ddot() - see:
```
aixtools@x064:[/data/prj/python/numpy-dev]nm -Ae /usr/lib/libblas.a | grep ddot
/usr/lib/libblas.a[ddot.o]: .ddot                T           0
/usr/lib/libblas.a[ddot.o]: TOC                  d         256
/usr/lib/libblas.a[ddot.o]: ddot                 D         264          12
/usr/lib/libblas.a[ddot.o]: ddot                 d         256           4
/usr/lib/libblas.a[drk1.o]: .ddot                U           -

```
Note: AIX does not provide (at least not by default) the include file `blas.h`
 > If so, is it (also) creating a throw-away site.cfg file, if yes, is there a way to save that: a) for debugging; b) for efficiency for the future?

No, the scripts set properties on the ccompiler class instance according to what is needed for the various blas variants in that file.

>  I do not know distutils all that well, and am trying to a negotiate aka learn how it works

Most everyone struggles with this. The hope is that someday something will come along and replace it, until then we are stuck with hacks. The general idea is that there are standard commands for each `ccompiler` class. There are general class attributes to add compilation flags, link flags, macros and more, and when instantiating an instance of that class you add desired flags.

> many other warnings about incorrect arguments to compilers.

Any warnings emitted by gcc on linux will fail the CI. We have not gotten to that point with clang/darwin or msvc/windows. PRs welcome, but they should be constructed as to leave the code readable (`#ifdef` followed by tens of lines of code and then `#else` and more heaps of code is an anti-pattern)

> However, AIX does have ddot()

That might signal that the routines are slightly different. You would have to try a shim to translate the names and check that it all works correctly. There are some accuracy tests in the test suite.
> 
> > I do not know distutils all that well, and am trying to a negotiate aka learn how it works
> 
> Most everyone struggles with this. The hope is that someday something will come along and replace it, until then we are stuck with hacks. The general idea is that there are standard commands for each `ccompiler` class. There are general class attributes to add compilation flags, link flags, macros and more, and when instantiating an instance of that class you add desired flags.
> 
Does it remember these values? Or would I need to create something similar in lapack/setup.py to try a quick compile to see if the underscore works, and if not try without underscore.

I have an idea for `a hack` and then maybe someone (you for example) could suggest something prettier.

> > many other warnings about incorrect arguments to compilers.
> 
> Any warnings emitted by gcc on linux will fail the CI. We have not gotten to that point with clang/darwin or msvc/windows. PRs welcome, but they should be constructed as to leave the code readable (`#ifdef` followed by tens of lines of code and then `#else` and more heaps of code is an anti-pattern)
> 
> > However, AIX does have ddot()
> 
> That might signal that the routines are slightly different. You would have to try a shim to translate the names and check that it all works correctly. There are some accuracy tests in the test suite.

Not a mathematician - too geek for me.

However, dping research I found this [post](https://www.math.utah.edu/software/lapack.html) from 2003 that shows the issue with the appended underscore was known even then (for AIX and hpux). And what I also gather is that there was not much consistency in the name convention for blas and/or lapack libraries.

Ultimately - TODAY - numpy is passing the flag -lblas (several times) and maybe it actually 'just works', i.e., the code does discover that it does NOT have CBLAS and just calls regular `ddot()` et al in the code - i.e., there are macros that set the actual name of the function that gets called.

I'll play with it a bit.

Resolving patch:
diff --git a/numpy/fft/setup.py b/numpy/fft/setup.py
--- a/numpy/fft/setup.py
+++ b/numpy/fft/setup.py
@@ -1,3 +1,4 @@
+import sys
 
 def configuration(parent_package='',top_path=None):
     from numpy.distutils.misc_util import Configuration
@@ -5,9 +6,12 @@ def configuration(parent_package='',top_path=None):
 
     config.add_data_dir('tests')
 
+    # AIX needs to be told to use large file support - at all times
+    defs = [('_LARGE_FILES', None)] if sys.platform[:3] == "aix" else []
     # Configure pocketfft_internal
     config.add_extension('_pocketfft_internal',
-                         sources=['_pocketfft.c']
+                         sources=['_pocketfft.c'],
+                         define_macros=defs,
                          )
 
     return config


Test patch: