Repository: spack/spack

Issue:
openblas: add an easy way to compile with DYNAMIC_ARCH=1 enabled.
This is useful in heterogeneous environments where you may be installing the same library for different Intel and/or AMD architectures.
"Installation issue: openblas: default blas/lapack 'openblas' can't build on RHEL6 with gcc@7.2.0 built by Spack
**snl-atdm-issue**

### Steps to reproduce the issue

```console
$ ./atdm-spack/spack/bin/spack install openblas %gcc@7.2.0
```

with build errors starting with:

```
==> Installing openblas
==> Searching for binary cache of openblas
==> Finding buildcaches in /ascldap/users/rabartl/Spack.base3/atdm-spack/atdm-spack-tarball-mirror/build_cache
==> No binary for openblas found: installing from source
==> Using cached archive: /home/rabartl/Spack.base3/atdm-spack/spack/var/spack/cache/openblas/openblas-0.3.5.tar.gz
==> Staging archive: /home/rabartl/Spack.base3/atdm-spack/spack/var/spack/stage/openblas-0.3.5-ase7rloe2w5cvr7egf4fglyx5kw2egrl/v0.3.5.tar.gz
==> Created stage in /home/rabartl/Spack.base3/atdm-spack/spack/var/spack/stage/openblas-0.3.5-ase7rloe2w5cvr7egf4fglyx5kw2egrl
==> No patches needed for openblas
==> Building openblas [MakefilePackage]
==> Executing phase: 'edit'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j32' 'CC=/home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc' 'FC=/home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'USE_OPENMP=0' 'USE_THREAD=0' 'libs' 'netlib' 'shared'

577 errors found in build log:
     1348    /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1 
             -DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_LN -DASMFNAME=dtrmm_kernel_LN_ -DNAME=dtrmm_kernel_LN_ -DCNAME=dtrmm_kernel_LN -DCHAR_NAME=\"dtrmm_kernel_LN_\" -DCHAR_CNAME=\"dtrmm_kernel_LN\" -D
             NO_AFFINITY -I.. -DDOUBLE  -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -DLEFT -UTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_LN.o
     1349    /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1 
             -DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_LT -DASMFNAME=dtrmm_kernel_LT_ -DNAME=dtrmm_kernel_LT_ -DCNAME=dtrmm_kernel_LT -DCHAR_NAME=\"dtrmm_kernel_LT_\" -DCHAR_CNAME=\"dtrmm_kernel_LT\" -D
             NO_AFFINITY -I.. -DDOUBLE  -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -DLEFT -DTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_LT.o
     1350    /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1 
             -DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_RN -DASMFNAME=dtrmm_kernel_RN_ -DNAME=dtrmm_kernel_RN_ -DCNAME=dtrmm_kernel_RN -DCHAR_NAME=\"dtrmm_kernel_RN_\" -DCHAR_CNAME=\"dtrmm_kernel_RN\" -D
             NO_AFFINITY -I.. -DDOUBLE  -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -ULEFT -UTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_RN.o
     1351    /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1 
             -DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_RT -DASMFNAME=dtrmm_kernel_RT_ -DNAME=dtrmm_kernel_RT_ -DCNAME=dtrmm_kernel_RT -DCHAR_NAME=\"dtrmm_kernel_RT_\" -DCHAR_CNAME=\"dtrmm_kernel_RT\" -D
             NO_AFFINITY -I.. -DDOUBLE  -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -ULEFT -DTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_RT.o
     1352    /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -c -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER
             =1 -DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrsm_kernel_LN -DASMFNAME=dtrsm_kernel_LN_ -DNAME=dtrsm_kernel_LN_ -DCNAME=dtrsm_kernel_LN -DCHAR_NAME=\"dtrsm_kernel_LN_\" -DCHAR_CNAME=\"dtrsm_kernel_LN\"
              -DNO_AFFINITY -I.. -DDOUBLE  -UCOMPLEX -DTRSMKERNEL -UCOMPLEX -DDOUBLE -DUPPER -DLN -UCONJ ../kernel/x86_64/../generic/trsm_kernel_LN.c -o dtrsm_kernel_LN.o
     1353    ../kernel/x86_64/dgemm_kernel_4x8_haswell.S: Assembler messages:
  >> 1354    ../kernel/x86_64/dgemm_kernel_4x8_haswell.S:1759: Error: no such instruction: `vpermpd $ 0xb1,%ymm0,%ymm0'
  >> 1355    ../kernel/x86_64/dgemm_kernel_4x8_haswell.S:1759: Error: no such instruction: `vpermpd $ 0x1b,%ymm0,%ymm0'
```

The STDOUT from spack is given in:

* [spack.install.openblas.gcc-7.2.0.rhel6..out.txt](https://github.com/spack/spack/files/3458812/spack.install.openblas.gcc-7.2.0.rhel6.out.txt)

The the build log file:

```
atdm-spack/spack/var/spack/stage/openblas-0.3.5-ase7rloe2w5cvr7egf4fglyx5kw2egrl/OpenBLAS-0.3.5/spack-build.out
```

is:

* [OpenBLAS-0.3.5-spack-build.out.txt](https://github.com/spack/spack/files/3458814/OpenBLAS-0.3.5-spack-build.out.txt)

### Platform and user environment

Please report your OS here:
```commandline
 $ uname -a
Linux crf450.srn.sandia.gov 2.6.32-696.30.1.el6.x86_64 #1 SMP Fri May 18 11:50:44 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -d
Description:    Red Hat Enterprise Linux Workstation release 6.9 (Santiago)
``` 

Here are the compilers (where gcc@7.2.0 was built by spack itself with gcc@4.4.7 on the system).

* [compilers.yaml.txt](https://github.com/spack/spack/files/3458813/compilers.yaml.txt)

### Additional information

Sometimes the issue benefits from additional details. In these cases there are
a few things we can suggest doing. First of all, you can post the full output of:

```console
./atdm-spack/spack/bin/spack spec --install-status openblas %gcc@7.2.0
Input spec
--------------------------------
 -   openblas%gcc@7.2.0

Concretized
--------------------------------
==> Error: String contains no specs: 
```

to show people whether Spack installed a faulty software or if it was not able to
build it at all. 



Discussion:
Poking into a (potential?) hornets nest, I'd even argue that DYNAMIC_ARCH=1 should be the default. Not only for heterogeneous environments, but also environments where the build host(s) may not match the execution hosts. 
Please verify this recipe on macOS: our own non-Spack builds of this package have encountered some problems, especially when building the LAPACK implementation with `DYNAMIC_ARCH=1`.
@jabl @healther Please check bec81e9.
@fcannini 

could you please fix indentation:

> var/spack/repos/builtin/packages/openblas/package.py:43: [E501] line too long (81 > 79 characters)
> var/spack/repos/builtin/packages/openblas/package.py:44: [E501] line too long (81 > 79 characters)
Hi everybody.
Do we have a decision on whether to set DYNAMIC_ARCH=1 as default?
IMHO  dealii/dealii#7891 is another reason to do so and avoid similar issues in the future.

@davydden Sure thing. However, the lines shown in the error message will need some rewording if it is decided to change the default `cpu_target`, so I would like to "kill two birds with one stone" and fix all issues at once.
Ping?
I just noticed that this has not been merged, and wanted to help. But I'm not sure what's going on here. The tests indicate that flake8 check has not passed, however, if I take the patch and do `spack flake8` locally, it passes:
```
=======================================================
flake8: running flake8 code checks on spack.

Modified files:
  var/spack/repos/builtin/packages/openblas/package.py
=======================================================

Flake8 checks were clean.
```
Given the recent support for specific microarchitectures, can't we go for the opposite route:
1. Remove the `cpu_target` variant
2. Add flags and features based on tests for the actual target (see [here](https://spack.readthedocs.io/en/latest/packaging_guide.html#querying-the-target-microarchitecture))

? Would that be possible with `openblas` or people here foresee issues with that?
@alalazo I don't think it solves the same problem. From what I understand, DYNAMIC_ARCH=1 allows to support for more than just one CPU type in a single openblas installation.
> I don't think it solves the same problem. From what I understand, DYNAMIC_ARCH=1 allows to support for more than just one CPU type in a single openblas installation.

I have the same understanding of what `DYNAMIC_ARCH=1` does. What I am wondering is if we have a use case for that in `openblas` now that we can query targets. What would be the meaning of:
```
openblas cpu_target=dynamic target=x86_64
``` 
vs. 
```
openblas cpu_target=dynamic target=knl
``` 
etc. ? Off the top of my head I would say that in general every option that involves architecture specific optimizations should be queried via `target` and set based on the queried property. I didn't think through all the corner cases though, so I was wondering if there was one here. Does that seem reasonable?
I would definitely advocate in favor of having DYNAMIC_ARCH=1 be default: it enables a single lowest-common-denominator binary package that supports microarchitecture-specific code when installed on machines with different CPUs.
My one caveat for this is that when enabling the full set of microarchitectures for macOS one runs into a command-line length issue when assembling the library. To ensure successful compilation on macOS one might wish to enable only those architectures known to be found on macs.

@alalazo I think I've understood your point. "Now that we have `target=x86_64` which is a totally generic target, maybe we can use it instead of `cpu_target`." 
Am I correct?
If yes, I see no problem.

About the use case of `DYNAMIC_ARCH=1`, this is very useful in heterogeneous environments, where you may have different micro-architectures, e.g.: a (usually small) cluster that has grown over time, each purchase bringing a different  micro-architecture to the mix. In this case you avoid madness by building just one .so to run them all.
I just hit a similar problem to what you're discussing here, though it is with a build that predated the microarchitecture stuff. On our heterogeneous architecture, the node on which I build OpenBLAS works fine, but other nodes (older presumably) that are used for CI but share the same filesystem fail due to illegal instructions. Will it be sufficient to recompile my toolchain with the generic `target=x86_64`? Or do I need to add an additional variant for OpenBLAS (i.e. `cpu_target=GENERIC`) to realize I want to be able to compile on other systems? :\
@sethrj : For the time being you can build the (somewhat poorly named) virtual_machine variant, which will build the DYNAMIC_ARCH=1 version of OpenBLAS which contains specialized kernels for all kinds of CPU's and the correct one selected at runtime.

(You might also need the target=x86_64 or cpu_target=GENERIC in order to get the common code to work everywhere?)
Thanks @jabl . Does anyone (@alalazo?) know of any effort to make this package choose flags based on the Spack-deduced microarchitecture settings? I think it would be worth refactoring the `openblas` package to respect the microarchitecture rather than trying to go its own way, leading to unexpected behavior when downstream codes encounter illegal instructions when running a package that claims to be compiled for the generic x86_64 😄 
I have added the bug tag back because of 

```
./atdm-spack/spack/bin/spack spec --install-status openblas %gcc@7.2.0
Input spec
--------------------------------
 -   openblas%gcc@7.2.0

Concretized
--------------------------------
==> Error: String contains no specs: 
```

It should be possible to get more info with `spack spec` (and it implies a bug in Spack if it isn't returning any info when that command is run).
@bartlettroscoe can you provide the output of 

```
./atdm-spack/spack/bin/spack -d --stacktrace spec --install-status openblas %gcc@7.2.0
```

? (i.e. rerun that command with `-d --stacktrace`)
@bartlettroscoe can you also provide the processor make of the node you ran on? I.E. skylake/haswell/sandybridge?
@scheibelp asked:

> can you provide the output of
>
> `./atdm-spack/spack/bin/spack -d --stacktrace spec --install-status openblas %gcc@7.2.0`
>
> ?

It is attached in [spack.d.stacktrace.install.openblas.out.txt](https://github.com/spack/spack/files/3468690/spack.d.stacktrace.install.openblas.out.txt).





@becker33 asked:

> @bartlettroscoe can you also provide the processor make of the node you ran on? I.E. skylake/haswell/sandybridge?

```
$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 79
Model name:            Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz
Stepping:              1
CPU MHz:               3192.902
BogoMIPS:              6384.74
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              25600K
NUMA node0 CPU(s):     0-7,16-23
NUMA node1 CPU(s):     8-15,24-31
```

@bartlettroscoe have you tried with other compilers and/or other versions of openblas?

The reason I ask is that's a weird error to be getting on a Broadwell chip with `-mavx2` flags (as shown in your output. The `A` version of the `vpermpd` operation is part of the avx2 standard, and I'm not quite sure why a broadwell chip would be rejecting it. So my first thought is if there's some bad combination of compiler/version that manifests a bug.
Is there anyone with ECP funding that can get on the SNL SRN system, log into one of these RHEL6 machines and then try to reproduce this?  It seems like we should not have to fight with installing the default BLAS/LAPACK on a RHEL6 machine.
I don't think I have access to any RHEL6 machines at the moment. I'm trying to build gcc@7.2.0 to see if I can reproduce on RHEL7, but building gcc takes a while
@becker33 said:

>  I'm trying to build gcc@7.2.0 to see if I can reproduce on RHEL7, but building gcc takes a while

Thanks @becker33!  I think @fryeguy52 reported that he got the same build error on his RHEL7 machine (but I can't find documentation for that).
Hi,
Redhat 6 has binutils 2.20 which doesn't support avx2 (or at least this instruction)
If you install a recent binutils and add it to the module section in your compiler.yaml it should work

on CentOs 6.10 I have:
as --version
GNU assembler version 2.20.51.0.2-5.48.el6_10.1 20100205
with asm-vpermd.s : 
       .text
        vpermpd $0xb1,%ymm0,%ymm0
as asm-vpermd.s
asm-vpermd.s: Assembler messages:
asm-vpermd.s:2: Error: no such instruction: `vpermpd $0xb1,%ymm0,%ymm0'

It works on CentOS7 with binutils 2.27


@laubry, Spack is already building binutils@2.31.1.  Why does the 'openblas' package not have dependence on 'binutils'?

@bartlettroscoe From what I understand, gcc is looking for gas on the PATH, unless you compile it explicitly with binutils.

It's possible the build will succeed if you add a depends_on("binutils") in openblas.
Openblas shouldn't depend on binutils, I bet it will compile fine with intel compiler or llvm.

The real problem is that "gcc -mavx2" will tell the build system that you  can compile avx2 instructions, but it doesn't check that the assembler it uses will support the instruction set.

The correct fix, should be for spack to provide an "as" wrapper as part of the "compiler" when providing a gcc toolchain.
My workaround is to load the binutils package as a module from the compiler definition.
@bartlettroscoe Since you built binutils, does adding the module in your compilers.yaml fix the build ?
ie:
```
compilers:
- compiler:
    environment: {}
    extra_rpaths: []
    flags: {}
    modules: ["binutils-2.31.1-gcc-x.y.z-wwwwww"]
    operating_system: rhel6
    paths:
      cc: /home/rabartl/Spack.base3/atdm-spack/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.7/gcc-7.2.0-
[...]
```
@laubry, thanks for the suggestion.  The problem is that we are building 'openblas' in the same `spack install` command as we are building `binutils`.  So do you think it is worth trying hacking `openblas/package.py` to add a call to `depends_on('binutils')`?
@bartlettroscoe It should work.

Make sure you do depends_on('binutils', type='build'), or you'll drag a runtime dependency.
@laubry, what is the difference between a 'build' dependency and a regular dependency?  There is no mention of the `type` argument at all for `depends_on()` in:

* https://spack.readthedocs.io/en/latest/tutorial_packaging.html

And what would a runtime dependency even mean for a library like 'openblas'?  It has no executbles (or at least none that we would care about).

@bartlettroscoe It is explained in the packaging guide : 
[https://spack.readthedocs.io/en/latest/packaging_guide.html#depends-on](https://spack.readthedocs.io/en/latest/packaging_guide.html#depends-on)

As you say, openblas doesn't need binutils at runtime, but I think that by default, the type for depends_on is type=("build", "link", "run")

Also, I'm not sure (maybe a dev can confirm), I think the hash for the package include link and run dependency, but not the build dep. Which means, if you drop the change at some point (because the problem is fixed) you won't trigger a rebuild of openblas.


@bartlettroscoe I should have read the link I gave you all the way... It's all explained : 

the default type is ('build', 'link'), but you should really use build only.
@laubry, the last thing I am worried about at this point is having to rebuild packages if dependencies change.  We have rebuilt these packages from scratch at least 100 times in the last 6 months trying to get all of this to work.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/openblas/package.py b/var/spack/repos/builtin/packages/openblas/package.py
--- a/var/spack/repos/builtin/packages/openblas/package.py
+++ b/var/spack/repos/builtin/packages/openblas/package.py
@@ -4,10 +4,10 @@
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
 
 import os
+import re
 
 from spack import *
 from spack.package_test import compare_output_file, compile_c_and_execute
-import spack.architecture
 
 
 class Openblas(MakefilePackage):
@@ -33,17 +33,9 @@ class Openblas(MakefilePackage):
     version('0.2.16', sha256='766f350d0a4be614812d535cead8c816fc3ad3b9afcd93167ea5e4df9d61869b')
     version('0.2.15', sha256='73c40ace5978282224e5e122a41c8388c5a19e65a6f2329c2b7c0b61bacc9044')
 
-    variant(
-        'shared',
-        default=True,
-        description='Build shared libraries as well as static libs.'
-    )
-    variant('ilp64', default=False, description='64 bit integers')
+    variant('ilp64', default=False, description='Force 64-bit Fortran native integers')
     variant('pic', default=True, description='Build position independent code')
-
-    variant('cpu_target', default='auto',
-            description='Set CPU target architecture (leave empty for '
-                        'autodetection; GENERIC, SSE_GENERIC, NEHALEM, ...)')
+    variant('shared', default=True, description='Build shared libraries')
 
     variant(
         'threads', default='none',
@@ -52,24 +44,6 @@ class Openblas(MakefilePackage):
         multi=False
     )
 
-    variant(
-        'virtual_machine',
-        default=False,
-        description="Adding options to build openblas on Linux virtual machine"
-    )
-
-    variant(
-        'avx2',
-        default=True,
-        description='Enable use of AVX2 instructions'
-    )
-
-    variant(
-        'avx512',
-        default=False,
-        description='Enable use of AVX512 instructions'
-    )
-
     # virtual dependency
     provides('blas')
     provides('lapack')
@@ -142,14 +116,80 @@ def check_compilers(self):
                     'OpenBLAS @:0.2.19 does not support OpenMP with clang!'
                 )
 
+    @staticmethod
+    def _read_targets(target_file):
+        """Parse a list of available targets from the OpenBLAS/TargetList.txt
+        file.
+        """
+        micros = []
+        re_target = re.compile(r'^[A-Z0-9_]+$')
+        for line in target_file:
+            match = re_target.match(line)
+            if match is not None:
+                micros.append(line.strip().lower())
+
+        return micros
+
+    @staticmethod
+    def _microarch_target_args(microarch, available_targets):
+        """Given a spack microarchitecture and a list of targets found in
+        OpenBLAS' TargetList.txt, determine the best command-line arguments.
+        """
+        args = []  # Return value
+
+        # List of available architectures, and possible aliases
+        openblas_arch = set(['alpha', 'arm', 'ia64', 'mips', 'mips64',
+                             'power', 'sparc', 'zarch'])
+        openblas_arch_map = {
+            'amd64': 'x86_64',
+            'powerpc64': 'power',
+            'i386': 'x86',
+            'aarch64': 'arm64',
+        }
+        openblas_arch.update(openblas_arch_map.keys())
+        openblas_arch.update(openblas_arch_map.values())
+
+        # Add spack-only microarchitectures to list
+        skylake = set(["skylake", "skylake_avx512"])
+        available_targets = set(available_targets) | skylake | openblas_arch
+
+        # Find closest ancestor that is known to build in blas
+        if microarch.name not in available_targets:
+            for microarch in microarch.ancestors:
+                if microarch.name in available_targets:
+                    break
+
+        arch_name = microarch.family.name
+        if arch_name in openblas_arch:
+            # Apply possible spack->openblas arch name mapping
+            arch_name = openblas_arch_map.get(arch_name, arch_name)
+            args.append('ARCH=' + arch_name)
+
+        if microarch.vendor == 'generic':
+            # User requested a generic platform, or we couldn't find a good
+            # match for the requested one. Allow OpenBLAS to determine
+            # an optimized kernel at run time.
+            args.append('DYNAMIC_ARCH=1')
+        elif microarch.name in skylake:
+            # Special case for renaming skylake family
+            args.append('TARGET=SKYLAKEX')
+            if microarch.name == "skylake":
+                # Special case for disabling avx512 instructions
+                args.append('NO_AVX512=1')
+        else:
+            args.append('TARGET=' + microarch.name.upper())
+
+        return args
+
     @property
     def make_defs(self):
+        spec = self.spec
+
         # Configure fails to pick up fortran from FC=/abs/path/to/fc, but
         # works fine with FC=/abs/path/to/gfortran.
         # When mixing compilers make sure that
         # $SPACK_ROOT/lib/spack/env/<compiler> have symlinks with reasonable
         # names and hack them inside lib/spack/spack/compilers/<compiler>.py
-
         make_defs = [
             'CC={0}'.format(spack_cc),
             'FC={0}'.format(spack_fc),
@@ -161,23 +201,15 @@ def make_defs(self):
         else:
             make_defs.append('MAKE_NB_JOBS=0')  # flag provided by OpenBLAS
 
-        if self.spec.variants['virtual_machine'].value:
-            make_defs += [
-                'DYNAMIC_ARCH=1',
-                'NUM_THREADS=64',  # OpenBLAS stores present no of CPUs as max
-            ]
-
-        if self.spec.variants['cpu_target'].value != 'auto':
-            make_defs += [
-                'TARGET={0}'.format(self.spec.variants['cpu_target'].value)
-            ]
-        # invoke make with the correct TARGET for aarch64
-        elif 'aarch64' in spack.architecture.sys_type():
-            make_defs += [
-                'TARGET=ARMV8'
-            ]
-        if self.spec.satisfies('%gcc@:4.8.4'):
-            make_defs += ['NO_AVX2=1']
+        # Add target and architecture flags
+        targetlist_name = join_path(self.stage.source_path, "TargetList.txt")
+        if os.path.exists(targetlist_name):
+            with open(targetlist_name) as f:
+                avail_targets = self._read_targets(f)
+        else:
+            avail_targets = []
+        make_defs += self._microarch_target_args(spec.target, avail_targets)
+
         if '~shared' in self.spec:
             if '+pic' in self.spec:
                 make_defs.extend([
@@ -201,11 +233,9 @@ def make_defs(self):
         if '+ilp64' in self.spec:
             make_defs += ['INTERFACE64=1']
 
-        if self.spec.target.family == 'x86_64':
-            if '~avx2' in self.spec:
-                make_defs += ['NO_AVX2=1']
-            if '~avx512' in self.spec:
-                make_defs += ['NO_AVX512=1']
+        # Prevent errors in `as` assembler from newer instructions
+        if self.spec.satisfies('%gcc@:4.8.4'):
+            make_defs.append('NO_AVX2=1')
 
         return make_defs
 


Test patch:
diff --git a/lib/spack/spack/test/llnl/util/cpu.py b/lib/spack/spack/test/llnl/util/cpu.py
--- a/lib/spack/spack/test/llnl/util/cpu.py
+++ b/lib/spack/spack/test/llnl/util/cpu.py
@@ -278,3 +278,14 @@ def test_version_components(version, expected_number, expected_suffix):
     number, suffix = llnl.util.cpu.version_components(version)
     assert number == expected_number
     assert suffix == expected_suffix
+
+
+def test_invalid_family():
+    targets = llnl.util.cpu.targets
+    multi_parents = Microarchitecture(
+        name='chimera', parents=[targets['pentium4'], targets['power7']],
+        vendor='Imagination', features=[], compilers={}, generation=0
+    )
+    with pytest.raises(AssertionError,
+                       matches='a target is expected to belong'):
+        multi_parents.family