mxcl/homebrew — Issue #8105: Optimization, as well as usage of {C,CXX,CPP,LD}FLAGS
Author: astiob
Created: 2011-10-14 02:10:46+00:00
State: open

I see multiple problems with the way Library/Homebrew/extend/ENV.rb currently works:

* Optimization flags are set in `case` statements without `else` clauses, relying on manual updates as new processors come out, and evidently these updates are made very rarely, as at the moment Arrandale and Sandy Bridge don’t get any SSE 4 on Snow Leopard and Lion and any sub-architecture optimization at all on Leopard. There also seems to be no particular ordering of values in `when` clauses, which is bad from a stylistic point of view.

* The initial optimization flags are decided based on the version of Mac OS X and assume that GCC 4.0 is used on Leopard and GCC 4.2 on newer systems, even though a different version of GCC or indeed Clang, which (at least if it is recent enough) supports newer and better optimization flags, might be used. (Note that LLVM-GCC accepts exactly the same options as the standard GCC.)

By the way, `-mfpmath=sse` seems to be automatically added even by GCC 4.0, even when run in 32-bit mode, even by the one shipped with Xcode as old as version 2.5 (which also wasn’t itself compiled for 64-bit). I did still perform all my tests on a 64-bit machine though, simply forcing 32-bit mode with `arch -i386` when needed.

* Compiler selection methods just remove flags the corresponding compilers do not support but do not add any other flags to replace them. So, for example, if you enable LLVM-GCC with `-O4` by default and then a formula calls `ENV.gcc`, you will be left with no optimization at all. The methods do not upgrade optimization either, so if you start out with GCC 4.0 and then call `ENV.gcc_4_2`, you will not get SSE 4 or Core 2 optimizations.

* Removing flags is done using simple string searches that do not check word boundaries, which may work now but is not generally protected from accidentally finding matches inside other flags.

* Optimization flags are added to CFLAGS and CXXFLAGS but not CPPFLAGS. Yet they normally cause preprocessor macros like `__OPTIMIZE__` and `__SSE__` to be defined, and they should still be defined even if a formula uses the preprocessor in a step separate from compilation (admittedly this is a rare sight). This of course means optimization options do need to go into CPPFLAGS too. In fact `ENV.libxml2` even adds an include path by calling `ENV.append_to_cflags`, which of course updates CFLAGS and CXXFLAGS but not CPPFLAGS, where include paths should actually go.

* Finally, `ENV.remove_cc_etc` does not remove CXXFLAGS, but this is of course a trivial error.

A lot of formulas also use *FLAGS incorrectly but that is probably a topic for another tracker issue. (Or is it?) (…Although I suppose formulas know that their build systems work with these *FLAGS even if the way they are set is not correct in the general case.)

I have a solution that

* modifies `append_to_cflags` and `remove_from_cflags` to also modify CPPFLAGS,
* adds methods `replace_in_cflags` and `set_cpu_cflags`,
* moves all CPU-specific logic inside compiler-specific methods in the form of hash arguments they pass to `set_cpu_cflags`,
* specially handles older CPUs and enables all optimizations on all newer ones with GCC to make sure future (and current!) processors automatically get the best optimization Apple’s GCC has to offer,
* makes Clang use `-march=native` (plus a flag to disable AES-NI on Nehalem because Clang enables it but Nehalem does not actually support AES-NI; yes, I do intend to submit a fix for this as well as a few other related issues to Clang sometime)
* and of course fixes `remove_cc_etc`.

CPU-specific flags are reset every time a compiler-specific method is called. I think this is reasonable and it is not likely that there is much code out there that relies on them not being reset. I will post my code later today (for some European definition of today).

What my solution does not (yet) do is make `remove` and `remove_from_cflags` operate on words. I am also concerned about the `fast` and `Og` methods adding flags that later invocations of other `O#` methods do not remove. Ideally the `llvm` and `clang` methods should also reset `-O4` if it is the default and if no particular `O#` method has been called yet. Actually, cannot we just pass `-O4` to non-LLVM GCC too (if the user has enabled it for LLVM-GCC by default)? I do not get any errors if I invoke it with `-O4`.