Always use the march flags. It was use only when we detect it, so the first time

As a side effect don't always loop over user provided cxx flags always when it contain march.
Don't merge elemwise that cause duplicate computation of broadcasted element.
Merge pull request #1639 from nouiz/march

Remove duplicate computation and compilation.