Add clang specific flag more selectively.