Harden the build a little bit

Add a number of the flags documented in https://wiki.ubuntu.com/ToolChain/CompilerFlags to our compiler flags so that we both link faster and give a little better protections.

Note: We don't FORTIFY_SOURCE because it seems to have some odd behavior which isn't worth tracking down ATM.