LTO: Disable unless explicitly told otherwise.

While this might kinda defeat the purpose of LTO, it's more
feasible than fixing the can of worms I've opened up now.