Revert "Revert "Disable LTO here so it works globally""

This reverts commit e1ba79af52614ce42c45c432bf3cd9f04e3cd75a.

I'm just too tired to figure this out right now.