BUILD: Remove simulator compiler flags

Not sure the need for these and they make the binaries very large.

Pat: Is there any reason I should be keeping these around?  I also plan to do similar changes upstream in VES.