Cleanup float complex optimizations.

This changes might have bad changes, we should take a closer look at the
diffs.