Catch one more unoptimizable case.
Fix Inf/-Inf/NaN handling.

Unbusts a range of spectests in Rakudo JVM.