Remove non-long vs long distinction

All numeric types are long. If this needs optimization it can be done
later.