Removed unnecessary allocations and casts

A lot of data was being allocated when reusing old segments was
possible... I took advantage of this as much as possible.  I also
optimized the distance normalization.