Backported 1.5.x memory leak fixes and removed FlatMap<V> usage

Backported memory leak fixes from 1.5.x branch
Removed FlatMap<V> was using a LOT of memory for marginal gains in 512x512 area around spawn. With multiple worlds, it adds up quickly... (~30MB/world or so)