Cut down on std::set usage, reorder vertices with ghosts at end