Proper field iteration

Slightly less hacky than #2574, not all that much better in the end due to map::erase() not acting as expected.