Sped up ActorMap.ActorsInBox.

- By ensuring both the add and remove actor lists are sets, we ensure the partitioning bins will contain only distinct actors. We can remove the HashSet and Distinct in ActorsInBox and ActorsInWorld which provides a nice speedup for queries. ActorsInBox sees nearly a 3x speedup in the RA shellmap.
Merge pull request #5853 from RoosterDragon/actormap-perf

Sped up ActorMap.ActorsInBox.