Oh, right. That's because the `Collision` object is the key of the set and `unordered_set` wants to enforce the keys staying constant to keep the internal hashing constant. I think it would be best to switch to `unordered_map`, have the `Collision` objects be the values and then use something else as the key. Either `std::pair` plus custom hash and comparison functions as `std::unordered_map` template parameters, or a custom `unordered_pair` template with accompanying hash and comparison functions.