core: refactoring the way sets work internally v2

This is a refactored solution for PR #616. Functionally this is still the same change, but it’s implemented a lot cleaner with less code and less changes to existing parts of TF.
Whoeps! Will fix as soon as any other comments are in... Thx...