Instead of actually using a set, we could preserve the functionality of getting index based on field value by using the following class:

    class IndexesSet(set):
        def __getitem__(self, name):
            return [value for field, value in self if field == name]
        
        def __setitem__(self, name, value):
            self.add((name, value))

Of course.. more methods has to be added to make this convenient. Though this may make it hard to figure out what the code doing.. not sure.