Keep track of open config windows