I think if we use message passing here and make all writes private to the timeout thread we can just use multiple datastructures and priority queues. I really think that there should only be a single thread modifying all datatstructures here and that thread pulls commands from a queue also in a single threaded fashion sending results back via callbacks - this goes together with my suggestion above related to `ThreadTimeoutState`