We have to lock both sets so t is not modified by another goroutine