Add mutexes to make the neighbors data structure thread safe