Replace `{add, remove}Clients` with mutexes.
Remove `logger.Fatalf()`.