Some smart mutexing to be more sure that things works