Only quit after returning from idle