CGIV-3821: Delay server restarts to ensure we only restart the minimum number of times