Fix slowdown of server/ unittests and prevent possible flaky test