Unfortunately these kind of tests are very unpredictable and can cause false positives.
As you can see this specific test is actually failing on our build server.

I would recommend instead to move these time based tests into the performance tests, see https://github.com/Particular/NServiceBus/tree/develop/src/NServiceBus.PerformanceTests