Try to detect the reload rather than just sleeping
Bypass assert_speedup on CI

It just seems to unreliable, probably because it's running on a VM so
doesn't have dedicated CPU.