Fix race condition while restarting (fixes #560)

It seems to happen much more often now (as described in #533)
Probably fixes #560