Fix swift restart problems

* Make FastI2I more thread-safe
* Make swift to restart correctly after crashing
Does _to_stop ever become True?