app: don't block on unit-remove

Closes #580.