Only supporting restartless add-ons at the moment