Quit the active app if it's deleted via HarnessServer