Handle HTTP 409 that can be returned when deleting app.