Throw a nice error if app not found