As with many things in life, what should be simple is a bit more complicated than needs to be.
View more: How To Remove an Unwanted Application from Azure AD