Removing an authorized user from one of your accounts could harm their credit score if the account is in good standing. Removing the person as an authorized user from the account will essentially take that good credit history off of their report, which may decrease their score, depending on their other credit history.