Simply being underwater on your mortgage won’t impact your credit score. However, if you walk away from the loan (that is, stop paying), short-sell or accept foreclosure, your credit score will take a major hit.