The mortgage company won't change your lender unless you start falling behind on your payments. At that point, the mortgage company may worry about whether it will be able to collect the debt from you, so it could sell your debt to a collection agency. At that point, the collection agency would technically own your debt, so they are the lender you need to repay.