If you did not accept the offer yet, you would generally not have to pay the broker fee unless your specific agreement or contract with the broker says that you have to under these circumstances. If you did accept the offer, thereby entering into a contract to sell the home, the broker has earned his or her fee and you would have to pay.