If you back out of buying a home after signing a purchase agreement for a reason not covered by a contingency, you could lose the money you spent on the transaction, such as inspection fees. You’ll also surrender any earnest money you provided to the seller. In some cases, the seller could sue you for damages. You can back out without penalties before your offer is accepted and an agreement is signed.