Typically, buyers are less likely to be sued for a breach of contract than sellers who back out of a purchase agreement, but buyers should still tread carefully.
It’s possible for a seller to sue a buyer for backing out of a sale, but the instances of this actually happening are rare. Your purchase agreement may even state that the seller is limited to keeping the earnest money as damages if the buyer backs out, and that by signing they agree to not pursue other legal remedies.