Ther terms and rights of the party's are contained in the card holder agreement, which, in most cases, provides that a fee can be levied for a miissed or late payment. In all likelyhood, the card issuer is merely exercising its rights under the agreement.  A debit card, however, would be treated differently, assuming that there existed suitable funds in the underlying account.