As long as the HOA was acting within its powers, your nonpayment could potentially result in late fees, prohibition on use of common areas, and a lien on your property.