It depends on what the siding material is. Cracks are a telltale sign of siding damage but discoloration, dents, and missing pieces may be signs of siding damage.