They can cause damage if:
Water collects around plants and penetrates into the foundation. Roots grow into the foundation. Vermin living in the greenery gains easy access to a crawl space.