Challenges of seed saving for heirloom varieties include cross-pollination with other varieties, maintaining purity, and preserving seed viability over time.