Aphids can cause significant damage to sweet peas by feeding on the sap of the plants, which can lead to yellowing and curling leaves, stunted growth, and reduced flowering. In severe cases, aphid infestations can kill sweet pea plants. It's essential to take action as soon as you notice an aphid problem to protect your sweet peas.