Note they went astray after they were born, not born astray. Their going astray consisted of speaking lies. Infants cannot speak lies therefore the passage does not refer to infants. And if all infants are born into the world speaking lies. Then it follows that all infants are born into the face the punishment of the lake of fire.
Transgression involves voluntary act.
Refers to Israel as a Nation, not to individuals. Its origin compared to a birth. There is none that understandeth, there is none that seeketh after God. Note, If man's heart becomes evil in youth, then it must have been pure before youth. Being the offspring of God would demand that a child is not depraved unless God is depraved.