In short, gravity from the planets shoved many icy planetesimals away from the Sun, and gravity from the galaxy likely caused them to settle in the borderlands of the solar system, where the planets couldn't perturb them anymore. And they became what we now call the Oort Cloud.