Question: why did rhaegar leave elia?
Answer: Rhaegar might have been a nice guy, but he didn't think through that scheme of his. He didn't like them. He annulled his marriage to Elia and made his children bastard so that he could live with Lyanna. Because its hard to do when you are dead.