It's possible to take up to $10,000 from your IRA for a qualified first-time home purchase without owing an early withdrawal penalty. However, you pay income tax on that money.