Borrowing from your 401(k) or IRA can provide the funds you need, but there are consequences. For a 401(k) loan, you'll pay interest back into your account, but you might miss out on potential market gains. If you don't repay the loan, it's considered a distribution and can be subject to taxes and penalties. With an IRA, you might qualify for a penalty-free withdrawal for a first-time home purchase, but you'd be depleting your retirement savings.