If you fail to repay your 403(b) loan as agreed, the IRS treats the unpaid balance as a taxable distribution. If you're under 59½, an additional 10% early withdrawal penalty may apply. You may also have to repay the full loan balance quickly if your employment is terminated.