You accumulate tax interests and fines. If you don’t pay your taxes on time, the IRS will charge you interest on the amount you owe. The interest rate on your loan is usually around 5% or 6%. You may face penalties and interest if you don’t pay your taxes on time. The IRS can also tack on a late payment fee of 0.5% per month, which can add up to a penalty of 25%. 2. You receive a letter of notice from the IRS after 1 to 3 months. 3. You may get tax liens and a collections visit from the IRS after 2 to 6 months.