The IRS charges an underpayment penalty of 0.5% on underpaid taxes. They will impose this charge each month or part of the month you do not clear the debt. Still, the penalty cannot exceed 25% of the delinquent taxes.