The IRS can abate penalties for filing and paying late if there is reasonable cause. Generally, interest charges may not be abated and continue to accrue until all assessed tax, penalties, and interest are paid in full. The law does provide exceptions for allowing abatement or suspension of interest.