Applying for penalty abatement is the best way to get the IRS to remove interests. You might qualify for this tax debt relief program if you paid all the taxes you owe or set up a payment plan.
The IRS may also approve a penalty abatement application if you requested a deadline extension or did not previously have to file returns.