In many cases you can have the penalty removed if you can show reasonable cause. That may not be so easy. Showing that you relied on erroneous written advice from the IRS is one reason. Another is reliance on a professional such as a CPA, tax attorney, etc. The penalty for filing a return or depositing taxes late may be abated because of a business catastrophe such as a fire or flood, but simply forgetting because you were busy isn't a good excuse.
Most tax professionals know of more reasons that can be used to escape the penalty.
Their bill should be more than offset by the penalty savings.