Not paying your taxes does not lead to criminal charges, but if you have unpaid taxes, you should contact the IRS and try to make payment arrangements. In most cases, when the IRS accuses someone of evading taxes, they are accusing them of filing tax returns that show a lower tax liability than they should. However, in rare cases, tax evasion can also occur just because someone took affirmative steps to avoid the payment of a tax once it was assessed.