If you refuse to pay your taxes, the IRS can place liens on your property, garnish your wages, or levy your bank accounts. It will typically only take these actions if you wilfully refuse to pay.