If you have an employer identification number or an EIN, write to the IRS to close the account. The EIN will still belong to the business, so you can use it again later. You should also make sure that the business has met all its state and local responsibilities.