In terms of income levels, the IRS in recent years has audited taxpayers with incomes below $25,000 and above $500,000 at higher-than-average rates, according to government data.