Although most organizational components within IRS are involved, the Tax Forms and Publications Division within IRS’s Media and Publications Division of the W&I Division is primarily responsible for creating and improving tax forms, instructions, and other documents. One goal of the Tax Forms and Publications Division is to make tax forms and instructions as clear and understandable as possible. It is divided into three branches— Individual Forms and Publications, Business Forms and Publications, and Tax Exempt/Government Entities and Specialty Forms and Publications. As of late January 2003, 103 persons were assigned to the Tax Forms and Publication Division, including about 15 persons whose primary responsibility was creating, revising, and reviewing individual income tax forms and instructions. Many tax forms and instructions are revised annually, often with short turnaround times and in response to tax law changes. IRS also periodically reviews tax forms and, when appropriate, schedules them for revision. According to IRS’s estimate, it revised about 450 tax forms and instructions in 2001 that affected individual and business tax returns. In addition to tax law changes, revisions to tax forms and instructions generally reflected procedural changes, legal rulings, and feedback from internal and external stakeholders about the understandability of forms and instructions. As illustrated in figure 1, the annual tax forms development process generally starts with a review of the current tax forms. IRS’s tax law specialists review the existing forms and instructions to determine what changes, if any, may be needed to reflect tax law changes and other requirements. The tax law specialists consider comments from a variety of sources both within and outside IRS. For example, comments may be obtained from IRS customer service staff in toll-free call centers who answer calls from taxpayers and have firsthand knowledge of particular forms or instructions that were confusing to taxpayers. IRS’s Taxpayer Advocate Service staff may also provide comments useful to the tax law specialists. The Tax Forms Coordinating Committee, comprised of representatives from all of IRS’s key components, the Department of the Treasury, and IRS’s Chief Counsel, reviews draft forms to help ensure that they are not overly burdensome and that they conform to legal and technical requirements. Draft forms are generally posted to IRS’s external website so that external stakeholders and taxpayers may review and comment on them. The Office of Management and Budget (OMB) is responsible for approving each form once every 3 years. The purpose of OMB’s approval is to assess IRS’s compliance with the Paperwork Reduction Act, which, among other things, requires agencies to assess the extent of burden the information they collect imposes on the public. Under the Paperwork Reduction Act, OMB must approve new forms and major revisions to existing ones. After OMB’s approval, the forms and instructions are sent to IRS’s vendors to be printed. Generally, IRS needs to have approved forms ready for printing by early October to ensure that they can be printed and distributed to the public when the tax filing season starts the following January. As shown in table 1, IRS used taxpayers or IRS employees to test the clarity of five individual income tax forms and instructions from July 1997 through June 2002. IRS relied primarily on focus groups to do the testing. Contractors using private citizens did three of the tests (i.e., Earned Income instructions, Child Tax Credit instructions, and Schedule D) and IRS did the other two using IRS employees. The two testing methods that IRS used—focus groups and one-on-one interviews—are among the commonly used methods to obtain data from individuals on whether documents such as forms and instructions are clear and understandable. Focus groups generally consist of a small number of participants—about 8 to 12 persons—-and are usually selected and organized around the focus group topic. Focus groups are a form of group interviewing that relies on interaction within the group to obtain the impressions of a group of people but not necessarily the impressions of each participant. One-on-one interviews, which aim to obtain individual attitudes, beliefs, and feelings, are used to probe individuals about specific difficulties they may have with completing a form or reading instructions. In some instances, these methods may be used in combination depending upon the particular circumstances of the test. As noted in table 1, IRS used both focus groups and one-on-one interviews in testing the Schedule D form. Testing written documents such as tax forms and instructions helps ensure they are clear, thereby benefiting taxpayers and IRS. Researchers from three federal agencies and a private research firm said testing leads to clearer documents as well as more accurate responses. Our guidance on developing and using questionnaires also recommends testing prior to distribution. IRS’s experience indicates that testing likely improves the clarity of tax forms and instructions and may therefore help reduce the number of errors taxpayers make. Although limited data were available on the costs and benefits of testing IRS forms and instructions, recent changes to Earned Income Credit (EIC) and Child Tax Credit forms illustrate the potential for benefits to significantly exceed the costs of testing. Researchers from the National Center for Health Statistics (NCHS), the Census Bureau, and the Bureau of Labor Statistics (BLS) said that testing helps ensure that their documents are clear, and therefore users are more likely to understand them and complete them correctly. Consequently, researchers from the agencies said they routinely test written documents, such as forms or surveys, prior to public distribution. A representative of a private research firm also said that testing ensures that documents communicate clearly and that his firm and many others perform such testing for a wide variety of private and public clients. The researchers from the federal agencies told us that the benefits of testing are difficult to quantify and, in some instances, may not be quantifiable. NCHS officials stated that the experience they have gained over time from testing and revising documents has helped them develop clearer forms from the outset, a benefit that may not be quantifiable. A Census researcher also agreed that testing is beneficial but difficult to quantify. Nevertheless, the Census researcher said that testing documents prior to public use helps ensure that they are clear and understandable, which gives the agency a greater chance of receiving accurate responses and which lessens the need for follow-up interviews. Similarly, a BLS researcher noted that, though difficult to quantify, testing benefits the agency by reducing errors made by respondents and those reductions can result in savings of time, money, and effort for BLS. See appendix II for additional information on the use of testing in these three agencies. The representative of the private research firm also provided some perspective of its experiences in conducting tests for public and private sector customers. This firm, like many similar companies, provides a variety of testing and data collection services to public and private sector customers. The firm arranges and conducts focus groups, one-on-one interviews, and other tests in order to ensure the clarity of forms, instruction manuals, surveys, and Web sites among other things. Focus groups generally involve 12 participants and cost around $6,700 to $7,600, excluding costs to develop the item to be tested but including incentive pay for participants that could range from $25 to $50 per participant. These costs may be higher, the representative said, if the participants come from special groups. For example, if the participants are medical doctors, incentive pay could be as high as $250 per participant. The researcher also said that private firms vary in how much they spend to test their documents. He estimated that firms generally spend between $300,000 and $500,000 to ensure that a form or document is clear and will meet the firm’s needs. In some cases, firms spend more than $500,000 to do a series of tests, making revisions between each test, before they arrive at a final version of a form or document. He also added that most firms do not spend at levels that would allow them to test all their forms and documents. While maintaining that testing is beneficial, researchers also stated that testing is not fail-safe. It can help identify particular parts of a form that are not clear, researchers said, but it cannot ensure that subsequent changes to the form will entirely resolve the clarity issue. In addition, testing may identify problems participants have in completing written documents, but the participants’ problems may be related to other issues, such as poor math skills, rather than confusing or unclear documents. Testing questionnaires before distribution is also recommended as a quality assurance measure in our guidance on developing and using questionnaires. According to our guidance, testing questionnaires before they are used is one of the best ways to ensure that the document actually communicates what it was intended to communicate and that users will uniformly interpret it. Testing increases the likelihood that respondents will provide the information needed and helps to alleviate inaccurate responses. Our guidance is also consistent with professional literature on survey design. According to professional literature, “reducing measurement error through better question design is one of the least costly ways to improve survey estimates. For any survey, it is important to attend to careful question design and pretesting.” IRS’s recent limited experience with testing indicates that testing may help ensure the clarity of tax forms and instructions. In 1999, IRS revised the forms and instructions related to the EIC and the Child Tax Credit, tested the revised forms and instructions, and revised them again based on the test results. The following year, when the revised and tested forms and instructions were used by taxpayers, the error rates for the EIC and the Child Tax Credit decreased by 28 and 35 percent, respectively, as shown in table 2. IRS officials told us they attribute part of the decrease in EIC errors to a new approach officials developed for structuring EIC forms and instructions and part to the improvements in the draft documents that resulted from testing the revised forms and instructions. Before the EIC forms and instructions were revised, the instructions included a definition and example describing a qualifying child that taxpayers had to interpret. Incorrectly claimed qualifying children have been a major source of EIC errors.