DEPARTMENT OF HOMELAND SECURITY 

Office of Inspector General 

Letter Report: 

Citizenship Test Redesign 

 

 

 

 

OIG-05-25 June 2005 

Office of Inspections and Special Reviews
 Office of Inspector General 

 U.S. Department of Homeland Security

 Washington, DC 20528

 

June 14, 2005 

MEMORANDUM TO: Eduardo Aguirre 

Director

U.S. Citizenship and Immigration Services 

 

FROM: Richard L. Skinner 

Acting Inspector General 

 

SUBJECT: Citizenship Test Redesign 

In 2001, the U.S. Immigration and Naturalization Service (INS), then part of the Department of 

Justice, began an initiative to redesign the English and U.S. history and civics tests delivered to 

naturalization applicants. This initiative has continued under the aegis of the U.S. Citizenship 

and Immigration Service (USCIS) within the Department of Homeland Security. Among the 

objectives of the redesign is the development of a standardized test to improve upon the current 

test delivery process and test content. The redesign is to address concerns that the current testing 

procedures are not uniform across all USCIS offices and do not effectively determine if a 

naturalization applicant has a meaningful understanding of the English language and U.S. history 

and civics. 

The redesign of the naturalization tests has significant implications for USCIS operations and the 

naturalization process experienced by millions of immigrants for years to come. We have 

tracked recent developments in the naturalization redesign process in line with our oversight 

responsibilities and out of concern for the potentially major impact of this initiative. Between 

December 2004 and February 2005, we met with USCIS officials, monitored naturalization 

interviews, reviewed relevant documentation, and attended public sessions of the National 

Research Council’s Committee on the U.S. Naturalization Test Redesign. Initially, we 

anticipated performing a traditional evaluation and review of USCIS’ test development initiative. 

However, due to the early stage of test development and likelihood that important aspects of the 

examination process will change, we are abbreviating our review and issuing this letter report 

instead. We share some of our observations for your consideration as you move forward in the 

test redesign process. 
Statutory, Regulatory, and Policy Basis for Testing

The statutory basis for testing naturalization applicants on English and U.S. history and civics is 

the Immigration and Naturalization Act of 1952, as amended. The Act states that for 

naturalization applicants to qualify for citizenship, they must demonstrate an understanding of 

the English language and a knowledge and understanding of the fundamentals of the history, and 

of the principles and form of government, of the United States. USCIS regulations and policy 

guidance further require applicants to demonstrate the ability to read, write, and speak words in 

ordinary usage in the English language while allowing certain exceptions for age, length of 

residence, or medical impairment. 

USCIS determines an applicant’s ability to speak English on the basis of his or her answers to 

questions normally asked in the naturalization interview. According to test administration 

guidelines, reading and writing skills are to be tested using excerpts from Federal Textbooks on 

Citizenship written at the elementary literacy level or other approved sources. Under current 

USCIS policy, an applicant passes the English language test if he or she can read one sentence 

and write another sentence to the satisfaction of an adjudicator. Each applicant is to be given 

three opportunities for both the reading and writing tests. 

USCIS regulations require the delivery of the U.S. history and civics test in an oral format giving 

due consideration to an applicant’s background, age, length of residence in the United States, and 

other factors. Applicants are to be given 10 questions on subject matter covered in the Federal 

Textbooks on Citizenship. Adjudicators are encouraged to randomly generate these 10 questions 

from the USCIS CLAIMS 4 database, which contains a list of 100 approved questions. To pass 

the test, an applicant must answer six or more of these questions correctly. 

Past Concerns with the Testing Process

S. COMMISSION ON IMMIGRATION REFORM 

Coopers and Lybrand (now IBM Business Consulting Services) studied the naturalization 

process on behalf of the INS in 1997. As a result of its review effort, Coopers and Lybrand 

expressed a number of concerns with the naturalization testing process. The firm found that, at 

the time, INS had no standard naturalization test content, testing instruments, test protocols, or 

scoring system. 

Also in 1997, the U.S. Commission on Immigration Reform reported that there was significant 

variation among INS district offices in the methods by which they administered the 

naturalization test and in the threshold number of correct answers required for passing. The 

report also asserted that, in some cases, INS adjudicators scaled the tests to the perceived 

educational abilities of applicants. The commission asserted that the tests used to evaluate the 

knowledge of U.S. history and civics did not do so in a meaningful way. The report noted that 

the civics test, for example, relies on memorization of discrete facts rather than on substantive 

understanding of the basic concepts of civic participation.1

 

 

 1 Becoming an American: Immigration and Immigrant Policy, U.S. Commission on Immigration Reform, 

Washington, DC, 1997, pp. 46-47.

 
 

The commission suggested that the tests be standardized and aim to evaluate a common core of 

information to be understood by all new citizens. The commission maintained that the U.S. 

history and civics test should assess whether applicants understand the basic principles of U.S. 

government: for example, what it means to have freedom of speech or the freedom to assemble. 

The commission further recommended that the English test accurately and fairly measure an 

immigrant’s ability to speak, read, and write in English.2

 

To increase the meaningfulness of the exams and foster greater standardization, the commission 

encouraged INS officials to develop a new U.S. history and civics exam, as well as, a new 

English language test. The commission encouraged INS officials to undertake this test 

development effort alongside “professional educators, pedagogical experts, and standardized test 

providers.”3

A July 2000, DOJ Office of Inspector General (OIG) report on INS’s Citizenship USA initiative4

found that the English and U.S. history and civics tests were unevenly administered throughout 

the INS. Among the cited contributing factors to this uneven test administration was a lack of 

clear standards to assess an applicant’s ability to speak, read, and write English. The system 

relied on the adjudicator’s discretion to determine the evaluation standard for each applicant. 

For the U.S. history and civics test, INS regulations explicitly authorized the use of discretion to 

give due consideration to the “education, background, age, length of residence in the United 

States” and the “opportunities and efforts made to acquire the requisite knowledge.”5

 

The DOJ OIG found that for both the English and U.S. history and civics tests, the use of 

discretion among INS adjudicators resulted in varying levels of test difficulty and inconsistent 

standards for what was considered a passing score. The OIG recommended that INS 

immediately develop a standard by which to evaluate an applicant's ability to read, write and 

speak words in ordinary usage in the English language. The OIG stated that this standard would 

have to be established through guidance to adjudicators and that it would have to define the 

questions asked in the naturalization interview and describe how those questions should be posed 

to applicants. For the U.S. history and civics exam, the OIG recommended that INS provide 

guidance and training to its adjudication officers concerning the nature and number of questions 

to be used to test an applicant's knowledge of U.S. history and civics. This would define the 

range of questions that an adjudication officer could choose from and how the officer should 

tailor the test to the individual applicant. The OIG also recommended that the guidance address 

the discretion that an adjudication officer has to choose questions, including the number of 

questions offered and whether there is a required passing percentage. 

In December 2000, INS developed a policy to standardize testing procedures in an effort to bring 

more consistency to the way adjudicators test and score applicants. The policy indicates the 

source materials to be used for test questions, how many questions are to be asked, how 

2

 Ibid., p. 47. 

3

 Ibid., p. 47. 

4

 Citizenship USA was an initiative to reduce the backlog of pending naturalization applications within one year to 

the point where an eligible applicant would be naturalized within six months of application. (DOJ OIG, An 

Investigation of the Immigration and Naturalization Service's Citizenship USA Initiative, July 2000) 5

 8 C.F.R. 312.2(c) (2) 
responses are to be evaluated, how many correct answers are required to pass the history and 

civics test, and how applicants are to be informed of the results. 

Despite the improved guidance, immigration stakeholder groups continued to criticize INS, and 

now USCIS, for inconsistently applying test procedures across offices and among adjudication 

officers. Stakeholders continue to articulate concerns that variation continues to exist in test 

delivery format, test content, level of difficulty, the number of questions asked, and the number 

of correct answers required for passing. Some of these stakeholders have expressed their desire 

for USCIS to improve testing consistency and standardization while maintaining flexibility to 

fairly address the differences across the broad spectrum of applicant characteristics including, 

among others, educational, professional, and cultural background, income level, age, and first 

language. 

Test Redesign Efforts

In 2001, INS began a redesign effort to address concerns with the testing process. INS 

contracted with a firm specializing in educational test development at an estimated cost of $3 

million over approximately three and a half years to restructure the naturalization test format and 

procedures. The deliverables of the contract included test content development with input from 

experts and stakeholders, research and study on test formats and test administration procedures, 

and pilot testing of sample English test questions. The contractor tested sample English 

questions at five USCIS offices from mid-March to mid-June, 2003. 

Then, in April 2004, USCIS contracted with the National Research Council’s Board on Testing 

Assessment (BOTA) to review the test redesign process and provide independent advice to assist 

USCIS and its test development contractor optimize the validity, reliability, and fairness of the 

redesigned tests. To accomplish this, BOTA established the Committee on the U.S. 

Naturalization Test Redesign, consisting of faculty members from various colleges and 

universities and representatives from research groups with expertise in education, testing, 

political science, psychology, history, and other relevant subject areas. Among its charges, the 

committee was asked to review ongoing USCIS test redesign decisions, examine pertinent data 

about possible testing options, discuss technical issues, and provide feedback and assessment on 

USCIS decisions. 

In its interim report issued in December 2004, the committee reviewed USCIS’s redesign efforts 

leading up to the first trial test of sample English questions. The committee expressed its 

concern that the test development process had not met its view of professional test development 

standards and that too many of the tests’ design decisions had been made by a small number of 

USCIS and contractor staff. Regarding the redesigned tests as potentially controversial, highstakes exams, the committee encouraged USCIS to apply an “open, transparent, and 

accountable” test development process.6

 

 

6 Redesigning the U.S. Naturalization Tests: Interim Report, Committee on the U.S. Naturalization Test Redesign, 

Board on Testing and Assessment, Center for Education, Division of Behavioral and Social Sciences and Education, 

National Research Council, The National Academies Press, Washington, DC, 2004, p. 1. 
 

The committee recommended that USCIS: 1) put in place an advisory structure to advise the 

agency in making important decisions about the naturalization test redesign; 2) create a detailed 

test development plan complying with testing standards with help from a technical advisory 

panel and review by an oversight committee; 3) cease work on developing the content 

frameworks until a clear, transparent, and publicly accountable process is defined and endorsed 

by an oversight group; and 4) once testing formats are determined, develop a detailed plan for 

standard setting (i.e., determining passing scores), with input from the technical advisory group 

and final recommendation by the oversight committee. In support of these recommendations, the 

committee asserted that the creation of a multi-tiered advisory structure overseeing the test 

development process would “meet professional standards of good testing practice and ... 

increase the likelihood that stakeholders [would] view the [new tests] as legitimate.” 7

USCIS considered the committee’s recommendation to create an advisory structure overseeing 

the test development effort and perceived that the implementation of such a structure would 

require adherence to Federal Advisory Committee Act (FACA) standards.8

 Federal agencies that 

sponsor advisory committees must adhere to the requirements established by the FACA and 

related administrative guidelines provided by the U.S. General Services Administration's (GSA) 

Committee Management Secretariat. 9 These requirements include, but are not limited to, the 

development of a charter requiring public notification and filing with Congress, development of 

ethics standards and procedures for the advisory body, identifying membership structure and 

potential members, and notifying members of selection. Once members have been selected, the 

sponsoring agency must solicit financial disclosure information from members, determine 

whether members have a conflict of interest, and collect personnel information associated with 

members’ status as special government employees. Then, when committee sessions begin, 

members are to receive an orientation to the sponsoring agency and staff, FACA requirements, 

their special employee status, and ethics policies and procedures. As committee meetings 

progress, the sponsoring agency has obligations to record meeting minutes, archive records, 

respond to public inquiries, advertise meetings, develop meeting agendas, prepare committee 

review materials, coordinate meeting logistics, and submit information for a comprehensive 

annual review by GSA. 

Apart from a staff support burden, advisory committee activities sometimes occur at the expense 

of timeliness. A committee’s activities require public notification and coordination among 

committee members and, therefore, can be time consuming, slowing the advisory committee’s 

ability to make recommendations and, ultimately, the sponsoring agency’s ability to make 

decisions. 

7

 Ibid., p. 7. 

8

 The Federal Advisory Committee Act (Public Law 92-463, 5 U.S.C., App) was enacted by Congress in 1972. The 

purpose of FACA is to ensure that advice rendered to the executive branch by various advisory committees be both 

objective and accessible to the public. FACA formalized a process for establishing, operating, overseeing, and 

terminating these advisory bodies that offer federal officials information and advice on a broad range of issues 

affecting federal policies and programs. (The Federal Advisory Committee Act (FACA) Brochure, General Services 

Administration, Office of Governmentwide Policy, Committee Management Secretariat, Washington, DC, 2004.) 9

 In 1977, Executive Order 12024 delegated to the Administrator of GSA all responsibilities of the President for 

implementing the FACA. 
Resolving differences of opinion within advisory committees also may prove difficult. FACA 

requires that committee memberships be "fairly balanced in terms of the points of view 

represented and the functions to be performed." In balancing committee memberships, agencies 

are expected to assure that major and sometimes strongly-opposing viewpoints are represented to 

provide a foundation for developing advice and recommendations that are fair and 

comprehensive.10 In light of the potential magnitude of public input and the range of strong 

opinions that currently exist with regard to immigration and naturalization issues, it is 

conceivable that the differences in opinion could become irreconcilable. 

After weighing the BOTA committee’s recommendations, USCIS opted to continue 

naturalization test redesign on its own. It did not renew the BOTA committee’s contract or 

fund its plans for future analysis and reporting on the test redesign effort. 

The BOTA committee correctly asserted that the naturalization test development process 

should be open, transparent, accountable, and technically sound. The committee also 

accurately maintained that its proposed advisory structure could have supported these test 

development goals. Nevertheless, a test development process that is sufficiently open, 

transparent, accountable, and technically sound can be achieved by other means. The pursuit of 

such alternative test development processes is USCIS’s prerogative. 

Future Challenges in Naturalization Test Redesign

Whatever the character of its test development process, USCIS will face at least three major 

challenges as it proceeds with the naturalization test redesign effort. First, USCIS management 

must ensure that the corresponding test development process reaches a timely conclusion and 

does so without undue expense. Second, while new examinations are being developed, USCIS 

should more strenuously enforce internal compliance with current test administration standards. 

Finally, in crafting new naturalization exams, USCIS must resolve important technical and 

policy issues stemming from tensions within and between statutory guidance and test 

development goals and objectives. 

Timeliness and Financial Considerations 

Given concerns with the current examination process, it is important for USCIS to advance the 

redesign effort in a timely fashion. Current plans already reflect delays in the test development 

process. The redesign effort began in 2001 and was initially projected to end in 2006. USCIS, 

however, recently announced January 2007 as its “flexible” target date for implementation. To 

advance the project in a timely way, USCIS should establish overall test redesign objectives with 

a clear set of project milestones and unambiguous lines of responsibility for accomplishing tasks 

and completing deliverables. USCIS should clearly define what it expects staff and contractors 

to accomplish and keep them focused on their tasks. 

USCIS also should continue to assess its test development plans to determine whether they are 

the most economical and effective means of meeting USCIS’s statutory obligations. USCIS 

 

10 The Federal Advisory Committee Act (FACA) Brochure, General Services Administration, Office of 

Governmentwide Policy, Committee Management Secretariat, Washington, DC, 2004. 
 

must continue to weigh the growing incremental costs of the redesign effort against the benefits 

of a redesigned test and test process. To the extent possible, we urge USCIS to use the products 

and knowledge obtained in its initial redesign efforts and evaluate the potential for using existing 

test materials and commercial off the shelf products. As USCIS has expended significant funds 

and time up to this point, we urge that some of this effort be salvaged. 

Interim Test Administration 

Current plans call for the new tests to be delivered more than 18 months hence. In the meantime, 

USCIS should ensure that its adjudication offices are implementing the standardization 

procedures described in its December 26, 2000 guidance memorandum. Adjudicators should be 

using the proper source materials, test format, procedures, and methods for determining passing 

scores. We strongly urge continuing and active training for adjudicators and monitoring of 

testing practices to assure compliance with current policies. 

Technical and Policy Challenges 

In moving the test redesign process forward, USCIS faces three major technical and policy issues 

that require management attention. 

Abstract U.S. History and Civics Concepts and Plain English 

The first of these challenges relates to the conflict between the need to test knowledge of abstract 

U.S. history and civics concepts and the limited test standard for English language proficiency. 

This conflict arises from the competing knowledge requirements set out in law and those 

articulated as test redesign objectives. The Immigration and Naturalization Act and its 

corresponding regulations set the threshold for successful demonstration of English language 

speaking, reading, and writing skills at an “ordinary usage” level. Despite this, much of the 

discussion surrounding the effort to increase the meaningfulness of the exams has centered on 

ensuring that applicants demonstrate an understanding of more complex aspects of U.S. history 

and civics than is presently required. Accordingly, if USCIS is to satisfy its statutory obligations 

while also meeting an important test redesign objective, it must arrive at an examination process 

that provides applicants who have a limited English language proficiency with an opportunity to 

effectively demonstrate their knowledge of more complex ideas than they may have the English 

language skills to understand or communicate. The tension between these competing 

requirements presents a major technical challenge for test developers and potential policy issues 

for USCIS management. 

Balancing “Due Consideration” Against Test Uniformity 

Another complicating factor in the naturalization test redesign effort stems from the current 

requirement that test administrators provide “due consideration” to applicant traits in 

administering part of the examination process while simultaneously ensuring that the tests are 

“uniformly” implemented. The U.S. Code states that, “the examination of applicants for 

naturalization ... shall be uniform throughout the United States.”11 This standard of uniformity 

11 8 USC § 1443 
 

is, however, somewhat undermined by current regulations related to the exam. These 

regulations provide test administrators with latitude in “choosing the subject matters, in 

phrasing questions and in evaluating responses” during the U.S. history and civics test. Under 

current regulations, test administrators are to exercise this latitude in “due consideration ... to the 

applicant's education, background, age, length of residence in the United States, opportunities 

available and efforts made to acquire the requisite knowledge, and any other elements or factors 

relevant to an appraisal of the adequacy of the applicant's knowledge and understanding.”12 This 

broad requirement to exercise latitude in administering the U.S. history and civics test presently 

amounts to a mandate to customize, rather than standardize test administration. Moreover, 

absent strict rules on how to apply “due consideration,” this regulatory guidance will contribute 

an element of capriciousness to the examination process and guarantee significant variation in 

how the test is administered from one test administrator to the next. Without detailed and 

carefully construed standards for applying “due consideration,” a redesigned naturalization test 

will not meet the requisite standard of uniformity. 

Achieving Meaningful Change without Affecting Test Difficulty 

Perhaps an even greater technical challenge to the test redesign effort is USCIS’ goal of making 

the tests more “meaningful” and yet no more difficult nor any easier than the current tests. This 

will be extremely difficult to accomplish due to the diversity in the applicant base. Any changes 

to the content or testing procedures will almost certainly have the effect of making the tests more 

difficult for some and easier for others. Some stakeholders perceive the tests to be too easy as 

they are, claiming that the U.S. history and civics test, for example, requires only memorization 

of facts, and does not test a deeper understanding of citizenship and civic participation in the 

United States. Others assert that the current tests are already too difficult and an attempt to make 

the tests more “meaningful” is, in actuality, an attempt to make the tests more difficult. 

Regardless, USCIS needs to articulate how it intends to achieve its goal of making the test 

neither more difficult nor any easier and how it will validate that test difficulty. This is a 

challenge, as USCIS has established little specific information on how different applicant 

populations fare under the current testing regime. Without detailed information on current test 

performance, there is no established baseline against which to compare new tests’ degree of 

difficulty. 

Furthermore, on the surface, the declared objective of fielding a new set of naturalization tests 

that are no more difficult, nor any easier than the current exams appears to call into question the 

need for substantial efforts toward a redesign, at all. USCIS must better communicate why it is 

seeking to reform the current tests when it is not seeking any change in the bottom-line impact 

the tests have on which applicants or what proportion of applicants pass the tests. 

12 8 C.F.R. 312.2(c)(2) 
We extend our appreciation to USCIS for the cooperation and courtesies extended to our staff. 

Should you have any questions, please call me, or your staff may contact Robert L. Ashbaugh, 

Assistant Inspector General for Inspections and Special Reviews, at (202) 254-4100. We hope 

our observations will be of assistance as you move forward. 
Additional Information and Copies 

To obtain additional copies of this report, call the Office of Inspector General 

(OIG) at (202) 254-4100, fax your request to (202) 254-4285, or visit the OIG 

web site at www.dhs.gov.

OIG Hotline 

To report alleged fraud, waste, abuse or mismanagement, or any other kind 

of criminal or noncriminal misconduct relative to department programs or 

operations, call the OIG Hotline at 1-800-323-8603; write to DHS Office of 

Inspector General/MAIL STOP 2600, Attention: Office of Investigations – 

Hotline, 245 Murray Drive, SW, Building 410, Washington, DC 20528; fax 

the complaint to (202) 254-4292; or email DHSOIGHOTLINE@dhs.gov. The 

OIG seeks to protect the identity of each writer and caller.