NISTIR 7990

Human Engineering Design Criteria 

Standards Part 3: Interim Steps

FY10-14 DHS S&T TSD Standards Project

Susanne Furman

Mary Theofanos

Hannah Wald

http://dx.doi.org/10.6028/NIST.IR.7990
NISTIR 7990

Human Engineering Design Criteria Standards 

Part 3: Interim Steps

FY10-14 DHS S&T TSD Standards Project

Susanne Furman

Mary Theofanos

Information Access Division

Information Technology Laboratory

Hannah Wald

Booz Allen Hamilton, Inc.

McLean, VA

http://dx.doi.org/10.6028/NIST.IR.7990

March 2014

U.S. Department of Commerce 

Penny Pritzker, Secretary

National Institute of Standards and Technology 

Patrick D. Gallagher, Under Secretary of Commerce for Standards and Technology and Director 
Page ii 3/2014

Table of Contents

EXECUTIVE SUMMARY..................................................................................................... 1

1 Introduction ................................................................................................................. 3

2 Background .................................................................................................................. 4

2.1 The DHS User Population................................................................................................... 4

2.2 Key Concepts .................................................................................................................... 5

2.2.1 Human Systems Integration (HSI).........................................................................................5

2.2.2 User-Centered Design (UCD) ................................................................................................5

2.2.3 Context of Use ......................................................................................................................5

3 HSI Gaps Within DHS..................................................................................................... 6

3.1 Interviewee Identified Challenges and Future Technology Needs........................................ 7

3.1.1 Environmental Factors Affecting Equipment........................................................................7

3.1.2 Software and Equipment Interfaces.....................................................................................7

3.1.3 Future Technology Needs.....................................................................................................8

3.2 Systemic Standards Gaps................................................................................................... 8

3.2.1 DHS Software Applications and Interfaces...........................................................................9

3.2.2 Mobile Devices and Touch Interfaces...................................................................................9

3.2.3 Biometrics...........................................................................................................................10

3.2.4 Accessibility.........................................................................................................................11

4 Recommended Interim Steps...................................................................................... 11

4.1 Integrate HSI Into the Technology Acquisition Process.......................................................11

4.1.1 ISO Standards for UCD ........................................................................................................12

4.1.2 NIST Common Industry Specification for Usability – Requirements ..................................12

4.1.3 NIST Handbook on Usability and Biometrics......................................................................13

4.2 Emphasize HSI Within the Organization ............................................................................15

5 Conclusion.................................................................................................................. 16

6 References.................................................................................................................. 18
Page 1 3/2014

EXECUTIVE SUMMARY

The Department of Homeland Security (DHS) requires general human systems integration (HSI) 

criteria for the design and development of human-machine interfaces for the technology, 

systems, equipment, and facilities employed by its user population. HSI is the relationship 

between humans and their environment and in particular how systems are designed and used 

relative to that relationship with the goal of ensuring a safe and effective environment that meets 

the mission. In general, HSI addresses hardware, software, and processes.

However, systematically adopting and applying HSI criteria within DHS will be a challenge 

because of the department’s large and extremely varied user population. The DHS personnel who 

operate and maintain the department’s technology and systems carry out a variety of different 

tasks in operating environments ranging from airports and border points of entry to subways and 

Coast Guard vessels. Other DHS users include public health officials; state and local first 

responders; travelers to be screened; bystanders; and the general public.

DHS sought help from the National Institute of Standards and Technology (NIST) Visualization 

and Usability Group (VUG) regarding standards and guidance for HSI criteria. Working in 

partnership with DHS, VUG designed a three-phase project that would help DHS meet its goals.

In Phase 1 of this project, VUG identified and reviewed the body of existing human factors and 

HSI standards, best practices, and guidelines. These standards are described in Part 1 of the

Human Engineering Design Criteria Standards Part series.

In Phase 2 of the effort, VUG researchers identified core, high-impact processes performed by 

different DHS directorates, then interviewed and (when possible) directly observed end users 

who performed those tasks. The results of the Phase 2 study highlighted five critical areas where 

there is a need for HSI standards related to interfaces, described in Part 2: 

1. Client-side interfaces for real-time, remote software applications, which are vital to many 

of the tasks performed by DHS end users. 

2. Hand-held and mobile devices, such as smartphones and tablet computers, which many 

interviewees identified as being critical to mission success in the future.

3. Touch interfaces, which are the default means of interaction with almost all 

contemporary mobile devices.

4. Interfaces for biometric collection devices, in terms of usability and languageindependent symbols for multi-cultural user populations.

5. Accessibility for both DHS agents and the populations with whom they interact; this 

applies to Web pages, application interfaces, and biometric collection processes.

Phase 3 of the project entails determining where DHS may use existing HSI standards and where 

there are gaps in those standards that the organization must address by creating new standards to 

fill its needs, particularly to address the gaps described above. NIST recognizes that developing 

and adopting appropriate HSI standards to address the previously described gaps will take time –

roughly three to five years. This document, Part 3, offers recommendations on steps DHS can 
Page 2 3/2014

take in the relatively near term to improve its usability posture and build an HSI-oriented 

organizational culture.

First, DHS should seriously consider conducting research (similar to the effort in Phase 2) in 

order to understand the context of use for the technologies employed by its end users. Context of 

use is a particularly important consideration for DHS, since users from different directorates may 

perform broadly similar tasks but do so under very different circumstances. For example, 

personnel from U.S. Citizenship and Immigration Services (USCIS) and the Coast Guard 

(USCG) both collect biometric information: however, USCIS agents primarily work in an office, 

while USCG agents work primarily on ships – and so must contend with sun glare, fog, darkness, 

rough waters, moisture, and other environmental conditions that can significantly affect their 

equipment and their ability to perform their tasks effectively.

Secondly, while DHS is very forward thinking with respect to technology – always looking for 

new technology solutions to help it carry out its missions – the organization needs to expand its 

focus to include the people who use that technology. This includes both primary users (DHS 

agents) and secondary users (the populations they work with). The most effective and impactful 

way to improve DHS’ usability posture is to explicitly address HSI and usability concerns during 

the technology procurement process. This document lists and summarizes standards and 

guidance that DHS can use to incorporate HSI into its technology acquisition process through 

user-centered design (UCD) activities.

Finally, DHS needs to conduct some type of user acceptance and usability testing of potential 

new technologies before deploying them in the field. Performing this kind of evaluation is part of 

the UCD process described earlier, but it deserves special emphasis because it can help DHS 

proactively address usability problems early in the technology acquisition process.

Following these recommendations will ensure that DHS equipment and processes are appropriate 

to their context of use, which will translate into fewer adverse effects on user health and safety; 

increased accessibility and sustainability; and improved user satisfaction as a whole. DHS end 

users will be able to complete their tasks more efficiently and effectively, and the organization as 

a whole will realize a greater return on its technology investments. 
Page 3 3/2014

1 INTRODUCTION

The Department of Homeland Security (DHS) requires general human systems integration (HSI) 

criteria for the design and development of human-machine interfaces for their technology, 

systems, equipment, and facilities. The goal of the DHS Science & Technology (S&T) Human 

Factors and Behavioral Science Division Human System Engineering Project is to identify, 

develop, and apply a standard process to enhance technology and system design, system safety, 

and operational efficiency.

The project manager partnered with the National Institute of Standards and Technology (NIST) 

Visualization and Usability Group (VUG) in furtherance of this effort. As part of its mission, 

NIST performs research to develop the technical basis for standards related to measurement, 

equipment specifications, procedures, and quality control benchmarks for industrial processes 

(among others), for organizations and users in industry, academia, government, and other 

sectors, while remaining objective and vendor-neutral. VUG, part of the NIST Information 

Technology Laboratory, conducts research in HSI and human-computer interaction (HCI) 

technologies. Members of VUG are also active on the International Organization for 

Standardization (ISO) Technical Committees Working Groups in HCI.

NIST’s work on this project consists of three phases:

1. Identify and review the body of publicly available existing human factors and HSI 

standards, best practices, and guidelines for applicability to DHS.

2. Apply a user-centered design (UCD) approach for the DHS organization in order to 

determine how existing HSI standards can be mapped to DHS needs, technology, and 

processes, and identify standards gaps.

3. Determine where DHS may use existing standards and where it may need to augment 

existing HSI standards and/or create new DHS HSI standards to address gaps and meet 

organizational needs. 

Simply put, HSI is the relationship between humans and their environment and in particular how 

systems are designed and used relative to that relationship with the goal of ensuring a safe and 

effective environment that meets the mission. In general, HSI includes the integration of 

hardware, software, and processes (including design and acquisition processes).

HSI design criteria, principles, and practices will benefit DHS by:

• improving performance of personnel,

• reducing skill and personnel requirements and training time,

• enhancing the usability, safety, acceptability, and affordability of technology and 

systems, and

• achieving the required reliability and productivity of personnel-equipment combinations

[1][3].
Page 4 3/2014

But most importantly for DHS, HSI Design Criteria Standards will foster design standardization 

and interoperability within and among DHS systems.

2 BACKGROUND

This is the third and final document in the Human Engineering Design Criteria Standards series. 

It should be noted that while Part 1 and Part 2 of this series align to Phase 1 and Phase 2 of this 

project respectively, this document – Part 3 – does not. 

In Phase 1 of this project, the NIST team identified and reviewed the body of existing human 

factors and HSI standards, best practices, and guidelines. These standards are described in 

Human Engineering Design Criteria Standards Part 1.

Phase 2 focused on identifying and interviewing DHS employees for the purpose of 

understanding the end users’ main tasks, technologies and devices they use to complete those 

tasks, and the context of use. These interviews allowed the research team to identify the “feature 

sets,” e.g., interface characteristics, of identified devices and technologies and then map them to 

the standards identified in Phase 1 of the project. The mapping exercise also helped the NIST 

team identify where DHS may need to augment existing HSI standards and/or create new 

standards to meet organizational needs. The results of the interviews and subsequent mapping are 

described in Human Engineering Design Criteria Standards Part 2.

This document, Part 3, does not align exactly to Phase 3 of the project. Completing that phase –

specifically, modifying and/or developing human factors and HSI standards to meet DHS’ 

organizational needs - will take a long time (between three and five years, based on the 

experience of similar organizations). Rather, Part 3 describes interim steps that DHS can take to 

improve its HSI posture and reduce the incidents of future usability issues while the long-term 

effort to develop needed HSI standards is underway.

2.1 THE DHS USER POPULATION

Although numerous Federal standards exist that establish general HSI and human engineering 

criteria for design and development of systems, equipment, and facilities (including DoD MILSTD-1472G Department of Defense Design Criteria Standard and NASA-STD-3000 ManSystems Integration Standards, among others), each of these standards also contains very 

domain-specific information and focuses on specialized populations, types of systems, and 

system functions.

In contrast, the DHS user populations’ characteristics are varied. The populations encompass not 

only Federal civil servants who operate and maintain the department’s technology and systems, 

but also a variety of other personnel, including public health officials; state and local first 

responders; travelers to be screened; bystanders; and the general public. Therefore DHS must 

consider a much broader range of user dimensions, characteristics, abilities, and ages than those 

populations addressed by the existing standards. DHS operating environments are also very 

diverse, ranging from airports and border points of entry to subways and Coast Guard vessels. 
Page 5 3/2014

Thus the existing standards may not be applicable based on differences in the populations and 

specific domains or context of use. 

2.2 KEY CONCEPTS

This section provides definitions of three critical (and interrelated) concepts that are referenced 

throughout this document: Human Systems Integration, User-Centered Design, and Context of 

Use. These concepts help to bring the usability issues described in Sec. 3 into sharper focus, and 

are foundational to the recommendations detailed in Sec. 4.

2.2.1 Human Systems Integration (HSI) 

HSI emphasizes human considerations as the top priority in systems design/acquisition to reduce 

life cycle costs and optimize system performance [6]. Essentially, HSI is the relationship 

between humans and their environment – particularly how systems are designed and used 

relative to that relationship – with the goal of ensuring a safe and effective environment that 

meets the mission. As previously described, HSI includes the integration of hardware, software,

and processes (including design and acquisition processes). In short, HSI is about improving the 

usability of technology. ISO defines usability as "the extent to which a product can be used by 

specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a 

specified context of use" [2].

2.2.2 User-Centered Design (UCD) 

UCD is a well-established design approach that concentrates on developing usable systems by 

focusing on the system users, their needs, and requirements. The approach applies principles of 

human factors and ergonomics, as well as usability knowledge and techniques. The goals of the 

UCD approach are to:

• Enhance effectiveness and efficiency

• Improve human well-being

• Increase user satisfaction 

• Improve accessibility and sustainability 

• Counteract possible adverse effects of use on human health, safety, and performance

There is a substantial body of knowledge in both human factors/ergonomics and usability 

demonstrating how user-centered design can be organized and applied effectively. In addition, 

UCD is supported by three sets of formal standards: ISO 9241-210, ISO/IEC TR 25060, and 

ISO/IEC 25062 (described in more detail in Sec. 4.1.1).

2.2.3 Context of Use 

For purposes of this study, the elements in the context of use for a particular product (which can 

be a technology, system, device, piece of equipment, or process) include: the intended users, 

their goals and tasks, associated equipment, and the physical and social environment in which the 

product can be used.
Page 6 3/2014

Understanding the context of use for a particular technology requires asking the right questions. 

For example: 

• Why is this technology being used? What task and/or process is it being used to 

accomplish?

• Who are the end users for this particular technology? What are the characteristics of the 

end user population (e.g., age, physical and mental capabilities, technical aptitude)?

• What are the characteristics of the technology itself? What are its component features? Is 

it fixed or portable?

• When is this technology used? What triggers the process/task the technology is used to 

carry out? At what point in the process/task is this technology used? How frequently is 

this technology used on an hourly, daily, monthly, etc. basis?

• Where is the technology used? Are there any environmental characteristics – such as 

dust, lighting conditions, or noise – that may impact the functioning or effectiveness of 

this technology?

The standards and guidelines described in Sec. 4.1 provide more detailed instructions on how to 

study, and capture data about, context of use.

3 HSI GAPS WITHIN DHS

The recommendations in this document are based on usability issues discovered during Phase 2 

of the project. In Phase 2, the NIST research team applied a UCD approach for the DHS 

organization in order to illustrate how existing HSI standards (identified in Phase 1) can be 

mapped to DHS needs, technology, and processes. Researchers identified core, high-impact 

processes performed by six different DHS directorates:

• U.S. Citizenship and Immigration Services (USCIS)

• Transportation and Security Administration (TSA)

• U.S. Coast Guard (USCG)

• Customs and Border Protection (CBP)

• U.S. Immigration and Customs Enforcement (ICE)

• Federal Emergency Management Agency (FEMA)

The research team identified, interviewed, and (when possible) directly observed end users who 

performed the tasks associated with those processes. The information collected during the 

interview process allowed the team to identify the feature sets (e.g., device interface 

characteristics) of the equipment used by end users, map those features to existing HSI standards, 

and begin to identify any gaps not addressed by those standards.

While it should be noted that Phase 2 was not an exhaustive gap analysis, the study results do 

highlight major areas of need for HSI standards and practices within the organization. Both 

general and specific areas of need uncovered during Phase 2 are described in the following 

subsections.
Page 7 3/2014

3.1 INTERVIEWEE IDENTIFIED CHALLENGES AND FUTURE TECHNOLOGY NEEDS

During the interviews conducted with end users in Phase 2, NIST researchers queried

interviewees about issues they have with the technology or devices they use to carry out their 

tasks and their perceptions of future technology needs. This section summarizes some of the 

most critical issues and future technology needs mentioned by interviewees in Part 2. All of these 

issues form the primary basis for the larger systemic HSI gaps described in Sec. 3.2, as well as 

some of the recommendations provided in Sec. 4.

3.1.1 Environmental Factors Affecting Equipment 

A common theme of issues highlighted by interviewees was that of equipment performance 

being negatively impacted by environmental factors. A number of interviewees said that 

fingerprint scanners need to be dust-resistant and/or easy to clean, since dust buildup presents a 

problem in both high-traffic indoor environments and outdoor environments.

1

In a similar vein, 

interviewees from multiple directorates said that ambient lighting conditions often negatively 

affect the performance of their cameras – specifically, their ability to take adequate facial 

photographs.

For the Coast Guard, environmental factors and their effects on equipment are a particularly 

acute problem. The conditions of the shipboard environment in which they work include sun 

glare, nighttime conditions, fog, extreme temperature differences, the possibility of fall damage, 

and limited space. Much of the equipment in their biometric capture kits was not designed with 

these conditions in mind, resulting in significant usability problems for Coast Guard end users. 

The findings of the Part 2 study indicate that environmentally related usability problems are 

fairly widespread, and that the negative impact of such problems – collectively speaking, if not 

on an individual basis – is fairly significant. Resolving such problems in existing equipment may 

often be difficult and costly. However, DHS can proactively address environmentally related

issues in new technologies by field-testing them in the environment of use before they are widely 

deployed (an approach actually recommended by some Phase 2 interviewees). This document 

covers testing in greater detail in Sec. 4.1.2 and Sec. 4.1.3.

3.1.2 Software and Equipment Interfaces 

Interviewees highlighted multiple issues related to software interfaces. ICE agents said that they 

must access multiple systems to enroll and/or check for detainee information (e.g., prior arrest 

records), and that it would be helpful to have a single, consolidated interface for this purpose 

instead. Similarly, Border Patrol agents said that they would prefer a more streamlined, easy-touse interface for their software. Additionally, interviewees from both organizations said that

more user-friendly software would help them minimize the time required to perform database 

 

1

This applies to other types of equipment as well, such as monitors and chemical detectors used by TSA agents.
Page 8 3/2014

queries and improve their task efficiency. It would also help them more rapidly identify 

potentially dangerous detainees, and thus reduce the risk to their safety. 

Biometric interfaces were also a matter of concern for interviewees from the Coast Guard and 

CBP Air and Marine. End users from both organizations frequently collect biometric information 

from individuals with little or no proficiency in English, and the language barrier between the 

agents and these “secondary users” can delay or interfere with the biometrics collection process. 

Coast Guard interviewees suggested that symbol-driven or language-independent biometric 

devices could help address this issue. 

DHS can build upon existing standards to improve its software interfaces, and could realize some 

significant benefits – gains in efficiency, reduction in errors, and (in some cases) improved user 

safety – by doing so. Developing more usable interfaces for biometric devices will be somewhat 

more difficult, since relevant standards do not yet exist. However, DHS has the potential to be a 

leader in this area: many of its end users are in a position to provide valuable input into a 

biometrics usability standards development effort. 

3.1.3 Future Technology Needs 

During interviews of DHS employees in Phase 2, interviewees discussed their needs for new

technologies that they could use to help them perform their tasks more effectively. For example, 

users from USCIS and CPB Air and Marine said that they needed electronic options to replace 

largely paper-driven processes. Interviewees from CBP Air and Marine also said that they could 

use Biometrics on the Move technology to replace tasks that require them to direct passengers 

through the biometrics capture process, e.g., “stand here”, “take off your hat”, “move your hair” 

and so on.

Interviewees from other organizations saw considerable potential in advanced biometric 

technologies as well. Members of the Coast Guard said they wanted to use facial recognition 

technologies, which could help quickly identify dangerous individuals and reduce safety risks. 

ICE agents said that they need mobile technology and devices for processing detainees in the 

field (including biometric collection), which will facilitate rapid identification of repeat offenders 

and/or dangerous individuals.

As an organization, DHS has always been eager to embrace new technologies that can help it 

better carry out its mission. It may be beneficial to the organization to take a “bottom up” 

approach to technology acquisition – in other words, to adopt technologies that fulfill its users’ 

most frequently expressed needs. 

3.2 SYSTEMIC STANDARDS GAPS

The issues described in the previous section, along with the research team’s observations, 

indicate certain critical systemic gaps in HSI technology and communications standards that 

DHS may wish to prioritize in its HSI standards development efforts. These are described in the 

following subsections. 
Page 9 3/2014

3.2.1 DHS Software Applications and Interfaces 

DHS application software packages are critical to many of the tasks performed by the end users 

interviewed in Phase 2. Some of the application types reported in that part of the study include 

real-time, client-server architectures, Web architectures, and mobile device applications. 

The particular issues cited by interviewees from ICE and the Border Patrol point to a need for 

DHS to adopt interface and information presentation standards for real-time, non-local service 

applications. Some existing standards for websites, such as those found in ISO 9241, could be 

applied for this purpose. By employing interface standards to create apps that are more 

streamlined and customized for context of use, DHS could reduce the amount of time required to 

perform tasks and reduce the incidence of omissions and mistakes. Using these standards could 

also ensure the usability and utility of applications that might be used to replace largely paperdriven tasks (such as those performed by USCIS or CBP Air and Marine) in the future.

DHS has the opportunity to use cloud computing technologies to support an integrated data 

model for certain types of applications that are used by multiple directorates, e.g., those that 

collect or reference biometric identity data. Various views of the data could be provided to end 

users via applications that have been customized for their specific context of use – in other 

words, the computing platform they are using, their role, the task they are performing, and their 

access privileges. While this integrated data model would present clear advantages for DHS, 

implementing it would require significant effort.

While standards and best practices exist for interface design, these are limited and do not address 

cloud-based services. DHS may want to partner with an accredited standards development 

organization to fill these gaps. The issue of interface design standards also applies to mobile 

technologies, which are discussed in more detail in the following subsection.

3.2.2 Mobile Devices and Touch Interfaces 

During Phase 2, interviewees from various directorates identified mobile technology (such as 

smartphones and tablet computers) as being critical to mission success in the future. In particular, 

Border Patrol and ICE officers said they need mobile technology that will allow them to process 

detainees in the field. Hand-held or mobile devices are already used in many directorates 

throughout DHS, such as the wand metal detectors used by TSA or mobile fingerprinting 

devices. All of these devices have some component or components not covered by the HSI 

standards identified in Phase 1. 

While there are a number of existing HSI standards that address generic mobile device 

components (such as displays, touch inputs, and operating systems), there are few if any 

standards that address specific mobile device interface features such as touch screen active area 
Page 10 3/2014

and size; display controls; separation of buttons or touch adjacent active areas for use 

with/without gloves; and display lighting that accommodates night or limited lighting conditions.

Another consideration is environmental conditions such as sun, rain, wind, or other elements, 

which can potentially impact the functionality of a mobile device. In addition, connectivity and 

response times for mobile devices can be impacted by use in the field. Long response times for 

database queries and lag time can subject field officers to undue risk when dealing with detainees 

(a concern raised by interviewees from ICE). Adopting and adhering to a threshold for 

acceptable response and/or lag time would increase the safety of the field officers. 

Authentication on mobile devices can also be challenging given the size of the interface and 

input device, and while there are a variety of authentication options that can work with desktop 

or laptop computers relatively easily (e.g., card reader, fingerprints, retina scan, voice 

recognition, or facial recognition systems) there are fewer options available to facilitate 

authentication on small mobile displays and keyboards.

3.2.3 Biometrics 

Since collecting biometric information is a critical part of many DHS processes – USCIS, 

USCG, CBP, and ICE all collect biometric information – standards regarding biometric 

collection and devices should be of particular importance to DHS. A significant challenge in this 

area is that, while standards do exist regarding biometrics, they fall short of addressing some of 

the considerations of biometrics collection at DHS.

One general issue is the usability of biometrics collection processes. Some of the end users 

interviewed during Phase 2 of the study said they found it difficult to capture facial images with 

cameras due to environmental conditions such as ambient light levels. To help ensure betterquality facial image captures, DHS should adopt standards that specify required components, 

capabilities, and features for cameras and the characteristics of the environment in which they 

are to be used. These standards should be dependent upon the context (for example, taking 

pictures at a customs station versus taking pictures aboard a Coast Guard cutter). Other types of 

biometrics collection factors may also require usability standards; for example, labeling, height, 

and angle specifications for electronic fingerprint scanners.2

Another issue is the multi-cultural population from whom DHS collects biometric information. A 

sizeable fraction of that population has little or no proficiency in English, which can significantly 

hinder the biometric collection tasks performed by various DHS directorates. One way to 

alleviate this problem is to use language-independent icons and graphics to represent concepts 

that will help individuals go through the biometric collection process. Such icons can reduce the 

need for translation of text to explain those same concepts. However, use of icons can be 

problematic when used cross-culturally in this way, so developing icons that will adequately 

 

2

Standards for usability in biometrics are being developed by the NIST M1.6 working group.
Page 11 3/2014

serve a multi-cultural population is a challenge. The International Standards Organization (ISO) 

JTC1/SC37/WG 6 develops icons for biometrics. 

3.2.4 Accessibility 

On a related note, DHS agents interact with different populations that may include individuals 

with some type of disability. Disabilities may include physical, sensory, or cognitive 

impairments as well as various types of chronic disease. These disabilities can impact an 

individual’s ability to carry out routine activities, which include reading Web pages, using 

applications, and understanding instructions. Individuals with disabilities may also experience 

difficulties undergoing biometric collection processes.

The World Wide Web Consortium (W3c) and Section 508.gov address accessibility standards 

for website design and applications. However, DHS also needs to consider accessibility for 

disabled individuals who interact with other technology and devices used by DHS agents, 

including those that involve biometric collection. There may be a need for additional 

accessibility standards addressing those technologies and devices.

4 RECOMMENDED INTERIM STEPS

It is reasonable to expect that identifying or developing and applying standards to address the 

current major HSI gaps described above would be a multi-year effort. In addition, DHS must 

consider emerging technologies its end users are likely to use in the near future, such as those 

mentioned by Phase 2 interviewees (e.g., mobile devices, emerging web applications, tools to 

support automation of tasks). As the technology and processes used within DHS change, new 

HSI standards will be needed – which means that iterative reviews of both standards and 

organizational processes will also be needed.

This section provides some recommendations on actions DHS can take while the organization 

addresses the identified systemic standards gaps. These actions will help DHS establish a more 

HSI-oriented organizational culture, so that it can effectively deal not only with existing HSI and 

usability issues but proactively address any similar issues that may come up in the future.

4.1 INTEGRATE HSI INTO THE TECHNOLOGY ACQUISITION PROCESS

One of the major findings that emerged from the Phase 2 study is that DHS needs to explicitly 

address HSI and usability considerations in its acquisition process. In particular, DHS needs to 

perform user acceptance and usability testing on new technologies before they are deployed in 

the field. Making such testing standard practice could help DHS proactively identify and address 

a number of usability issues like those described in Sec. 3.1.

In particular, DHS should add language to its procurement contracts to address:

• Definition of context of use

• Usability testing as defined in ISO/IEC 25062 (see Sec. 4.1.1)
Page 12 3/2014

• Usability requirements as defined in NIST IR 7432: Common Industry Specification for 

Usability (see Sec. 4.1.2)

• User acceptance testing as defined in the NIST Handbook Usability and Biometrics: 

Ensuring Successful Biometric Systems (see Sec. 4.1.3)

This section presents helpful standards and guidelines (including those in the list above) for 

UCD-oriented technology acquisition processes, including the user acceptance and usability 

testing described earlier.

4.1.1 ISO Standards for UCD 

UCD is supported by three sets of formal standards: ISO 9241-210, ISO/IEC TR 25060, and 

ISO/IEC 25062. 

ISO 9241-210 provides requirements and recommendations for user-centered design principles 

and activities throughout the life cycle of computer-based interactive systems. 

ISO/IEC TR 25060 describes a potential family of International Standards, named the Common 

Industry Formats (CIF), that document the specification and evaluation of the usability of 

interactive systems. The Technical Report focuses on documenting design and development 

elements of usable systems. It does not prescribe a specific process and is intended for use with 

ISO 9241 standards. 

ISO/IEC 25062 standardizes the types of information captured with user testing. The level of 

detail allows the same or another organization to replicate the test procedure. Major variables 

include: user demographics, task descriptions, test context (including the equipment used, the 

testing environment, and the participant and test administrator’s interaction protocol), and the 

metrics chosen to code the study findings. Advantages of using standardized reporting format 

include: (1) a reduction in training time for usability staff since an individual only needs to learn 

to use one form regardless of how many companies he works for and (2) enhanced potential for 

increased communication between vendors and purchasing organizations since readers of CIFcompliant reports will share a common language and expectations.

4.1.2 NIST Common Industry Specification for Usability – Requirements 

NIST IR 7432: Common Industry Specification for Usability – Requirements (CISU-R) sets 

standards for specifying usability requirements, which include three types of information [4]:

• The context of use: the intended users, their goals and tasks, associated equipment, and 

the physical and social environment in which the product can be used.

• Performance and satisfaction criteria: measures of usability for the product.

• The test method and context of testing: the method to be used to test whether the 

usability requirements have been met and the context in which the measurements will be 

made.
Page 13 3/2014

More specifically, the CISU-R provides guidance and structure to support the following 

activities related to gathering and articulating usability requirements during software 

development projects:

• Document the context of use for a product, including definitions of the expected 

technical, physical, and social environments, user groups, goals for use of the product and 

scenarios of use.

• Write usability requirements in sufficient detail to make an effective contribution to 

design and development.

• Relate usability requirements to stakeholder requirements (including user, customer, and 

business) for successful use of a product and increased productivity.

• Define usability criteria that can be empirically validated. 

• Define the method for testing the product against the criteria.

• Create requirements that are useful throughout the product design and development 

process, providing input to the design process early in a project and adding more detailed 

information about criteria and methods, as it is available.

The CISU-R complements and expands upon the ISO standards for UCD described in Sec. 4.1.1, 

so it is possible (but not necessary) to use it in conjunction with those standards. In this and other 

respects, it is designed to be flexible and accommodate any type of development process. It 

supports different levels of compliance so that it can “match the level of detail and formality of 

the usability requirements to business needs” in a given project [4]. It can also be applied to 

contractual relationships between organizations – particularly useful for DHS, which works with 

private-sector organizations to develop most of its technological solutions. 

4.1.3 NIST Handbook on Usability and Biometrics 

The NIST Handbook Usability and Biometrics: Ensuring Successful Biometric Systems details a 

UCD design methodology specifically for the development of biometric technologies [5]. It 

incorporates the ISO standards described in the previous section. The design process consists of 

four steps:

• Define the Context of Use

• Determine User and Organizational Requirements

• Develop the Design Solution

• Conduct the Evaluation

As Fig. 1 below demonstrates, the latter three steps in the Handbook are best treated as a cycle. 

Evaluation should generate feedback that helps define new requirements and/or further elaborate 

existing ones. Changes to requirements spur changes in system design, and the modified system 

must be tested again.
Page 14 3/2014

Figure 1: The NIST Usability and Biometrics Development Process

Step 1 is to Define the Context of Use. The Usability and Biometrics handbook provides 

guidance on how to conduct research with users in order to:

• Identify user populations (both primary and secondary)

• Capture user demographics

• Capture characteristics of the environment of use

• Explicitly define user goals and tasks

The output from Step 1 informs the two subsequent steps. In Step 2, Determine the User and 

Organizational Requirements (i.e., requirements analysis), information about context of use 

should form the basis of user, environmental/physical, and functional requirements. An example 

of each type of requirement is described in the list below:

• User requirement: Foreign nationals who have little or no English proficiency need a 

fingerprint scanner with a language-independent, symbol-driven interface to help guide 

them through the process.

• Environmental/physical requirement: Cameras used by members of the Coast Guard to 

capture facial photos of illegal aliens at sea must automatically adapt to ambient light 

levels, so that picture quality will not be impacted by low-light conditions or glare.

• Functional Requirement: Black lights used by TSA agents to check IDs should have a 

long battery life, be easily rechargeable, or both.

Step 3 is to Develop the Design Solution. For DHS this may mean selecting an available 

technology that best fills the requirements elaborated in Step 2. Regardless of whether the 

technological solution is designed or acquired, however, user demographics and goals captured 

in Step 1 should be used as the basis for interaction design, interface design, and any associated 

training materials. Also, DHS should not neglect secondary users, i.e., non-DHS employees who 

participate in DHS processes.
Page 15 3/2014

Step 4, Conduct the Evaluation, is where usability and user acceptance testing occurs. According 

to the handbook: 

“A well-conducted and well-planned project will have several rounds of 

evaluation, at varying levels of fidelity. By incorporating user feedback 

throughout the design of a system, it is easier to identify major problems or flaws 

at a much earlier stage.” [5]

There are two types of evaluation: qualitative and quantitative. Qualitative evaluation involves:

• Asking users about their expectation of what the system will do and how it will function

• Observing users interacting with a system while “thinking aloud” and noting areas that 

cause user confusion or frustration

• Probing for suggestions from users and asking users about their level of satisfaction with 

the system

Quantitative evaluation involves measuring the following:

• Task Completion Rates: Percent of users who successfully complete each task

• Time on Task: Time it takes for users to perform a task from beginning to end

• Error Rates: Number of errors made during the course of a task

• Satisfaction Rating: Satisfaction scores for the system

A balanced approach combines both qualitative and quantitative evaluation. Generally, 

qualitative evaluation is appropriate for earlier stages of the design process, while quantitative

evaluation is best applied during later stages. As indicated in Fig. 1, the cycle of evaluation, 

feedback, and modification should be repeated as many times as is practical.

In addition to outlining this four-step UCD process, the handbook contains detailed guidance on 

how to carry out the activities associated with each step.

4.2 EMPHASIZE HSI WITHIN THE ORGANIZATION

To effectively address its HSI and usability issues, DHS, like any organization, must first 

understand the context of use for the technologies it deploys in the field. The best way to do this 

is to use the study described in Part 2 as an “exemplar methodology”: in other words, to use its 

approach and associated questionnaire as the basis for longer-term, more intensive UCD field 

studies. In particular, these studies should focus capturing any problems that end users have with 

the equipment they use to perform their tasks and the environment in which they do so. 

Performing such UCD-based field studies is an excellent way to gather data on, and understand, 

the context of use for particular technologies, whether they are fingerprint scanners at U.S. ports 

of entry or cameras used on Coast Guard cutters. The context of use for a particular product 

(which can be a technology, system, device, piece of equipment, or process), as defined in Sec. 

2.2.3, includes: the intended users, their goals and tasks, associated equipment, and the physical 

and social environment in which the product can be used.
Page 16 3/2014

For an organization like DHS, context of use is an important consideration. It is clear from the 

data collected during Phase 2 that end users in different directorates may perform broadly similar 

tasks. For example, five of the directorates’ (i.e., USCIS, USCG, CBP, ICE, FEMA) end users 

use some type of camera as well as a fingerprint capture device. However, the context of use in 

which these tasks are performed may be very different. There is a significant difference between 

fingerprinting an applicant at a USCIS center and a (potentially dangerous) illegal alien at the 

border or at sea.

Also, many of the issues highlighted by interviewees in Phase 2 concerned ways in which tasks

or equipment were compromised by the conditions of the working environment, i.e., the context 

of use. One example is fingerprint scanners: in certain environments, sun glare, moisture, or dust 

often affected the quality of captured fingerprints. A number of interviewees said that they 

encountered similar problems when using cameras to take photographs: certain environmental 

characteristics affected device functionality and/or picture quality. These kinds of problems 

result from mismatches between the equipment being used for a certain task and the context of 

use for that task. Thus, while theoretically it makes sense to use the same type of equipment 

(e.g., fingerprint scanner, camera) across the organization for similar or identical tasks performed 

by end users in different directorates, the reality is that different environments of use necessitate 

at least some degree of equipment customization. 

Field studies like the one conducted in Part 2 should also include asking end users about the 

technology they currently use – in essence, whether it meets their requirements – and also about 

technology they would like to have, or anticipate having, in the near future (e.g., mobile devices, 

biometrics on the move). The subsequent findings can be used as the basis for a variety of HSIrelated organizational efforts, including user requirements gathering and acceptance testing (as 

described in Sec. 4.1.3). DHS can also use such findings to more accurately predict, and 

proactively address, emerging technology- and usability-related needs. 

5 CONCLUSION

DHS is very forward-thinking with respect to technology: the organization is always looking for 

new technology solutions to help it carry out its mission. But if DHS wants to maximize its 

return on investment in new technologies, it needs to expand its focus to include the people who 

use those technologies – both department agents and the populations they work with. In the end, 

technology is only as useful as it is usable.

As previously described, DHS can significantly improve the usability of its technologies by 

studying and understanding context of use; integrating UCD practices into its technology 

acquisition processes; and performing usability and user acceptance testing of new technologies 

before they are deployed. DHS can take these actions while applying existing HSI standards 

(detailed in Phase 1) to its currently deployed technologies and addressing the issues described 

by DHS end users. Making HSI and usability important considerations in the acquisition process 

will allow DHS to identify and mitigate potential usability issues before they create significant 

difficulties for end users out in the field.
Page 17 3/2014

By following UCD standards and best practices during the technology acquisition process, DHS 

can help its users complete their tasks more efficiently and effectively, with fewer adverse 

effects to their health and safety. Integrating HSI and usability into technology acquisition will 

also help DHS realize a greater return on its investments in new technologies, while pursuing 

new standards to address gaps identified in Phase 2.
Page 18 3/2014

6 REFERENCES

[1] Buie, E., & Murray, D. (2012). Usability in government systems: User experience design 

for citizens and public servants. Waltham, MA: Morgan Kaufmann. 

[2] ISO 9241-210:2010 Ergonomics of human-system interaction – Part 210: Humancentered design for interactive systems

[3] Sherman, P. (2006). Usability success stories: How organizations improve by making 

easier-to-use software and websites. Hampshire, England: Gower House.

[4] Theofanos, M. & Industry Usability Reporting Project Working Group. U.S. Department 

of Commerce, National Institute of Standards and Technology (NIST). (2007). Common 

industry specification for usability – requirements (NIST IR 7432). Retrieved March 1, 

2013 from http://www.nist.gov/customcf/get_pdf.cfm?pub_id=51179.

[5] Theofanos, M., Stanton, B., & Wolfson, C.A. U.S. Department of Commerce, National 

Institute of Standards and Technology (NIST). (2008). Usability and biometrics: 

Ensuring successful biometric systems. Retrieved February 28, 2013 from 

http://www.nist.gov/customcf/get_pdf.cfm?pub_id=152184.

[6] U.S. Navy. (2012, July 03). Naval postgraduate school – HSI. Retrieved from 

http://www.nps.edu/or/hsi/.