NISTNCSTAR1-7B

Federal Building and Fire Safety Investigation of the

World Trade Center Disaster

Technical Documentation for Survey

Administration: Questionnaires,

Interviews, and Focus Groups

Johanna Zmud

National Institute of Standards and Technology • Technology Administration U.S. Deportment of Commerce

NIST NCSTAR1-7B

Federal Building and Fire Safety Investigation of the

World Trade Center Disaster

Technical Documentation for Survey

Administration: Questionnaires,

Interviews, and Focus Groups

Johanna Zmud

NuStats Partners, LP

September 2005

U.S. Department of Commerce

Carlos M. Gutierrez, Secretary

Technology Administration

Mictiele O'Neill, Acting Under Secretary for Technology

National Institute of Standards and Technology

William Jeffrey, Director
Disclaimer No. 1

Certain commercial entities, equipment, products, or materials are identified in this document in order to describe a procedure or concept adequately or to trace the history of the procedures and practices used. Such identification is not intended to imply recommendation, endorsement, or implication that the entities, products, materials, or equipment are necessarily the best available for the purpose. Nor does such identification imply a finding of fault or negligence by the National Institute of Standards and Technology.

Disclaimer No. 2 The policy of NIST is to use the International System of Units (metric units) in all publications. In this document,

however, units are presented in metric units or the inch-pound system, whichever is prevalent in the discipline.

Disclaimer No. 3 Pursuant to section 7 of the National Construction Safety Team Act, the NIST Director has determined that certain evidence received by NIST in the course of this Investigation is "voluntarily provided safety-related information" that is "not directly related to the building failure being investigated" and that "disclosure of that information would inhibit the

voluntary provision of that type of information" (15 USC 7306c).

In addition, a substantial portion of the evidence collected by NIST in the course of the investigation has been

provided to NIST under nondisclosure agreements.

Disclaimer No. 4 NIST takes no position as to whether the design or construction of a WTC building was compliant with any code

since, due to the destruction of the WTC buildings, NIST could not verify the actual (or as-built) construction, the

properties and condition of the materials used, or changes to the original construction made over the life of the

buildings. In addition, NIST could not verify the interpretations of codes used by applicable authorities in determining

compliance when implementing building codes. Where an Investigation report states whether a system was

designed or installed as required by a code provision, NIST has documentary or anecdotal evidence indicating

whether the requirement was met, or NIST has independently conducted tests or analyses indicating whether the requirement was met.

Use in Legal Proceedings

No part of any report resulting from a NIST investigation into a structural failure or from an investigation under the

National Construction Safety Team Act may be used in any suit or action for damages arising out of any matter mentioned in such report (15 USC 281a; as amended by P.L. 107-231).

National Institute of Standards and Technology National Construction Safety Team Act Report 1-7B

Natl. Inst. Stand. Technol. Natl. Constr. Sfty. Tm. Act Rpt. 1-7B, 79 pages (September 2005)

CODEN: NSPUE2

U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON: 2005

For sale by the Superintendent of Documents, U.S. Government Printing Office

Internet: bookstore.gpo.gov — Phone: (202) 512-1800 — Fax: (202) 512-2250

Mail: Stop SSOP, Washington, DC 20402-0001
Abstract

This report documents the methods and outcomes of the telephone survey, face-to-face interviews, and

focus groups that were conducted in support of the federal building and fire safety investigation of the

World Trade Center disaster. In total, 803 telephone interviews were completed, 220 face-to-face

interviews were completed, and 6 focus groups involving 28 individuals were completed. All interview

material was transferred to National Institute of Standards and Technology investigators for analysis.

Keywords: Survey, face-to-face interviews, focus groups. World Trade Center.

NISTNCSTAR 1-7B, WTC Investigation ni
This page intentionally left blank.

•i

NISTNCSTAR 1-7B, WTC Investigation
Table of Contents

Abstract iii

List of Figures vii

List of Tables ix

List of Acronyms and Abbreviations xi

Preface xiii

Executive Summary xxiii

Chapter 1 Introduction 1 LI Objective 1

1.2 Protection of Human Subjects 1

1 .3 Paperwork Reduction Act 2

L4 Toil-Free Hotline 2

Chapter 2 Summary of Data Collection Methods 3

2. 1 Telephone Survey 3

2.1.1 Sampling Approach 4

2. 1 .2 Instrument Development 5

2.1.3 CATI Interviev^ing 6

2.2 Face-to-Face Interviews 7

2.2. 1 Sampling Approach 7

2.2.2 Instrument Development 9

2.2.3 Face-to-Face Interviews with Occupants 1

1

2.2.4 Face-to-Face Interviews with Family Members 12

2.3 Focus Group Interviews 13

2.3.1 Focus Group Recruitment and Interview Protocol 14

Chapter 3 Summary of Data Collection Outcomes 15

3 . 1 Telephone Interviews 15

3.2 Face-to-Face Interviews.... 17

3.3 Focus Group Interviews 18

NISTNCSTAR 1-7B, WTC Investigation v
Table of Contents

Chapter 4

Data File Deliveries 19

4.1 Telephone Survey Interviews 19

4.2 Face-to-Face Interviews 19

4.3 Focus Group Interviews 19

Appendix A

Work Plan 21

Appendix B

Data Collection Methodology for World Trade Center Evacuation and Emergency

Response: Telephone Interviews, Face-to-Face Interviews, Focus Groups and

Population Sampling 41

vi NISTNCSTAR 1-7B, WTC Investigation
List OF Figures

Figure P-1. The eight projects in the federal building and fire safety investigation of the WTC

disaster xv

Figure A-1 . Example pen tablet PC 27

NISTNCSTAR 1-7B, WTC Investigation vii
List of Figures

This page intentionally left blank.

viii NISTNCSTAR 1-7B, WTC Investigation
List of Tables

Table P-1. Federal building and fire safety investigation of the WTC disaster xiv

Table P-2. Public meetings and briefings of the WTC Investigation xvii

Table 2-1. Example tabular face-to-face interview data entry 10

Table 3-1. Disposition of the CATI sample and the total sample by tower 16

Table 3-2. Summary disposition rates by tower 17

NISTNCSTAR 1-7B, WTC Investigation ix
List of Tables

This page intentionally left blank.

X NISTNCSTAR 1-7B, WTC Investigation
List of Acronyms and Abbreviations

Arronx/mQ

ADA Americans with Disabilities

BSIT Behavioral Sequence Intei'view Teclinique

CAPI Computer-Assisted Personal Interviewing

CATI Computer-Assisted Telephone Interviewing

CIM Cognitive Interviewing Method

DHHS Department of Health and Human Services

DTAP Dissemination and Technical Assistance Program

FDNY Fire Department of New York City

FEMA Federal Emergency Management Agency

FWA Federalwide Assurance

IRB Institutional Review Board

NCST National Construction Safety Team

NIST National Institute of Standards and Technology

OHRP Office of Human Research Protections

0MB Office of Management and Budget

R&D Research and Development

WTC 1 World Trade Center 1 (North Tower)

WTC 2 World Trade Center 2 (South Tower)

WTC 7 World Trade Center 7

NIST NCSTAR 1-7B, WTC Investigation XI
List of Acronyms and Abbreviations

This page intentionally left blank.

xii NISTNCSTAR 1-7B, WTC Investigation
Preface

Genesis of This Investigation

Immediately following the terrorist attack on the World Trade Center (WTC) on September 1 1 , 2001 , the

Federal Emergency Management Agency (FEMA) and the American Society of Civil Engineers began

planning a building performance study of the disaster. The week of October 7, as soon as the rescue and

search efforts ceased, the Building Performance Study Team went to the site and began its assessment. This was to be a brief effort, as the study team consisted of experts who largely volunteered their time

away from their other professional commitments. The Building Performance Study Team issued its report in May 2002, fulfilling its goal "to determine probable failure mechanisms and to identify areas of

future investigation that could lead to practical measures for improving the damage resistance of buildings

against such unforeseen events."

On August 21, 2002, with funding from the U.S. Congress through FEMA, the National Institute of

Standards and Technology (NIST) announced its building and fire safety investigation of the WTC

disaster. On October 1, 2002, the National Constmction Safety Team Act (Public Law 107-231), was

signed into law. The NIST WTC Investigation was conducted under the authority of the National

Construction Safety Team Act.

The goals of the investigation of the WTC disaster were:

• To investigate the building construction, the materials used, and the technical conditions that contributed to the outcome of the WTC disaster.

• To serve as the basis for: - Improvements in the way buildings are designed, constructed, maintained, and used; - Improved tools and guidance for industry and safety officials; - Recommended revisions to current codes, standards, and practices; and - Improved public safety.

The specific objectives were:

1. Determine why and how WTC 1 and WTC 2 collapsed following the initial impacts of the

aircraft and why and how WTC 7 collapsed;

2. Determine why the injuries and fatalities were so high or low depending on location,

including all technical aspects of fire protection, occupant behavior, evacuation, and

emergency response;

3. Determine what procedures and practices were used in the design, construction, operation,

and maintenance of WTC 1, 2, and 7; and

4. Identify, as specifically as possible, areas in current building and fire codes, standards, and

practices that warrant revision.

NISTNCSTAR 1-7B, WTC Investigation Xlll
Preface

NIST is a nonregulatory agency of the U.S. Department of Commerce's Technology Administration. The

purpose of NIST investigations is to improve the safety and structural integrity of buildings in the United

States, and the focus is on fact finding. NIST investigative teams are authorized to assess building

perfonnance and emergency response and evacuation procedures in the wake of any building failure that has resulted in substantial loss of life or that posed significant potential of substantial loss of life. NIST

does not have the statutory authority to make findings of fault nor negligence by individuals or

organizations. Further, no part of any report resulting from a NIST investigation into a building failure or from an investigation under the National Construction Safety Team Act may be used in any suit or action

for damages arising out of any matter mentioned in such report (15 USC 281a, as amended by Public

Law 107-231).

Organization of the Investigation

The National Construction Safety Team for this Investigation, appointed by the then NIST Director, Dr. Arden L. Bement, Jr., was led by Dr. S. Shyam Sunder. Dr. William L. Grosshandler served as Associate Lead Investigator, Mr. Stephen A. Cauffman served as Program Manager for Administration,

and Mr. Harold E. Nelson served on the team as a private sector expert. The Investigation included eight

interdependent projects whose leaders comprised the remainder of the team. A detailed description of

each of these eight projects is available at http://wtc.nist.gov. The purpose of each project is summarized

in Table P-1, and the key interdependencies among the projects are illustrated in Fig. P-1.

Table P-1. Federal building and fire safety investigation of the WTC disaster.

Technical Area and Project Leader Project Purpose

Analysis of Building and Fire Codes and

Practices; Project Leaders: Dr. H. S. Lew

and Mr. Richard W. Bukowski

Document and analyze the code provisions, procedures, and

practices used in the design, construction, operation, and

maintenance of the structural, passive fire protection, and

emergency access and evacuation systems of WTC 1, 2, and 7. Baseline Structural Performance and

Aircraft hnpact Damage Analysis; Project

Leader: Dr. Fahim H. Sadek

Analyze the baseline performance of WTC 1 and WTC 2 under

design, service, and abnonnal loads, and aircraft impact damage on

the structural, fire protection, and egress systems.

Mechanical and Metallurgical Analysis of

Structural Steel; Project Leader: Dr. Frank

W. Gayle

Determine and analyze the mechanical and metallurgical properties

and quality of steel, weldments, and connections from steel recovered from WTC 1, 2, and 7. Investigation of Active Fire Protection

Systems; Project Leader: Dr. David

D. Evans; Dr. William Grosshandler

Investigate the performance of the active fire protection systems in WTC 1,2, and 7 and their role in fire control, emergency response, and fate of occupants and responders.

Reconstruction of Thermal and Tenability

Environment; Project Leader: Dr. Richard

G. Gann

Reconstruct the time-evolving temperature, thermal environment,

and smoke movement in WTC 1, 2, and 7 for use in evaluating the

structural performance of the buildings and behavior and fate of

occupants and responders.

Structural Fire Response and Collapse

Analysis; Project Leaders: Dr. John

L. Gross and Dr. Therese P. McAllister

Analyze the response of the WTC towers to fires with and without

aircraft damage, the response of WTC 7 in fires, the perfonnance

of composite steel-trussed floor systems, and deteraiine the most

probable structural collapse sequence for WTC 1, 2. and 7. Occupant Behavior, Egress, and Emergency

Communications; Project Leader: Mr. Jason D. Averill

Analyze the behavior and fate of occupants and responders, both

those who survived and those who did not, and the performance of

the evacuation system.

Emergency Response Technologies and

Guidelines; Project Leader: Mr. J. Randall

Lawson

Document the activities of the emergency responders from the time

of the terrorist attacks on WTC 1 and WTC 2 until the collapse of

WTC 7, including practices followed and technologies used.

xiv NISTNCSTAR 1-7B, WTC Investigation
Preface

NIST WTC Investigation Projects

Baseline Performance & Impact

Damage

Video/

Photographic

Records

Oral History Data

Thermal and

Tenability Environment

Analysis of Codes and

Practices Active Fire Protection Emergency

Response

Records

Figure P-1. The eight projects in the federal building and fire safety

investigation of the WTC disaster.

National Construction Safety Team Advisory Committee

The NIST Director also established an advisory committee as mandated under the National Construction

Safety Team Act. The initial members of the committee were appointed following a public solicitation. These were:

• Paul Fitzgerald, Executive Vice President (retired) FM Global, National Construction Safety

Team Advisory Committee Chair

• John Barsom, President, Barsom Consulting, Ltd.

• John Bryan, Professor Emeritus, University of Maryland

• David Collins, President, The Preview Group, Inc.

• Glerm Corbett, Professor, John Jay College of Criminal Justice

• Philip DiNenno, President, Hughes Associates, Inc.

NISTNCSTAR 1-7B, WTC Investigation XV
Preface

• Robert Hanson, Professor Emeritus, University of Michigan

• Charles Thornton, Co-Chainnan and Managing Principal, The Thomton-Tomasetti Group,

Inc.

• Kathleen Tiemey, Director, Natural Hazards Research and Applications hiformation Center,

University of Colorado at Boulder

• Forman Williams, Director, Center for Energy Research, University of California at San

Diego

This National Construction Safety Team Advisory Committee provided technical advice during the

Investigation and commentary on drafts of the Investigation reports prior to their public release. NIST

has benefited from the work of many people in the preparation of these reports, including the National

Construction Safety Team Advisory Committee. The content of the reports and recommendations,

however, are solely the responsibility of NIST.

Public Outreach

During the course of this Investigation, NIST held public briefings and meetings (listed in Table P-2) to

solicit input from the public, present preliminary findings, and obtain comments on the direction and

progress of the Investigation from the public and the Advisory Committee.

NIST maintained a publicly accessible Web site during this Investigation at http://wtc.nist.gov. The site contained extensive information on the background and progress of the Investigation.

NIST's WTC Public-Private Response Plan

The collapse of the WTC buildings has led to broad reexamination of how tall buildings are designed,

constructed, maintained, and used, especially with regard to major events such as fires, natural disasters,

and terrorist attacks. Reflecting the enhanced interest in effecting necessary change, NIST, with support

from Congress and the Administration, has put in place a program, the goal of which is to develop and

implement the standards, technology, and practices needed for cost-effective improvements to the safety

and security of buildings and building occupants, including evacuation, emergency response procedures,

and threat mitigation.

The strategy to meet this goal is a three-part NIST-led public-private response program that includes:

• A federal building and fire safety investigation to study the most probable factors that contributed to post-aircraft impact collapse of the WTC towers and the 47-story WTC 7 building, and the associated evacuation and emergency response experience.

• A research and development (R&D) program to (a) facilitate the implementation of

recommendations resulting from the WTC Investigation, and (b) provide the technical basis

for cost-effective improvements to national building and fire codes, standards, and practices

that enhance the safety of buildings, their occupants, and emergency responders.

xvi NISTNCSTAR 1-7B, WTC Investigation
Preface

Table P-2. Public meetings and briefings of the WTC Investigation.

Date Location Principal Agenda

June 24, 2002 New York City, NY Public meeting: Public comments on the Draf! Plan for the pending WTC Investigation.

August 21. 2002 Gaithersburg, MD Media briefing announcing the formal start of the Investigation.

December 9. 2002 Washington, DC Media briefing on release of the Public Update and NIST request

for photographs and videos. April 8, 2003 New York City, NY Joint public forum with Columbia University on first-person interviews. April 29-30, 2003 Gaithersburg, MD NCST Advisory Committee meeting on plan for and progress on WTC Investigation with a public comment session. May 7, 2003 New York City, NY Media briefing on release of May 2003 Progress Report.

August 26-27, 2003 Gaithersburg, MD NCST Advisory Committee meeting on status of the WTC

investigation with a public comment session. September 17, 2003 New York City. NY Media and public briefing on initiation of first-person data

collection projects.

December 2-3. 2003 Gaithersburg, MD NCST Advisory Committee meeting on status and initial results and release of the Public Update with a public comment session. February 12, 2004 New York City, NY Public meetmg on progress and preliminary findings with public

comments on issues to be considered in formulating final recommendations.

June 18. 2004 New York City, NY Media/public briefing on release of June 2004 Progress Report. June 22-23, 2004 Gaithersburg, MD NCST Advisory Committee meeting on the status of and

preliminary findings from the WTC Investigation with a public

comment session. August 24, 2004 Northbrook, IL Public viewing of standard fire resistance test of WTC floor system at Underwriters Laboratories, Inc. October 19-20, 2004 Gaithersburg, MD NCST Advisory Committee meeting on status and near complete

set of preliminary findings with a public comment session. iNovemoer J.z, /uu4 Gaithersburg, MD iNt^oi /VQVisory ^^ommiuee uiscussion on uraii dimudi icpoii lo Congress, a public comment session, and a closed session to discuss pre-draft recommendations for WTC Investigation.

April 5, 2005 New York City, NY Media and public briefing on release of the probable collapse

sequence for the WTC towers and draft reports for the projects on

codes and practices, evacuation, and emergency response. June 23, 2005 New York City, NY Media and public briefing on release of all draft reports for the WTC towers and draft recommendations for public comment.

September 12-13,

2005

Gaithersburg, MD NCST Advisory Committee meeting on disposition of public

comments and update to draft reports for the WTC towers. September 13-15,

2005

Gaithersburg, MD WTC Technical Conference for stakeholders and technical community for dissemination of findings and recommendations

and opportunity for public to make technical comments.

• A dissemination and technical assistance program (DTAP) to (a) engage leaders of the

construction and building community in ensuring timely adoption and widespread use of

proposed changes to practices, standards, and codes resuhing from the WTC Investigation

and the R&D program, and (b) provide practical guidance and tools to better prepare faciUty owners, contractors, architects, engineers, emergency responders, and regulatory authorities

to respond to future disasters. The desired outcomes are to make buildings, occupants, and first responders safer in future disaster

events.

NISTNCSTAR 1-7B, WTC Investigation xvii
Preface

National Construction Safety Team Reports on the WTC Investigation

A final report on the collapse of the WTC towers is being issued as NIST NCSTAR 1 . A companion

report on the collapse of WTC 7 is being issued as NIST NCSTAR I A. The present report is one of a set that provides more detailed documentation of the Investigation findings and the means by which these

technical results were achieved. As such, it is part of the archival record of this Investigation. The titles of the full set of Investigation publications are:

NIST (National Institute of Standards and Technology). 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Final Report on the Collapse ofthe World Trade

Center Towers. NIST NCSTAR 1 . Gaithersburg, MD, September.

NIST (National Institute of Standards and Technology). 2006. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Final Report on the Collapse of World Trade Center 7. NIST NCSTAR 1 A. Gaithersburg, MD.

Lew, H. S., R. W. Bukowski, and N. J. Carino. 2005. Federal Building and Fire Safety Investigation of

the World Trade Center Disaster: Design, Construction, and Maintenance ofStructural and Life Safety

Systems. NIST NCSTAR 1-1. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Fanella, D. A., A. T. Derecho, and S. K. Ghosh. 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Design and Construction ofStructural Systems.

NIST NCSTAR 1-1 A. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Ghosh, S. K., and X. Liang. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Comparison ofBuilding Code Structural Requirements. NIST

NCSTAR 1-lB. National Institute of Standards and Technology. Gaithersburg, MD, September.

Fanella, D. A., A. T. Derecho, and S. K. Ghosh. 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Maintenance and Modifications to Structural

Systems. NIST NCSTAR 1-lC. National Institute of Standards and Technology. Gaithersburg,

MD, September.

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Fire Protection and Life Safety Provisions Applied to the Design and

Construction of World Trade Center 7, 2. and 7 and Post-Construction Provisions Applied after Occupancy. NIST NCSTAR 1- ID. National Institute of Standards and Technology. Gaithersburg,

MD, September. ,

Razza, J. C, and R. A. Grill. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Comparison of Codes, Standards, and Practices in Use at the Time ofthe

Design and Construction of World Trade Center 1, 2, and 7. NIST NCSTAR 1-lE. National

Institute of Standards and Technology. Gaithersburg, MD, September.

Grill, R. A., D. A. Johnson, and D. A. Fanella. 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Comparison ofthe 1968 and Current (2003) New

XVlll NIST NCSTAR 1-7B, WTC Investigation
Preface

York City Building Code Provisions. NIST NCSTAR 1-lF. National Institute of Standards and

Technology. Gaithersburg, MD, September.

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safely Investigation ofthe World

Trade Center Disaster: Amendments to the Fire Protection and Life Safety Provisions ofthe New

York City Building Code by Local Laws Adopted While World Trade Center J, 2, and 7 Were in Use. NIST NCSTAR 1-lG. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation of the World

Trade Center Disaster: Post-Construction Modifications to Fire Protection and Life Safety Systems

of World Trade Center 1 and 2. NIST NCSTAR 1-lH. National Institute of Standards and

Technology. Gaithersburg, MD, September.

Grill, R. A., D. A. Johnson, and D. A. Fanella. 2005. Federal Building and Fire Safety Investigation

ofthe World Trade Center Disaster: Post-Construction Modifications to Fire Protection, Life

Safety, and Structural Systems of World Trade Center 7. NIST NCSTAR l-ll. National Institute of

Standards and Technology. Gaithersburg, MD, September.

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Design, Installation, and Operation ofFuel System for Emergency Power in World Trade Center 7. NIST NCSTAR 1-1 J. National Institute of Standards and Technology.

Gaithersburg, MD, September.

Sadek, F. 2005. Federal Building and Fire Safety Investigation ofthe World Trade Center Disaster:

Baseline Structural Performance and Aircraft Impact Damage Analysis ofthe World Trade Center

Towers. NIST NCSTAR 1-2. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Faschan, W. J., and R. B. Garlock. 2005. Federal Building and Fire Safety Investigation ofthe

World Trade Center Disaster: Reference Structural Models and Baseline Performance Analysis of

the World Trade Center Towers. NIST NCSTAR 1-2A. National Institute of Standards and

Technology. Gaithersburg, MD, September.

Kirkpatrick, S. W., R. T. Bocchieri, F. Sadek, R. A. MacNeill, S. Holmes, B. D. Peterson, R. W. Cilke, C. Navarro. 2005. Federal Building and Fire Safety Investigation ofthe World Trade

Center Disaster: Analysis ofAircraft Impacts into the World Trade Center Towers, NIST

NCSTAR 1-2B. National Institute of Standards and Technology. Gaithersburg, MD, September.

Gayle, F. W., R. J. Fields, W. E. Luecke, S. W. Banovic, T. Foecke, C. N. McCowan, T. A. Siewert, and

J. D. McColskey. 2005. Federal Building and Fire Safety Investigation ofthe World Trade Center

Disaster: Mechanical and Metallurgical Analysis ofStructural Steel. NIST NCSTAR 1-3. National

Institute of Standards and Technology. Gaithersburg, MD, September.

Luecke, W. E., T. A. Siewert, and F. W. Gayle. 2005. Federal Building and Fire SafetyInvestigation ofthe World Trade Center Disaster: Contemporaneous Structural Steel

Specifications. NIST Special Publication 1-3 A. National Institute of Standards and Technology.

Gaithersburg, MD, September.

NIST NCSTAR 1-7B, WTC Investigation xix
Preface

Banovic, S. W. 2005. Federal Building and Fire Safety Investigation ofthe World Trade Center

Disaster: Steel Inventory and Identification. NIST NCSTAR 1-3B. National Institute of Standards

and Technology. Gaithersburg, MD, September.

Banovic, S. W., and T. Foecke. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Damage and Failure Modes ofStructural Steel Components. NIST

NCSTAR 1-3C. National Institute of Standards and Technology. Gaithersburg, MD, September.

Luecke, W. E., J. D. McColskey, C. N. McCowan, S. W. Banovic, R. J. Fields, T. Foecke,

T. A. Siewert, and F. W. Gayle. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Mechanical Properties ofStructural Steels. NIST NCSTAR 1-3D.

National Institute of Standards and Technology. Gaithersburg, MD, September.

Banovic, S. W., C. N. McCowan, and W. E. Luecke. 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Physical Properties ofStructural Steels. NIST

NCSTAR 1-3E. National Institute of Standards and Technology. Gaithersburg, MD, September.

Evans, D. D., R. D. Peacock, E. D. Kuligowski, W. S. Dols, and W. L. Grosshandler. 2005. Federal

Building and Fire Safety Investigation ofthe World Trade Center Disaster: Active Fire Protection

Systems. NIST NCSTAR 1-4. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Kuhgowski, E. D., D. D. Evans, and R. D. Peacock. 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Post-Construction Fires Prior to September II, 2001. NIST NCSTAR ]-4A. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Hopkins, M., J. Schoenrock, and E. Budnick. 2005. Federal Building and Fire Safety Investigation

ofthe World Trade Center Disaster: Fire Suppression Systems. NIST NCSTAR 1-4B. National

Institute of Standards and Technology. Gaithersburg, MD, September.

Keough, R. J., and R. A. Grill. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Fire Alarm Systems. NIST NCSTAR I-4C. National Institute of Standards

and Technology. Gaithersburg, MD, September.

Ferreira, M. J., and S. M. Strege. 2005. Federal Building and Fire Safety Investigation ofthe

World Trade Center Disaster: Smoke Management Systems. NIST NCSTAR 1-4D. National

Institute of Standards and Technology. Gaithersburg, MD, September.

Gann, R. G., A. Hamins, K. B. McGrattan, G. W. Mulholland, H. E. Nelson, T. J. Ohlemiller,

W. M. Pitts, and K. R. Prasad. 2005. Federal Building and Fire Safety Investigation ofthe World Trade

Center Disaster: Reconstruction ofthe Fires in the World Trade Center Towers. NIST NCSTAR 1-5. National Institute of Standards and Technology. Gaithersburg, MD, September.

Pitts, W. M., K. M. Butler, and V. Junker. 2005. Federal Building and Fire Safety Investigation of

the World Trade Center Disaster: Visual Evidence, Damage Estimates, and Timeline Analysis.

NIST NCSTAR 1-5A. National Institute of Standards and Technology. Gaithersburg, MD,

September.

XX NIST NCSTAR 1-7B, WTC Investigation
Preface

Hamins, A., A. Maranghides, K. B. McGrattan, E. Johnsson, T. J. Ohlcmiller, M. Donnelly,

J. Yang, G. Mulholland, K. R. Prasad, S. Kukuck, R. Anleitner and T. McAllister. 2005. Federal

Building and Fire Safety Investigation ofthe World Trade Center Disaster: Experiments and

Modeling ofStructural Steel Elements Exposed to Fire. NIST NCSTAR 1 -5B. National Insliiutc of

Standards and Technology. Gaithersburg, MD, September.

Ohlemiller, T. J., G. W. Mulholland, A. Maranghides, J. J. Filliben, and R. G. Gann. 2005. Federal

Building and Fire Safety Investigation ofthe World Trade Center Disaster: Fire Tests ofSingle

Office Workstations. NIST NCSTAR 1-5C. National Institute of Standards and Technology.

Gaithersburg, MD, September.

Gann, R. G., M. A. Riley, J. M. Repp, A. S. Whittaker, A. M. Reinhom, and P. A. Hough. 2005.

Federal Building and Fire Safety Investigation ofthe World Trade Center Disaster: Reaction of

Ceiling Tile Systems to Shocks. NIST NCSTAR 1-5D. National Institute of Standards and

Technology. Gaithersburg, MD, September.

Hamins, A., A. Maranghides, K. B. McGrattan, T. J. Ohlemiller, and R. Anleitner. 2005. Federal

Building and Fire Safety Investigation ofthe World Trade Center Disaster: Experiments and

Modeling ofMultiple Workstations Burning in a Compartment. NIST NCSTAR 1-5E. National

Institute of Standards and Technology. Gaithersburg, MD, September.

McGrattan, K. B., C. Bouldin, and G. Fomey. 2005. Federal Building and Fire Safety

Investigation ofthe World Trade Center Disaster: Computer Simulation ofthe Fires in the World

Trade Center Towers. NIST NCSTAR 1-5F. National Institute of Standards and Technology.

Gaithersburg, MD, September.

Prasad, K. R., and H. R. Baum. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Fire Structure Interface and Thermal Response ofthe World Trade Center

Towers. NIST NCSTAR 1-5G. National Institute of Standards and Technology. Gaithersburg,

MD, September.

Gross, J. L., and T. McAllister. 2005. Federal Building and Fire Safety Investigation ofthe World Trade

Center Disaster: Structural Fire Response and Probable Collapse Sequence ofthe World Trade Center

Towers. NIST NCSTAR 1-6. National Institute of Standards and Technology. Gaithersburg, MD,

September.

Carino, N. J., M. A. Stames, J. L. Gross, J. C. Yang, S. Kukuck, K. R. Prasad, and R. W. Bukowski.

2005. Federal Building and Fire Safety Investigation ofthe World Trade Center Disaster: Passive

Fire Protection. NIST NCSTAR 1-6A. National Institute of Standards and Technology.

Gaithersburg, MD, September.

Gross, J., F. Hervey, M. Izydorek, J. Mammoser, and J. Treadway. 2005. Federal Building and

Fire Safety Investigation ofthe World Trade Center Disaster: Fire Resistance Tests ofFloor Truss

Systems. NIST NCSTAR 1-6B. National Institute of Standards and Technology. Gaithersburg,

MD, September.

Zarghamee, M. S., S. Bolourchi, D. W. Eggers, O. O. Erbay, F. W. Kan, Y. Kitane, A. A. Liepins,

M. Mudlock, W. I. Naguib, R. P. Ojdrovic, A. T. Sarawit, P. R Barrett, J. L. Gross, and

NIST NCSTAR 1-7B, WTC Investigation XXI
Preface

T. P. McAllister. 2005. Federal Building and Fire Safety Investigation ofthe World Trade Center

Disaster: Component, Connection, and Subsystem Structural Analysis. NISTNCSTAR 1-6C.

National Institute of Standards and Technology. Gaithersburg, MD, September.

Zarghamee, M. S., Y. Kitane, O. O. Erbay, T. P. McAllister, and J. L. Gross. 2005. Federal

Building and Fire Safety Investigation ofthe World Trade Center Disaster: Global Structural

Analysis of the Response ofthe World Trade Center Towers to Impact Damage and Fire. NIST

NCSTAR 1-6D. National Institute of Standards and Technology. Gaithersburg, MD, September.

McAllister, T., R. W. Bukowski, R. G. Gann, J. L. Gross, K. B. McGrattan, H. E. Nelson, L. Phan,

W. M. Pitts, K. R. Prasad, F. Sadek. 2006. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: Structural Fire Response and Probable Collapse Sequence of World Trade

Center 7. (Provisional). NIST NCSTAR 1-6E. National Institute of Standards and Technology.

Gaithersburg, MD.

Gilsanz, R., V. Arbitrio, C. Anders, D. Chlebus, K. Ezzeldin, W. Guo, P. Moloney, A. Montalva,

J. Oh, K. Rubenacker. 2006. Federal Building and Fire Safety Investigation ofthe World Trade

Center Disaster: Structural Analysis ofthe Response of World Trade Center 7 to Debris Damage

and Fire. (Provisional). NIST NCSTAR 1-6F. National Institute of Standards and Technology.

Gaithersburg, MD.

Kim, W. 2006. Federal Building and Fire Safety Investigation ofthe World Trade Center

Disaster: Analysis ofSeptember 11, 2001, Seismogram Data. (Provisional). NIST NCSTAR 1-6G.

National Institute of Standards and Technology. Gaithersburg, MD.

Nelson, K. 2006. Federal Building and Fire Safety Investigation ofthe World Trade Center

Disaster: The Con Ed Substation in World Trade Center 7. (Provisional). NIST NCSTAR I-6H.

National Institute of Standards and Technology. Gaithersburg, MD.

Averill, J. D., D. S. Mileti, R. D. Peacock, E. D. Kuligowski, N. Groner, G. Proulx, P. A. Reneke, and

H. E. Nelson. 2005. Federal Building and Fire Safety Investigation ofthe World Trade Center Disaster:

Occupant Behavior, Egress, and Emergency Communication. NIST NCSTAR 1-7. National Institute of

Standards and Technology. Gaithersburg, MD, September.

Fahy, R., and G. Proulx. 2005. Federal Building and Fire Safety Investigation ofthe World Trade

Center Disaster: Analysis ofPublished Accounts ofthe World Trade Center Evacuation. NIST

NCSTAR 1-7A. National Institute of Standards and Technology. Gaithersburg, MD, September.

Zmud, J. 2005. Federal Building and Fire Safety Investigation ofthe World Trade Center

Disaster: Technical Documentation for Sw\'ey Administration. NISTNCSTAR 1-7B. National

Institute of Standards and Technology. Gaithersburg, MD, September.

Lawson, J. R., and R. L. Vettori. 2005. Federal Building and Fire Safety Investigation ofthe World

Trade Center Disaster: The Emergency Response Operations. NIST NCSTAR 1-8. National Institute of

Standards and Technology. Gaithersburg, MD, September.

xxn NISTNCSTAR 1-7B, WTC Investigation
Executive Summary

Survey administration involved interviews with survivors of the World Trade Center (WTC) attacks using

three methods, telephone interviews, face-to-face interviews, and focus groups, and the delivery of the

resuhant data. This report documents the methods and outcomes of these interviews. Analyses of the

resuhant data were conducted by National Institute of Standards and Technology (NIST) investigators.

The interviews were carried out by a data collection team consisting of NuStats Partners, LP, Austin,

Texas, as the prime contractor, with the assistance of subcontractors - NuStats DataSource, San Marcos,

Texas, to conduct telephone interviews; GeoStats, Atlanta, Georgia, to provide data collection devices

and programming services for the face-to-face interviews; and Diversity Services, New York City, to recruit surveyors and provide office space.

The goals of the NIST World Trade Center investigation were to investigate the building construction, the

materials used, and the technical conditions that contributed to the outcome of the WTC disaster. The

results of the investigation were meant to serve as the basis for improvements in the way buildings are designed, constructed, maintained, and used; improved tools, guidance for industry and safety officials; revisions to codes, standards, and practices; and improved public safety.

This research was conducted in compliance with NIST requirements regarding the protection of human

subjects. The Office of Human Research Protections (OHRP) in the Department of Health and Human

Services (DHHS) registered Federalwide Assurance (FWA) numbers for both NuStats Partners, LP and

NuStats DataSource, LP. For both of these FWA entities, the Institutional Review Board (IRB)

designated for review was Essex IRB, Lebanon, New Jersey

NuStats prepared a comprehensive IRB package for all data collection protocols, and submitted the

package to Essex IRB. Subsequent to NIST review, Essex approved a revised IRB package. An

important partner in the provision of human subjects protection was Jamie Abelson, a senior research

associate affiliated with the University of Michigan Institute for Social Research. Based on her vast experience in dealing with sensitive interviewing situations, she provided hands-on mental health support,

as needed, to both surveyors and respondents. NIST further worked with NuStats to ensure that the data

collection met the requirements of the Paperwork Reduction Act and was awarded an Office of

Management and Budget (0MB) approval number (0693-0044).

Three data collection methodologies were employed for this investigation, including: (1) a telephone

interview of 803 occupants of the WTC towers, utilizing a scientific probability sample of sur\dving

occupants was used to identify survey participants; (2) a qualitative study of 220 face-to-face interviews

of occupants and family members of victims, utilizing a snowball quota sample of key survivor types and

the toll-free hotline were used to identify potential participants; and (3) six focus groups of occupants,

relying on a snowball quota sampling methodology for recruiting key survivor participants.

The primary goal of the telephone interview was to provide quantitative information on occupant

behavior during the evacuation experience and capture unique investigative observ^ations, particular to the events at the WTC on September 11, 2001. This data collection activity also sought to estimate the

occupant population in the WTC at the time of the attacks on September 1 1 . The mode of data collection was computer-assisted telephone interviewing (CATI).

NISTNCSTAR 1-7B, WTC Investigation XXUl
Executive Summary

The interview was designed to measure five primary content areas, presented to the respondent in the

following order:

1. Preparedness and Training - These questions collected information on the respondents'

degree of evacuation or emergency training pertinent to their September 1 1 experience. The

questions also addressed famiharity with the WTC buildings based on length employment at the location and also whether the respondents were at the WTC in 1993 during that prior

bombing incident.

2. Initial September 1 1 Experience - These questions captured data on respondents' initial awareness that something had happened to the building - where they were, what they were

doing, how they first became aware, sources of information about the event (both in terms of

other persons and also in terms of their physical environment).

3. Interim September 1 1 Experience - These questions addressed the span between the time

respondents first became aware of the WTC incident and when they began evacuating - data

items included their social environment, physical environment, information sources, and

factors influencing their decision to evacuate.

4. Evacuation Experience on September 1 1 - These questions captured information on the

respondents' egress from the building, including the factors that facilitated their egress and

those that hindered their egress.

5. Respondent Characteristics - Information collected included age, gender, language, and

mobility impairments.

The objective of the face-to-face interviews was to gather first-hand accounts and observations of the

activities and events inside the WTC buildings on the morning of September 1 1 . The qualitative nature of

the face-to-face interviews identified heretofore unknown information, evaluated technical hypotheses,

and explored conscious and subconscious motivations for occupant and responder behaviors, while

allowing for qualitative comparisons to the telephone interview data. The face-to-face interview

methodology was a synthesis of the Behavioral Sequence Interview Technique originally developed by

Keating and Loftus, and the Cognitive Interviewing Methods, originally developed by Fisher and

Gieselman.

The following special groups of building occupants were targeted in the face-to-face interviews. NIST

and the expert panel selected these special groups for the face-to-face interviews because these groups were believed to hold unique pieces of information to add to the investigation. They either belong to special subpopulation groups or were in locations during the evacuation that enabled them to experience

distinctive situations. Because these respondents were expected to represent relatively rare types of

occupants or other targeted groups, a snowball sampling approach was used to locate respondents.

Snowball sampling relies on referrals from initial subjects to generate additional subjects. The selection

of this approach, a non-probability design, eliminated the potential for generalizing to the population of

inference (i.e., occupants of WTC towers on September 11), but the design provided access to rare, hardto-find persons with unique information to provide to the investigation. The special groups targeted in the

face-to-face interviews were: (a) people near the floors of impact; (b) people in the lobby who witnessed

fireballs; (c) families of victims who talked to victim during disaster; (d) Occupants from WTC 7; (e)

XXIV NIST NCSTAR 1-7B, WTC Investigation
Executive Summary

persons with building responsibilities; (f) people who used or were trapped in elevators; (g) people with

mobility challenges; and (h) random evaeuees in WTC 1 and WTC 2.

The face-to-face interview instrument was drafted as a computer-assisted personal interview (CAPl)

application that would operate on Pen Tablet PCs. With CAPI technology, high quality qualitative data were electronically coded, organized, and stored directly in an electronic fonnat, with all relevant error cheeking performed automatically. Not only was this application more appropriate for the prescribed data

collection protocol, this process simultaneously eliminated unnecessary data entry costs and associated

key entry errors that typically occur in paper-based field surveys. Both the Pen Tablet PCs and the

application were provided by GeoStats.

The goal of the focus groups was to elicit group representations of specific events or themes. Distinct

categories of persons were provided by NIST based on preliminary analyses of CATI survey and face-toface interview data. These groups were prioritized based on expectations of sample availability and

information value.

Six focus groups were completed. The groups were:

1 . Random survivors from WTC 1

;

2. Random survivors from WTC 2;

3. Mobihty challenged individuals;

4. Persons having building responsibility;

5. Persons near the floors of impact; and

6. Floor wardens.

The venues for the focus groups were changed from Manhattan to outside of Manhattan depending on the

preferences of focus group respondents. Among respondents who were willing to participate in the focus

group, several no longer traveled into Manhattan. Of the persons who participated in the groups, all were

active in the discussion and willing to share their experiences, thoughts, and recommendations.

NuStats was responsible for delivery of databases containing information resulting from the interviews

with survivors. These databases comprised various formats, specific to the type of interview conducted.

In addition to delivering databases, NuStats also provided database support services. These services

included locating a subcontractor to provide qualitative analysis support for the faee-to-face interviews

and providing statistical expertise to support model development using the CATI data.

NIST NCSTAR 1-7B, WTC Investigation XXV
Executive Summary

This page intentionally left blank.

XXVI NISTNCSTAR 1-7B, WTC Investigation
Chapter 1 Introduction

Survey administration involved interviews with survivors of the World Trade Center (WTC) attacks using

three methods, telephone interviews, face-to-face interviews, and focus groups, and the delivery of the

resultant data. This report documents the methods and outcomes of these interviews. (Appendix A

contains the work program that guided the work conducted herein.') Analyses of the resultant data were

conducted by National Institute of Standards and Technology (NIST) investigators. The interviews were

carried out by a data collection team consisting of NuStats Partners, LP, Austin, Texas, as the prime

contractor, with the assistance of subcontractors—^NuStats DataSource, San Marcos, Texas, to conduct

telephone interviews; GeoStats, Atlanta, Georgia, to provide data collection devices and programming

services for the face-to-face interviews; and Diversity Services, New York City, to recruit surveyors and

provide office space.

1.1 OBJECTIVE

The goals of the NIST WTC investigation were to investigate the building construction, the materials

used, and the technical conditions that contributed to the outcome of the WTC disaster. The results of the

investigation were meant to serve as the basis for improvements in the way buildings are designed,

constructed, maintained, and used; improved tools, guidance for industry and safety officials; revisions to codes, standards, and practices; and improved public safety. While multiple sources and types of data were sought to meet the needs of the NIST investigation, the scope of services under this contract provided important first-hand accounts of events and experiences on September 1 1

, 200 1 , through

telephone interviews, face-to-face interviews, and focus groups. This data collection effort evaluated the

role of occupant behavior and evacuation technologies and practices for tall buildings, including decisionmaking and situation awareness, time-constrained evacuation strategies, communications, role of floor

wardens and fire safety directors, and issues concerning people with disabilities. Additionally,

observations of fire and smoke conditions and/or structural damage from within the building were sought.

Families of the victims, or others who communicated with loves ones inside the towers before collapse, were also interviewed primarily to determine the nature of the environment above the floors of impact.

1 .2 PROTECTION OF HUMAN SUBJECTS

This research was conducted in comphance with NIST requirements regarding the protection of human

subjects. The Office of Human Research Protections (OHRP) in the Department of Health and Human

Services (DHHS) registered Federalwide Assurance (FWA) numbers for both NuStats Partners, LP

(FWA00000562) and NuStats DataSource, LP (FWA00005084). For both of these FWA entities, the

Institutional Review Board (IRB) designated for review was Essex IRB, Lebanon, New Jersey. Essex

' Whereas the work program provided in Appendix A refers to interviews with first responders, subsequent agreements between NIST and the City of New York led to NIST personnel conducting these interviews. This level of effort by the contractor was

replaced by the provision of other services.

NISTNCSTAR 1-7B, WTC Investigation 1
Chapter 1

IRB is registered with OHRP and is in good standing with OHRP. Essex's IRB identification number is IRB00001742.

NuStats prepared a comprehensive IRB package for all data collection protocols, and submitted the

package to Essex IRB in September 2003. Essex approved this package on September 26. In October 2003, NuStats revised the IRB package based on NIST legal review, and then NuStats submitted

the revised IRB package to Essex IRB for review. On November 10, Essex approved this revised IRB

package. Based on subsequent issues raised by NIST legal, NuStats revised the IRB package once again,

and submitted this revised package to Essex IRB for review. On November 17, Essex approved this revised IRB package.

An important partner in the provision of human subjects protection was Jamie Abelson, a senior research

associate affiliated with the University of Michigan Institute for Social Research. Based on her vast experience in dealing with sensitive interviewing situations, she provided hands-on mental health support,

as needed, to both surveyors and respondents.

The Paperwork Reduction Act of 1995, effective October 1, 1995, requires that Office of Management

and Budget (0MB) approve each collection of information by a Federal agency before it can be

implemented. NIST prepared a package of materials for the telephone data collection. NuStats reviewed

these 0MB materials to ensure consistency with survey and sample design parameters and with 0MB

reporting requirements. In September 2003, NIST was granted 0MB approval number 0693-0044.

At NIST's request, NuStats set up a toll-free (1-800) number with a staff of telephone operators. This line was used to capture requests for information about the NIST investigation, as well as to screen potential

respondents for the telephone, face-to-face, or focus group interviews. A screener interview was

conducted with hotline callers to cull those who fit the necessary profile for unique groups to be

interviewed, primarily in the face-to-face interviewing effort. Summaries of hotline contacts by day were

prepared for NIST review.

1.3 PAPERWORK REDUCTION ACT

1.4 TOLL-FREE HOTLINE

2 NISTNCSTAR 1-7B, WTC Investigation
Chapter 2

Summary of Data Collection Methods

Three data collection methodologies were employed for this investigation, as presented in the bullets below.

• Telephone Survey: a telephone interview survey of 800 occupants of the World Trade

Center (WTC) towers; a scientific probability sample of surviving occupants was used to identify survey participants.

• In-Depth Personal Interviews . a qualitative study of approximately 220 face-to-face

interviews of occupants and family members of victims; a snowball quota sample of key

survivor types and the toll-free hotline were used to identify potential participants.

• Focus Groups: five focus groups of occupants were conducted; this method also relied on a snowball quota sampling methodology for recruiting key survivor participants.

Prior to the NuStats contract. National Institute of Standards and Technology (NIST) staff and contractor experts selected this multi-method approach for several reasons." First, multiple methods would increase

confidence in the conclusions and findings when more than one method arrived at the same conclusions, a process known as triangulation. Second, the multiple objectives of the investigation mandated

complementary approaches to accomplish all the goals. In other words with only one method, it would be

difficult to establish a scientific foundation for general findings while also broadly investigating and

establishing new facts and discovering unique events. Finally, concerns associated with the time latency

since September 11, 2001, suggested the use of different approaches and techniques in order to increase

memory recall and accuracy.^ The original methodology paper is included in Appendix B.

2.1 TELEPHONE SURVEY"*

The primary goal of the telephone interview was to provide quantitative information on occupant

behavior during the evacuation experience and capture unique investigative observations, particular to the events at the WTC on September 11, 2001 . This data collection activity also sought to estimate the

' The NIST project staff primarily involved in this effort included Jason Averill, Erica Kuligowski, Randy Lawson, Richard

Peacock, and Paul Reneke. Contractor experts included Dr. Norman Groner, John Jay College of Criminal Justice in New

York City, Dr. Dennis Mileti, Director of the National Hazards Research and Applications Information Center within the

Institute of Behavioral Science at the University of Colorado at Boulder, and Dr. Guylene Proulx, Research Officer from the

Institute for Research in Construction at the National Research Council of Canada.

^ From "Data Collection Methodology for World Trade Center Evaluation and Emergency Response: Telephone Interviews, Face-to-Face Interviews, Focus Groups and Population Sampling." Jason Averill, Erica Kuligowski. Randy Lawson, Richard

Peacock, and Paul Reneke, National Institute of Standards and Technology, Gaithersburg, MD and Norman Groner, Dennis

Mileti, and Guylene Proulx. The information in this section relies on the content of NIST NCSTAR 1-7, prepared by NIST from information provided by

NuStats.

NIST NCSTAR 1-7B, WTC Investigation 3
Chapter 2

occupant population in the WTC at the time of the attacks on September 1 1 . The mode of data collection was computer-assisted telephone interviewing (CATI).

The contract required the collection of 800 CATI interviews with persons occupying either of the two

WTC towers (WTC 1, WTC 2) at the time of the terrorist attacks on September 11, 2001. Attempts were made to equally divide the respondents between WTC 1 and WTC 2 occupants (i.e., n = 400 occupant

intei'views from each tower). A proportionate stratified sample design drew samples of survivors within

each of the WTC buildings. This maximized the precision associated with contrasts between the two

building-specific groups of occupants, and provided a nominal level of precision for statistically

analyzing the characteristics of survivors from each building. Robert Santos, NuStats, developed this sampling approach and the design parameters for it.

2.1.1 Sampling Approach

The sampling frame (i.e., the list from which the sample was drawn) consisted of the names of occupants

from badge lists of WTC 1 and WTC 2. All occupants of the WTC were required to provide personal

data in support of issuing badges to clear through the security station at the entrance of each tower. The

badge lists were provided to NIST by the Port Authority of New York /New Jersey. For sampling

purposes, the lists provided name, floor of occupancy, employer, and social security number.

Unfortunately, contact information was not provided, which added unanticipated complexity to the CATI

survey task. In addition, the badge list contains September 1 1 occupants, occupants who were absent on

the day of the attacks, decedents, and fonner occupants.

As mentioned, a proportionate stratified sample design was developed for this survey effort. The sample was stratified on two variables: (1) Building Category (WTC 1 vs. WTC 2); and (2) Floor of Occupancy

(tliree groupings). This resulted in six pvimaiy strata as defined below:

• WTC 1 - floors 0-43

• WTC 1 - floors 44-76

• WTC 1 - floors 77 and above

• WTC 2 - floors 0-43

• WTC 2 - floors 44-76

• WTC 2 - floors 77 and above.

Secondary stratification involved two categories of Tenant Square Footage of a Floor: (1) occupants

employed by tenants occupying up to 40 percent of a floor's square footage, and (2) occupants employed

by tenants occupying more than 40 percent of a floor's square footage.

Given that the sampling frame contained both eligible and ineligible respondents, an intensive screening

effort was needed to identify "eligible" badge list members - namely, those who were inside WTC 1 or WTC 2 during the attacks. Moreover, the absence of telephone numbers for the badge holders on the list

4 NISTNCSTAR 1-7B, WTC Investigation
Summary of Data Collection Outcomes

necessitated a tracking/locating effort. The primary tracking mechanism was to search public databases

using the LexisNexis program." This necessitated a large sample to generate the 800 desired interviews.

The number of occupant selections drawn into the sample was contingent on four key design parameters:

• the percentage of individuals from badge listings for whom a working telephone number can be found (Initial estimate: 80 percent tracking success)

• the percentage of badge listings that correspond to a surviving WTC 1 , WTC 2 occupant on September 1 1 (Initial estimate: 14 percent)

• the cooperation rate for screening the occupants (Initial estimate: 65 percent)

• the interview response rate among 9/1 1 survivors (Initial estimate: 50 percent).

Given these design parameters, a total sample of approximately 22,735 persons from the badge list needed to be tracked and located to generate the desire 800 completed interviews. A reserve sample of

20 percent (or about n=4,550) was added in the event additional sample was needed due to unanticipated

circumstance (e.g., the eligibility rate is lower than anticipated). The reserve was held "in reserve" while

the main sample was worked. Working the main sample allowed preliminary estimates of all design

parameters to be monitored so that an infomied decision could be made on the necessity of releasing none, some or all of the reserve. Equal samples were drawn from the collections of badge holders in WTC 1 and WTC 2. That is, half of the 22,735 selections were allocated to the WTC 1 sample, and the

other half went to the WTC 2 sample. The reserve samples were similarly drawn.

2.1.2 Instrument Development

NuStats developed the telephone instrument in close association with NIST project staff and the expert

panel members. A one-day meeting of these individuals was held at NIST in July 2003 to review an

instrument draft. Subsequent to this meeting, NuStats prepared a conceptual map of the instmment to

prioritize measurement areas. Subsequent in-person and teleconference meetings relied on this conceptual map to refine the measurement areas and finalize the draft instrument. The draft instrument was provided to Dr. Jon Krosnick, a cognitive psychologist and nationally recognized expert in questionnaire design, for review in terms of controlling for non-response and measurement errors and to Jamie Abelson for review for human subjects and surveyor training implications.^ After their review and

subsequent final reviews by NIST project staff and the expert panel, the CATI instrument was finalized.

The interview was designed to measure five primary content areas, presented to the respondent in the

following order:

• Preparedness and Training - These questions collected information on the respondents'

degree of evacuation or emergency training pertinent to their September experience. The

^ LexisNexis is a web-based search product that enables subscribers to search nationwide databases to locate information on

individuals, verify addresses, and find telephone numbers.

* Dr. Jon Krosnick, Ohio State University Departments of Psychology and Political Science; Jamie Abelson, University of Michigan Institute for Social Research.

NISTNCSTAR 1-7B, WTC investigation 5
Chapter 2

questions also addressed familiarity with the WTC buildings based on length of employment

at the location and also whether the respondents were at the WTC in 1993 during that prior

bombing incident.

• Initial September 1 1 Experience - These questions captured data on respondents' initial awareness that something had happened to the building - where they were, what they were

doing, how they first became aware, sources of information about the event (both in terms of

other persons and also in terms of their physical environment).

• Interim September 1 1 Experience - These questions addressed the span between the time

respondents first became aware of the WTC incident and when they began evacuating - data

items included their social environment, physical enviroimient, information sources, and

factors influencing their decision to evacuate.

• Evacuation Experience on September 1 1 - These questions captured information on the

respondents' egress from the building, including the factors that facilitated their egress and

those that hindered their egress.

• Respondent Characteristics - Infomiation collected included age, gender, language, and

mobility impairments.

A pilot test of the CATI instrument was conducted in November 2003, directed by Heather Contrino,

NuStats. Eleven interviews were completed. In the pilot, all aspects of the data collection plan were

evaluated, including, instrument flow, respondent comprehension, interviewer administration,

identification of questions needing probes or clarifications, data coding and processing, and review of

survey data. Only very minor modifications to the instrument were required based on pilot outcomes.

2.1.3 CATI Interviewing

Highly experienced interviewers at DataSource conducted the telephone interviews using a computer

program which provided questions and answer categories. Prior to calling, subjects received an advance

letter that outlined the scope and purpose of the investigation, the purpose of the interview, and the

telephone call that came several days later. A full informed consent statement appeared in the letter, as

well. When interviewers reached the respondents by telephone, respondents were provided a description

of the survey, the confidentiality of responses, the length of the interview, and the voluntary nature of

participation. Respondents were then asked if they wished to participate, thereby obtaining oral informed

consent.

Interviewer training was held on December 1 and 2, 2003, and actual dialing of the sample began on December 3. Interviewing continued until January 23, 2004, with a short hiatus during the December

holidays. The actual productivity of the CATI interviewing was lower than initial design parameters.

Interviewing productivity rates are provided in Chapter 3 of this report. In December 2003, prenotification letters were inadvertently mailed to approximately 47 decedent persons. This event occurred

because the badge list was not properly flagged for decedents. Once the error surfaced, the database of

respondents (i.e., the badge list) was manually matched and checked against known decedents to ensure

that all potential decedents in the sample were flagged and removed from the active sample. Letters of

6 NISTNCSTAR 1-7B, WTC Investigation
Summary of Data Collection Outcomes

apology were mailed to decedent households. In total, 803 CAT! interviews were complelcd. Details on

the outcomes of the CATI interviewing are presented in Chapter 3.

2.2 FACE-TO-FACE INTERVIEWS

The objective of the face-to-face interviews was to gather first-hand accounts and observations of the

activities and events inside the WTC buildings on the morning of September 1 1 . The qualitative nature of

the face-to-face interviews identified heretofore unknown information, evaluated technical hypotheses,

and explored conscious and subconscious motivations for occupant and responder behaviors, while

allowing for qualitative comparisons to the telephone interview data.

The contract required that the face-to-face interviews be conducted via a relative new methodology, that was a synthesis of the Behavioral Sequence Interview Technique (BSIT) originally developed by Keating

and Loftus,' and the Cognitive Interviewing Methods (CIM), originally developed by Fisher^ and

Gieselman.^ Both approaches begin by allowing the respondent to retell an unimpeded account without

interruption from the interviewer, and both initially employ a chronological retelling of infonnation. The

BSIT was designed to yield a database of qualitative infonnation that could be subjected to systematic

analysis and consolidation, while CIM was designed to facilitate investigative interviews. Since the WTC

investigation was pursuing both goals (i.e., creation of a database of evacuation-related behaviors and an

investigatory attempt to capture information relevant to outcomes), the face-to-face methodology was

designed to combine these two approaches.

2.2.1 Sampling Approach

The following special groups of building occupants were targeted in the face-to-face interviews. NIST

and the expert panel selected these special groups for the face-to-face interviews because these groups were beheved to hold unique pieces of information to add to the investigation. They either belong to special subpopulation groups or were in locations during the evacuation that enabled them to experience

distinctive situations. Because these respondents were expected to represent relatively rare types of

occupants or other targeted groups, a snowball sampling approach was used to locate respondents.

Snowball sampling relies on referrals from initial subjects to generate additional subjects. In the case,

referrals came from NIST public announcements (at press conferences in New York City and on the NIST

web site), from CATI interview respondents, and from associated parties (i.e., NIST, Port Authority of

New York/ New Jersey). The selection of this approach, a non-probabihty design, eliminated the

potential for generalizing to the population of inference (i.e., occupants of WTC towers on September 1 1) but the design provided access to rare, hard-to-fmd persons with unique information to provide to the

investigation. The special groups targeted in the face-to-face interviews are described as follows.

^ Keating, F.P.; Loftus, E. L. Post Fire Interviews. 1984. Development and Field Validation ofBeJiavioral Sequence Inten'iew Technique. Final Report. NBS GCR84-477. Washington, DC: National Bureau of Standards.

^ Fisher, R.P., Falkner, K.L., Trevisan, M., McCauley, M.R. 2000. Adapting the cognitive interview to enhance (35 years) recall ofphysical activities. Journal of Applied Psychology, 85(2), 1 80-1 89.

^ Gieselman, R.E., Fisher, R., Mackinnon, D., Hallond, H. 1986. Enhancement ofeyewitness memoiy with the cognitive

interview. American Journal of Psychology, 99, 385-401

.

NIST NCSTAR 1-7B, WTC Investigation 7
Chapter 2

Occupants Subgroup A: People near the Floors of Impact

• 50 people from floors 90 - 94 in WTC 1 at the time of WTC 1 impact.

• 50 people from floors 75 - 110 in WTC 2 at the time of WTC 2 impact.

Rationale for inclusion: These people witnessed fire, smoke, structural/ system damage, heard

PA aimouncements, and walked the greatest distance to evacuate the buildings.

Occupants Subgroup B: People in the Lobby who Witnessed Fireballs

• 10 people who witnessed fireballs.

Rationale for inclusion: Estimating the size and source of the fireball to identify the path the fuel traveled, how much fuel was consumed, and how much damage to the lobby resulted.

Occupants Subgroup C: Families of Victims who Talked to Victim during Disaster.

• 30 family members or others.

Rationale for inclusion: These people who communicated with victims may be the sole source of

"first-hand" information from above the floors of impact.

Occupants Subgroup D: People in WTC Building 7

• 1 5 people who evacuated

• 5 people with some level of decision-making authority over the evacuation.

Rationale for inclusion: Determine the total evacuation time for Building 7, total number of

people evacuated, density of people in the stairwells, and evaluate when fires may have started.

Occupants Subgroup E: Persons with Building Responsibility

• 50 floor wardens, fire safety directors, or building management staff.

Rationale for inclusion: Involved in decision-making process on September 1 1 about who and

how to evacuate the buildings. Know who had what information, when they had it, and what they

did with it.

Occupants Subgroup F: People w ho Used or were Trapped in Elevators

• 25 people trapped in elevators or used elevators after their building was hit.

Rationale for inclusion: Estimating the number of people who may have been killed inside

elevators and what the failure mechanism of the elevator system was.

8 NISTNCSTAR 1-7B, WTC Investigation
Summary of Data Collection Outcomes

Occupants Subgroup G: People with Disabilities

• 20 people who experienced a mobility impairment prior to, or during, their evacuation -

anything from a disability under the Americans with Disabilities Act (ADA) to pregnancy,

overweight, age, etc.

Rationale for inclusion: Document the challenges faced by people with disabilities in order to better address those needs in future building incidents as well as identify pro-social behaviors,

rescue, or devices that contributed to their ultimate evacuation success.

Occupants Subgroup H: Random Evacuee

• 25 people from each of the lower and middle thirds of WTC 1

• 25 people from each of the lower and middle thirds of WTC 2 (that is, not from the floors near the impact). 4 zones x 25 people = 100 people.

Rationale for inclusion: Document typical evacuation experience. Ensure unanticipated events

or experiences were captured.

2.2.2 Instrument Development

NuStats developed the face-to-face instrument in close association with NIST project staff and the NIST

expert panel members. However, after this draft was prepared it became apparent that the prescribed data

collection protocol for the face-to-face interviews would determine the instrument structure and content. Thus, the instrument was drafted as a computer-assisted personal interview (CAPI) application that would

operate on Pen Tablet PCs rather than as a paper document (see Appendix A, Section A. 3. 2.1). With

CAPI technology, high quality qualitative data were electronically coded, organized, and stored directly

in an electronic format, with all relevant error checking performed automatically. Not only was this application more appropriate for the prescribed data collection protocol, this process simultaneously

ehminated unnecessary data entry costs and associated key entry errors that typically occur in paper-based

field surveys. Both the Pen Tablet PCs and the application were provided by GeoStats.

The CAPI application captured information from participants in four (4) steps.

Step 1: Uninterrupted Narrative Account

The participants were asked to relate their experience on September 1 1 from the time that they first knew

something was wrong to the time when they exited the building. Researchers and practitioners involved

with cognitive interviewing believe that starting the face-to-face interviews in this manner both improves

recall and helps build rapport between the participant and the interviewer. The surveyors were trained to

listen intently and to take notes during this stage.

Step 2: Collaborative, Structured Action-Cue-Goal Account

After the participant completed their story, the respondent was asked to re-visit their experience.

Interviewers worked cooperatively with participants to record entries into a CAPI table (see Table 2-1).

NISTNCSTAR 1-7B, WTC Investigation 9
Chapter 2

During this collaboration, interviewers recorded the infonnation, guided the process, and asked questions

to clarify and define the content. In the structured table, each row of the table was a triad comprised of an

action, a cue, and a goal.

Table 2-1. Example tabular face-to-face interview data entry.

Action Cue Goal

1 stumbled in the dark towards

where I thought the voice came

I heard someone yell "I've found

a clear path"

So that I could find a way to escape So 1 called out to whoever yelled, "I'm near the reception area. Where are you?"

My path was blocked by debris To try to get a better idea of

where the person was

• Action. An action was something that the participant did as a reaction to a cue. Actions

should be specific enough for a reader to visualize.

• Cue. A cue was something that starts an action. Cues can be external (something one sees, hears, smells, feels) or something internal (a decision, a memory ).

• Goal. Goals were what the participant was trying to accomplish by taking the action.

Step 3: Review and Edit Account of September 11 Experience

After the table was completed collaboratively, the interviewer asked the participant to review what had

been recorded to ensure that it accurately captured the participant's experience.

Step 4: Topic Cards

After completing the negotiated, structured narrative account, the interviewer showed the participant

"Topics" cards and asked the participant to identify those topics that were relevant to his/her experience.

The topics were intended to capture specific information of particular value to the investigation. If a topic was relevant, the interviewer recorded the open-ended statements of the participants in text boxes that were associated with the relevant A-C-G row in the structured table. The interviewing application was programmed to enable this to be easily done.

On September 9 and 10, 2003 in New York City, NuStats interviewed three WTC building occupants

with the purpose of beta testing the face-to-face CAPI application. These beta test interviews were

conducted by Dr. Johanna Zmud and Ms. Delia Santos. Results of the beta test were compiled and

delivered to NIST. The three individuals were paid as consultants to NuStats because IRB approval had

not been obtained at the time. The interviews ranged from 1 to 3 hours. Respondents reacted quite

positively to the "cue", "action", and "reason" (C-A-R) protocol. The act of being interviewed on their

September 1 1 experiences was an emotionally trying activity for the respondents. Respondents arrived at the interview site emotionally stressed. Their unimpeded narratives were spoken with emotion and

sometimes their eyes teared. The C-A-R protocol, however, seemed to objectify the experience for them.

Thus, the negotiated description of their experience was done with much less emotion than the unimpeded

narrative and greater detail resulted. Generally, the unimpeded narratives took approximately one-quarter

the time of the negotiated C-A-R activity. The respondents appeared genuinely to be engaged in working

with the interviewer to complete the C-A-R table. Respondents also took ownership of the content of the

C-A-R table. The beta test identified some mechanical problems with the application and also some

10 NISTNCSTAR 1-7B, WTC Investigation
Summary of Data Collection Outcomes

conceptual issues pertaining to clarifying C-A-R. Revisions to the application and to the conceptual

underpinnings of the CAP! application were made.

A pilot test of the revised CAPI application was conducted on November 5 and 6 in NYC. NuStats

NY-based coordinator (Ms. Maria Elena Ramos) recruited five respondent consultants for these pilot

interviews. The interviews were conducted by Dr. Johanna Zmud and Delia Santos. The pilot revealed

that the modifications to the application enhanced its user-interface.

2.2.3 Face-to-Face Interviews with Occupants

The face-to-face interviews were conducted by surveyors recruited by a subcontractor, Diversity Services

in New York City. Surveyors were sought who had some social work, interviewing, or client-interaction

experience. Training of the surveyors was held over a 3-day period in early November 2003 in New York

City. Surveyors were required to attend all three days of training. Surveyors were responsible for setting

up the interview appointments, completing the interviews, and uploading interview files to the secure NuStats file transfer protocol (FTP) site for data editing and processing.

Through Diversity Services, NuStats trained 12 surveyors of which ten continued interviewing through

mid- February 2004. A skeleton crew of five surveyors continued interviewing through the end of

April 2004 for completing face-to-face interviews with really hard-to-find special groups, to recruit focus

group participants, and to conduct coding of 9-1-1 tapes and transcripts of interviews with first responders

as a special assignment from NIST.

Respondents for the face-to-face interviews were identified through three avenues: (1) NIST, (2) Hotline,

(3) CATI survey. From these sources, persons were identified with whom a DataSource interviewer

would conduct a telephone screener interview. Respondents who successfully completed a screener interview were coded based on their affiliation with one or more of the target groups and placed into the

sampling frame. The day-to-day management of this sample was critical to completing interviews with

this hard-to-reach population. The sample file was housed on the NuStats' FTP site. The NYC-based

coordinator downloaded sample information daily and distributed the new sample to surveyors. Surveyors scheduled interviews and emailed the information back to the coordinator, who then posted it on the NuStats' FTP site so that the information on potential and completed interviews was kept in real- time. The information on completed, missed, or rescheduled interviews was tracked at NuStats and a

report was prepared three-times per week for NuStats and NIST project staff.

Face-to-face interviewing began in mid-November. Each surveyor was monitored in a "live" interview

prior to being allowed to operate independently. A group debrief of surveyors was held in late November

to trouble-shoot any challenges that had arisen in the first few interviews and also to ensure that all surveyors understood the interviewing requirements and were successfully carrying these out. The face- to-face interviews began slowly due to lack of sample. Most of the referrals for these interviews were to come from the CATI survey and that did not start until early December 2003. During this interim period,

interviews were conducted with the trickle of respondents who entered the frame through the Hotline.

Interviewing continued through the end of April to maximize the ability to locate and find potential faceto-face interview respondents.

NIST NCSTAR 1-7B, WTC Investigation II
Chapter 2

In all, NuStats had access to 744 potential face-to-face interview respondents, the majority of these

coming from the screening of respondents in the CATI survey. The face-to-face sample pool included:

• 456 cases from the CATI,'"

• 140 cases from the published accounts database,"

• 106 cases from the Port Authority's badge list for WTC 7, and

• 42 referrals from a variety of sources including NIST.

All of these cases, other than the cases from the CATI, required intensive tracking using LexisNexis to locate contact infonnation. Approximately 70 percent of persons in the sample pool were located and

contacted. One of the anticipated challenges in this interviewing effort was the "no-show" rate for

interview appointments, given the sensitive nature of the interviews. During December and January,

NuStats experienced a no-show rate of 1 in 4 (25 percent). However, starting in February the no-show

rate continued to increase to more than 1 in 3 (36 percent). This was attributed to a shrinking pool of

available participants; most of them being persons who previously cancelled or were no-shows. As

people were successfully interviewed, the more reluctant subjects were the ones who remained to be

interviewed; thus, increasing the likelihood of a "no-show" appointment. In March and April,

interviewers were trying to interview the "hardest-to-get" cases.

Interviewing was done at various locations, depending on the preference of the respondent. The goal was

to maximize participation in the face-to-face interviews by making the interviewing situation as comfortable and convenient as possible for the potential respondents. Interviewing locations included the

Diversity Services offices in midtown Manhattan, NuStats' procured space in downtown Manhattan, as well as the work or residential locations of respondents. About 20 percent of interviews were conducted

at off-site locations. Surveyors were given personal Pen Tablet PC's to accommodate interviewing at these multiple locations. The average length of the face-to-face interviews was 3 hours.

2.2.4 Face-to-Face Interviews with Family Members

The conduct of the face-to-face interviews with family members was particularly delicate to implement

because of the emotional duress that many of the individuals still felt. The names and contact information

(for some not all) of 16 potential respondents were provided to NuStats by NIST. NuStats mailed prenotification letters to those cases where the LexisNexis search provided address of known names of

family members of victims. These cases were followed up by a telephone call to the person. In all but one case, the 8 completed interviews with the family members of victims were conducted by telephone by

Delia Santos, NuStats. One interview was conducted in-person by Dr. Johanna Zmud, NuStats.

These were persons who met the screening criteria and agreed to be interviewed again using the face-to-face protocol. " The published accounts database was developed by Dr. Rita Fahy and Dr. Guylene Proulx, and published as NIST

NCSTAR 1-7A.

12 NIST NCSTAR 1-7B, WTC Investigation
Summary of Data Collection Outcomes

2.3 FOCUS GROUP INTERVIEWS

The goal of the focus groups was to elicit group representations of specific events or themes. Distinct

categories of persons were provided by NIST based on preliminary analyses of CATI survey and face-to- face interview data. These groups were prioritized based on expectations of sample availability and

information value as presented below. The goal was to recruit five to six persons for each group.

Priority Group 1

1. Mobihty Challenged - WTC 1 only. At least half of the participants were to be wheelchair bound

or use some type of device to aid their mobility. Half could have less visible mobility challenges,

such as asthma. Questioning priorities were to (1) have them tell their stories, (2) talk about the building design or evacuation procedure policies that aided their evacuation or (3) hindered their evacuation, and (4) what would they like to see in terms of building design or evacuation

procedures policies in the future. These persons would be recruited from the face-to-face

interview sample.

2. Persons near the floor of impact in WTC 1 (floors 88 - 94). This group provided explanatory

power to the causal models on evacuation behavior. Dependent variables in the models are: (1) length of time between their decision to evacuate and their actual evacuation time, and

(2) time out of the building once they started their evacuation. NIST provided variables that were

identified as independent variables in the model so that these could be probed during the group

discussion. These persons would be recruited from the face-to-face interview sample.

3. WTC 1 respondents who started evacuating 2 standard deviations after the mean evacuation time.

The "glue" of this group was to examine why some people took so long to start evacuating.

These persons would be recruited from the CATI sample.

Priority Group 2

1. 12 Persons from floor 78 (above impact) in WTC 2 who survived the impact. It would be

important to interview these persons as face-to-face interviews first to know what they did to survive. A Lexis Nexus search was performed in order to locate these individuals.

2. Group who stayed behind in WTC I, floor 68 for 20-45 minutes after impact. Information

priority was to find out why they stayed behind and did not start evacuating right away. These

persons would be recruited from CATI or face-to-face completed interviews.

3. Floor wardens in either WTC 1 or WTC 2. These were persons who "tried" to perform their

warden duties. Information priorities were to probe on why people might have ignored them,

what they felt might have been a better evacuation procedures than the one that they employed, if they thought the evacuation procedures / policies they followed were sound and should be

guidance for future events. These persons would be recruited from CATI or face-to-face

completed interviews.

Priority Group 3

1 . People redirected in stairs in WTC I . These were persons who were evacuating down the stairs and were redirected to another stairwell. The information value would be in knowing who did the

redirecting and why. These persons would be recruited from face-to-face interview sample.

2. People who were evacuating down the stairwells and were stopped on the sky lobby on floor 78

in WTC 2 before the plane hit and told to return to their floors. Questions would focus on who

NISTNCSTAR 1-7B, WTC Investigation 13
Chapter 2

did the redirecting; why; and people's reactions to being redirected. These persons would be

recruited from the face-to-face interviews.

3. Building support staff. These were elevator operators, maintenance persons, etc., to get their assessment of the building design features that aided or hindered successful evacuation. They

would be recruited through snowball sampling, outside of the CATI or face-to-face interview

sample.

4. Evacuation helpers. These were persons in either WTC 1 or 2 who helped an injured person, a mobility challenged person, or anyone else during their evacuation to get the helper perspective on what building design or evacuation procedure policies helped or hindered their efforts. They

would be recruited from face-to-face interview sample.

5. Persons who observed the fireball in lobby of WTC 1 . This was a low priority because of the

lack of sample to constitute a group.

2.3.1 Focus Group Recruitment and Interview Protocol

After prioritizing and analyzing the availability of sample with which to recruit focus group participants,

it was decided to conduct six focus groups: (1 ) WTC 1 occupants on March 9, 2004, (2) WTC 2 occupants on April 1, 2004, (3) mobility challenged persons on April 3, 2004, (4) persons with building

responsibility on April 27, 2004, (5) persons near the floors of impact on April 28, 2004, and (6) persons

with floor warden responsibilities on May 12, 2004. Even though these groups were vetted for inclusion

based on the availability of sample, it was still a challenge to locate persons who were willing and able to

attend. The original plan was to conduct all the groups during one week in New York City. But due to the challenge in recruiting participants, the groups were spaced out as noted above. In most cases, participants had already completed either a CATI interview or a face-to-face interview. Recruitment of

focus group participants was conducted by the face-to-face interviewers; whenever possible, the

recruitment contact was made by the surveyor who had conducted the face-to-face interview with the

respondent.

The interview protocols for the focus groups followed the same basic structure, but each was customized

for specific infonnation priorities. Dr. Carlos Arce, NuStats, moderated the first group. The remaining

five groups were moderated by Dr. Johanna Zmud, NuStats. Because audio taping or videotaping was not allowed, two note takers were present at each group to record the discussion. Each focus group lasted approximately three hours, and participants were reimbursed for their travel with a stipend of $100.

14 NISTNCSTAR 1-7B, WTC Investigation
Chapter 3

Summary of Data Collection Outcomes

Interviews were completed with surveyors of the WTC attacks via CATI, CAPI, and focus groups. The

information resulting from these interviews provided unique first-hand accounts of experiences in the towers on the morning of September 1 1, 2001. The level of effort required to complete interviews with

survivors of the WTC attacks was greater than anticipated. There were several factors that influenced

NuStats' ability to locate, contact, and complete interviews, including availability of sample and the willingness of survivors to revisit their experiences in an interview format. The sections below document

outcomes of the interviewing efforts and the factors that influenced these outcomes.

3.1 TELEPHONE INTERVIEWS

A total sample of 26,000 was drawn, comprising 13,000 names for each tower. From this sample, CATI

interviews were completed with 803 persons. Of which, 427 persons were in WTC 1 on September 11,

and 376 persons were in WTC 2. Using guidehnes of the Council of American Survey Research

Organizations (CASRO), the estimated response rate was 23 percent. This response rate was driven by an

inability to contact and screen eligible respondents, rather than by refusals. Of those contacted, only

7 percent refused to be interviewed. On the other hand, only 47 percent of sampled persons were

contacted, of which only 1 8 percent were eligible to be interviewed (that is, they were in the towers on the

morning of September 11, 2001).

Table 3-1 summarizes the final disposition of the CATI sample and the total (locating) sample. The table

is comprised of two sets of rows. The top set pertains to the CATI sample and represents those sample

persons for whom an initial telephone number was identified prior to commencing the CATI survey

operations. The bottom set of rows with the heading "Total Sample Disposition" represents the results of

locating/tracking efforts used to identify usable telephone numbers associated with the sample subjects.

(Recall that only name, SSN, and employer were available; no other contact information was readily

available.)

The top set of rows in Table 3-1 presents the final disposition of the sample by tower as well as for the

overall sample. Several statistics in the percentage distribution (rightmost) column are notable. First,

subjects for half the sample (50.5 percent) could not be contacted, due either to failures to answer the

phone, answering machines, unusable numbers (e.g., wrong number, disconnected, business), etc. Most

of these telephone numbers represent "unloadable" subjects - subjects for whom the initial telephone

number was incorrect. It bears reiterating that substantial additional research during CATI operations was

conducted using powerful subscription-based web-based search engines. Unfortunately, little information was available for these individuals.

NISTNCSTAR 1-7B, WTC Investigation 15
Chapter 3

'able 3-1. Disposition of the CATI samp e and the total samp e by tower.

CATI Dispositiont \\/T<~' 1

^ W 1 l„ 1 W i z I OTal % Dints

Interview 07C OU3 4.0 %

Partial Interview A 7 07 Q A o4 0.4 %

9/1 1 decedent on A Ci 4U bU 0.3 %

Other decedent A O jy Q Q

cSo 0.4 %

Not hligiDle o 7 ^ o O 7CO /,4b4 37.5 %

Langu3ge Barrier 1 oo 1 zy OK/1 1 .3 %

Flinihip Rpfiic:prl to Intprvtpw 138 139 277 1 .4 %

Other Refusal 224 181 405 2.0 %

Respondent not Interviewed 247 168 415 010/ Z. 1 /o Can't contact/locate Respondent 4,987 5,076 10,063 OU . D /o CATI TOTAL 9,986 9,937 19,923 100.0 %

Total Sample Disposition WTC 1 WTC 2 Total 7o Dints

Found initial telephone number 9,986 9,937 19,923 76.6 %

Unable to find a telephone

number 3,014 3,063 6,077 23.4 %

SAMPLE TOTAL 13,000 13,000 26,000 100 %

a. Table data is weighted. Tower location as indicated in the badge list and may differ from reported tower location.

The bottom set of rows shows that telephone numbers were identified for just over three quarters

(76.6 percent) of the sainpled subjects. Moreover, this rate was fairly uniform across towers. The

19,923 individuals with an initial telephone number were then loaded into the CATI sample management

system for calling. Ultimately, all reserve samples were used in the telephone survey. In the initial design parameters, it was assumed that 82 percent of the subjects would be locatable. While 76.7 percent

is close, many of the numbers were obsolete (e.g., disconnect, wrong number) and necessitated additional

tracking during CATI operations. UUimately, by the end of data collection, only half the sample

represented confinned contacts with subjects.

A second result of interest is the prevalence of ineligible subjects—those not in the building on the

morning of September 1 1, 2001. A third result is the existence of decedents—some from the

September 1 1 attack and others from causes not necessarily related to September 1 1 (e.g., cause unknown, natural causes). Most of the September 1 1 decedents were encountered due to a difference in the full (formal) name of the subject and the name that appeared on the badge list (e.g., the badge list sometimes contained maiden names, middle names, nicknames, misspelled first or last names, out of

sequence names, titles, and so on). This impeded the ability to remove known decedents prior to calling.

The outcome of CATI operations on the final outcome rates is presented by tower in Table 3-2. The table shows screening rates, interview rates, and rates of eligible occupants (among those who responded to the

screening questions). The first row shows that screening response rates were relatively uniform across towers at about 46 percent. A screening response rate of 65 percent had been planned. Similarly,

interview response rates (among screened eligible subjects) were relatively stable across towers at about

49 percent. This is consistent with the planned interview response rate of 50 percent.

16 NISTNCSTAR 1-7B, WTC Investigation
Summary of Data Collection Outcomes

The eligibility rates were higher than expected—about 18 percent overall compared to the 14 percent

expected. The eligibility rate among WTC 1 subjects was slightly higher than those of WTC 2. However, the overall response rates are essentially uniform across towers, at 22.6 percent.

Table 3-2. Summary disposition rates 3y tower.

Disposition Rate" WTC 1 WTC 2 Total

Screen 46.5 % 45.8 % 46.1 %

Interview 48.6 % 49,5 % 49.0 %

Eligibility 18.9% 16.7% 17.8 %

Overall 22.6 % 22.7 % 22.6 %

a. Definitions for "Rates" consistent with American Association of Public Opinion Research (AAPOR) Standards, which may be found at http://www.aapor.org/pdfs/standarddefs2004.pdf

3.2 FACE-TO-FACE INTERVIEWS

The face-to-face interviews were conducted via non-probability methods. Thus, response rate estimates

cannot be derived. In total, 220 interviews were completed between mid-November and mid-April. The

following is the breakdown of completes by group affiliation:'"

28 = Near floors of impact

2 = Persons in the lobby who saw fireballs

8 = Family of victim

7 = Persons in building 7

33 = Persons with building responsibility

15 = Persons in elevators

13 = Persons with disabilities

• 114 = Persons who evacuated from WTC 1 or 2 not otherwise coded.

Most of the interviews were conducted with persons who had also agreed to complete a CATI interview.

The provision of sample, outside of the CATI interviewees, was less than originally anticipated.

Snowball sampling proved less than effective as survivors were reluctant to provide names of other

persons who may have qualified for a face-to-face interview out of respect for the persons' privacy or mental health. The biggest challenge during the face-to-face interviews was the "no show" rate for scheduled appointments, which ranged from 25 percent at the start of interviewing to 36 percent as interviewing ended. Interviewers averaged 10 interviews per week in December and January. But this productivity dwindled to less than three per week in February, March, and April as available sample and

Some individuals may have had more than one group affiliation, but each individual was coded to a primary group for tracking purposes. A hierarchy was used to do the primary coding based wherein an individual was coded to the hardest-to-locate group.

NISTNCSTAR 1-7B, WTC Investigation 17
Chapter 3

willing respondents diminished. Interviewers also found that they were required to travel to residences or work places outside of Manhattan for these harder-to-interview persons.

3.3 FOCUS GROUP INTERVIEWS

Six focus groups were completed. Planning and recruiting for all focus group participants began in midMarch. Recruiting for some focus groups was more challenging than others. The easiest to recruit groups were with survivors from WTC 1 and WTC 2 on their evacuation experiences. These groups were held on March 9, 2004 and April 1, 2004. Six individuals were recruited for the group of WTC 1 survivors,

and all six showed at the facility. Eight persons were recruited for the group of WTC 2 survivors and

seven of the eight showed.

For the remaining focus groups, it was challenging to track, locate, and recruit individuals. The focus

group with mobility challenged individuals was held on April 3, 2004. Five individuals were recruited

(of which two were wheelchair-bound), and three individuals showed up, of which one was wheelchairbound. Focus groups with persons having building responsibility and with persons near the floors of

impact were held on consecutive days, April 27 and 28, 2004. Eight individuals were recruited for the

focus group of persons with building responsibility, and six individuals participated. Six persons were

recruited for the group of persons near the floors of impact, and four participated. Finally, the floor wardens focus group was the most difficult to recruit. Only four persons were recruited, and two

showed up.

The venues for the focus groups were changed from Manhattan to outside of Manhattan depending on the

preferences of focus group respondents. Among respondents who were willing to participate in the focus

group, several no longer traveled into Manhattan. Of the persons who participated in the groups, all were

active in the discussion and willing to share their experiences, thoughts, and recommendations.

18 NISTNCSTAR 1-7B. WTC Investigation
Chapter 4

Data File Deliveries

NuStats was responsible for delivery of databases containing information resulting from the interviews

with survivors. These databases comprised various formats, specific to the type of interview conducted,

In addition to delivering databases, NuStats also provided database support services. These services

included locating a subcontractor to provide qualitative analysis support for the face-to-face interviews

and providing statistical expertise to support model development using the CATl data.

4.1 TELEPHONE SURVEY INTERVIEWS

The CATI data was delivered in SPSS and DBF formats in late January 2004, after completing processing

and quality review. After delivery of the final data, NuStats participated in a weeklong analysis session

with NIST to finalize the statistical analysis procedures. During this time and subsequently, NuStats

provided data support to NIST for reporting writing and final data review. NuStats also provided

statistical analysis support during the months of February and March for the development of a dataset that was consistent with the causal modeling requirements and the running of necessary statistical programs to support quantitative analysis and causal modeling.

4.2 FACE-TO-FACE INTERVIEWS

NIST selected ATLAS.ti as the software system for qualitative analysis. NuStats provided assistance to NIST in locating and managing a qualitative research expert to support its qualitative analyses.

Subcontractors for this effort were the co-directors of the survey laboratory of the University of Chicago

for the National Opinion Research Center (NORC), Dr. Martha van Haitsma and Dr. Virginia Bartot. These two individuals provide support to NIST in developing the data analysis plan and coding scheme,

as well as in ATLAS.ti software usage. After interview data cleaning and editing, which was done in Excel, the face-to-face interview data was converted to conform to the specifications of the ATLAS.ti

software. NuStats prepared and delivered face-to-face interview data on a continuous basis throughout

the data collection period.

4.3 FOCUS GROUP INTERVIEWS

Focus group interview summaries were delivered to NIST as documents in Microsoft Word. Data

confidentiality constraints prevented audiotaping or videotaping of the focus group sessions. Thus,

summaries were prepared by the focus group facilitator from detailed notes provided by obser\'ers at the groups.

NISTNCSTAR 1-7B, WTC Investigation 19
Chapter 4

This page left intentionally blank.

20 NISTNCSTAR 1-7B, WTC Investigation
Appendix A

Work Plan

This document provides documentation of the NuStats work plan, following a kick-off meeting with

NIST staff and three outside experts on June 9-10, 2003. This work plan updates both the NlST-produced

white paper, "Collection Methodology for Work Trade Center Evacuation and Emergency Response:

Telephone Interviews, Face-to-Face Interviews, Focus Groups and Population Sampling" (included as

part of the original contract solicitation), and the Scope of Work (task-by-task description) that was

included in NuStats proposal.

A.1 BACKGROUND

The goal of the National Institute of Standards and Technology's World Trade Center Investigation is to investigate the building construction, the materials used, and the technical conditions that contributed to the outcome of the World Trade Center (WTC) disaster. The results of the Investigation will serve as the

basis for improvements in the way buildings are designed, constructed, maintained, and used; improved

tools, guidance for industry and safety officials; revisions to codes, standards, and practices; and

improved public safety. The primary objectives of the NIST-led technical investigation of the WTC

disaster are to:

1 . Determine why and how WTC 1 and 2 collapsed following the initial impacts of the aircraft and why and how WTC 7 collapsed;

2. Determine why the injuries and fatalities were so high or low depending on location,

including all technical aspects of fire protection, occupant behavior, evacuation, and

emergency response;

3. Determine what procedures and practices were used in the design, construction, operation,

and maintenance of WTC 1, 2, and 7; and

4. Identify, as specifically as possible, areas in building and fire codes, standards, and practices

that are still in use and warrant revision.

The NIST Investigation Plan can be found at http://wtc.nist.gov, including a description of Projects 7 and 8. Under Project 7, Occupant Behavior, Egress, and Emergency Communications, first-hand

accounts of the events of September 11, 2001, from inside WTC 1, 2, and 7 will be collected. This data

collection effort will evaluate the role of occupant behavior and evacuation technologies and practices for

tall buildings, including decision-making and situation awareness, time-constrained evacuation strategies,

communications, role of floor wardens and fire safety directors, and issues concerning people with

disabilities. Additionally, NIST will seek specific observations of fire and smoke conditions and/or

structural damage from within the building. Families of the victims, who communicated with loved ones

inside the towers before collapse, will be interviewed to determine the nature of the environment above

the floors of impact. The objectives of Project 8, Fire Service Technology and Guidelines, are to build

upon work already done by the Fire Department of New York (FDNY) and McKinsey & Company by:

(1) fully documenting what happened during the response by the fire services to the attacks on the

NISTNCSTAR 1-7B, WTC Investigation 21
Appendix A

World Trade Center, up to the time of collapse of WTC 7; (2) identifying issues that need to be addressed

in changes to practice, standards and codes; (3) identifying alternative practices and/or technologies that may address these issues; and (4) identifying R&D needs that advance the safety of the fire service in responding to massive fires in tall buildings. Thus, a subset of the emergency responders who were

present at the WTC complex will be asked to voluntarily participate in the face-to-face interview or focus

group phases of this project. Only first responders who participated in fire suppression, operational, or search and rescue activities prior to the building collapse will be considered for inclusion in these phases

of the study.

A.2 OVERVIEW OF METHODOLOGICAL APPROACH

The WTC interviews are being conducted to assist the NIST-investigation team in discovering "what

happened" from the time of the first airplane hit until the collapse of the buildings (WTC 1, WTC 2, and

WTC 7). The interviews are focused on behaviors and observations that took place within the buildings

and up to a block radius surrounding the buildings. The interviews are not concerned with events subsequent to the collapse of the buildings or outside of this spatial radius. The data collection objective

is to conduct 800 telephone interviews with occupants of WTC 1 and WTC 2; up to 600 face-to-face

interviews with occupants of WTC 1, WTC 2, and WTC 7, 30 face-to-face interviews with families of

victims, and 150 face-to-face interviews of first responders; and 10 focus group sessions with first responders and five (5) focus group sessions with building occupants. The types of information to be

derived from these data collection activities include:

• Identifying physical and human impedances and facilitators to building evacuation,

• Developing a better fundamental understanding of:

o Egress

o Human behavior

o Emergency communications

o Emergency response procedures

o Building/ response technology.

This multi-methodological approach was selected to uncover the required information for several reasons.

First, multiple methodologies increase confidence in the conclusions and findings when more than one methodology arrives at the same conclusions. Second, the varied objectives of the Investigation mandate

complementary approaches to accomplish all the goals. Finally, concerns associated with the time latency

since September 11, 2001, suggested the use of different approaches and techniques in order to increase

memory recall and accuracy.

^

22 NISTNCSTAR 1-7B, WTC Investigation
VJork Plan

A.3 TASK 1 : INSTRUMENT DEVELOPMENT, PROTOCOLS AND DATABASE

DESIGN

A.3.1 Telephone Interview: Survey Instrument

One of the data collection methodologies is the telephone interview. The collection mechanism utilized in this phase of the study will be a computer-assisted telephone interview. It is expected interview will range from 20 - 30 minutes, with an average length of 25 minutes. The primary goal of the telephone

interview is to provide statistical estimates that will inform both the NIST Investigation and evacuation

theory, particularly as it pertains to disasters involving fires. The categories of information to be captured

in the telephone interviews include:

• Means of first alert (i.e., emergency notification)

• Situational awareness

• Factors associated with evacuation decision

• Delay period and activities associated with delay in evacuation

• Evacuation route choice and reason for choice

• Positive and negative aspects of egress

• Pre-existing injuries or injuries incurred during evacuation

• Occupant characteristics and traits

• Employer-related information relevant to the investigation, such as existence or provision of

emergency training

• Physical building elements and human-related impedances or facilitators encountered during

evacuation.

NuStats will use information obtained during the kick-off meeting to draft the telephone interview

instrument. Such infonnation includes verbal communications with NIST staff and the outside experts,

the documents identifying key questions of the Investigation by project, the document developed by E. Kuligowski on Sample WTC Questions, the notes from the brainstorming session of the WTC project

leaders and team members pertaining to survey content, and questionnaires provided by G. Proulx for prior surveys on this topic. The questions will flow in a logical order in relation to the chronology of the

events, as suggested in the literature.

NuStats will follow standard telephone interview construction techniques. The first draft of the telephone

interview instrument will be reviewed in a joint meeting with NIST staff and the outside experts and

subsequently revised. Prior to this joint meeting, our consultants. Dr. Jon Krosnick and Jamie Abelson.

will review the questionnaire for content, flow, and question construction and validity. Once the final draft instrument has been developed, NuStats (Heather Contrino) will assist NIST in submitting the

instrument package for 0MB approval.

NISTNCSTAR 1-7B, WTC Investigation 23
Appendix A

A.3.1.1 Telephone Interview: Survey Population

The persons who were in WTC 1 and WTC 2 immediately prior to the first aircraft impact on

September 11, 2001, will constitute the population to be sampled in this study segment. The sampling

plan of the September 11, 2001, occupants is a stratified probability sample of building occupants.

Stratifying the population. The proposed stratification criteria are based on the assumption that occupants experienced unique egress behavior, observed different aspects of the structural integrity of the

buildings and suppression systems, and had varied interactions with first responders based on their

location in the buildings at the time of the airline impacts. The capture of these unique circumstances,

observations and experiences are critical for NIST in understanding the egress process and the structural integrity of the buildings, its suppression protocols, and first responder impact leading up to the time of

collapse.

A primary stratification criterion will be building of occupancy - namely, WTC 1 and WTC 2. A

secondary stratification criterion will be location within the towers as defined by three zones that reflect distance from ground floor and proximity to point of impact. The zones are defined according to the

location of the mechanical floors. Separate zones approximately comprise thirds of the buildings:

• the top floors (floors 77 to 91 in WTC 1 and floors 77 to 1 10 in WTC 2),

• the middle floors (floors 43 to 74 in WTC 1 and WTC 2), and

• the lower floors (floors 9 to 40 in WTC 1 and WTC 2). A third stratification criterion is tenant size. The tenant size criterion represents a floor as one of two

levels:

• large tenant floor (a single tenant occupies greater than 40 percent of the usable square

footage of a floor), and

• small tenant floor (all other tenant-occupied floors).

Enumerating the population. At this time, NuStats expects to use the security badge list as the sampling

frame. The list would be sorted to approximate the stratification criteria noted above. The utihty of this

list is pending further investigation of the completeness and reliability of the contact information. If this

list is used, it will constitute the sampling frame of WTC 1 and WTC 2 occupants.

This list is highly attractive because of its full coverage of the September 1 1 WTC occupants. The

downside of the list is that it also contains a large number of occupants who were not in the WTC on the

day of the tragedy. The list is unable to distinguish who was present in the WTCs on September 11. This means that considerable, unanticipated screening will be needed to identify September 1 1 occupants.

Ideally, security badge list will contain home address and telephone number as of September 11. If it does not, then considerable unanticipated resources will be needed to locate the subjects. Even with the

September 11 addresses and telephone numbers, we anticipate that a high proportion of subjects will have

moved or changed telephone numbers and this will require an unanticipated incremental locating effort.

24 NISTNCSTAR 1-7B, WTC Investigation
Work Plan

Screening. It is important to note that telephone screening will be needed to determine whether an

individual on the list is eligible - that is, whether he/she was an occupant of WTC 1 or 2 at the time of the

first aircraft impact on September 11, 2001. It is estimated that a total of 10,000 to 14,000 people were

inside WTC 1 and WTC 2 at the time of the first impact.

Selecting the sample. Stratified probability sampling of individuals will be conducted from the frame to yield a total of 800 telephone interviews. Oversampling of certain subgroups is expected, but the delineation of those groups has not yet been finalized. Oversampled groups might include:

• occupants near the point of impact

• occupants in each tower (e.g., equal allocation to WTC 1 and WTC 2)

• others to be specified.

It is assumed that there will be an approximate 30 percent participation rate among those asked to participate. A final data set of 800 cases will ensure a 0.05 level of significance and power of 0.80. NuStats will make every effort to increase participation above 30 percent.

A.3.1.2 Telephone Interview: Pilot Testing

For the CATI pilot, NuStats will randomly select 10 building occupants from the list provided and

attempt to interview them. Our subcontractor, DataSource, will conduct these interviews. The Task

Manager, Heather Contrino, will be on-hand to directly interact with interviewers during the pilot phase.

DataSource has technology for remote monitoring of interviews so that NIST staff can assess the

interview administration, if requested. All aspects of the data collection plan will be tested including:

• Instrument flow;

• Respondent comprehension;

• Interviewer administration;

• Identification of questions needing probes or clarifications;

• Data coding and processing; and

• Review of survey data.

A.3.2 Face-to-Face Interviews: Survey Instrument

The objectives of the face-to-face interview segment is to gather richness of detail about the human

evacuation experience, to obtain first-hand accounts and observations of the activities and events inside

the buildings on the morning of September 1 1 for specialized investigatory topics, and to capture the

views, experiences, and first-hand accounts from sub-populations of high interest to the Investigation. It

is estimated that the average face-to-face interview will last approximately two hours, with some lasting

significantly longer.

NISTNCSTAR 1-7B, WTC Investigation 25
Appendix A

The face-to-face instrument design is guided by the proposed methodology, which is a synthesis of the

Behavioral Sequence Interview Technique (BSIT) originally developed by Keating and Loftus and the

Cognitive Interviewing Method (CIM), originally developed by Fisher and Geiselman. These two

interviewing methodologies were developed with the purpose of assisting persons in retrieving more

comprehensive and accurate memories of incidents, and sharing important attributes. Both approaches

begin by allowing the informant to retell an unimpeded account without interruption from the interviewer,

and both initially employ a chronological retelling of information. However, BSIT was designed to yield

a database of qualitative infonnation that could be subjected to systematic analysis and consolidation,

while CIM was designed to facilitate investigative interviews. Since the Investigation is pursuing both

goals (i.e., creation of a database of evacuation-related behaviors and an investigatory attempt to capture

information relevant to outcomes), the proposed methodology combines these two approaches.

Cognitive interviewing has been the subject of many empirical investigations. Fisher, et al summarized

these findings, demonstrating that the methodology significantly increases the amount of information

recalled without affecting rate of errors. Interviewing a large number of informants will allow

corroboration of infonnation, thereby compensating for the likely increase in the absolute number of

errors. Accordingly, it is likely that this approach will be productive in achieving a holistic view of the

building evacuations.

NuStats will design three paths within the face-to-face questionnaire: one for occupants, one for family

members of the victims who communicated with loved ones inside the towers before collapse and one for

first responders. Depending on initial screening questions, the respondent will proceed through a specialized version of the questionnaire directed at their survey population. This will increase design

efficiency and allow all responses to be stored in one database.

The occupant questionnaire will be used primarily to capture the egress process, including occupant

behavior and emergency communications in rich detail. The face-to-face questionnaire for family

members will collect data that describe, define and measure the nature, extent and timing of

communications between family members of the victims and their loved ones inside the towers before

collapse. The first responder questionnaire will collect data on emergency communications, emergency

response procedures, and building / response technology. As with the CATI instrument. Dr. Jon Krosnick

and Jamie Abelson will review the questions to ensure they will meet the research objectives and, at the same time, minimize unit and item non-response.

A.3.2.1 Face-to-Face Interviews: Computer-Assisted Personal Interview (CAPI) Format

NuStats will automate the interview structure to increase efficiency and reduce error. The specific CAPI

technology recommended is the use of Tablet PCs. By using this state-of-the art technology, high quality

survey data can be electronically coded and organized, and stored directly in an electronic format, with all relevant error checking performed automatically. This process will simultaneously eliminate unnecessary

data entry costs and associated key entry errors that typically occur after a traditional paper-based field survey is complete. In fact, through the use of a Tablet PC for survey administration, the entire survey can be captured and coded in real time making data delivery faster and easier.

Tablet PCs offer laptop PC functionality and processing power - in a fonn factor that is significantly

smaller than a laptop. An example of a Tablet PC can be seen in Figure A-1.

26 NISTNCSTAR 1-7B, WTC Investigation
Work Plan

Figure A-1. Example pen tablet PC.

This pen tablet PC weighs only 3.1 pounds, has a 10.4-inch viewable screen, and comes standard with an 800 MHz CPU, 128 MB of RAM, and a 20 GB hard drive. In addition to the standard touch screen

display, the latest generation of Tablet PCs includes a wireless keyboard to facilitate data entry. The

Tablet PC shown in Figure 1 converts easily from a Tablet PC to a Notebook PC form factor. Our

proposed instrument design using this technology is described in the following paragraphs. The

instrument will be developed (i.e., programmed) to collect the fact-to-face interview data in three stages,

following the steps of the combined BSIT / CIM interview process.

Step 1: Unimpeded open-ended narrative account. Both BSIT and CIM begin the process by asking

the participant to chronologically recount his or her story. The proposed starting point is when it became

apparent that something unusual had occurred on the morning of September 1 1, 2001. The proposed

ending point is when the participant feels that he or she reached a location where they felt safe (or, alternatively, when he or she successfully reached the exterior of the building). Researchers and

practitioners involved with cognitive interviewing beheve that starting the face-to-face interviews in this manner both improves recall and helps build rapport between the participant and the interviewer.

NuStats will expand on the guidehnes for step 1 by organizing the interview process using the sensory

approach. Respondents will be asked something like.- I'd like you to share your evacuation experience

with mefrom the time youfirst became aware that there was something unusual occurring until you were

safety out ofthe building. I 'm particularly interested in specific things you saw, heard, smelled, or touched during your evacuation.

The instrument for this phase is a list of sensory experiences that the respondent may mention in their narrative. The interviewer only "taps" the screen to record that a particular type of sensory experience was mentioned. For example, the participant might briefly mention an odd odor which the interviewer

makes note of During the negotiated phase (step 2 below) the interviewer will want to prove this fact to determine whether the smell might have been that ofjet fuel, smoke, or of some other origin as yet

unknown.

NISTNCSTAR 1-7B, WTC Investigation 27
Appendix A

Step 2: Structured (negotiated) narrative account. After participants complete their stories,

interviewers will prompt them to go through the story again, but this time they will work cooperatively

with the interviewer to record entries into a table. This approach is employed by BSIT for three primary

reasons: (1) to yield a structured account that can be entered into a database without further processing;

(2) to avoid the biasing effects of having interviewers ask specific questions; and, (3) to enhance the effort

at recall put forward by participants by encouraging their active collaborative participation, an advantage

to open-ended formats as noted by Fisher, et al.

Each row of the table will represent a single action in a sentential format, meaning that each action is expressed as a grammatical sentence. The approach is based on the hypothesis that people encode

episodic memories in a manner consistent with this format, thus facilitating both recall and data entry.

Each coluinn of the table represents three essential components of actions: a cue, an action, and the reason

for taking that action (see Table A-1).

Table A-1. Example tabular face-to-face interview data entry.

Cue Action Reason

1 heard but couldn't see someone

yell "I've found a clear path"

So 1 stumbled in the dark towards

where 1 thought the voice came

So that I could find a way to escape My path was blocked by debris So ] called out to whoever yelled, "I'm near the reception area. Where are you?

To try to get a better idea of

where the person was

Cues can be either external (e.g., signs of a fire, someone saying something) or internal (e.g.,

remembering about another means of escape). For purposes of this research, cues will be sensory

experiences. Actions are expressed using specific action verbs (i.e., ran. instead of went) and may include

artifacts (e.g., a fire extinguisher) used by the infonnant. Reasons are the intentional, goal-directed base

for the action. The interviewer will encourage the participants to use their own words to the greatest extent possible.

Experimental findings in psychological research on memory suggest that when people perform actions,

their abilities to verbally recall those actions are significantly improved. Script theory suggests that people naturally organize their knowledge of actions using narrative sequences of actions structured

around their pursuit of goals. However, gaps in the narrative are anticipated, especially given the long

period of time that will have elapsed between the event and the interview. The information entered by the

interviewer in Step 1 on the sensory experiences recounted in the unstructured narrative will be used to "populate" a table prior to the start of the negotiated structured narrative report. This pre-populated table

will be used to structure the re-telling of the respondent's story. In addition, interviewers will assist the

participants to fill in these gaps by asking them to recall events in reverse order, an approach used in CIM. Interviewers will, however, encourage participants to report only those memories about events or

incidents that they are confident really occurred to them.

Step 3: Probing for specific information. After completing the negotiated, structured narrative account,

interviewers will ask specific open-ended questions (probes) intended to provide specific information of

particular value to the investigation. While some of this infonnation is likely to be part of the structured

narrative account, participants may be able to recall other valuable information as well. In this step,

interviewers will ask questions related to topics that have been identified a priori as being priority topics.

28 NISTNCSTAR 1-7B, WTC Investigation
\Nork Plan

This will ensure that all information of significant importance to the Investigation will be examined in the face-to-face portion.

Depending on the population, probes may be used to try to elicit information including, but noi linuicd lo:

• Location of the informant at the time of certain marker events (e.g., location in WTC 1 when

WTC 2 collapsed)

• Fire conditions (e.g., fire and smoke);

• Other cues of interest (e.g., the smell ofjet fuel);

• Presence and activities of persons with disabilities;

• Use of elevators by self or others; and,

• Knowledge of any obstacles to their progress while using the stairs.

Because information about many of these areas of concern requires precise responses, questions for openended probes will be developed collaboratively between NuStats and NIST. Responses to probes may be

recorded using standardized formats where feasible. For example, all participants who observed smoke

may be asked to estimate the smoke density using an encodable scale, such as visibility distance. To help

standardize scales or indexes used and also the reporting of locational information, graphical prompts will

be included in the software to allow for better specification.

A.3.2.2 Face-to-Face Interviews: Survey Population

The populations to be sampled in this study segments include the following types of people:

• individuals who were in WTC 1, WTC 2, and WTC 7 immediately prior to the first aircraft impact on September 1 1 , 2001

;

• individuals who communicated with members of their families who were in WTC 1, WTC 2, and WTC 7 immediately prior to the first aircraft impact on September 11, 2001, and who

never made it out of the buildings safely;

• individuals who were among the first responders (i.e., firefighters and police) to the WTC

disaster.

The sample will be selected on the basis of NIST knowledge of populations that may have unique pieces

of information to add to the Investigation. These are persons who belong to special subpopulation groups

or were in locations during the evacuation that enabled them to experience distinctive situations. This

method of sampling is called purposive or judgmental sampling. The populations of interest for the faceto-face interviews are:

• Occupants near the floors of impact (WTC 1 and WTC 2)

• Occupants of WTC 7

NISTNCSTAR 1-7B, WTC Investigation 29
Appendix A

• Floor wardens, fire safety directors, and other persons with building responsibility

• People in elevators or lobbies

; • Occupants with disabilities -

• Family members of victims who called out of the towers

• First responders (members of the Fire Department of New York, Port Authority Police

Department, New York Police Department, and other having operational responsibility.

While the exact numbers of interviews for each group has not been finalized, the total number of face-toface interviews will be 750 intei'views, of which 150 will be with first responders.

Enumerating the population. The population will include the entire occupant, management, and first responder population of World Trade Center WTC 1, 2, and 7.

Selecting the sample. The sample for the face-to-face interviews is purposive. The potential

respondents will be located via a number of methods. For example, NIST will provide the names and

contact information for the first responder sample. NuStats expects to receive approximately 300 names

from which to select these respondents. Other special populations can be identified through media

accounts. NuStats will work with Guylene Proulx to identify potential respondents as she is the contractor for another WTC-related study of media coverage. A number of the respondents for the faceto-face interviews will be identified via the CATI interviews. During CATI interview, it will be noted

when a respondent has characteristics or experiences matching the subpopulations of interest. In such

cases, the respondent will be interviewed via CATI and then an appointment will be set up for a face-to- face interview.

A.3.2.3 Face-to-Face Interviews: Pilot Testing

The pilot of the face-to-face instrument will be done in New York City at MBC Research Center. Johanna

Zmud (project director), Robert Santos (senior methodologist), and Delia Santos (task manager), will

conduct the pilot face-to-face interviews. A total of five face-to-face interviews will be conducted among

occupants, family members and / or first responders. The viewing room at MBC Research will enable

NIST staff to observe the pilot interviews. As with the CATI interviews, all six (6) aspects of data

collection will be tested.

A.3.3 Focus Groups: Interview Protocol

The goal of the focus group interviews is to elicit accurate group representations of specific events or themes. Williams reports that in a group setting, people provide cues that evoke memories in others, and

that social pressures mediate against reporting misrepresentations of what they recall. The interview

protocols will build upon the concepts and operationalizations used in the CATI and face-to-face

instruments. Focus group protocols will be designed to help facilitate recall and elicit group

representations of specific events, communications, and egress behavior. This will be accomplished

through a tiered focus group protocol that begins with less sensitive topics and progresses into more

detailed and difficult information as the session proceeds.

30 NISTNCSTAR 1-7B, WTC Investigation
V\/ork Plan

A.3.3.1 Focus Groups: Target Population

Two distinct populations will voluntarily participate in the focus groups: occupants and first respondcrs.

The first set of focus group interviews will be the occupant sample. Distinct categories of people will be

selected for inclusion in this study. The objective of this study is to capture the experience of people in unique places in WTC 1, 2, and 7. Some of these groups have been identified by NIST: the 1 6 people

who were above floor 78 who made it safely out of WTC 2, disabled persons, persons who saw the lobby

fireball, and persons who were trapped in the elevator. Every effort will be made to include no less than

5 people in each of these categories in this study, with 10 people constituting the preferred focus group

size. NuStats anticipates conducting five occupant focus groups.

First responders will constitute a second set of focus group interviews. The set of first responders will include FDNY, NYPD, PAPD, and other groups identified as having operational or command authority at the World Trade Center on September 1 1th. The focus group size will be determined as an operating unit

size, if applicable. An operating unit may be a Fire Department company, for example. This project

proposes 10 focus groups, each containing five people.

Sample selection. The people selected for inclusion in the focus groups will be selected using nonprobability sampling procedures. Respondents contacted for the CATI interviews or face-to-face

interviews may be eligible for participation in the focus groups. In addition, respondents in these other

studies will be asked to provide the names and contact information for people they know in each of the categories of interest. The contractor will collect names until at least five people in each category have

agreed to participate in an occupant focus group, with a preference for 10 people. It is expected that the

potential participant list for the first responder focus groups will be provided by NIST. The same process

will occur for selection of the first responder samples, with a preference for inclusion of entire operating

units (about 5 people per unit).

A.3.4 Instrument Evaluations

Each instrument will be evaluated prior to pilot testing. The instrument review process will benefit from

using cognitive interviews to identify, measure, and/or reduce survey response errors. The administration

of this survey requires respondents to accurately report factual data for which knowing, remembering, and

placing events in time are critical. Cognitive interviews of likely respondents would serve to identity and

assess the potential for survey response errors by examining comprehension of the survey item, retrieval of relevant information, judgment based on recall, and mapping and reporting of a response.

Cognitive testing is relatively new to the survey industry (it has come into use only in the past 15 years or

so). It involves the observation and testing of prototype respondents in a controlled laboratory setting to provide insight into the cognitive survey process that is undertaken by a subject who is interviewed by an

interviewer. The cognitive survey response process literally refers to what happens in a subject's mind

between the time he/she is asked a question, up to and including the point that he/she provides a response.

It is composed of four stages: (a) comprehension; (b) retrieval of information; (c) response formation;

and (d) response editing. Cognitive testing provides insight into these areas in a way that can lead to significant improvements in question wording and/or validation of research constructs.

Not all questions in an instrument require cognitive testing. Dr. Jon Krosnick is an expert in the cognitive

survey process and he will assist in the identification of the questions that do require cognitive testing.

NISTNCSTAR 1-7B, WTC Investigation 31
Appendix A

Our approach would be to identify in the review process a subset of critical questions and constructs that might benefit from cognitive testing (regardless of the mode of administration - CAT!, face-to-face, or focus groups), and to employ a small number of cognitive interviews with likely respondents to address

them. Cognitive tests will be done with individuals known to NuStats who were building occupants on

September 11.

A.3.5 Finalize Survey Instruments and Interview Protocols

After completion of the pilot tests, NuStats will provide a report that details recommended changes in the

questionnaire/ protocols and training materials. This report will include a discussion of recommendations

for modifying question order / sequencing, introductory statements, response categories and question

wording that might be problematic for the respondent. The report will also include a discussion of

recommendations for modifying interviewer instructions.

A.3.6 Pre-Field Work Meeting

A working session with NIST and its outside experts will be held in which NuStats details problems,

identifies questionnaire items that need revision, and suggests corrections for problems encountered

during the pilot. Following this meeting, NIST will notify NuStats of any recommended changes in the

survey instrument and protocols. NuStats will incorporate only NIST-approved changes to the

instruments and in-person protocols before going into the field.

A.3.7 Protection of Human Subjects and IRB Approval

This data gathering effort will ensure that all precautions required by the Common Rule for the Protection

of Human Subjects are met or exceeded by the contractor. Participation in any part of this project by any

person will be strictly voluntary. Interviewers will be trained to establish a rapport with participants

based on a compassionate interest in their story and will ensure participants that information provided will

be of value in preventing casualties in future building emergencies. During the briefing, interviewers will

provide information to participants about where and how to receive counseling and about the fact that participants may stop the interview at any time without explanation. Interviewers will also be trained to recognize signs of post-traumatic stress. Similar services will be offered to participants of focus groups

and to people taking the telephone interview. NuStats will take the necessary precautions to ensure the

safety of contract employees administering, collecting, or otherwise involved in this data collection effort. Finally, Nustats will ensure that the identities of the subjects are held in the strictest confidence.

The human subjects protocols to be used in this project by NuStats and DataSource will be reviewed by

Essex Institutional Review Board.

A.3.8 Training of Surveyors

NuStats will train a team of survey specialists for CATI and face-to-face administration as well as focus

group moderators. Training protocols will be prepared by each of the Task Managers (Heather Contrino,

Delia Santos, Kim Hilsenbeck) under the direction of the Project Director, Dr. Johanna Zmud. All

interviewers will receive an eight-hour training session and will be required to perform simulated

interviews before beginning actual data collection activities. NuStats produces project-specific

32 NISTNCSTAR 1-7B, WTC Investigation
Work Plan

interviewer-training manuals for each project. The basic elements of the manual include an overview of

the study, its objectives, glossary of ternis, and questions and answers typically posted by the respondent.

Another aspect of training deals with the intricacies of the survey instruments themselves, with separate

training for each survey instrument involved.

In this case, training materials and sessions will include specific provisions to address the unique needs of

this study including:

• Informed consent;

• Privacy/confidentiality;

• Elements of risk; and

• Provisions for the protection of human subjects.

Mock interviews are used to help surveyors quickly become familiar with the survey and nuances of the

formatting and layout. The mock interviews will also serve to help face-to-face interviewers practice

handling encounters with different types of respondents. Surveyors will not begin working on this study

until they have passed a project-specific test.

The telephone interviewers will be regular staff of DataSource. Interviewers for the face-to-face

interviews will be contracted social work practitioners. Even though the social workers will be sensitive

to context in which the interviews are being done, all interviewers and moderators will receive specialized

training. Our consultant, Jamie Abelson, will be particularly active in this training component. This

component will include techniques for dealing with emotionally traumatized respondents, as well as voice

tone and emotional mediation techniques. The contracted social workers will also be trained specifically

in BSIT and CIM techniques using in-house experts and outside practitioners.

CATI interviewers will be trained at DataSource, NuStats' subcontractor, while face-to-faee inter\'iewers

will be trained at NuStats' space in New York City. Since focus group moderators are all in-house staff, Ms. Abelson will brief them at the NuStats' offices in Austin or in the D.C. area if the NIST staff would

care to attend.

Both NuStats key staff and DataSource supervisors will continually monitor interviewers and moderators

involved in the study to ensure the highest level of quality is maintained. Interviewer debriefings will be

held weekly to address any data collection issues that arise and to heighten interviewers' performance.

It should be noted that NuStats will require all staff working on this project to sign new legally binding

pledges of confidentiality specific to this project, in addition to the pledges all staff sign as part of their employment contract with the firm. Any additional subcontractors or consultants retained for this project

will be required to do the same.

A.3.9 Database Design

NuStats will design a database, i.e., an encoded table of results, based on the developed coding scheme of

all concepts and their operationalized measures used in the telephone and face-to-face approaches. Thus,

NISTNCSTAR 1-7B, WTC Investigation
Appendix A

the database will consist of two parts: CATI data and face-to-face data. Focus group infomiation will be

delivered to NIST as court transcripts. NuStats will design the database structures for the telephone and

face-to-face surveys according to the NIST specifications. Upon final approval of the questionnaires and

moderation guides for each component of the study, the NuStats team will develop the database structures and fully document variable names, structure, and skip patterns in a comprehensive Data Matrix. One

data matrix will cover each of the two databases. The Data Matrix will include all variable specifications

including built in quality assurance measures (such as edit, range, and consistency checks) and will serve

as the final road map for database development. The number of unique variables comprising the data

bases has not be specified, however, the RFP states that there will be no more than 75 encoding variables.

To protect the confidentiality of participants, each data record will be assigned a unique identifying

number. This will ensure that the names of study participants are not associated with responses to questions. Moreover, the data file containing the link between name and ID number are stored separately

from the data files containing question responses. All confidential information will be stored in password-protected files by the holders of this information.

A.3.10 Additional Data Collection

NuStats understands that the scenario may arise where an individual critical to developing an

understanding of the events of September 11, 2001, may be unavailable or unidentified during the period

of performance of the Contractor. Thus, NIST will obtain NIST Institutional Review Board (IRB)

approval to conduct a limited number of face-to-face interviews with individuals deemed by NIST likely

to contribute significantly to the outcome of the Investigation. The scope, objectives, and procedures

used in the additional data collection will be similar to the scope, objectives, and procedures used by the

contractor. Telephone interviews and focus groups will not be conducted in this additional data collection

effort. It is anticipated that the number of face-to-face interviews conducted by NIST will be less than 10

percent of the number of face-to-face interviews conducted by the contractor. The contractor will incur no duties or obligations related to the additional data collection.

A.4 TASK 2: SURVEY ADMINISTRATION

A.4.1 CATI Field Work

NuStats subcontractor, DataSource, will reach a participation rate of 30 percent to conduct a minimum of

800 interviews with building occupants. Due to the length and nature of the interview, we will not

include partial interviews as completes.

DataSource will use a small, well-trained team of 8-12 interviewers to conduct the CATI interviews.

Each interviewer will be experienced in dealing with sensitive subjects, well versed in mental health

protections and confidentiality issues, and will have passed an extensive training exam before being

allowed to dial on the survey.

It is expected that data collection will last for eight weeks. This is to allow for the time it will take to reach a particular respondent (research, initial contact, follow-up contact/ interview) and to track those not

easily reached to achieve the desired participation rate. Dialing hours will be 10 a.m. to 9 p.m. (Eastern

34 NISTNCSTAR 1-7B, WTC Investigation
Work Plan

Time) and calling will vary across days, nights, and weekends to maximize response. NIST will provide

the sampling frame to the NuStats team in the format requested by NuStats.

A phone call will be made to the sampled individuals. NuStats will vary the day and time of the phone

call to increase the likelihood that contact with the individual will be made. If an individual chooses to participate, he/ she will be asked to voluntarily disclose a preferred means of follow-up contact. Regardless of the preferred means, all contacted individuals will receive a personalized mailing that details the purpose and goals of the study, the client, and the types of the questions that will be asked.

The mailing of the letters will be done prior to the expected day / time of the follow-up contact. Individuals will be instructed to phone a toll-free number at DataSource should they require further

information.

Heather Contrino, task manager, will monitor progress through well-established communications with the

DataSource field team. NuStats project managers receive nightly dialing reports (documenting the

number of completed interviews) and weekly disposition reports (a cumulative disposition file of all numbers dialed to date) from DataSource. Ms. Contrino will customize this reporting to suit NIST

progress reporting needs.

NuStats is always concerned with non-response, and understands that the response rate for this sur\'ey is a

critical element of the data collection process. We will use time-tested methods for minimizing refusals

without infringing on the voluntary nature of this survey. Techniques to combat non-response will

include:

• Advance notification

• The use of Lexis/Nexis to locate "missing" respondents selected for the study; and

• A toll-free respondent hotline and a respondent web page.

Advance notification has been proven in studies such as this one to minimize refusals. Whenever

possible, we will send a pre-notice letter describing the survey. The letter will be short, written in simple

language, will explain the survey relevance, distinguish it from other surveys investigating the WTC

disaster, and will be personalized to the extent possible. Respondents will be encouraged to contact DataSource through the respondent hotline if they have questions.

In some cases, contact information will be outdated or nonexistent. NuStats anticipates that we may face

this and other barriers in locating some of the sampled occupants. We propose to use our in-house

resource Lexis/ Nexis, for tracking subjects selected in the sample but form whom inadequate or no

contact information is available.

A member of our trained interviewer team will work at a toll-free respondent hotline during calling hours.

Since many respondents will have received advance notification, we anticipate that some will have

questions or concerns about participating. Making staff available to answer respondent concerns is a

critical element in building trust and establishing validity in the survey design. NuStats will design and

informational (with Q&A capabilities) web page for respondents who seek quick answers to their

questions.

NISTNCSTAR 1-7B, WTC Investigation 35
Appendix A

A.4.2 Face-to-Face Field Work

NuStats will conduct 750 face-to-face interviews, with occupants, family members, and first responders.

NuStats will use the synthesized BSIT and CIM technique in conjunction with CAPI technology to create a database of evacuation-related behaviors and capture information relevant to outcomes. NuStats will

recruit the face-to-face intei^viewers from the available pool of psychiatric social workers in the New

York metropolitan area. We expect this team will be trained (and experienced) in dealing with sensitive

topics and additional mental health provisions. Their experience will be supplemented with projectspecific training for the WTC investigation. This will include training to recognize signs of posttraumatic stress. They will also be able to provide sources for additional mental health counseling, should

a respondent request it.

NuStats expects to recruit a team of 8-12 interviewers and to conduct approximately 6-8 interviews per

day (seven days a week), with approximately 2 hours allocated for each interview. This is a larger faceto-face interviewer pool than we identified in the RFP to ensure that the data collection in completed

within the necessary timeframe. This will require more Pen Tablet PCs than we had budgeted.

Interviews will not be video- or audio-taped to protect the privacy and confidentiality of the respondent.

NuStats will try to schedule the location for the interviews to be as convenient as possible. NuStats will

employ a variety of locations to ensure that a convenient location for each face-to-face respondent can be

achieved. Locations include NIST space, MBC research space, space in lower Manliattan at 181 Mott

Street, as well as the respondent's home or work place, if so desired.

Della Santos, task manager, will manage all fieldwork. DataSource will handle the recruiting and

scheduling for all face-to-face interviews. Because DataSource interviewers will go through intense, in- depth training on the WTC project, we believe they will be proficient in dealing with this sensitive topic,

and will be able to best answer questions about study objectives and how results will be used.

A confirmation letter will be mailed to each individual who agrees to participate in the study. The letter will explain the purpose of the interview, describe how the interview will be conducted, confirm the time

and date of the interview, and provide directions to the interview location. Reminder calls will be made

to these persons by the actual face-to-face interviewer to whom the case has been assigned. This will help

establish a relationship between the intei^viewer and respondent prior to the actual interview.

Through the use of the CAPI technology, data will be entered and coded almost in real-time. This will

enable NuStats to provide data from these interviews on an interim schedule.

A.4.3 Focus Group Field Work

Because the focus groups are expected to be longer than typical (approximately 4 hours) and emotional in content, NuStats will employ a team of experienced, in-house focus group moderators to conduct the

focus groups with occupants and first responders. These moderators include Dr. Carlos Arce, Dr. Johanna Zmud, and Kim Hilsenbeck. Ms. Hilsenbeck will also manage this task, ensuring that all groups

are organized and scheduled as required.

DataSource will handle the recruiting, scheduling, and confirmation for all focus groups (for the same

reasons noted for the face-to-face interviews). Focus groups will be held at MBC Research Center on

36 NISTNCSTAR 1-7B, WTC Investigation
Work Plan

Madison Avenue. A confirmation letter will be mailed to each respondent that explains the purpose of the focus group and the protocols in place for identify protection, describes how the session will be

conducted, and provides directions to the focus group facility. DataSource interviewers will also make all the reminder calls.

Each moderator will employ a moderator protocol for the response population. Discussion guidelines

provide a general framework for questioning and discussion during a focus group. NuStats will develop

this guideline in conjunction with NIST and the outside experts. While the focus group moderators

follow guides as close as possible, each is trained to understand that additional, relevant topics may

surface during the course of a focus group session. Flexibility is critical to gaining the most valuable

insight from respondents. If participants diverge into topics that would provide useful data for the study, our moderators are experienced in allowing the divergence without going too far off course.

NuStats anticipates that focus groups will last approximately four hours, with hourly breaks provided.

The moderators will provide an introduction at the outset of each session, explaining the voluntary nature of respondents' participation. This introduction will also provide the basic ground rules for discussion as well as disclose any court transcription or viewers. The moderator will also relate the purpose of the study and how the focus group results will be used. NuStats expects that there may be questions about

privacy issues, and we will use this time to reassure all participants about the high security protocols in place to protect the identity of and information provided by respondents.

A.4.4 Status Reporting and Data Deliveries

Monthly status reports will be delivered to NIST to monitor progress in accordance with the statement of

work. These reports will advise NIST of work completed during the performance period, and work

forecasted for the next performance period. NuStats will also provide data deliveries with these status reports. Data deliveries (for CATI and face-to-face interviews and transcripts of focus groups) will be

delivered at the end of September, October, and November.

NuStats expects to have a meeting with NIST and the outside experts after the delivery of the September

progress report and data to review the data and any issues that have arisen with the fieldwork.

A.4.5 Completion of Field Work

Fieldwork will be deemed complete with NuStats has accomplished all of the sampling requirements set forth in this finalized work plan. We expect the vast majority of fieldwork to be completed by the end of

November. There may be some leakage of field work into early December but expect this to be minimal.

At the completion of fieldwork, NIST will have received the majority of all required data through the

interim data deliveries. Three separate databases will be present. Each data set will be cleaned prior to delivery. If NIST finds any data cleaning issues, these will be dealt with promptly. To facilitate data

quality assurance, NuStats will prepare a data quality assurance plan for NIST review prior to the start of

fieldwork.

At the completion of all fieldwork, NuStats will prepare a summary analysis for NIST review. The

summary analysis will contain three parts. The first part is a compilation of on-going (monthly) statistical summaries of survey progress, consistency checks, plausibility checks, and other indicators of data quality

NISTNCSTAR 1-7B, WTC Investigation 37
Appendix A

for the CATI and CAPI data. Second, a summary of the fielding for the CATI and CAPI survey response

will be summarized, including number of contacts, participation rates, completed responses in each cell of

the sampling stratification, completed response rates by question. Third a descriptive accounting of the CATI and CAPI data will be provided, including frequency distributions and calculations of the mean and

variance of the sample for all data elements.

A.4.6 Debrief

At the conclusion of data collection, NuStats, NIST and the outside experts will meet for a project debrief

This meeting will allow for discussion of survey resuhs. From NuStats, participants will include, among

others. Dr. Johanna Zmud, Project Director, and each of the Task Managers Heather Contrino, Delia

Santos, and Kim Hilsenbeck. Dr. Carlos Arce and Robert Santos will also be likely attendees from

NuStats. Prior to coming to this debrief meeting, each of the Task Managers will have debriefed

surveyors within their span of control. They will compile and bring to the meeting the insights and other

feedback garnered directly from those involved in the actual interviewing.

A.4.7 Final Field Work Report

NuStats will provide two data sets - one for the CATI data and one for the CAPI data. Focus group data

will be comprised of transcriptions of the focus group sessions.

In addition, NuStats will prepare a comprehensive draft final report summarizing our work under the

contract. This final report will fiilly document the study and sample design including instrument, sample,

or methodological changes, result of the cognitive interviews and pilot, documentation of data collection

period and protocols, data processing procedures, and any conclusions. Response rates and call dispositions will also be included. In addition, the report will include all materials used in the study

including the CATI script, CAPI script, focus group protocols, and data file structures.

NIST will have 30 days to comment on the draft final report. NuStats will incorporate these comments

any format requirements into the final report and deliver four copies within 15 days of receipt of NIST

comments.

A.5 TASK 3: DATABASE COMPLETION AND DELIVERY

NuStats will deliver the final databases as noted in prior sections of this final work plan. NuStats will not conduct any data analysis as part of this contract. Data will be delivered in a forniat that will be most

usable to NIST. All analysis will be limited to statistical calculation of the data, and any content analysis

required to categorize open-ended responses.

A.5.1 Technical Assistance by Database Expert

NuStats' database experts will assist NIST after database delivery to facilitate NIST developing an

understanding of the structure, architecture, limitations, and use of the database (this can include interim

database delivery if requested by NIST). NuStats will also deliver a Data User's Guide as part of final deliverable to NIST. While this Guide will not replace the need for the on-site consultation by our

database expert, our past experience has shown that it greatly facilitates client use of the database. The

38 NISTNCSTAR 1-7B, WTC Investigation
Work Plan

Guide will include descriptions of the data file, sample tables, data file codebooks, glossary of terms,

discussion of estimated sampling errors, and additional background information.

A.6 SCHEDULE

The following provides a schedule of project milestones and meetings.

Task Description Due Date

Days from

Award

Contract Award 6/2 0

1 Final Work Plan 6/16 13 CATl and CAPl Instrument Drafts 7/1 35

Meeting with NIST / Experts to Review Drafts 7/8 42

Revised CATI and CAPl Instruments and Focus Guide Protocols 7/22 56

Respondent Materials 7/22 56

IRB Approval 8/5 70

NIST IRB Approval 8/19 84

CATI Pilot Training 8/21 86

CATI Pilot 8/25 90

CAPl Pilot 8/26-28 93

Pre-Field Work Meeting 8/28-29 94

Finalize Instruments and Procedures 9/15 110

Training CAPl 9/17-18 113

0MB Approval 9/22 117

Training CATI 9/23-24 119

2 Begin CAPl Field Work 9/23 119

Begin CATI Field Work 10/1 127

First Data Delivery and Status Report 10/3 129

First Focus Group Sessions 10/6-9 135 Data Review Meeting 10/10 136

Second Focus Group Sessions 10/27-30 156

Second Data Delivery and Status Report 11/3 160

Third Focus Group Sessions 11/3-6 163

Fourth (Last) Focus Group Sessions 11/17-20 177 End CATI Field Work 11/27 184

Third Data Delivery and Status Report 12/3 190

End CAPl Field Work 12/12 199

Debrief on Field Work and Delivery of Summary Reports 12/19 204

Final Field Work Report 12/19 204

3 Final Data Delivery 12/19 206

Data Users Guide 12/19 206

NISTNCSTAR 1-7B, WTC Investigation 39
Appendix A

This page intentionally left blank.

40 NISTNCSTAR 1-7B, WTC Investigation
Appendix B

Data Collection Methodology for World Trade Center

Evacuation AND Emergency Response: Telephone Interviews,

Face-to-Face Interviews, Focus Groups and Population

Sampling

1. Background

The goal of the National Institute of Standards and Technology's World Trade Center Investigation is to investigate the building construction, the materials used, and the technical conditions that contributed to the outcome of the World Trade Center (WTC) disaster. The results of the Investigation will serve as the

basis for improvements in the way buildings are designed, constructed, maintained, and used; improved

tools, guidance for industry and safety officials; revisions to codes, standards, and practices; and

improved public safety. The primary objectives of the NIST-led technical investigation of the WTC

disaster are to:

1 . Determine why and how WTC 1 and 2 collapsed following the initial impacts of the aircraft and

why and how WTC 7 collapsed;

2. Detennine why the injuries and fatalities were so high or low depending on location, including all technical aspects of fire protection, occupant behavior, evacuation, and emergency response;

3. Determine what procedures and practices were used in the design, construction, operation, and

maintenance of WTC 1, 2, and 7; and

4. Identify, as specifically as possible, areas in building and fire codes, standards, and practices that are still in use and warrant revision.

The NIST Investigation Plan can be found at http://v^c.nist.gov, including a description of Projects 7 and

8. Under Project 7, "Occupant Behavior, Egress, and Emergency Communications," first-hand accounts of the events of September 11, 2001 from inside WTC 1, 2, and 7 will be collected. This data collection

effort will evaluate the role of occupant behavior and evacuation technologies and practices for tall buildings, including decision-making and situation awareness, time-constrained evacuation strategies,

communications, role of floor wardens and fire safety directors, and issues concerning people with

disabilities. Additionally, NIST will seek specific observations of fire and smoke conditions and/or

structural damage from within the building. Families of the victims, who communicated with loved ones

inside the Towers before collapse, will be interviewed to determine the nature of the environment above

the floors of impact.

The nature of the communications between and among different groups within the World Trade Center

has been identified as being a potentially significant factor in determining the outcome of the evacuation

and emergency response. The project will investigate the content and timing of communications among

the occupants and authorities within the buildings, as well as people outside the buildings. The figure

below, a hypothetical demonstration of the extraordinary flow of information on the morning of

September 1 1"^, reinforces the need to understand the role of information transfer in explaining occupant

and responder actions. In addition to the intergroup communications, communications within each group,

particularly the building occupants, are potentially important to understanding the events of

September 11 '^

NISTNCSTAR 1-7B, WTC Investigation 41
Appendix B

Emergency Communications

The objectives of Project 8, "Fire Service Technology and Guidehnes," are to build upon work already

done by the Fire Department of New York (FDNY) and McKinsey & Company by: ( 1 ) fully documenting

what happened during the response by the fire services to the attacks on the World Trade Center, up to the

time of collapse of WTC 7; (2) identifying issues that need to be addressed in changes to practice,

standards and codes; (3) identifying ahemative practices and/or technologies that may address these

issues; and (4) identifying R&D needs that advance the safety of the fire service in responding to massive

fires in tall buildings. Thus, a subset of the emergency responders who were present at the World Trade

Center complex will be asked to voluntarily participate in the face-to-face interview or focus group

phases of this project. Only first responders who participated in fire suppression, operational, or search

and rescue activities prior to the building collapse will be considered for inclusion in the population of

face-to-face interviewees.

The data collection will be conducted by a yet-to-be-selected contractor and is planned to begin as soon as the necessary pre-work is complete. This includes preparation of the telephone interview schedule, faceto-face interview protocol, focus group protocol, training of contractor staff, and approval by NIST and

the appropriate Institutional Review Board (IRB) to assure compliance with federal requirements for the

protection of human subjects. NIST will use established procedures to review all survey and interview

questions, data collection methods, and safeguards for maintaining privacy and confidentiality of all instruments before proceeding with these critical data collection efforts.

Note that this paper identifies specific populations and the size of samples to be included in the data

collection effort. The exact numbers and populations may be modified to better suit the Investigation as

additional details of the methodology are finalized by NIST and the yet-to-be-chosen contractor.

42 NISTNCSTAR 1-7B, WTC Investigation
Data Collection Paper

2. Overview of Methodological Approach

A multidisciplinary, triangulated approach, including telephone interviews, face-to-face interviews, as well as focus group interviews has been selected. The multi-methodological approach was selected for several reasons. First, multiple methodologies increase confidence in the conclusions and findings when

more than one methodology arrives at the same conclusions. Second, the varied objectives of the Investigation mandate complementary approaches to accomplish all the goals. Finally, concerns

associated with the time latency since September 1 1, 2001 suggest the use of different approaches and

techniques in order to increase memory recall and accuracy. A discussion of each methodology and

statistical sampling will follow.

NIST intends to solicit experienced contractors to perfonri the telephone interviews, face-to-face

interviews, and focus groups. The contractor will meet or exceed all Federal requirements regarding the Common Rule for the Protection of Human Subjects,' including Institutional Review Board (IRB) and

NIST approvals. The objective is to perform up to 600 face-to-face interviews of occupants from areas of

interest, approximately 150 face-to-face interviews of first responders using selected groups, approximately 800 telephone interviews covering selected floors of WTC 1 and WTC 2. Additionally,

NIST will contract for the conduct of up to 10 focus group sessions with first responders, including the

Fire Department of New York (FDNY), the New York City Police Department (NYPD), the Port Authority Police Department (PAPD), or other groups identified as having operational or command

authority at the World Trade Center on September 11, 2001. Finally, up to five focus group sessions will

be conducted with selected building occupants and management.

2.1 Telephone Interview Format

The first data collection instrument is the telephone interview. The collection mechanism will be a computer-assisted telephone interview. The primary goal of the telephone interview is to provide

qualitative and quantitative occupant behavior and egress data which can be generalized. A secondary

goal will be to provide unique, investigative observations, particular to the events at the World Trade

Center on September 1 1"^. The telephone interview schedule will be closely linked to the evacuation

experience of the occupants.

The questions will flow in a logical order in relation to the chronology of the events, as suggested in the

literature."'^ Significant topic areas proposed for the telephone interview include, but are not limited to: occupant demographics and inherent traits, chronology of occupant activity, observations and perceptions

during evacuation, and environmental, social, psychological, physiological, information, frequency, and

source attributes. As the precise content of the telephone interviews, face-to-face interviews, and focus

groups has not yet been established, these factors are subject to review and change.

NIST will follow standard telephone interview construction techniques.'* These techniques suggest that the project team identify the scope and objectives of the telephone interview schedule. The question type

and format which best accomplishes the scope and objectives will then be selected. The first draft of the

telephone interview schedule will then be reviewed and revised. Cognitive and pilot testing of the

telephone interview with the informants will then occur. After further revision, the procedures to administer the study will be specified.

The persons who were in WTC 1 and 2 immediately prior to the first aircraft impact on September 1 1

,

2001, will constitute the population to be sampled in this study segment, hereafter, to be referred to as the

"selected floors study." The sampling plan of the September 1 1, 2001 occupants is a multi-stage

statistically representative sample with two stages. The first stage will stratify floors by area, population,

and number of tenants. The second stage will select occupants from the floors selected in the first stage.

Stratifying the population . Stage one is an area sample of floors. WTC 1 and 2 will be segmented into

three zones each according locations of the sky lobbies. These zones will approximately represent the top

(TS'" - 1 10"" floors), middle (44* to 78* floors), and lower (ground to 43"* floor) thirds of WTC 1 and 2.

NISTNCSTAR 1-7B, WTC Investigation 43
Appendix B

This will result in a total of six building zones from WTC 1 and 2. Each zone will then be further

stratified by tenant number and occupant density. Tenant number will include two levels: single tenant

floors and multi-tenant floors. Occupant density will include three levels: low, medium, and high density.

Definitions of low, medium, and high density will be formed after the total building population is identified. Thus, each zone will contain six floors, if possible. The second stage is a random sample

without replacement of occupants from the floors selected in the first stage.

Enumerating the population. A population list of all the people in each of these 6 building strata immediately prior to first impact on September 11, 2001 will then be enumerated. It is estimated that a

total of 10,000 - 14,000 people were inside WTC 1 and 2 at the time of the first impact,^ and there are unsubstantiated accounts of between 4000 and 5000 persons in WTC 7 on the morning of September 1 1"^. For the purposes of sampling and estimation of the number of telephone interviews, face-to-face

interviews, and focus groups, this project will assume an initial population of 18,500 occupants. NIST

will provide to the contractor an enumeration of persons on the selected floors.

Selecting the sample. The sampling plan for the selected floors study will be constructed such that a total of 800 telephone interviews are obtained from people included in the study. It is assumed that there will

be an approximate 30 percent participation rate among those asked to participate. A total of 800 telephone

interviews is selected since this is twice the estimated number required to obtain a 0.05 level of statistical significance.^ The contractor will make every reasonable effort to increase participation above 30

percent.

Data collection. This segment of the study will use a computer assisted telephone interview to obtain data

from those who choose to participate in the study.

2.2 Face-to-face interview Format

The objective of the face-to-face interview segment is to gather first-hand accounts and observations of

the activities and events inside the buildings on the morning of September 1

1'''. This approach will

identify unknown information, evaluate technical hypotheses, and explore conscious and subconscious

motivations for occupant and responder behaviors, while allowing for comparisons to the telephone

interview data. There will be no verbatim record of the face-to-face interviews, other than random

selections for quality control purposes. It is estimated that the average face-to-face interview will last approximately two hours, with some lasting significantly longer.

The proposed methodology for the face-to-face interviews is a synthesis of the Behavioral Sequence

Interview Technique (BSIT) originally developed by Keating and Loftus,' and the Cognitive Interviewing

Method (CIM), originally developed by Fisher^ and Geiselman.^ These two interviewing methodologies were developed with the purpose of assisting persons in retrieving more comprehensive and accurate memories of incidents, and sharing important attributes. Both approaches begin by allowing the informant

to retell an unimpeded account without interruption from the interviewer, and both initially employ a chronological retelling of information. However, BSIT was designed to yield a database of quahtative

information that could be subjected to systematic analysis and consolidation, while CIM was designed to

facilitate investigative interviews. Since the Investigation is pursuing both goals (i.e., creation of a database of evacuation-related behaviors and an investigatory attempt to capture information relevant to outcomes), the proposed methodology combines these two approaches.

Cognitive interviewing has been the subject of many empirical investigations. Fisher, et al.'" summarized

these findings, demonstrating that the methodology significantly increases the amount of information

recalled without affecting rate of errors. Interviewing a large number of informants will allow

coiToboration of infonnation, thereby compensating for the likely increase in the absolute number of

errors. Accordingly, it is likely that this approach will be productive in achieving a holistic view of the

building evacuations.

44 NISTNCSTAR 1-7B, WTC Investigation
Data Collection Paper

The face-to-face interview methodology, hereinafter referred to as the "areas of interest study," will involve face-to-face interviews of occupants and first responders who may have observed (knowingly or unknowingly) events important to completion of the objectives of the hivestigation.

EnumeratinR the population. The population will include the entire occupant, management, and first responder population of World Trade Center WTC 1, 2, and 7.

Selecting the sample. The areas of interest sample will identify individuals using the snowball quota

sample approach whose constituency may resemble individuals selected for the "Specialized Groups

Study" sampling methodology (see below). A snowball quota sample approach asks individuals for the names of other people who may meet the selection criteria for the study. These people are subsequently

contacted and asked the same question. The process continues until the quota has been reached. The goal

is to perfonn approximately 600 face-to-face interviews with occupants, 30 face-to-face interviews with

family members who communicated with victims inside the building during the event, and 150 first responders. The 150 first responders will be divided among the Fire Department of New York

(firefighters, company officers, and operational command officers). Port Authority Police Department,

New York Police Department, and other responsible parties. Additional individuals may be randomly

selected from strata previously defined in the whole buildings study in order to compare the face-to-face

interview results with the results of the telephone interviews.

Data Collection. The face-to-face interviews will follow a four step technique, including unimpeded,

open-ended narrative, a structured narrative, technical probes, and closed-ended questions. Each step is described more fully below.

Step 1: Unimpeded open-ended narrative account. Both BSIT and CIM begin the process by asking

the participant to chronologically recount his or her "story." The proposed starting point is when it became apparent that something unusual had occurred on the morning of September 1 1 , 200 1 . The

proposed ending point is when the participant feels that he or she reached a location where they feh safe

(or, alternatively, when he or she successfully reached the exterior of the building). Researchers and

practitioners involved with cognitive interviewing believe that starting the face-to-face interviews in this manner both improves recall and helps build rapport between the participant and the interviewer. Fisher et

al.'^ also noted that asking questions may interfere with recall because a participant must divide his or her

mental resources between recall and listening to the interviewer's questions.

During the open-ended narrative account, the interviewer can record notable information that can be used

for the probing phase conducted later. For example, the participant might briefly mention an odd odor to which the interviewer will want to return to determine whether the smell might have been that ofjet fuel,

smoke, or of some other origin as yet unknown.

Step 2: Structured narrative account. After participants complete their stories, interviewers will prompt

them to go through the story again, but this time they will work cooperatively with the interviewer to record entries into a table. This approach is employed by BSIT for three primary reasons: (1) to yield a structured account that can be entered into a database without further processing; (2) to avoid the biasing

effects of having interviewers ask specific questions; and, (3) to enhance the effort at recall put forward

by participants by encouraging their active collaborative participation, an advantage to open-ended

formats as noted by Fisher, et al.'" Each row of the table will represent a single action in a sentential format, meaning that each action is expressed as a grammatical sentence. The approach is used based on the hypothesis that people encode

episodic memories in a manner consistent with this format, thus facilitating both recall and data entry.

Each column of the table represents three essential components of actions: a cue, an action, and the reason

for taking that action. Cues can be either external (e.g., signs of a fire, someone saying something) or

internal (e.g., remembering about another means of escape.) Actions are expressed using specific action

verbs (i.e., "ran" instead of "wenf) and may include artifacts (e.g., a fire extinguisher) used by the

NISTNCSTAR 1-7B, WTC Investigation 45
Appendix B

informant. Reasons are the intentional, goal-directed base for the action. The interviewer will encourage

the participant to use their own words to the greatest extent possible.

A hypothetical example of actions recorded in this manner is:

Cue Action Reason

1 heard but couldn't see someone yell "I've found a

clear path"

So 1 stumbled in the dark towards

where 1 thought the voice came

So that 1 could find a way to escape

My path was blocked by debris So 1 called out to whoever yelled,

"I'm near the reception area. Where are you?"

To try to get a better idea

about where the person was

Table 1: Example Tabular Face-to-face interview Data Entry

Experimental findings in psychological research on memory' ' suggest that when people perform actions,

their abilities to verbally recall those actions are significantly improved. Script theory'" suggests that people naturally organize their knowledge of actions using narrative sequences of actions structured

around their pursuit of goals. However, gaps in the narrative are anticipated, especially given the long

period of time that will have elapsed between the event and the interview. Interviewers will assist the

participants to fill in these gaps by asking them to recall events in reverse order, an approach used in CIM. Inter\'iewers will, however, encourage participants to report only those memories about which they

are confident really occurred to them.

Step 3: Probing for specific information. After completing the structured narrative account,

interviewers will ask specific open-ended questions (probes) intended to provide specific information of

particular value to the investigation. While some of this information is likely to be part of the structured

narrative account, participants may be able to recall other valuable information as well.

Interviewers may use "context reinstatement" from CIM to improve recall of important information,

because laboratory experiments have demonstrated that contextual cues enhance recall of related

information. Fisher et al. explain that context reinstatement may enhance recall because people use muhisensory coding of events. Using this mnemonic method, interviewers will ask participants to "mentally recreate the external environment, and their affective, physiological, cognitive, and emotional

states that existed at the time of original event." Depending on the population, probes may be used to try to elicit information including, but not limited to:

• Location of the informant at the time of certain marker events (e.g., location in WTC 1 when

WTC 2 collapsed) • Fire conditions (e.g., fire and smoke);

• Other cues of interest (e.g., the smell ofjet fuel); • Presence and activities of persons with disabilities; • Use of elevators by self or others; and,

• Knowledge of any obstacles to their progress while using the stairs. Because information about many of these areas of concern requires precise responses, questions for openended probes will be developed collaboratively between the contractor and NIST. Responses to probes

may be recorded using standardized formats where feasible. For example, all participants who observed

smoke may be asked to estimate the smoke density using an encodable scale, such as visibility distance.

Step 4: Close-ended telephone interview items. Participants will be asked to complete at least some

items from the telephone interview. These items are likely to be placed at the end of the face-to-face

interview session to avoid biasing open-ended responses. The purpose of administering telephone

46 NISTNCSTAR 1-7B, WTC Investigation
Data Collection Paper

interview items is to determine whether there are systematic differences among the representative samples

used for telephone interviews, face-to-face interviews, and focus groups.

Quality Control for Face-to-face interviews. To ensure that mtcrviewcrs are complying with the facc-to- face interview techniques and administrative requirements, this project is proposing that interviews be

videotaped, with the following provisions: the participant signs a form accurately describing the reasons

for and retention of videotapes and granting pennission for the face-to-face interview to be videotaped;

the contractor will destroy the video record at the conclusion of the investigation; review for quality

control will be the only purpose for videotaping the face-to-face interview; no transcriptions will be made.

NIST and the contractor will periodically review videotapes to ensure that interviewers precisely follow

the protocol and confonn to administrative requirements.

2.3 Focus Groups

WiUiams'^^ reports that in a group setting, people provide cues that evoke memories in others, and that social pressures mediate against reporting misrepresentations of what they recall. Thus, the goal of the

focus group interviews is to elicit accurate group representations of specific events or themes. Two

distinct populations will voluntarily participate in the focus groups: occupants and first responders. The

first set of focus group interviews will be the occupant sample. Distinct categories of people will be

selected for inclusion in this study, hereafter referred to as the "specialized groups study." The objective

of this study will be to capture the experience of people in unique places in WTC 1, 2, and 7. These

groups will be defined by the NIST Investigation team. Every effort will be made to include no less than 5 people in each of these categories in this study, with 10 people constituting the preferred focus group size. NIST anticipates conducting approximately 5 occupant focus groups.

First responders will constitute a second set of focus group interviews. The set of first responders will

include FDNY, NYPD, PAPD, and other groups identified as having operational or command authority at the World Trade Center on September 1 1''\ The focus group size will be determined as an operating unit

size, if applicable. An operating unit may be a Fire Department company, for example. This project

proposes 10 focus groups, each containing 5 people.

Sample selection . The people selected for inclusion in this study will be selected using non-probability

sampling procedures. The contractor will use a snowball quota sample.'''"'"'' Respondents contacted or

face-to-face interviewed for other reasons will be asked to provide the names and contact information for people they know in each of the categories in the specialized groups study. Names will be collected by the contractor until at least 5 people in each category have agreed to participate in an occupant focus group,

with a preference for 1 0 people. The same process will occur for selection of the first responder samples,

with a preference for inclusion of entire operating units (about 5 people per unit).

Data collection . Focus groups will be conducted with the members of each group selected for inclusion in each of the specialized categories included in this study segment. The data collected in this study will

produce qualitative and detailed narrative accounts of the experiences of each category of people. The

focus group discussion will be moderated by a trained and experienced contractor.

3. Database

The contractor will provide to NIST at the conclusion of the project a database of encoded survey results. Each telephone interview, face-to-face interview and focus group will result in an encoded table of resuhs

which can be analyzed using standard data analysis techniques, such as averages, multivariate regression,

and statistical significance. The specific identity of the encoding variables will be generated jointly by

the contractor and NIST and is subsequent to the actual content of the survey instruments, which will also

be developed by the contractor, subject to input and approval from NIST. The number of encoding

variables is anticipated to be less than 75. This database will need to be consistent with an analysis of

third-party and media accounts which NIST will generate and code independently of any contractors. Analysis of all data and any conclusions derived therein will be the sole responsibility of NIST.

NISTNCSTAR 1-7B, WTC Investigation 47
Appendix B

However, it is anticipated that a database expert from the survey contractor will assist NIST after database

delivery in developing an understanding of the structure, architecture, limitations, and use of the database.

4. Latency and Accuracy of Recall

The accuracy of participants' memories of events is a consideration, especially given the period of time

that will have elapsed between the September 1 1 attacks and the data gathering activities. Empirical

investigations reveal that greater amounts of information are recalled using CI methods without

increasing the rate of errors. For example, as compared to traditional epidemiological interviews, Fisher et

al. were able, with CI methods, to elicit many more responses and more precise responses from people

asked to recall daily physical activities from 35 years earlier. NIST will address latency in two ways. First, multiple participants who would have experienced similar

situations will be used to corroborate as much of the infonnation as possible. Thus, information that cannot be reconciled with other evidence may be discounted. Second, the proposed investigative

approaches are expected to increase the accuracy of the data collected. In a review of research, Pezdek

and Taylor'^ concluded that people retain fairly accurate memories of directly experienced events. They

hypothesized that participation in events leads to coherent well-structured narrative memories. Because

NIST will only be asking about directly experienced events, and will be asking participants to recall events in a manner compatible with their naturally occurring internal representations, the accuracy of

recall should be acceptable.

5. Protection of Human Subjects

This data gathering effort will ensure that all precautions required by the Common Rule for the Protection

of Human Subjects are met or exceeded by the contractor. Participation in any part of this project by any

person will be strictly voluntary. Interviewers will be trained to establish a rapport with participants

based on a compassionate interest in their story and will ensure participants that information provided will

be of value in preventing casualties in future building emergencies. During the briefing, interviewers will

provide information to participants about where and how to receive counseling without charge, and that participants may stop the interview at any time without explanation. Interviewers will also be trained to recognize signs of post-traumatic stress. Similar services will be offered to participants of focus groups

and to people taking the telephone interview. Finally, the contractor will take the necessary precautions

to ensure the safety of contract employees administering, collecting, or otherwise involved in this data

collection effort.

6. Additional Data Collection

The scenario may arise where an individual critical to developing an understanding of the events of

September 11, 2001, may be unavailable or unidentified during the period of performance of the Contractor. Thus, NIST will obtain NIST IRB approval to conduct a limited number of face-to-face

interviews with individuals deemed by NIST likely to contribute significantly to the outcome of the

Investigation. The scope, objectives, and procedures used in the additional data collection will be similar

to the scope, objectives, and procedures used by the contractor. Telephone interviews and focus groups

will not be conducted in this additional data collection effort. It is anticipated that the number of face-toface interviews conducted by NIST will be less than 10% of the number of face-to-face interviews

conducted by the contractor. The contractor will incur no duties or obligations related to the additional

data collection.

7. Conclusions and Summary

Table 1 summarizes the NIST Investigation survey method. NIST proposes a triangulated,

multidisciplinary survey methodology to analyze and document the events of September 11, 2001, at the

WTC 1, 2, and 7. The three strategies include telephone interviews, face-to-face interviews, and focus

groups. The triangulated approach was selected in order to increase confidence in the conclusions.

48 NISTNCSTAR 1-7B, WTC Investigation
Data Collection Paper

complete dual objectives of generalization and investigation, and increase memory recall and accuracy. The methodology and enumeration are summarized below.

The data collection will be conducted by a yet-to-be-selected contractor and is planned to begin as soon as the necessary pre-work is complete. This includes preparation of the telephone interview schedule, facc- to-face interview protocol, focus group protocol, training of contractor staff, and approval by NIST and

the appropriate Institutional Review Board (IRB) to assure compliance with federal requirements for the protection of human subjects. NIST will use established procedures to review all survey and face-to-facc

interview questions, data collection methods, and safeguards for maintaining privacy and confidentiality

of all instruments before proceeding with these critical data collection efforts. As additional details of the

survey methodology are finalized, populations to be included in this project may be modified.

The telephone interview approach is described as the selected floors study. WTC 1 and 2 will be divided

into three zones, low, medium, and high. The zones will be further stratified into combinations of singleand multi-tenant floors, as well as floors with low, medium, and high occupant densities at the time of the

first aircraft impact. This is anticipated to resuh in a 30 percent response rate, yielding approximately 800

respondents. Eight hundred respondents represent a safety factor of two for the population necessary to achieve 0.05 level of statistical significance.

The areas of interest study will be conducted with face-to-face interviews of up to 600 people. The

potential respondents will include:

• up to 200 people near floors of impact,

• up to 150 floor wardens, fire safety directors and persons with responsibility,

• up to 100 people in elevators or lobbies,

• up to 100 people from WTC 7,

• up to 30 family members of victims who called out of the towers, and

• up to 20 people with disabilities. The Behavioral Sequence Interview Technique and Cognitive Interview Method will be combined in the

face-to-face interview sessions. This approach will maximize the investigative return in order to identify

unknown information, evaluate technical hypotheses, and explore conscious and subconscious

motivations for occupant and responder behaviors, while allowing for comparisons to the telephone

interview data. NIST will also conduct face-to-face interviews with members of the Fire Department of

New York, Port Authority Police Department, New York Police Department, and others having

operational responsibilities. This approach will face-to-face interview approximately 150 people, with the

population being stratified among firefighters, company officers, and operational command officers.

The third approach will employ focus groups. NIST anticipates creating five focus groups of building

occupants with approximately 10 people per group. The population will be generated using the snowball

quota sample approach. Additionally, NIST will create approximately 10 focus groups with first responders, with each focus group containing approximately five individuals. The population will be

generated using the snowball quota sample approach.

NISTNCSTAR 1-7B, WTC Investigation 49
Appendix B

Method

Intended

Number of

Respondents

Intended

Response

Rate

Sampling

Strategy

Population

Telephone

Questionnaire Selected Floors

(Occupants)

800 30%

Statistically

Representative

Floors

WTC 1 and 2

o

Areas of Interest (Occupants)

570 N/A

Snowball Quota

and Randomly

Selected

WTC 1,2,

and 7

Faee

Int

(Families)

30 N/A Snowball Quota WTC 1 and 2

Face

to Areas of Interest 150 N/A Snowball Quota

FDNY,

NYPD,

PAPD, others

Focus

Groups

Specialized

Groups

(Occupants)

50 N/A Snowball Quota

WTC 1,2,

and 7

Specialized

Groups (First

Responders)

50 N/A Snowball Quota

FDNY,

NYPD,

PAPD, others

Table 2: Summary of Methods

50 NISTNCSTAR 1-7B, WTC Investigation
Data Collection Paper

' 15 CFR Subtitle A (1-1-99 Edition) Part 27.

" Dillman, D. 1978. Mail and Telephone Snn'ey. New York: Wiley Press.

^ Lindzey, D., Aronson, E., (Eds). 1985. The Handbook ofSocial Psychology, 3"' Ed. New York: McGraw-Hill.

Shaughnessy, J.J, Zechmeister, B.E. 1994. Research Methods in Psychology, 3'^'' Ed. New York: McGraw-Hill,

Inc.

' Cauchon, D. December 20, 2001. For many on Sept. 11. survival was no accident. USA Today.

^ Kalton, G. 1983. Introduction to Sun'ey Sampling. Quantitative Applications in the Social Sciences Series. London: Sage Publications.

^ Keating, J. P.; Loftus, E. L. Post Fire Interviews. 1984. Development and Field Validation of the Behavioral

Sequence Inten'iew Technique. Final Report. NBS GCR 84-477. Washington, DC: National Bureau of Standards.

^ Fisher, R.P., Falkner, K.L., Trevisan, M., McCauley, M.R. 2000. Adapting the cognitive interview to enhance long- term (35 years) recall ofphysical activities. Journal of Applied Psychology, 85(2), 180-189.

^ Geiselman, R.E., Fisher, R., Mackinnon, D. and Holland, H. 1986. Enhancement ofeyewitness memory with (he

cognitive inten>iew. American Journal of Psychology, 99. 385-401.

Fisher, R.P., Brennan, K.H., and McCauley, M.R. 2002. The Cognitive Intei-view Method to Enhance Eyewitness

Recall. In Eisen, M.L., Quas, J. A., and Goodman, G.S. (Eds). Memoiy and Suggestibility in the Forensic Inteniew.

Mahweh, New Jersey: Lawrence Erlbaum. " Nillson, L. 2000. Remembering actions and words. In E. Tulving and F.I.M. Craik (eds.) The Oxford Handbook of

Memoiy. New York: Oxford University Press. Schank, R. C, and Abelson, R. 1977. Scripts. Plans, Goals, and Understanding. Hillsdale NJ: Erlbaum.

Williams, F.D. 1990. SLAM: The Influence ofS.L.A. Marwill on the United States Army, in TRADOC. Ft. Monroe, VA: U.S. Anny.

Blalock, H. 1 972. Social Statistics. New York: McGraw Hill. Cochrane, W.G. 1977. Sampling Techniques. New York, John Wiley.

Pesdek, K. and Taylor, J. 2002. Memoiyfor traumatic events in children and adults. In Eisen, M.L., Quas, J. A., and Goodman, G.S. (Eds). Memoiy and Suggestibility in the Forensic Inten'iew. Mahweh, New Jersey: Lawrence

Erlbaum.

NISTNCSTAR 1-7B, WTC Investigation 51