Homeland Security -- Reducing the Vulnerability of Public and Private Information Infrastructures to Terrorism: An Overview
Released: 2004-02-10T00:00:00

Introduction

Analysis of the effects of the terrorist attacks of September 11, 2001, suggests various "lessonslearned" concerning public and private information infrastructures. What results are some insightsto the range of challenges and opportunities facing policymakers as they seek to identify relevanttechnical solutions to homeland security concerns. This report considers these homeland securityissues in the context of critical information infrastructure protection and continuity of operations (COOP) planning.

As part of the congressional and presidential efforts to develop and implement a comprehensive response to homeland security concerns, the role of information technology (IT) has become anincreasingly important focus. In July 2002, the President released the country's first NationalStrategy for Homeland Security , outlining the strategic objectives, critical mission areas, andinitiatives in support of the Administration's proposed Department of Homeland Security. (1) Thestrategic objectives included: preventing terrorist attacks within the United States; reducingAmerica's vulnerability to terrorism; and minimizing the damage and recover from attacks that dooccur. In November 2002 Congress passed, and the President signed into law, the HomelandSecurity Act of 2002 ( P.L. 107-296 ) creating the Department of Homeland Security. Common toboth the national strategy document and the legislation is an emphasis on developing informationsharing initiatives and fostering partnerships between and within the levels of government and thesectors of industry.

The heavy reliance upon information technology to carry out mission critical tasks and provideother citizen services highlights the need to ensure these assets are protected, backed up, and resilientto attack. Moreover, the growth of the use of electronic government (e-government) applicationsto conduct government-to-citizen interactions, as well as government-to-business and government-to-governmenttransactions, has put additional pressure on the need to reconstitute systems quicklyto minimize any disruptions and financial costs associated with a major infrastructure failure. (2) Inaddition, renewed emphasis is being placed on reducing the vulnerability of the nation's criticalinformation infrastructures while more fully integrating and utilizing public and private informationtechnology assets. (3) Taken together, these issuesdemonstrate the importance of ensuring thereliability and continuity of information technology systems, as part of the government's overallapproach to homeland security. The accounts of successes and failures regarding how agencies andbusinesses responded to the September 11, 2001 attacks provide an unusual opportunity to examineoptions for further improving the nation's emergency preparedness.

Relevance and Context of the September 11, 2001 Attacks

In addition to the destruction of buildings and the loss of life, the September 11, 2001 attacks on the World Trade Center (WTC) and the Pentagon inflicted heavy damage on elements of thecountry's information and communication infrastructure. This, in turn, affected how both public andprivate organizations were able to respond to the events of the day. First responders experienceddifficulties communicating among themselves. (4) Citizens and some governmental officialsexperienced problems communicating by telephone due to overloaded and destroyed circuits. Asdescribed in the sections below, some agencies and businesses directly affected by the attacks haddifficulties recovering and reestablishing data operations due to inadequate infrastructure and/or thelack of backup systems. Faced with an overloaded telecommunications system, many turned to theInternet, which continued to function as designed, to send and receive e-mail messages regarding thesafety of family, friends, and colleagues. However, even in areas not directly affected by the attacks,citizens and government employees sometimes found a dearth of information because some agenciesshut down Web sites or did not use them to provide information regarding available resources andinstructions on when and where to report for work. For example, it was reported that as of nearly48 hours after they occurred, neither the General Services Administration (GSA) or the CentralIntelligence Agency (CIA) had posted any information regarding the terrorist attacks. It was alsoreported that of the Office of Personnel Management (OPM) shut down its Web site due to concernsrelated to cyberterrorism. (5)

It is important to note that many of the technology-related problems that emerged from the September attacks have less to do with the capabilities of the technology itself than with how it wasimplemented. For example, phone lines can be disrupted, so organizations with critical functionsneed to secure redundant, but separate, means to communicate. Data can be stored and sent nearlyanywhere, but agencies need to establish protocols for regularly backing up important informationto secure, remote centers. The mixed performance of information infrastructures suggests that boththe public and private sectors need to reexamine their information planning and practices so that theycan better weather and rebound from catastrophic events. The damage sustained by two importanteconomic and military locations, combined with efforts to restore services and prepare contingencyplans, also raise questions regarding the federal government's role and the private sector's ability toensure the protection and continuity of the country's information infrastructure (e.g.,telecommunications, computer networks, Internet, etc.) in the future.

It is likely that the full extent of the damage to information technology resources will not be made public due to concerns about national security and business continuity. Traditionally, bothpublic and private sector organizations have been very reluctant to reveal publicly the extent towhich their operations are affected by computer viruses and worms, hacker attacks, or similarsecurity weaknesses. This reluctance to share information occurs for two primary reasons. The firstis the interest in maintaining the confidence of customers/constituents, and, by extension, in the caseof publicly traded companies, maintaining market value. The second reason is concern over beingidentified as a target for future attacks, and the possibility of revealing (unwittingly or not) othervulnerabilities. However, despite the validity of these concerns, the reluctance to share informationwith the appropriate actors can serve as an impediment to recovery and prevention planning byfurther embedding potential weaknesses into the information infrastructure the country has becomeincreasingly dependent upon. Despite the imperfect nature of the information available, a numberof lessons learned can be identified and are discussed below. To place these lessons in context, thenext section provides a brief synopsis of how the public and private information infrastructuresperformed and were affected in the wake of the initial destruction and the immediate reaction byindividuals, businesses, and the federal government.

Summary of the Events and Impact of September 11, 2001

Overview

Due to the scope of the situation, it is not possible to provide a comprehensive accounting of all the organizations affected by the terrorist attacks. However, a variety of examples are discussedin context to provide a sense of the range of issues facing the public and private sectors as theycontinue to review and implement new initiatives. One means to gain an overall sense of theimmediate impact of the September 11, 2001 attacks is to consider how people and organizationscommunicated.

The attacks spurred a tremendous spike in telephone calls that overloaded the capacity of some networks. Verizon normally handles 115 million calls per day in New York City and 35 million inWashington, DC, for a normal daily total of 150 million calls. Following the attacks, the combinedtotal jumped to 340 million calls. Similarly, Cingular Wireless said its call volume jumped 400%. Requests were made to international telecommunications carriers, such as France Telecom, to controlthe flow of calls to the United States in an effort to keep trans-Atlantic links open. (6)

Many people turned to cellphone-based text messaging, Internet-based instant messaging, and the use of two-way radio features of cellphones to get around the congested phone networks. AOLreported a 20% jump in instant messaging volume, handling 1.2 billion messages on September 11,2001. (7)

The National Communications System (NCS) activated the Government Emergency Telecommunications Service (GETS). Using a special phone number and a personal identificationnumber, GETS calls receive priority handling before all other calls on phone lines operated by ATT,Sprint, and WorldCom. During the week following the attacks, 3,000 GETS calls were made inWashington. An additional 4,000 GETS calls to and from Manhattan were completed with a 95%success rate. (8)

The General Services Administration (GSA) also provided mobile communications centers that supported several agencies, including the Federal Bureau of Investigation (FBI). In addition, theGSA Federal Technology Service (FTS) made 500 computers available to agencies within the firsttwo days of the attacks. (9)

Many agencies used their Web sites to keep the public informed and provide information on how to help the victims. One of those was the Federal Emergency Management Agency (FEMA)Web site. FEMA was one of the first agencies to post information about the attacks on the morningof September 11, 2001. On September 12, the FEMA site had 3.4 million visitors, an all-time highfor the agency. GSA used its site to notify people about the status of various governmentalbuildings. Many agencies, including the Department of Defense, used their sites to keep employeesinformed of changes. (10) Some Members ofCongress, including Senators Charles Schumer andHillary Rodham Clinton of New York, and Senator George Allen of Virginia, also turned theircongressional sites into information centers regarding the attacks. (11)

FirstGov's Web staff, who were evacuated from their Washington, DC, offices, worked at home immediately following the attacks, collecting information, phone numbers, and URLs for relevantsites. They posted this information to the FirstGov site on September 12. Many commercial newssites and other government sites then posted links to the FirstGov site (12) , helping drive 448,552unique visitors accounting for 1.75 million page hits during the week of September 9-15. FirstGovalso changed its site update schedule from every two weeks to updating it every 12 hours. (13)

New York - the World Trade Center

At 8:45 AM, American Airlines Flight 11 crashed into the north tower of the World Trade Center. Eighteen minutes later, United Airlines Flight 175 crashed into the south tower. Theeventual collapse of both towers inflicted heavy damage to the surrounding buildings andinfrastructure, ultimately resulting in the collapse of other buildings on the site and the deaths ofnearly 3,000 people. The attacks displaced large numbers of both public and private sectoremployees. The World Trade Center contained an estimated 430 tenants with 50,000 employees (notall present at the time of the attack), and typically received another 140,000 visitors on a dailybasis. (14)

According to the General Services Administration (GSA), more than 2,800 federal employees worked in offices leased by the GSA in Buildings 6 and 7 of the World Trade Center complex. Building 6 had over 2,000 federal employees from a variety of agencies, including the CustomsService, Bureau of Alcohol, Tobacco, and Firearms (ATF), the Occupational Safety and HealthAdministration (OSHA), the Export-Import Bank, the Foreign Commercial Service of theDepartment of Commerce, and the Pension and Welfare Benefits Administration of the LaborDepartment. (15) Among the tenants of Building7 were 760 federal employees from agencies includingthe Secret Service, the Equal Employment Opportunity Commission (EEOC), the Department ofDefense, and the Internal Revenue Service (IRS). (16) Another 25,000 federal employees wereevacuated from four nearby buildings; 26 Federal Plaza, 290 Broadway, 40 Centre Street, and 500Pearl Street. (17)

The attacks also inflicted heavy damage on elements of the city's information and communication infrastructure, including both land lines and wireless services. One switching facility,which handled 40% of the lower Manhattan phone lines and 20% of the New York Stock Exchange's(NYSE) traffic, was damaged when steel beams from a collapsing building punctured the switchingstation, flooding it with water and debris. A second switching facility, which normally handles 80%of the NYSE's 15,000 phone and data lines, did not suffer direct damage, but was renderedinoperable by intermittent power outages. (18) Inaddition, several wireless cell sites were destroyedand others were rendered inactive by power outages. Communication between the New York FireDepartment, the Emergency Medical Systems (EMS), and the New York Police Department werealso interrupted due to the loss of an antenna that had been on 1 World Trade Center. (19) By January2002, Verizon had restored service to 99% of the affected area. (20)

Local television stations were also affected. Nearly all of the broadcasters had their main antennas located on the roof of the north tower of the World Trade Center. Two stations had backupantennas on the Empire State Building, allowing their signals to still be received by most citizenswho were not wired for cable television. The other broadcasters had to utilize towers in more distantlocations, such as Alpine, NJ, where they could only reach portions of the New York metropolitanarea. Several broadcasters were later able to establish a makeshift tower site on the Empire StateBuilding. Broadcasters worked with lawmakers to try to identify a site in either New Jersey or NewYork for a common tower that would be located within an appropriate radius of the World TradeCenter site, necessary to reach the local residents while not interfering with broadcasts inPhiladelphia or Boston. In May, 2003 it was reported in the New York Times that the MetropolitanTelevision Alliance signed an agreement with the developer of the World Trade Center site to installup to 22 antennas on Freedom Tower, which is anticipated to be completed in 2008. (21)

Virginia - the Pentagon

At 9:38 AM, American Airlines Flight 77 crashed into the west face of the Pentagon, killing 64 passengers on board and 125 additional people on the ground. The crash and ensuing firedestroyed an estimated 10% of the Pentagon's office space, and reportedly disrupted one of thePentagon's two major communications lines. (22) The Navy lost 70% of its Pentagon offices, includinga portion of the Navy's budget office, the Office of the Chief of Naval Operations, and itstelecommunications operation center. (23) The U.S.Army's Information Management Support Centeralso received significant damage, losing most of its desktop computers, its entire central help desk,and apparently was unable to access its backup tapes. (24) In addition, the Defense Finance andAccounting Service was damaged. Computerworld magazine suggested that many of the Navy'stop-secret network operations were probably damaged, although it was believed to be unlikely thatthis affected the Navy's ability to communicate sensitive information to Navy vessels. (25)

The attack on the Pentagon caused the loss of knowledge assets, including hard copies and data on workstations and servers that were not duplicated or backed up and stored in a different physicallocation, according to media reports. (26) It is notclear how much information may have beenpermanently lost or to what degree a lack of backed up information hampered efforts to continue theoperations of the affected offices. One office that was able to resume its functions quickly was theDefense Finance and Accounting Office, which maintains servers located in Ohio. (27) Also, the Navywas able to utilize its Navy Marine Corp Intranet (NMCI) contract to assist its efforts to resumeoperations. Using the NMCI contract, the Navy relocated approximately 1,000 of its displacedpersonnel to temporary offices in Arlington, VA, and had 860 laptop computers, 335 desktopcomputers, and 30 servers, routers, and cabling delivered and installed in just over one week. TheNavy's Budget Office, which was in the middle of preparing its budget that was due October 6 whenthe attacks occurred, lost part of its server farm. However, it had 50 computers and its server farmrestored by Sunday, September 16. (28) Efforts toreconstruct and repair the 400,000 square feet ofdamaged Pentagon offices, dubbed Project Phoenix, progressed rapidly, with the first group ofpeople moving back into their rebuilt offices on August 15, 2002. (29) The entire E-ring portion of thedamaged Wedge 1 area was restored for use by September 11, 2002. (30)

Lessons Learned

In the months following September 11, 2001, there were a number of accounts of successes,failures, and 'lessons learned' regarding continuity and disaster recovery planning. In many cases,these descriptions are specific to a particular organization or business activity. However, one canidentify some observations and lessons learned that are widely applicable and that policymakers andbusiness leaders may wish to consider as they develop and implement new homeland securityinitiatives.

Continuity of operations (COOP) and disaster recovery planning are not new concepts. However, surveys have shown that only about half of American businesses have disastermanagement plans in place. In many cases, past threats are often the motivating influences fororganizations to make these plans. For example, Morgan Stanley, one of the tenants in the southtower of the World Trade Center, adopted thorough plans in response to bomb threats being madeduring the Persian Gulf War in 1991, and reinforced those plans following the 1993 bombing of theWorld Trade Center. The financial services firm's regular evacuation drills were credited as one ofthe reasons why nearly all of its approximately 3,500 employees were able to escape before thebuildings collapsed. (31)

In the case of information technology disaster recovery planning, preparation for the year 2000 transition (Y2k) has been cited by a number of private and public sector organizations as one of themain reasons they were able to respond and recover quickly from the September 11, 2001, attacks. (32) Y2k planning began substantially in the 1990s, led primarily by the private sector and followed byfederal and state government agencies. (33) Y2kpreparation spurred many organizations tooperationalize strategies for backing up data, enabling remote working, and enhancingcommunication links between organizations, employees, customers, and vendors. (34) Many tenantsof the World Trade Center also cited the 1993 bombing of the building as their rationale for havingdeveloped extensive disaster recovery plans, which they, in turn, attributed to their success inevacuating employees and preserving vital data. (35)

The amount of time and resources spent on information technology disaster planning can vary with the size and type of organization. According to the Gartner Group, a research and advisoryfirm, an average company allocates approximately three percent of its annual information technologybudget to disaster recovery. In contrast, financial services companies, which have to meetrequirements set by the Federal Reserve Board and/or the Securities and Exchange Commission(SEC), spend an average of seven to eight percent. (36)

However, if it may seem costly to dedicate a significant portion of one's budget to planning for an event with a low chance of occurring, the financial consequences for being unprepared can beeven higher. In the case of financial companies, many, if not most, had invested in data backup anddisaster recovery facilities to one degree or another. While this allowed them to save a significantamount of customer and business-critical data, it was estimated that these firms will still spend $3-5billion over the two years following the attack to replace their destroyed information technologyinfrastructure. (37) For example, Dow Jones Inc.,a global financial news company and publisher of TheWall Street Journal and Barron's publications, was expected to spend $2 million to replaceinformation technology hardware and office equipment. (38) Although it had relatively few offices inor around the World Trade Center and a portion of the affected area of the Pentagon was notoccupied, the federal government was also expected to spend a significant amount to replace lost anddamaged information technology systems. Input, a Web-based information technology marketresearch and marketing services firm, predicted that the federal government would spend $75million, with the Customs Service (now part of the Department of Homeland Security) aloneexpected to account for $15 million of that amount. (39)

While information technology disaster recovery planning is often compared to the preparations for Y2k, it is important to recognize that these scenarios are qualitatively different. Y2k had a finitetime line with a clear indicator of success or failure. In contrast, the war on terrorism appears to bean open-ended and evolving process. As the examples below demonstrate, measures of success arerelative, and the task of planning is never truly done. The lessons learned today can help prepare fortomorrow, but they do not represent the final word on information technology disaster recoveryplanning.

Lessons Regarding Continuity and Recovery Planning and Practices

The events of September 11, 2001, have brought a new urgency to continuity and recoveryplanning and practices. While attention has been growing over time, this multifaceted undertakingcan often be a very challenging and frustrating process as planners try to coordinate disparate partsof their organizations while trying to strike a balance between how much they cannot afford to beunprepared and how much they can afford to spend on resources they may never use. Furthercomplicating matters has been the tendency for organizations to "stovepipe" the different protectionsrelevant to information technology disaster recovery planning. For example, information securityhas often been handled independently from physical security. Similarly, the compartmentalizationof an organization's units and processes can contribute to a fractured planning process that can leavean organization vulnerable. (40)

One reported example indicating the possible costs and consequences when an organization does not have a fully integrated plan is the May Davis Group. The privately held financial servicescompany had its offices on the 87th floor of one of the World Trade Center towers. In addition tolosing $100,000 of equipment, the firm apparently lost some regulatory documents and $1 millionin revenue due to data loss and downtime. (41)

A less severe example is the Secret Service field office, located in 7 World Trade Center, which did have a contingency plan, but had not fully implemented it at the time of the attack. The agency'sfield office was able to resume operations the following day at an alternative location. It reportedlylost some of the information it was collecting on criminal suspects. (42)

Another example is Deloitte Consulting, whose primary telecommunications hub in New York City, located in the World Financial Center next to the World Trade Center towers, was put out ofservice, affecting an estimated one thousand employees in the area. As a consulting firm, Deloitteconsiders its ability to share information with clients to be its core business. During the weeks ittook to rebuild its land line communication center, Deloitte's New York area employees used cellphones to stay in contact with customers and fellow employees. (43)

Even organizations that have planned extensively can sometimes overlook small details that appear insignificant, but later prove to be important. One such example is American InternationalGroup (AIG) Inc. The financial services company has offices and information technology operationsnear the World Trade Center site. Using its backup business center in Parsippany, NJ, and a secondpreviously contracted emergency facility in Livingston, NJ, AIG was able to use data from itsautomated backup system to be operational the day after the attacks. However, not all of its serverswere part of the automated backup system, and the backup tapes for these servers reportedly wereleft behind when its building was evacuated. (44)

Augment Disaster Recovery Plans

In many respects, the September 11, 2001, attacks established a new standard for disasterrecovery plans - the complete destruction and loss of a building. Even among some of the mostprepared organizations, their plans sometimes presumed the ability to eventually return to theiroffices, even if only to retrieve equipment and paper files. (45) Modern fire safety and constructionmethods have largely made the possibility of a building collapse less likely, so many organizationsdeveloped plans that focused on the movement of data to be used temporarily at a backup facility. These same businesses and agencies must now consider whether to augment their disaster recoveryplans to include the movement of people, the rapid acquisition of equipment and furniture, networkconnectivity, adequate workspace, and more. Some organizations may need to be able to not juststore, but also to run mission critical applications, at their backup sites, and staff may need to betrained to implement such a plan. (46) The possibleneed for more sophisticated data backup facilitiessuggests organizations may consider establishing a 'hot site'. A hot site is a facility that has all ofthe data, equipment, software, connectivity, furniture, and office space assembled and ready to useso that an organization can continue its computer operations uninterrupted in the event of a disaster. In some cases, organizations will mirror their data directly to the hot site as an additional backup orin lieu of using backup tapes at the primary site. While hot sites provide the greatest amount ofredundancy and readiness, the cost of establishing and maintaining such a site can cost millions ofdollars, with additional yearly maintenance costs. (47) This raises cost effectiveness issues. Some hotsites, however, might serve a dual purpose, such as a secondary data site or as part of acomprehensive backup system. (48)

Organizations with more detailed recovery plans were often able to respond better to the events of September 11, 2001. In the case of the Occupational Safety and Health Administration (OSHA),the 21-member office had a contingency plan for its contingency plan. After evacuating, the OSHAemployees discovered their first designated meeting site was inaccessible, so they regrouped at anearby regional office. After this office was evacuated, they gathered at a third site to implementtheir plan to provide advice and technical assistance to businesses and agencies in an effort to protectworkers from being exposed to hazardous substances and other safety risks at ground zero. (49)

A second example is the New York Board of Trade (NYBOT), which handles trading for commodities such as coffee, sugar, and orange juice. NYBOT had invested in a hot site in Queens,NY, which included limited space for a reduced number of trading pits. According to one report,NYBOT had invested $1.75 million to establish the facility, which cost an additional $300,000 peryear to maintain. However, while an extended outage would have resulted in the rapid loss oftrading contracts to other exchanges in the United States and abroad, NYBOT's investment paid off,allowing it to resume modified trading operations one day after the attacks. (50)

A corollary to this lesson learned is the need to ensure that recovery procedures are welldocumented and safeguarded. Employees need to be well-informed and practiced for theresponsibilities they are expected to carry out. Correspondingly, just as one would not keep the solebackup tapes of important information at the primary data center in which they were created, aprudent step would be to keep one or more copies of the recovery plans available at the appropriateoff-site locations.

Backing Up Data and Backing Up Applications

A second lesson learned that is related to continuity and disaster planning and practices is theneed to have a comprehensive backup system that captures more than just an organization's datafiles. As discussed above, most of the financial services firms in or near the World Trade Center hadcomprehensive data management and recovery systems in place. Some highly automated systemswill begin backing up data to a remote center when a significant temperature change or power lossin the building is detected. (51) Cantor Fitzgerald,one of the hardest hit financial services companies,losing 733 of its approximately 1,000 World Trade Center employees, including 150 informationtechnology workers, lost none of its data due to a system that mirrored all of its software and datato its data center in Rochelle Park, NJ. (52) Incontrast, the U.S. Customs Service was not as wellprepared. In addition to experiencing difficulties finding alternative office space for its 800displaced employees, the office located in the World Trade Center complex did not have all of itsfiles backed up to computers in its Washington, DC, headquarters. Several months after the attack,the Customs Service reportedly was still working to re-create some of its files from scratch whileothers were considered permanently lost. (53)

In addition to backing up data, organizations also need to backup the data catalogs, directories, and software applications used with the data. Organizations that saved only their raw data had tospend a significant amount of time re-creating their applications, organizing the data, andreestablishing user permissions to access the data. (54) One company that apparently experienced thisproblem was NYBOT. Despite its extensive hot site contingency plan, NYBOT lost some financialrecords, applications, and e-mail files that were not backed up to the site. Some of this informationwas backed up to tapes, but the tapes were stored in a fireproof safe kept in their World Trade Centertower office. (55)

In addition to taking a broader view of the digital tools and assets to backup and preserve, organizations - such as regulatory agencies and insurance companies - that still rely heavily onpaper files may wish to consider digitizing some of their documents as they are received. Despitemuch touting of the so-called paperless office, the blizzard of paper that accompanied the dust anddebris with the collapse of the towers suggests many organizations are still heavily dependent ontheir physical documents. However, the high cost of digital imaging requires companies andagencies to consider carefully which documents are most critical and often used.

One company that did have a comprehensive digital imaging system in place before September 11, 2001, was Empire Blue Cross Blue Shield. Developed over the past ten years, starting withclaims forms, the insurance carrier's optical storage system captures almost all of its paperdocuments. As a result, the company lost only about two days' worth of paper mail. (56)

One company that decided to accelerate its plans to digitize and automate its paper-based information is Kemper Casualty Company, which had offices on the 35th and 36th floorsof OneWorld Trade Center. The Kemper data backup system had all but the previous day's and thatmorning's data backed up on tape at its headquarters. It was also able to re-create the losttransactions for the missing data. However, according to media reports, Kemper lost thousands ofpaper documents, including innumerable insurance policy applications. The company had to spenda significant amount of time and resources to re-create the information by going back to customersand Kemper agents. (57)

The Securities and Exchange Commission, whose New York regional office was located in 7 World Trade Center, also did not digitize its paper records. While the regulatory agency did havea significant amount of its data and files backed up electronically, case files that took years tocompile, informal notes written down from interviews and analysis, and other documents were lost. Some of these lost records reportedly were part of pending cases, including those related toinvestigations of insider trading and financial fraud. While the SEC can have parties submit newcopies of documents previously provided, and it may be able to obtain some documents fromagencies conducting parallel investigations, documents and other evidence from older cases andsmaller companies that are not in operation any longer can prove to be difficult to recover. The lossand reconstruction of the files generally can slow down the progress of some investigations andpossibly result in others being discontinued. (58)

A corollary to the lessons learned regarding comprehensive data storage plans is the need to fully and regularly test backup sites and media. Organizations that rely on 'cold sites' - backup sitesthat are not always in use and may require the organization to install hardware, software, or load datato become functional - in the event of a disaster, could experience further problems if theydiscover their tapes are corrupted or equipment does not work. One organization that regularlytested its backup systems and information was NYBOT. The exchange tested its backup sitemonthly and practiced its recovery plan every 60 days to assure its systems were working and itsemployees were familiar with the procedures. (59)

Lessons Regarding Decentralization

A second major category of lessons learned concerns the decentralization of operations and theeffectiveness of distributed communications. The rise of networked computing and the Internet hasprovided the opportunity to connect far-flung locations around the country and the world. Manypublic and private sector organizations have used this technology to reach outward to new markets,deliver new services, and reduce communications costs. However, the lessons learned discussedbelow suggest there is also a need for organizations to turn this technology inward to reduce thevulnerabilities of internal operations and to strengthen communications links with internal, as wellas external, constituencies.

Decentralize Operations

The maxim, 'don't put all of your eggs in one basket,' can be applied to a variety of situations: college applications, job searches, and investment portfolios. It can also be applied to the structureand location of an organization's operations. Although many of the tenants and neighbors of theWorld Trade Center had backup facilities, some of these facilities were located within a few blocksof their primary location, resulting in the loss, or at least the inaccessibility, of data at both siteswhen it was needed most. According to some industry experts, recovery sites should be located atleast 20-50 miles way from the primary data center. In addition, some observers suggest that humanresources should also be located in more than one spot to reduce the potential for losing a significantportion of one's workforce in a single event. (60)

As was mentioned above, Cantor Fitzgerald lost 733 of its employees, including 150 of its information technology workers. However, despite this devastating loss, the firm was able tocontinue its operations in part by relying on employees in its other offices to assume some of theresponsibilities of its World Trade Center office. (61) Another company that benefitted fromdecentralization was Blackwood Trading LLC. The brokerage firm's offices were located six blocksfrom the World Trade Center, part of the area that lost power and communications service, but itmirrored all of its data at a remote site in Jersey City, NJ. It purposely chose a site physically distantfrom its offices to reduce the chance of a complete loss of data in the event of a terrorist attack ora natural disaster. (62)

Since September 11, 2001, several other firms have decided to heed the decentralization lesson. One of those organizations is Dow Jones. It had maintained offices at One World Financial Center,adjacent to the World Trade Center, which served as a hub for 800 employees and its data productioncenter. While the damage to the building was not significant enough to prevent its eventual repairand reoccupation, the damage did render approximately 100 servers, 400 workstations, and anestimated "millions of dollars" of the company's networking equipment inoperable. As part of itsnew strategy to reduce its vulnerability, Dow Jones reportedly decided to permanently move thecompany's data center to its backup facilities in South Brunswick, NJ, and move approximately halfof its employees back to its original Manhattan location, with the remainder at other locations. Thefinancial information company plans to rely on a network of news and data centers distributedaround the country. It will also rely more heavily on remote offices, encourage telecommuting, andestablish a new backup facility in Secaucus, NJ, separate from its South Brunswick data center. (63)

Empire Blue Cross Blue Shield, which benefitted significantly from its extensive backup system, has also decided to decentralize the 1,900 employees from its World Trade Center officesinto three different facilities. In addition to spreading the risk, the insurance carrier also observedthat it is easier to conduct a data recovery operation for a smaller portion of its operations ascompared to the larger whole in the event of a disaster. (64)

Ensure the Ability to Communicate with Internal and External Constituencies

Another lesson learned related to decentralization is the need to ensure the ability to communicate with internal and external constituencies in the event of an emergency. The attacks ofSeptember 11, 2001, forced many companies to rely on ad hoc networks to communicate withemployees. The overload of telecommunications networks, including cellular voice networks, leftmany people scrambling to find a way to get messages out. Some have suggested relying onwireless data backup systems as an alternative to voice networks in the case of an emergency. Onesuch network that proved useful for many people is a data network, which is used to support theBlackBerry pager, a mobile e-mail and paging device. (65) In the weeks following the attacks, theCommittee on House Administration supplied BlackBerry devices and monthly service to all 435Members. (66) Demand for BlackBerry pagerdevices by other federal agencies has jumpeddramatically since September 11, 2001. (67)

Most organizations are aware of the importance of communications with external constituencies: citizens and customers. In the case of government agencies, this function isbecoming associated with electronic government. Many people searching for information andguidance during the uncertainty that followed the attacks turned to corporate and agency Web sites. However, the performance of these sites was mixed. Some sites immediately transformed themselvesinto centers for crisis information, while others were shut down altogether. The Office of PersonnelManagement (OPM) initially shut down its Web site citing cyber attack concerns. The site wasbrought back online late in the afternoon of September 11, 2001, with the announcement thatagencies in Washington, DC would be open the next day. (68) The Federal Aviation Administration's(FAA) Web site was also offline for a significant portion of the day. (69)

The Department of Defense used its DefenseLink site to provide information and pictures about the attack on the Pentagon. DefenseLink, the Defense Department's main Web site, experienced a243% increase in page hits the week following the attacks. In order to handle the increased traffic,DoD tripled the sites' bandwidth capacity. Other sites that experienced surges in Web trafficincluded the FBI, the Department of Justice, and FirstGov.

FEMA regularly updated its Web site with information. The number of visitors to the FEMA site demonstrates the expectations of citizens to be able to find information online. FEMA normallyhas 500,000 visitors to its site every day. On September 11, 2001, that number soared to 2.3 million. FirstGov was a bit slower in responding, but, by late in the day of September 12, had cobbledtogether a page providing links to information from a variety of federal, state, and local agencies, aswell as non-governmental organizations helping with the crisis. The FBI established a special siteto collect tips about the terrorists. Of the 80,000 tips reportedly received within the first five daysof the attack, more than half, or 47,052, were received via the Web site. (70) Following the anthraxincidents, the Centers for Disease Control Web site also experienced a notable increase in visitors. (71)

Lessons Regarding Redundancy and Planning of Communications

The third category of lessons learned involves the institutionalization of redundancy ininformation infrastructures. Redundancy, as used here, includes having computer or network systemcomponents, including hardware, software, and telecommunications links, installed and ready to useas a backup in the event primary resources fail. A related aspect of redundancy is the ability toreplace and/or reconstruct hardware and software quickly and easily when necessary to preventextended periods of downtime.

Employment of Redundant Service Providers

Redundancy, or the lack thereof, proved to be critical to many agencies and businesses in lower Manhattan. As described earlier, most of the area's telecommunications lines were connectedthrough two primary switching stations, one of which was destroyed and the other rendered uselessdue to a lack of electricity. Some organizations that thought they had redundant connections bycontracting with two different service providers discovered that both providers used the same centralswitching office, leaving the organization without service. Approximately 40 of the competitivelocal exchange carriers in the area relied on Verizon's 140 West Street facility to provide theirservices. (72)

One group of businesses that did not lose access to their data networks were the tenants of the New York Information Technology Center, an office building located at 55 Broad Street inManhattan, six blocks from the World Trade Center. Among the services provided by the building'smanagement company is a full telecommunications infrastructure served by 14 voice and datacarriers. As a result, it was reported that none of the tenants lost access to their data networks, andthe few tenants who did use Verizon's voice services were able to switch to another of the building'sproviders within 24 hours. (73)

Two other companies that had redundant networks were Lehman Brothers and Empire Blue Cross Blue Shield. Lehman Brothers, which had offices both in and around the World Trade Center,had fully redundant networks in Manhattan and in Jersey City, NJ, along with duplicative wide-arealinks that kept all 45 of its branches connected. (74) Empire Blue Cross Blue Shield, whose backupsystem was referred to earlier, also had multiple voice and data carriers that connected its WorldTrade Center offices to its redundant data centers outside of Manhattan, which allowed the insurancecarrier to continue to serve its 4.5 million customers. (75) It was reported that the insurance carrier wasconsidering plans to install satellite receivers on the roof of its new building in Brooklyn so it wouldbe able to transmit data between facilities worldwide. In addition to being a less expensivealternative to using several high speed land lines, the satellite receivers would also enable thecompany to continue to transmit data in the event that the city's land line infrastructure experiencesa disruption. (76)

Use of Generic Replaceable Technology

A related lesson that some organizations have cited as valuable to their ability to rebuild their systems quickly is the use of generic, replaceable technology. Agencies and financial firms facedwith the need to rebuild their systems quickly -- in some cases, in a matter of days -- received asignificant amount of support from many of the major technology vendors, including, but not limitedto, Compaq, Dell, IBM, and Sun Microsystems. Drawing from their existing stock of equipment,and ramping up production, the vendors were able to supply large amounts of equipment on shortnotice. Several information technology companies donated equipment and services to federalagencies in Virginia and New York to assist with recovery efforts. (77) Many vendors also provideddiscounts, sometimes as high as 80%, for their commercial clients. (78) Vendor support also came inthe form of emergency help desk support, cross-country equipment deliveries, and on-site technicalsupport. For example, IBM helped Empire Blue Cross Blue Shield replace over 2,200 desktops and413 laptops, while Compaq replaced the insurance carrier's 256 servers. (79) Dell and Compaq alsoprovided the American Red Cross with desktops, laptops, servers, and other equipment to assist withthe relief efforts. (80) Based on the experience ofSeptember 11, 2001, the ability to replace equipmentquickly with easy-to-find products that do not require significant customization is likely to be oneof the factors affecting organizations' future continuity and disaster recovery planning decisions.

Future Considerations

Although there are undoubtedly additional lessons learned from the September 11, 2001 attacks,the lessons highlighted in the previous pages provide a broad sense of the breadth and depth of theissues facing public and private sector organizations. While not all-inclusive, they emphasize threegeneral approaches: the establishment and practice of comprehensive continuity and recovery plans,the decentralization of operations, and the development of system redundancies to eliminate singlepoints of weakness. The lessons learned from September 11, 2001 build, in part, upon the lessonslearned from the 1993 World Trade Center bombing and the preparation for the Y2k transition. However, as agencies and businesses move ahead with continuity planning and implementation,there are indicators that the character of this preparation is changing.

Emphasis on Business Continuity Over Disaster Recovery

One change is that the new lessons learned appear to represent the shift to a

higher standard of continuity and disaster recovery planning. (81) Comprehensive contingency plans,perhaps once viewed, at the least, as optional and, at the most, as a prudent measure, may now beseen as an integral part of developing and maintaining an organization's information technologyinfrastructure. Once considered a remote possibility, the permanent loss of a facility, while stillunlikely, must now be taken more seriously. Consequently, an increasing number of organizations,including small and mid-sized companies who often have more limited resources, have begun tofocus not just on disaster recovery, but on business continuity. In a networked economy, the costsof network downtime can be measured in tens of thousands of dollars per hour, and as high as onemillion dollars per hour for highly technology-dependent entities such as infrastructure services firmsand energy companies. (82) Other concerns, suchas the loss of electricity, the increased frequency ofcomputer viruses, and high-profile hacking attempts have also spurred many organizations to focuson comprehensive business continuity planning rather than disaster recovery alone. (83)

In addition to these higher standards, there is also greater recognition of the qualitative change in preparation. Whereas some organizations may have felt their Y2k readiness measures providedadequate protection, there is now a greater realization that continuity and disaster recovery planningis an open-ended and evolving process, requiring reinforced and redundant infrastructures, regularpractice exercises, and testing of data backups and systems. For example, the Pentagon, whichalready had contingency plans in place, embarked on its Command Communications SurvivabilityProject in an effort to redesign its information technology contingency plans. (84) In the 107th Congress,Senator Ted Stevens introduced an amendment (SA 2450) to the Department of DefenseAppropriations Act, 2002 ( P.L. 107-117 ) on December 7, 2001 that would have required agenciesto have "redundant and physically separate" telecommunications systems in an attempt to maintainthe operability of communications of government offices in the event of an attack or catastrophe. The amendment passed on a voice vote in the Senate but did not pass in the House ofRepresentatives. (85)

Information Sharing and Collaboration

Another change is the potential for increased information sharing between federal, state, and local government, as well as between the public and private sectors. While information sharingfigures prominently in plans for law enforcement-related homeland security activities, it also mayplay an important role in continuity planning and critical information infrastructure protection. Forexample, the Chief Information Officers (CIO) Council, which serves as an interagency forum forthe CIOs of thirty federal departments and agencies, decided to include a representative from theNational Association of State Chief Information Officers (NASCIO) in its activities. In addition tocollaborating on issues such as interoperability in wireless communications and electronicgovernment initiatives, several state CIOs are working with federal officials on efforts related tohomeland security, information sharing, and protecting against terrorism. (86)

Another example is Operation Dark Screen, initiated by Representative Ciro Rodriguez. (87) Thethree-phase exercise was conducted over several months during 2002 and 2003 as a partnershipbetween federal, state, and local government, and the private sector. It was designed to test thepartners' preparedness to protect critical infrastructures from cyberattacks. The organizers conductedboth a tabletop and a live exercise. (88) As additionalactivities such as Operation Dark Screen arecarried out, and organizations continue to rebuild and reinforce their information technology assets,it is anticipated that further lessons learned will be added, providing a fuller assessment of our stateof readiness, and guidance for the development of future homeland security initiatives.

For Further Reading

CRS Reports

CRS Report RL31594 , Congressional Continuity of Operations (COOP): An

Overview of Concepts and Challenges , by [author name scrubbed] and Jeffrey W.

Seifert.

CRS Report RL31857 , Continuity of Operations (COOP) in the Executive Branch:

Background and Issues for Congress , by [author name scrubbed].

CRS Report RL30153 , Critical Infrastructures: Background, Policy, and

Implementation , by [author name scrubbed].

CRS Report RL31534 , Critical Infrastructures Remote Control Systems and the

Terrorist Threat , by [author name scrubbed].

CRS Terrorism Electronic Briefing Book EBTER129, Information and

Telecommunications Infrastructure , by [author name scrubbed] and [author name scrubbed].

CRS Report RL31978 , Emergency Preparedness and Continuity of Operations

(COOP) Planning in the Federal Judiciary , by [author name scrubbed].

CRS Report RL31493 , Homeland Security: Department Organization and

Management , by [author name scrubbed].

CRS Report RL31513, Homeland Security: Side-by-Side Comparison of H.R. 5005

and S. 2452, 107th Congress , by the CRS Homeland Security Team.

CRS Report RL31465(pdf) , Protecting Critical Infrastructure from Attack: A Catalog of

Selected Federal Assistance Programs , coordinated by [author name scrubbed].

Other Resources

Dorobek, Christopher J., "Web Sites that Worked," Federal Computer Week , 1

October 2001, p.18.

INPUT, Attack on America: The Impact of the September 11 Terrorist Attacks on the

Federal Government , 3 October 2001,

http://www.input.com/article_printver.cfm?article_id=310 .

National Research Council. 2002. The Internet Under Crisis Conditions: Learning

from September 11 . Washington, DC: National Academy Press.

Pentagon Renovation Program Web Site http://renovation.pentagon.mil/ .

Schulz, James, "New Urgency for Disaster Recovery Planning," Washington

Technology , 8 October 2001, p.18.

"Special Coverage: Attack on America," Computerworld ,

http://www.computerworld.com/news/special/pages/0,10911,1446,00.html .

"Special Report: September 11, 2001," Government Executive Magazine ,

http://www.govexec.com/091101report.htm .